From coleen.phillimore at oracle.com Thu Sep 1 00:28:16 2016 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Wed, 31 Aug 2016 20:28:16 -0400 Subject: [9] RFR (X): 8163880: Constant pool caching of fields inhibited/delayed unnecessarily In-Reply-To: <671e26f0-f4c6-49ba-610c-00ddcc2bb9ec@oracle.com> References: <7063271f-879a-1a0f-5505-38f32f29e1d5@oracle.com> <57BB426A.4000505@oracle.com> <521712d6-f90a-a7cf-88c8-df28393fc228@oracle.com> <075c6e95-1992-23c9-1f5f-d44a2912204b@oracle.com> <17acbbe4-f115-9809-fd02-cb123ec3edfe@oracle.com> <8709b35a-97ef-b274-1df3-fec1fc2bdc4e@oracle.com> <671e26f0-f4c6-49ba-610c-00ddcc2bb9ec@oracle.com> Message-ID: On 8/30/16 7:24 AM, Zolt?n Maj? wrote: > Hi Coleen, > > > thank you for the review! Please see details below. > > On 08/29/2016 10:31 PM, Coleen Phillimore wrote: >> >> http://cr.openjdk.java.net/~zmajo/8163880/webrev.02/src/share/vm/interpreter/interpreterRuntime.cpp.udiff.html >> >> >> I thought we always want to suppress resolving a static putfield, and >> only want to give the error if >= classfile version 53. > > Yes, we throw an IllegalAccessError() only for class file version >= > 53. For class file version < 53 we do not to throw an error. > Delaying CP cache resolution is therefore not necessary for class file > version < 53. That is why we have the classfile version check in place. I see where my confusion was now. thanks, Coleen > > If we remove the classfile version check, we'll delay CP caching for > class file version < 53 as well. That won't hurt correctness and is > will not hurt performance either (I did some experiments to verify that). > > Unfortunately, I pushed the fixed before your mail arrived. But we > can open a new bug to handle the issue you have raised. Please let me > know if you'd like that and I can take care of the bug. > >> >> *+bool has_initialized_final_update = >> info.field_holder()->major_version() >= 53 &&* >> *+ info.has_initialized_final_update();* >> >> >> So, not check the classfile version here. >> >> If I understand correctly, this only inhibits the quickening for when >> functions other than are updating the final field, right? > > If inhibits quickening if methods other than *or * are > updating the final field. > >> I am surprised this hurt performance because I would have thought >> initializing final fields were an infrequent operation. > > Here are some more details to answer that. > > The CP cache does caching for get and put bytecodes individually. > Normally, the caching for put (in 'put_code') should be set up by put > instructions, the caching for get (in 'get_code') should be set up by > get instruction. The optimization is to let get instructions set up > caching for put bytecodes in some select cases. > > 1) The table below shows the way that worked before the regression > (i.e., before JDK-8160527). The table shows if put_code is set (S) or > not set (NS) after InterpreterRuntime::resolve_get_put(). The table > applies to both static and instance fields. Furthermore, the table > assumes that class initialization has already taken place (i.e., > uninitialized_static is false). > > | Instruction triggering resolution > |=================================== > Field type | get | put > =================================================== > final | NS | S > non-final | S | S > > http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/29ed49c42486#l1.38 > > 2) Unfortunately, JDK-8160527 changed the table to: > > | Instruction triggering resolution > |=================================== > Field type | get | put > =================================================== > final | NS |*NS* > non-final | *NS* | S > > http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/29ed49c42486#l1.30 > > The cause of the regression reported in JDK-8163880 is that > > (1) For final fields, setting up the CP cache for put_code is > inhibited (Row 1). In most class files we have to deal with, only > initializers (, ) update final fields. Inhibiting CP > caching for static final fields will most likely not impact > performance too much, as methods are executed infrequently. > Inhibiting CP for instance fields has most likely have a larger > performance impact, as methods are executed more frequently. > > (2) For non-final fields, setting up the CP cache for put_code is not > set if the triggering instruction is a get (i.e., setting up the CP > cache is delayed until a put instruction attempts to set up the CP > cache), which can cause a number of extra calls into the runtime. > > The regression is due to both (1) and (2). You pointed out (and I > agree) that (1) has a small effect if the field is static. I think > the effect of (1) is larger if the field is non-static. Also, there > might be a large effect due to (2). But we'd need to do some > experiments to confirm that. > > Currently, the system works is as follows. > > | Instruction triggering resolution > |=================================== > Field type | get | put > =================================================== > final | NS | S (for fields in class filed version < > 53) / NS (for field in class file version >= 53 that have > initialized final updates) > non-final | S | S > > We can change the final/put case to "S for all fields except ones with > initialized final updates)", as you suggested. That is not necessary > for correctness, but won't hurt performance either and could make code > more readable. Please let me know if you'd like a bug to be filed for > that -- I'd be happy to take care of it. > > Thank you! > > Best regards, > > > Zoltan > >> >> thanks, >> Coleen >> >> >> On 8/29/16 1:28 AM, Zolt?n Maj? wrote: >>> Thank you, Vladimir, for the review! >>> >>> Best regards, >>> >>> >>> Zoltan >>> >>> >>> On 08/26/2016 07:58 PM, Vladimir Kozlov wrote: >>>> Looks good. >>>> >>>> Thanks, >>>> Vladimir >>>> >>>> On 8/26/16 4:38 AM, Zolt?n Maj? wrote: >>>>> Hi Vladimir, >>>>> >>>>> >>>>> thank you for looking at this issue once more. >>>>> >>>>> On 08/25/2016 09:05 PM, Vladimir Kozlov wrote: >>>>>> Okay I think I understand what you are trying to say but it was >>>>>> not my >>>>>> point. >>>>> >>>>> Oh, I see. Then I misunderstood you. >>>>> >>>>>> I was questioning why put_code is set when bytecode could be get >>>>>> instruction. Reading comments and code around I think I >>>>>> understand now. >>>>>> We set both get_code and put_code marking that the field is >>>>>> resolved. >>>>>> But in some cases we delay marking to force field's reresolution. >>>>> >>>>> Yes, that is also my understanding. >>>>> >>>>>> >>>>>> In next code you can use is_static: >>>>>> >>>>>> bool uninitialized_static = ((bytecode == >>>>>> Bytecodes::_getstatic || >>>>>> bytecode == Bytecodes::_putstatic) && >>>>>> !klass->is_initialized()); >>>>> >>>>> OK, I've updated the code. >>>>> >>>>>> >>>>>> Code could be rearranged to be more compact (similar to original >>>>>> code): >>>>>> >>>>>> Bytecodes::Code put_code = (Bytecodes::Code)0; >>>>>> Bytecodes::Code get_code = (Bytecodes::Code)0; >>>>>> if (!uninitialized_static) { >>>>>> get_code = ((is_static) ? Bytecodes::_getstatic : >>>>>> Bytecodes::_getfield); >>>>>> if ((is_put && !has_initialized_final_update) || >>>>>> !info.access_flags().is_final()) { >>>>>> put_code = ((is_static) ? Bytecodes::_putstatic : >>>>>> Bytecodes::_putfield); >>>>>> } >>>>>> } >>>>> >>>>> I agree and have updated the code accordingly. >>>>> >>>>> Here is the updated webrev: >>>>> http://cr.openjdk.java.net/~zmajo/8163880/webrev.02/ >>>>> >>>>> JPRT testing passes (testset hotspot). >>>>> >>>>> Thank you! >>>>> >>>>> Best regards, >>>>> >>>>> >>>>> Zoltan >>>>> >>>>>> >>>>>> Thanks, >>>>>> Vladimir >>>>>> >>>>>> On 8/23/16 5:31 AM, Zolt?n Maj? wrote: >>>>>>> Hi Vladimir, >>>>>>> >>>>>>> >>>>>>> thank you for the feedback! Please see details below. >>>>>>> >>>>>>> On 08/22/2016 08:20 PM, Vladimir Kozlov wrote: >>>>>>>> Typo (double 'not'): >>>>>>>> + // and the required IllegalAccessError would not not be >>>>>>>> thrown. >>>>>>> >>>>>>> OK, I've corrected that. >>>>>>> >>>>>>>> >>>>>>>> I think has_initialized_final_update should be part of assert >>>>>>>> to see >>>>>>>> it in hs_err: >>>>>>>> >>>>>>>> + if (has_initialized_final_update) { >>>>>>>> + assert(info.access_flags().is_final(), "Fields with >>>>>>>> initialized >>>>>>>> final updates must be final"); >>>>>>>> + } >>>>>>> >>>>>>> Yes, I've updated that as well. >>>>>>> >>>>>>>> >>>>>>>> Why is_put check is not done in outer check? It is missing for >>>>>>>> is_final(): >>>>>>>> >>>>>>>> + if (!uninitialized_static) { >>>>>>>> + if ((is_put && !has_initialized_final_update) || >>>>>>>> !info.access_flags().is_final()) { >>>>>>> >>>>>>> I think it's an optimization to do it this way: For >>>>>>> non-final-fields, we >>>>>>> want to set up CP caching faster (i.e., we want to reduce the >>>>>>> number of >>>>>>> times we call into the runtime to >>>>>>> LinkResolver::resolve_field_access()).We had that optimization >>>>>>> in place >>>>>>> before JDK-8160527, but not afterwards, which caused a performance >>>>>>> regression. Please see some details below. >>>>>>> >>>>>>> The CP cache does caching for get and put bytecodes individually. >>>>>>> Normally, the caching for put (in 'put_code') should be set up >>>>>>> by put >>>>>>> instructions, the caching for get (in 'get_code') should be set >>>>>>> up by >>>>>>> get instruction. The optimization is to let get instructions >>>>>>> set up >>>>>>> caching for put bytecodes in some select cases. Here is the way >>>>>>> that >>>>>>> (not) workedbefore/after JDK-8160527. >>>>>>> >>>>>>> >>>>>>> 1) The table below describes the way resolution worked before >>>>>>> JDK-8160527. The table shows if put_code was set (S) or not set >>>>>>> (NS) >>>>>>> after InterpreterRuntime::resolve_get_put(). >>>>>>> >>>>>>> | Instruction triggering resolution >>>>>>> |=================================== >>>>>>> Field type | get | put >>>>>>> =================================================== >>>>>>> final | NS |S >>>>>>> non-final | S | S >>>>>>> >>>>>>> http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/29ed49c42486#l1.38 >>>>>>> >>>>>>> >>>>>>> 2) Unfortunately, JDK-8160527 changed the table to: >>>>>>> >>>>>>> | Instruction triggering resolution >>>>>>> |=================================== >>>>>>> Field type | get | put >>>>>>> =================================================== >>>>>>> final | NS |*NS* >>>>>>> non-final | *NS* | S >>>>>>> >>>>>>> http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/29ed49c42486#l1.30 >>>>>>> >>>>>>> So the VM does not set up caching in *2additional cases*, which >>>>>>> causes a >>>>>>> performance regression. >>>>>>> >>>>>>> >>>>>>> 3) The current webrev proposes the following: >>>>>>> >>>>>>> | Instruction triggering resolution >>>>>>> |=================================== >>>>>>> Field type | get | put >>>>>>> =================================================== >>>>>>> final | NS |*S (unless >>>>>>> has_initialized_final_update >>>>>>> is true for the field)* >>>>>>> non-final | S | S >>>>>>> >>>>>>> That table looks the same as what we had before JDK-8160527, >>>>>>> except for >>>>>>> fields with initialized final updates. For those fields we need >>>>>>> to defer >>>>>>> setting up CP caching so that the appropriate exception can be >>>>>>> thrown at >>>>>>> the right place (by LinkResolver::resolve_field_access()). >>>>>>> >>>>>>> >>>>>>> 4) If we would do what you suggested (I hope I understood your >>>>>>> suggestion correctly)thenwe'd move the is_put condition to the >>>>>>> outer >>>>>>> check: >>>>>>> >>>>>>> + if (!uninitialized_static && is_put) { >>>>>>> + if (!has_initialized_final_update || >>>>>>> !info.access_flags().is_final()) { >>>>>>> >>>>>>> Then the table would look: >>>>>>> >>>>>>> | Instruction triggering resolution >>>>>>> |=================================== >>>>>>> Field type | get | put >>>>>>> =================================================== >>>>>>> final | NS |S(unless >>>>>>> has_initialized_final_update is >>>>>>> true for the field) >>>>>>> non-final | *NS* | S >>>>>>> >>>>>>> So we would have *still one case* where CP caching is not set up >>>>>>> and >>>>>>> that would most likely result in some performance regression >>>>>>> (relative >>>>>>> to pre-JDK-8160527). >>>>>>> >>>>>>> >>>>>>> I hope I have not overlooked anythingabove. >>>>>>> >>>>>>> Here is the updated webrev: >>>>>>> http://cr.openjdk.java.net/~zmajo/8163880/webrev.01/ >>>>>>> >>>>>>> JPRT testing is running. >>>>>>> >>>>>>> Thank you! >>>>>>> >>>>>>> Best regards, >>>>>>> >>>>>>> >>>>>>> Zoltan >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Vladimir >>>>>>>> >>>>>>>> On 8/22/16 4:20 AM, Zolt?n Maj? wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> >>>>>>>>> please review the fix for JDK-8163880. >>>>>>>>> >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8163880 >>>>>>>>> >>>>>>>>> Problem & solution: The performance regression reported by the >>>>>>>>> bug is >>>>>>>>> caused by unnecessary delay (or even inhibition) of constant pool >>>>>>>>> caching for fields targeted by put instructions. For a more >>>>>>>>> detailed description of the problem and possible solutions, >>>>>>>>> please >>>>>>>>> see the following comment in the bug report [1]. >>>>>>>>> >>>>>>>>> Webrev: >>>>>>>>> http://cr.openjdk.java.net/~zmajo/8163880/webrev.00/ >>>>>>>>> >>>>>>>>> Testing: >>>>>>>>> - performance evaluation with the affected benchmark shows >>>>>>>>> that fix >>>>>>>>> above solves the performance regression; >>>>>>>>> - all hotspot test with -Xmixed and -Xcomp (running). >>>>>>>>> >>>>>>>>> Thank you! >>>>>>>>> >>>>>>>>> Best regards, >>>>>>>>> >>>>>>>>> >>>>>>>>> Zoltan >>>>>>>>> >>>>>>>>> [1] >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8163880?focusedCommentId=13990030&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13990030 >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>> >>>>> >>> >> > From zoltan.majo at oracle.com Thu Sep 1 05:59:25 2016 From: zoltan.majo at oracle.com (=?UTF-8?B?Wm9sdMOhbiBNYWrDsw==?=) Date: Thu, 1 Sep 2016 07:59:25 +0200 Subject: [9] RFR (X): 8163880: Constant pool caching of fields inhibited/delayed unnecessarily In-Reply-To: References: <7063271f-879a-1a0f-5505-38f32f29e1d5@oracle.com> <57BB426A.4000505@oracle.com> <521712d6-f90a-a7cf-88c8-df28393fc228@oracle.com> <075c6e95-1992-23c9-1f5f-d44a2912204b@oracle.com> <17acbbe4-f115-9809-fd02-cb123ec3edfe@oracle.com> <8709b35a-97ef-b274-1df3-fec1fc2bdc4e@oracle.com> <671e26f0-f4c6-49ba-610c-00ddcc2bb9ec@oracle.com> Message-ID: Hi Coleen, On 09/01/2016 02:28 AM, Coleen Phillimore wrote: > > > On 8/30/16 7:24 AM, Zolt?n Maj? wrote: >> Hi Coleen, >> >> >> thank you for the review! Please see details below. >> >> On 08/29/2016 10:31 PM, Coleen Phillimore wrote: >>> >>> http://cr.openjdk.java.net/~zmajo/8163880/webrev.02/src/share/vm/interpreter/interpreterRuntime.cpp.udiff.html >>> >>> >>> I thought we always want to suppress resolving a static putfield, >>> and only want to give the error if >= classfile version 53. >> >> Yes, we throw an IllegalAccessError() only for class file version >= >> 53. For class file version < 53 we do not to throw an error. >> Delaying CP cache resolution is therefore not necessary for class >> file version < 53. That is why we have the classfile version check >> in place. > > I see where my confusion was now. Thank you for looking into this issue! Best regards, Zoltan > thanks, > Coleen >> >> If we remove the classfile version check, we'll delay CP caching for >> class file version < 53 as well. That won't hurt correctness and is >> will not hurt performance either (I did some experiments to verify >> that). >> >> Unfortunately, I pushed the fixed before your mail arrived. But we >> can open a new bug to handle the issue you have raised. Please let me >> know if you'd like that and I can take care of the bug. >> >>> >>> *+bool has_initialized_final_update = >>> info.field_holder()->major_version() >= 53 &&* >>> *+ info.has_initialized_final_update();* >>> >>> >>> So, not check the classfile version here. >>> >>> If I understand correctly, this only inhibits the quickening for >>> when functions other than are updating the final field, right? >> >> If inhibits quickening if methods other than *or * are >> updating the final field. >> >>> I am surprised this hurt performance because I would have thought >>> initializing final fields were an infrequent operation. >> >> Here are some more details to answer that. >> >> The CP cache does caching for get and put bytecodes individually. >> Normally, the caching for put (in 'put_code') should be set up by put >> instructions, the caching for get (in 'get_code') should be set up by >> get instruction. The optimization is to let get instructions set up >> caching for put bytecodes in some select cases. >> >> 1) The table below shows the way that worked before the regression >> (i.e., before JDK-8160527). The table shows if put_code is set (S) or >> not set (NS) after InterpreterRuntime::resolve_get_put(). The table >> applies to both static and instance fields. Furthermore, the table >> assumes that class initialization has already taken place (i.e., >> uninitialized_static is false). >> >> | Instruction triggering resolution >> |=================================== >> Field type | get | put >> =================================================== >> final | NS | S >> non-final | S | S >> >> http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/29ed49c42486#l1.38 >> >> 2) Unfortunately, JDK-8160527 changed the table to: >> >> | Instruction triggering resolution >> |=================================== >> Field type | get | put >> =================================================== >> final | NS |*NS* >> non-final | *NS* | S >> >> http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/29ed49c42486#l1.30 >> >> The cause of the regression reported in JDK-8163880 is that >> >> (1) For final fields, setting up the CP cache for put_code is >> inhibited (Row 1). In most class files we have to deal with, only >> initializers (, ) update final fields. Inhibiting CP >> caching for static final fields will most likely not impact >> performance too much, as methods are executed infrequently. >> Inhibiting CP for instance fields has most likely have a larger >> performance impact, as methods are executed more frequently. >> >> (2) For non-final fields, setting up the CP cache for put_code is not >> set if the triggering instruction is a get (i.e., setting up the CP >> cache is delayed until a put instruction attempts to set up the CP >> cache), which can cause a number of extra calls into the runtime. >> >> The regression is due to both (1) and (2). You pointed out (and I >> agree) that (1) has a small effect if the field is static. I think >> the effect of (1) is larger if the field is non-static. Also, there >> might be a large effect due to (2). But we'd need to do some >> experiments to confirm that. >> >> Currently, the system works is as follows. >> >> | Instruction triggering resolution >> |=================================== >> Field type | get | put >> =================================================== >> final | NS | S (for fields in class filed version >> < 53) / NS (for field in class file version >= 53 that have >> initialized final updates) >> non-final | S | S >> >> We can change the final/put case to "S for all fields except ones >> with initialized final updates)", as you suggested. That is not >> necessary for correctness, but won't hurt performance either and >> could make code more readable. Please let me know if you'd like a >> bug to be filed for that -- I'd be happy to take care of it. >> >> Thank you! >> >> Best regards, >> >> >> Zoltan >> >>> >>> thanks, >>> Coleen >>> >>> >>> On 8/29/16 1:28 AM, Zolt?n Maj? wrote: >>>> Thank you, Vladimir, for the review! >>>> >>>> Best regards, >>>> >>>> >>>> Zoltan >>>> >>>> >>>> On 08/26/2016 07:58 PM, Vladimir Kozlov wrote: >>>>> Looks good. >>>>> >>>>> Thanks, >>>>> Vladimir >>>>> >>>>> On 8/26/16 4:38 AM, Zolt?n Maj? wrote: >>>>>> Hi Vladimir, >>>>>> >>>>>> >>>>>> thank you for looking at this issue once more. >>>>>> >>>>>> On 08/25/2016 09:05 PM, Vladimir Kozlov wrote: >>>>>>> Okay I think I understand what you are trying to say but it was >>>>>>> not my >>>>>>> point. >>>>>> >>>>>> Oh, I see. Then I misunderstood you. >>>>>> >>>>>>> I was questioning why put_code is set when bytecode could be get >>>>>>> instruction. Reading comments and code around I think I >>>>>>> understand now. >>>>>>> We set both get_code and put_code marking that the field is >>>>>>> resolved. >>>>>>> But in some cases we delay marking to force field's reresolution. >>>>>> >>>>>> Yes, that is also my understanding. >>>>>> >>>>>>> >>>>>>> In next code you can use is_static: >>>>>>> >>>>>>> bool uninitialized_static = ((bytecode == >>>>>>> Bytecodes::_getstatic || >>>>>>> bytecode == Bytecodes::_putstatic) && >>>>>>> !klass->is_initialized()); >>>>>> >>>>>> OK, I've updated the code. >>>>>> >>>>>>> >>>>>>> Code could be rearranged to be more compact (similar to original >>>>>>> code): >>>>>>> >>>>>>> Bytecodes::Code put_code = (Bytecodes::Code)0; >>>>>>> Bytecodes::Code get_code = (Bytecodes::Code)0; >>>>>>> if (!uninitialized_static) { >>>>>>> get_code = ((is_static) ? Bytecodes::_getstatic : >>>>>>> Bytecodes::_getfield); >>>>>>> if ((is_put && !has_initialized_final_update) || >>>>>>> !info.access_flags().is_final()) { >>>>>>> put_code = ((is_static) ? Bytecodes::_putstatic : >>>>>>> Bytecodes::_putfield); >>>>>>> } >>>>>>> } >>>>>> >>>>>> I agree and have updated the code accordingly. >>>>>> >>>>>> Here is the updated webrev: >>>>>> http://cr.openjdk.java.net/~zmajo/8163880/webrev.02/ >>>>>> >>>>>> JPRT testing passes (testset hotspot). >>>>>> >>>>>> Thank you! >>>>>> >>>>>> Best regards, >>>>>> >>>>>> >>>>>> Zoltan >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Vladimir >>>>>>> >>>>>>> On 8/23/16 5:31 AM, Zolt?n Maj? wrote: >>>>>>>> Hi Vladimir, >>>>>>>> >>>>>>>> >>>>>>>> thank you for the feedback! Please see details below. >>>>>>>> >>>>>>>> On 08/22/2016 08:20 PM, Vladimir Kozlov wrote: >>>>>>>>> Typo (double 'not'): >>>>>>>>> + // and the required IllegalAccessError would not not be >>>>>>>>> thrown. >>>>>>>> >>>>>>>> OK, I've corrected that. >>>>>>>> >>>>>>>>> >>>>>>>>> I think has_initialized_final_update should be part of assert >>>>>>>>> to see >>>>>>>>> it in hs_err: >>>>>>>>> >>>>>>>>> + if (has_initialized_final_update) { >>>>>>>>> + assert(info.access_flags().is_final(), "Fields with >>>>>>>>> initialized >>>>>>>>> final updates must be final"); >>>>>>>>> + } >>>>>>>> >>>>>>>> Yes, I've updated that as well. >>>>>>>> >>>>>>>>> >>>>>>>>> Why is_put check is not done in outer check? It is missing for >>>>>>>>> is_final(): >>>>>>>>> >>>>>>>>> + if (!uninitialized_static) { >>>>>>>>> + if ((is_put && !has_initialized_final_update) || >>>>>>>>> !info.access_flags().is_final()) { >>>>>>>> >>>>>>>> I think it's an optimization to do it this way: For >>>>>>>> non-final-fields, we >>>>>>>> want to set up CP caching faster (i.e., we want to reduce the >>>>>>>> number of >>>>>>>> times we call into the runtime to >>>>>>>> LinkResolver::resolve_field_access()).We had that optimization >>>>>>>> in place >>>>>>>> before JDK-8160527, but not afterwards, which caused a performance >>>>>>>> regression. Please see some details below. >>>>>>>> >>>>>>>> The CP cache does caching for get and put bytecodes individually. >>>>>>>> Normally, the caching for put (in 'put_code') should be set up >>>>>>>> by put >>>>>>>> instructions, the caching for get (in 'get_code') should be set >>>>>>>> up by >>>>>>>> get instruction. The optimization is to let get instructions >>>>>>>> set up >>>>>>>> caching for put bytecodes in some select cases. Here is the way >>>>>>>> that >>>>>>>> (not) workedbefore/after JDK-8160527. >>>>>>>> >>>>>>>> >>>>>>>> 1) The table below describes the way resolution worked before >>>>>>>> JDK-8160527. The table shows if put_code was set (S) or not >>>>>>>> set (NS) >>>>>>>> after InterpreterRuntime::resolve_get_put(). >>>>>>>> >>>>>>>> | Instruction triggering resolution >>>>>>>> |=================================== >>>>>>>> Field type | get | put >>>>>>>> =================================================== >>>>>>>> final | NS |S >>>>>>>> non-final | S | S >>>>>>>> >>>>>>>> http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/29ed49c42486#l1.38 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> 2) Unfortunately, JDK-8160527 changed the table to: >>>>>>>> >>>>>>>> | Instruction triggering resolution >>>>>>>> |=================================== >>>>>>>> Field type | get | put >>>>>>>> =================================================== >>>>>>>> final | NS |*NS* >>>>>>>> non-final | *NS* | S >>>>>>>> >>>>>>>> http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/29ed49c42486#l1.30 >>>>>>>> >>>>>>>> >>>>>>>> So the VM does not set up caching in *2additional cases*, which >>>>>>>> causes a >>>>>>>> performance regression. >>>>>>>> >>>>>>>> >>>>>>>> 3) The current webrev proposes the following: >>>>>>>> >>>>>>>> | Instruction triggering resolution >>>>>>>> |=================================== >>>>>>>> Field type | get | put >>>>>>>> =================================================== >>>>>>>> final | NS |*S (unless >>>>>>>> has_initialized_final_update >>>>>>>> is true for the field)* >>>>>>>> non-final | S | S >>>>>>>> >>>>>>>> That table looks the same as what we had before JDK-8160527, >>>>>>>> except for >>>>>>>> fields with initialized final updates. For those fields we need >>>>>>>> to defer >>>>>>>> setting up CP caching so that the appropriate exception can be >>>>>>>> thrown at >>>>>>>> the right place (by LinkResolver::resolve_field_access()). >>>>>>>> >>>>>>>> >>>>>>>> 4) If we would do what you suggested (I hope I understood your >>>>>>>> suggestion correctly)thenwe'd move the is_put condition to the >>>>>>>> outer >>>>>>>> check: >>>>>>>> >>>>>>>> + if (!uninitialized_static && is_put) { >>>>>>>> + if (!has_initialized_final_update || >>>>>>>> !info.access_flags().is_final()) { >>>>>>>> >>>>>>>> Then the table would look: >>>>>>>> >>>>>>>> | Instruction triggering resolution >>>>>>>> |=================================== >>>>>>>> Field type | get | put >>>>>>>> =================================================== >>>>>>>> final | NS |S(unless >>>>>>>> has_initialized_final_update is >>>>>>>> true for the field) >>>>>>>> non-final | *NS* | S >>>>>>>> >>>>>>>> So we would have *still one case* where CP caching is not set >>>>>>>> up and >>>>>>>> that would most likely result in some performance regression >>>>>>>> (relative >>>>>>>> to pre-JDK-8160527). >>>>>>>> >>>>>>>> >>>>>>>> I hope I have not overlooked anythingabove. >>>>>>>> >>>>>>>> Here is the updated webrev: >>>>>>>> http://cr.openjdk.java.net/~zmajo/8163880/webrev.01/ >>>>>>>> >>>>>>>> JPRT testing is running. >>>>>>>> >>>>>>>> Thank you! >>>>>>>> >>>>>>>> Best regards, >>>>>>>> >>>>>>>> >>>>>>>> Zoltan >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Vladimir >>>>>>>>> >>>>>>>>> On 8/22/16 4:20 AM, Zolt?n Maj? wrote: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> please review the fix for JDK-8163880. >>>>>>>>>> >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8163880 >>>>>>>>>> >>>>>>>>>> Problem & solution: The performance regression reported by >>>>>>>>>> the bug is >>>>>>>>>> caused by unnecessary delay (or even inhibition) of constant >>>>>>>>>> pool >>>>>>>>>> caching for fields targeted by put instructions. For a more >>>>>>>>>> detailed description of the problem and possible solutions, >>>>>>>>>> please >>>>>>>>>> see the following comment in the bug report [1]. >>>>>>>>>> >>>>>>>>>> Webrev: >>>>>>>>>> http://cr.openjdk.java.net/~zmajo/8163880/webrev.00/ >>>>>>>>>> >>>>>>>>>> Testing: >>>>>>>>>> - performance evaluation with the affected benchmark shows >>>>>>>>>> that fix >>>>>>>>>> above solves the performance regression; >>>>>>>>>> - all hotspot test with -Xmixed and -Xcomp (running). >>>>>>>>>> >>>>>>>>>> Thank you! >>>>>>>>>> >>>>>>>>>> Best regards, >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Zoltan >>>>>>>>>> >>>>>>>>>> [1] >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8163880?focusedCommentId=13990030&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13990030 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>> >>>> >>> >> > From david.holmes at oracle.com Thu Sep 1 07:53:33 2016 From: david.holmes at oracle.com (David Holmes) Date: Thu, 1 Sep 2016 17:53:33 +1000 Subject: RFR: JDK-8157453 - Convert DependencyContext_test to GTest In-Reply-To: <2249b326-9265-1cd0-5622-8ec53f8b8d22@oracle.com> References: <2249b326-9265-1cd0-5622-8ec53f8b8d22@oracle.com> Message-ID: <9da51973-1686-e9e4-f013-063e275f4ebc@oracle.com> On 30/08/2016 6:38 PM, Jesper Wilhelmsson wrote: > Hi, > > Good catch Kirill! > > Den 30/8/16 kl. 06:26, skrev David Holmes: >> Hi Kirill, >> >> On 30/08/2016 5:32 AM, Kirill Zhaldybin wrote: >>> Jesper, >>> >>> As far as I understand test_dependencyContext could be run in both >>> product and debug. >>> But in the class DependencyContext >>> >>> 148 #ifndef PRODUCT >>> 149 void print_dependent_nmethods(bool verbose); >>> 150 bool is_dependent_nmethod(nmethod* nm); >>> 151 bool find_stale_entries(); >>> 152 #endif //PRODUCT >>> >>> so is_dependent_nmethod and find_stale_entries are compiled only in >>> non-product configs but they are used in test_dependencyContext >> >> I presume this is why these calls are wrapped in the ASSERT_TRUE and >> ASSERT_FALSE macros - I expect them to be no-ops in a product build. >> But I can't >> find the definitions of these ASSERT_xxx macros ?? > > No, Kirill is right, this won't compile in product. I completely forgot > that these tests are run in product mode these days (they weren't when I > did this test conversion a while ago). > > ASSERT_TRUE and ASSERT_FALSE are part of the GTest framework. They are > two of the handful of ASSERT_X macros that we can use in the tests. > These are defined in /test/fmw/gtest/include/gtest/gtest.h Not familiar with Gtest itself but this seems like a design flaw to me - the GTest macros should be obviously part of GTest! :( > A new webrev that I have verified with product is found here: > > http://cr.openjdk.java.net/~jwilhelm/8157453/webrev.01/ > > The test will still do some things with dependency contexts in product > mode but it won't verify anything. I chose not to use ifndef PRODUCT > around the entire test to highlight what parts need to be fixed if the > test should do some verification in product as well. I leave it up to > the team who owns the DependencyContext code to decide if that is > important or not. I'd probably prefer #ifndef PRODUCT blocks instead of multiple lines of NOT_PRODUCT, but won't insist. Thanks, David > > Thanks, > /Jesper > > >> >> David >> ----- >> >> >> >> >> >> >>> Have you tried to build VM and run this test in product config? >>> >>> http://cr.openjdk.java.net/~jwilhelm/8157453/webrev.00/test/native/code/test_dependencyContext.cpp.html >>> >>> >>> >>> >>> 86 TEST(code, dependency_context) { >>> >>> According to "almost existing naming convention" the test should be >>> named (, ). >>> >>> Thank you. >>> >>> Regards, Kirill >>> >>> On 25.08.2016 20:53, Jesper Wilhelmsson wrote: >>>> Hi, >>>> >>>> Please review this test conversion of the DependencyContext tests to >>>> GTest. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8157453 >>>> Webrev: http://cr.openjdk.java.net/~jwilhelm/8157453/webrev.00/ >>>> >>>> Thanks, >>>> /Jesper >>> From aph at redhat.com Thu Sep 1 10:31:53 2016 From: aph at redhat.com (Andrew Haley) Date: Thu, 1 Sep 2016 11:31:53 +0100 Subject: 8162338: AArch64: Intrinsify fused mac operations In-Reply-To: References: <57910421.7090300@redhat.com> Message-ID: <1f6a69d1-7b99-2ab7-f8bd-2364df9a00e3@redhat.com> On 26/08/16 22:21, Vladimir Kozlov wrote: > Here is final Hotspot webrev for shared and x86 changes: > > http://cr.openjdk.java.net/~kvn/8154122/webrev.03/ > > Please, update aarch64 changes according to it so they will be ready > when 8154122 is approved and pushed. Quick review please! http://cr.openjdk.java.net/~aph/8162338.2/ Just to save you time: the only change is to vm_version_aarch64.cpp and globals_aarch64.hpp. This is because Vladimir slightly changed the way that UseFMA is declared. Andrew. From rwestrel at redhat.com Thu Sep 1 10:41:08 2016 From: rwestrel at redhat.com (Roland Westrelin) Date: Thu, 01 Sep 2016 12:41:08 +0200 Subject: 8162338: AArch64: Intrinsify fused mac operations In-Reply-To: <1f6a69d1-7b99-2ab7-f8bd-2364df9a00e3@redhat.com> References: <57910421.7090300@redhat.com> <1f6a69d1-7b99-2ab7-f8bd-2364df9a00e3@redhat.com> Message-ID: > http://cr.openjdk.java.net/~aph/8162338.2/ That looks good to me. Roland. From kim.barrett at oracle.com Thu Sep 1 14:36:49 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 1 Sep 2016 10:36:49 -0400 Subject: Request for review (s) - 8155948: Add message for CMS deprecation for some internal builds In-Reply-To: <74eccd9a-1525-a9ef-1608-294de99d4483@oracle.com> References: <74eccd9a-1525-a9ef-1608-294de99d4483@oracle.com> Message-ID: > On Aug 31, 2016, at 2:06 PM, Jon Masamitsu wrote: > > Adds the support to emit a deprecation message in some builds. > > https://bugs.openjdk.java.net/browse/JDK-8155948 > > A data structure is moved to a header file to allow > sharing and a stub function is added which can be > specialized for different builds. > > http://cr.openjdk.java.net/~jmasa/8155948/webrev.00/ Mostly looks ok. A style issue: ------------------------------------------------------------------------------ src/share/vm/runtime/arguments.hpp 45 typedef struct { ... 50 } SpecialFlag; This is C++ code, not C. Rather than using typedef of an anonymous struct, we should be using a named struct, e.g. struct SpecialFlag { ... }; ------------------------------------------------------------------------------ There are some other similar typedefs in arguments.cpp that aren?t touched by this change. From david.holmes at oracle.com Fri Sep 2 01:54:48 2016 From: david.holmes at oracle.com (David Holmes) Date: Fri, 2 Sep 2016 11:54:48 +1000 Subject: Request for review (s) - 8155948: Add message for CMS deprecation for some internal builds In-Reply-To: <74eccd9a-1525-a9ef-1608-294de99d4483@oracle.com> References: <74eccd9a-1525-a9ef-1608-294de99d4483@oracle.com> Message-ID: Hi Jon, So this simply allows for extending the definitions of "special flags" to custom extensions - ok. Other than Kim's suggestion to use a named struct, this looks fine to me too. Thanks, David On 1/09/2016 4:06 AM, Jon Masamitsu wrote: > Adds the support to emit a deprecation message in some builds. > > https://bugs.openjdk.java.net/browse/JDK-8155948 > > A data structure is moved to a header file to allow > sharing and a stub function is added which can be > specialized for different builds. > > http://cr.openjdk.java.net/~jmasa/8155948/webrev.00/ > > Thanks. > > Jon > > From jon.masamitsu at oracle.com Fri Sep 2 03:33:06 2016 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Thu, 1 Sep 2016 20:33:06 -0700 Subject: Request for review (s) - 8155948: Add message for CMS deprecation for some internal builds In-Reply-To: References: <74eccd9a-1525-a9ef-1608-294de99d4483@oracle.com> Message-ID: David, On 9/1/2016 6:54 PM, David Holmes wrote: > Hi Jon, > > So this simply allows for extending the definitions of "special flags" > to custom extensions - ok. Yes, that is correct. > Other than Kim's suggestion to use a named struct, this looks fine to > me too. Thanks for the review. I'll send out an updated webrev with Kim's suggestion. Jon > > Thanks, > David > > On 1/09/2016 4:06 AM, Jon Masamitsu wrote: >> Adds the support to emit a deprecation message in some builds. >> >> https://bugs.openjdk.java.net/browse/JDK-8155948 >> >> A data structure is moved to a header file to allow >> sharing and a stub function is added which can be >> specialized for different builds. >> >> http://cr.openjdk.java.net/~jmasa/8155948/webrev.00/ >> >> Thanks. >> >> Jon >> >> From doko at ubuntu.com Mon Sep 5 11:10:39 2016 From: doko at ubuntu.com (Matthias Klose) Date: Mon, 5 Sep 2016 13:10:39 +0200 Subject: [patch] fix compiler flag for linux sparc targets Message-ID: at least in 9b134 the build system currently picks up the wrong compiler flag on linux sparc targets. The patch below only applies it for solaris builds. --- a/common/autoconf/flags.m4 +++ b/common/autoconf/flags.m4 @@ -921,7 +921,7 @@ # Set some additional per-CPU defines. if test "x$OPENJDK_$1_OS-$OPENJDK_$1_CPU" = xwindows-x86; then $2JVM_CFLAGS="[$]$2JVM_CFLAGS -arch:IA32" - elif test "x$OPENJDK_$1_CPU" = xsparcv9; then + elif test "xx$OPENJDK_$1_OS-$OPENJDK_$1_CPU" = xsolaris-sparcv9; then $2JVM_CFLAGS="[$]$2JVM_CFLAGS -xarch=sparc" elif test "x$OPENJDK_$1_CPU" = xppc64; then if test "x$OPENJDK_$1_OS" = xlinux; then Is the linux sparc port supposed to work in 9? It apparently does in jdk8, but the build later fails with /<>/src/hotspot/src/os_cpu/linux_sparc/vm/os_linux_sparc.cpp: In function 'bool checkByteBuffer(address, address, u_char**)': /<>/src/hotspot/src/os_cpu/linux_sparc/vm/os_linux_sparc.cpp:442:49: error: 'thread' was not declared in this scope *stub = SharedRuntime::handle_unsafe_access(thread, npc); ^~~~~~ lib/CompileJvm.gmk:201: recipe for target '/<>/build/hotspot/variant-server/libjvm/objs/os_linux_sparc.o' failed Matthias From erik.joelsson at oracle.com Mon Sep 5 11:33:53 2016 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Mon, 5 Sep 2016 13:33:53 +0200 Subject: [patch] fix compiler flag for linux sparc targets In-Reply-To: References: Message-ID: <0f524f4a-4ae6-4fb1-ac92-768bdbdb9579@oracle.com> Patch looks ok. To my knowledge, nobody is actively keeping linux-sparc working in 9, so I would expect it to be broken. /Erik On 2016-09-05 13:10, Matthias Klose wrote: > at least in 9b134 the build system currently picks up the wrong compiler flag on > linux sparc targets. The patch below only applies it for solaris builds. > > --- a/common/autoconf/flags.m4 > +++ b/common/autoconf/flags.m4 > @@ -921,7 +921,7 @@ > # Set some additional per-CPU defines. > if test "x$OPENJDK_$1_OS-$OPENJDK_$1_CPU" = xwindows-x86; then > $2JVM_CFLAGS="[$]$2JVM_CFLAGS -arch:IA32" > - elif test "x$OPENJDK_$1_CPU" = xsparcv9; then > + elif test "xx$OPENJDK_$1_OS-$OPENJDK_$1_CPU" = xsolaris-sparcv9; then > $2JVM_CFLAGS="[$]$2JVM_CFLAGS -xarch=sparc" > elif test "x$OPENJDK_$1_CPU" = xppc64; then > if test "x$OPENJDK_$1_OS" = xlinux; then > > Is the linux sparc port supposed to work in 9? It apparently does in jdk8, but > the build later fails with > > /<>/src/hotspot/src/os_cpu/linux_sparc/vm/os_linux_sparc.cpp: In > function 'bool checkByteBuffer(address, address, u_char**)': > /<>/src/hotspot/src/os_cpu/linux_sparc/vm/os_linux_sparc.cpp:442:49: > error: 'thread' was not declared in this scope > *stub = SharedRuntime::handle_unsafe_access(thread, npc); > ^~~~~~ > lib/CompileJvm.gmk:201: recipe for target > '/<>/build/hotspot/variant-server/libjvm/objs/os_linux_sparc.o' failed > > > Matthias From doko at ubuntu.com Mon Sep 5 12:03:28 2016 From: doko at ubuntu.com (Matthias Klose) Date: Mon, 5 Sep 2016 14:03:28 +0200 Subject: [patch] add zero support for x86_64-linux-gnux32 target Message-ID: <44e0be8d-f78b-0143-ebf7-d23937becdfd@ubuntu.com> The attached patch adds support for building zero for the x86_64-linux-gnux32 target, having changes in the build system, hotspot and jdk. - the build system currently only derives the target from the cpu in PLATFORM_EXTRACT_VARS_FROM_CPU; that is not enough for the new target, which only differs by the ending of the triplet. However the $host macro should be available anywhere. - the hotspot part just handles the new "cpu" - GensrcX11Wrappers.gmk assumes that there is a black/white decision about -m32/-m64. The patch works around it. However the real patch should be to get these flags from the build system, and not hardcode itself. - the sysctl system call is unsupported in the x32 kernel, and just the include leads to a build error. From my point of view the header is not needed. I had successful builds on all other targets without including it. If you want to keep the include, then it should be guarded with #if !(defined(_ILP32) && defined(__x86_64__)) Matthias -------------- next part -------------- A non-text attachment was scrubbed... Name: zero-x32.diff Type: text/x-diff Size: 2767 bytes Desc: not available URL: From erik.joelsson at oracle.com Mon Sep 5 12:11:45 2016 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Mon, 5 Sep 2016 14:11:45 +0200 Subject: [patch] add zero support for x86_64-linux-gnux32 target In-Reply-To: <44e0be8d-f78b-0143-ebf7-d23937becdfd@ubuntu.com> References: <44e0be8d-f78b-0143-ebf7-d23937becdfd@ubuntu.com> Message-ID: <3e7b40fa-acab-b631-7b36-ba6d6ea41813@oracle.com> Build changes look ok. /Erik On 2016-09-05 14:03, Matthias Klose wrote: > The attached patch adds support for building zero for the x86_64-linux-gnux32 > target, having changes in the build system, hotspot and jdk. > > - the build system currently only derives the target from > the cpu in PLATFORM_EXTRACT_VARS_FROM_CPU; that is not enough > for the new target, which only differs by the ending of the > triplet. However the $host macro should be available anywhere. > > - the hotspot part just handles the new "cpu" > > - GensrcX11Wrappers.gmk assumes that there is a black/white > decision about -m32/-m64. The patch works around it. However > the real patch should be to get these flags from the build > system, and not hardcode itself. > > - the sysctl system call is unsupported in the x32 kernel, and > just the include leads to a build error. From my point of view > the header is not needed. I had successful builds on all other > targets without including it. If you want to keep the include, > then it should be guarded with > #if !(defined(_ILP32) && defined(__x86_64__)) > > Matthias From sgehwolf at redhat.com Mon Sep 5 12:16:12 2016 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Mon, 05 Sep 2016 14:16:12 +0200 Subject: [patch] add zero support for x86_64-linux-gnux32 target In-Reply-To: <44e0be8d-f78b-0143-ebf7-d23937becdfd@ubuntu.com> References: <44e0be8d-f78b-0143-ebf7-d23937becdfd@ubuntu.com> Message-ID: <1473077772.2884.11.camel@redhat.com> On Mon, 2016-09-05 at 14:03 +0200, Matthias Klose wrote: > The attached patch adds support for building zero for the x86_64- > linux-gnux32 > target, having changes in the build system, hotspot and jdk. > > ?- the build system currently only derives the target from > ???the cpu in PLATFORM_EXTRACT_VARS_FROM_CPU; that is not enough > ???for the new target, which only differs by the ending of the > ???triplet. However the $host macro should be available anywhere. > > ?- the hotspot part just handles the new "cpu" > > ?- GensrcX11Wrappers.gmk assumes that there is a black/white > ???decision about -m32/-m64. The patch works around it.??However > ???the real patch should be to get these flags from the build > ???system, and not hardcode itself. > > ?- the sysctl system call is unsupported in the x32 kernel, and > ???just the include leads to a build error.??From my point of view > ???the header is not needed. I had successful builds on all other > ???targets without including it. If you want to keep the include, > ???then it should be guarded with > ???#if !(defined(_ILP32) && defined(__x86_64__)) > > Matthias I've filed this bug for this: https://bugs.openjdk.java.net/browse/JDK-8165440 Unfortunately, I have no way of testing it. Cheers, Severin From sgehwolf at redhat.com Mon Sep 5 12:24:04 2016 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Mon, 05 Sep 2016 14:24:04 +0200 Subject: [patch] Fix zero builds for non-listed architectures In-Reply-To: <1706a231-0993-3059-0c75-248d0f7efbcb@debian.org> References: <1706a231-0993-3059-0c75-248d0f7efbcb@debian.org> Message-ID: <1473078244.2884.13.camel@redhat.com> On Tue, 2016-08-30 at 14:33 +0200, Matthias Klose wrote: > The zero build for architectures not explicitly listed fails currently in > hotspot, because of an unset macro, and the compiler complains about a single -D > option passed. > > Here are two proposals how to fix these: > > ?- Not passing the -D if it is not set. Afaik the define > ???is only used for hotspot builds, not for zero builds. > > ?- Provide a fall back for "unknown" zero architectures. > > Matthias I've filed this bug for this: https://bugs.openjdk.java.net/browse/JDK-8165441 Cheers, Severin From sgehwolf at redhat.com Mon Sep 5 12:28:53 2016 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Mon, 05 Sep 2016 14:28:53 +0200 Subject: [patch] Fix zero builds for non-listed architectures In-Reply-To: <1473078244.2884.13.camel@redhat.com> References: <1706a231-0993-3059-0c75-248d0f7efbcb@debian.org> <1473078244.2884.13.camel@redhat.com> Message-ID: <1473078533.2884.16.camel@redhat.com> On Mon, 2016-09-05 at 14:24 +0200, Severin Gehwolf wrote: > On Tue, 2016-08-30 at 14:33 +0200, Matthias Klose wrote: > > > > The zero build for architectures not explicitly listed fails > > currently in > > hotspot, because of an unset macro, and the compiler complains > > about a single -D > > option passed. > > > > Here are two proposals how to fix these: > > > > ?- Not passing the -D if it is not set. Afaik the > > define > > ???is only used for hotspot builds, not for zero builds. > > > > ?- Provide a fall back for "unknown" zero architectures. > > > > Matthias > > I've filed this bug for this: > https://bugs.openjdk.java.net/browse/JDK-8165441 Sorry, duplicate of: https://bugs.openjdk.java.net/browse/JDK-8165158 From david.holmes at oracle.com Mon Sep 5 23:01:59 2016 From: david.holmes at oracle.com (David Holmes) Date: Tue, 6 Sep 2016 09:01:59 +1000 Subject: [patch] add zero support for x86_64-linux-gnux32 target In-Reply-To: <1473077772.2884.11.camel@redhat.com> References: <44e0be8d-f78b-0143-ebf7-d23937becdfd@ubuntu.com> <1473077772.2884.11.camel@redhat.com> Message-ID: <490d3972-937f-893a-dd97-ae8759493e59@oracle.com> Hi Severin, Matthias, On 5/09/2016 10:16 PM, Severin Gehwolf wrote: > On Mon, 2016-09-05 at 14:03 +0200, Matthias Klose wrote: >> The attached patch adds support for building zero for the x86_64- >> linux-gnux32 >> target, having changes in the build system, hotspot and jdk. >> >> - the build system currently only derives the target from >> the cpu in PLATFORM_EXTRACT_VARS_FROM_CPU; that is not enough >> for the new target, which only differs by the ending of the >> triplet. However the $host macro should be available anywhere. >> >> - the hotspot part just handles the new "cpu" >> >> - GensrcX11Wrappers.gmk assumes that there is a black/white >> decision about -m32/-m64. The patch works around it. However >> the real patch should be to get these flags from the build >> system, and not hardcode itself. >> >> - the sysctl system call is unsupported in the x32 kernel, and >> just the include leads to a build error. From my point of view >> the header is not needed. I had successful builds on all other >> targets without including it. If you want to keep the include, >> then it should be guarded with >> #if !(defined(_ILP32) && defined(__x86_64__)) >> >> Matthias > > I've filed this bug for this: > https://bugs.openjdk.java.net/browse/JDK-8165440 Please note that as a P4 issue this can not be fixed given we have hit RDP1: http://mail.openjdk.java.net/pipermail/jdk9-dev/2016-August/004777.html Further this is filed as "bug" but seems to clearly be an enhancement, so you would need approval for it to come in post-Feature-Complete. Please consider if this is something that must be fixed for 9 or can be deferred. Otherwise you will need to follow additional approval processes. Sorry. David (just the messenger!) > Unfortunately, I have no way of testing it. > > Cheers, > Severin > From david.holmes at oracle.com Mon Sep 5 23:07:59 2016 From: david.holmes at oracle.com (David Holmes) Date: Tue, 6 Sep 2016 09:07:59 +1000 Subject: [patch] fix compiler flag for linux sparc targets In-Reply-To: <0f524f4a-4ae6-4fb1-ac92-768bdbdb9579@oracle.com> References: <0f524f4a-4ae6-4fb1-ac92-768bdbdb9579@oracle.com> Message-ID: <3884ee4b-1d90-8bad-4c2f-a4aefa09b238@oracle.com> On 5/09/2016 9:33 PM, Erik Joelsson wrote: > Patch looks ok. > > To my knowledge, nobody is actively keeping linux-sparc working in 9, so > I would expect it to be broken. It is not a listed platform for 9 or 8 (not even listed as known to be broken): https://wiki.openjdk.java.net/display/Build/Supported+Build+Platforms These patches likely fall out of scope now that we have hit RDP1. David ----- > /Erik > > On 2016-09-05 13:10, Matthias Klose wrote: >> at least in 9b134 the build system currently picks up the wrong >> compiler flag on >> linux sparc targets. The patch below only applies it for solaris builds. >> >> --- a/common/autoconf/flags.m4 >> +++ b/common/autoconf/flags.m4 >> @@ -921,7 +921,7 @@ >> # Set some additional per-CPU defines. >> if test "x$OPENJDK_$1_OS-$OPENJDK_$1_CPU" = xwindows-x86; then >> $2JVM_CFLAGS="[$]$2JVM_CFLAGS -arch:IA32" >> - elif test "x$OPENJDK_$1_CPU" = xsparcv9; then >> + elif test "xx$OPENJDK_$1_OS-$OPENJDK_$1_CPU" = xsolaris-sparcv9; then >> $2JVM_CFLAGS="[$]$2JVM_CFLAGS -xarch=sparc" >> elif test "x$OPENJDK_$1_CPU" = xppc64; then >> if test "x$OPENJDK_$1_OS" = xlinux; then >> >> Is the linux sparc port supposed to work in 9? It apparently does in >> jdk8, but >> the build later fails with >> >> /<>/src/hotspot/src/os_cpu/linux_sparc/vm/os_linux_sparc.cpp: >> In >> function 'bool checkByteBuffer(address, address, u_char**)': >> /<>/src/hotspot/src/os_cpu/linux_sparc/vm/os_linux_sparc.cpp:442:49: >> >> error: 'thread' was not declared in this scope >> *stub = SharedRuntime::handle_unsafe_access(thread, npc); >> ^~~~~~ >> lib/CompileJvm.gmk:201: recipe for target >> '/<>/build/hotspot/variant-server/libjvm/objs/os_linux_sparc.o' >> failed >> >> >> Matthias > From per.liden at oracle.com Tue Sep 6 12:17:49 2016 From: per.liden at oracle.com (Per Liden) Date: Tue, 6 Sep 2016 14:17:49 +0200 Subject: RFR: 8151600: Syntax for -Xlog complicates shell scripts In-Reply-To: <61cf5c4b-4913-3a67-81fe-2742bb11b76a@oracle.com> References: <8dc40379-cfe7-29df-fefa-1fdb1e79dc2c@oracle.com> <9148902b-c03a-5254-9c4e-9a531d1d0477@oracle.com> <5a44e2ab-f503-55ce-a9a3-fcd5d5aa700f@oracle.com> <57C69E49.8070504@oracle.com> <57C6AFF7.7000702@oracle.com> <61cf5c4b-4913-3a67-81fe-2742bb11b76a@oracle.com> Message-ID: Hi, On 2016-08-31 21:43, Chris Plummer wrote: > On 8/31/16 4:44 AM, Stefan Karlsson wrote: >> >> >> On 2016-08-31 12:22, Ioi Lam wrote: >>> >>> >>> On 8/31/16 2:07 AM, Claes Redestad wrote: >>>> Hi, >>>> >>>> On 2016-08-30 21:22, Robbin Ehn wrote: >>>>> Hi Coleen, thanks for having a look at this, >>>>> >>>>> On 08/30/2016 04:48 PM, Coleen Phillimore wrote: >>>>>> >>>>>> Honestly, I think I can live with "*" rather than "+" and definitely >>>>>> better than "%" (the latter seems a bit meaningless). >>>>> >>>>> I'm not removing asterisk, only adding a more shell friendly wildcard. >>>> >>>> Too bad! I think asterisk implies something different intuitively than >>>> what it actually does. >>>> >>>> Example: -Xlog:class* means exactly "the union of all tag sets >>>> containing the tag class", say: class, class+load, class+init >>>> >>>> I had to read the JEP long and hard to realize it doesn't mean, say, >>>> partial matches or similar expressions. >>>> >>>> IMHO we would more succinctly express the intent with a new meta-tag >>>> any: >>>> >>>> -Xlog:class+any >>>> -Xlog:gc+any >>>> >>>> Optionally leave * as an alias for +any, since it's likely entrenched >>>> already, but if there are compatibility issues with including * as >>>> a control character in command lines maybe now is the (last possible) >>>> time to consider dropping this? >>>> >>> >>> I vote for dropping * in favor of +any. >> >> My vote: I don't want to write -Xlog:gc+any and prefer to write >> -Xlog:gc+ or -Xlog:gc*. > +1 +1 /Per > > ...or maybe -Xlog:gc+* (I'll duck now). > > Chris >> >> Thanks, >> StefanK >> >>> >>> Let's not make the same mistake as MS in picking the "\" for >>> directory sepatator :-). Yes, it can be escaped, but everybody hates >>> you. >>> >>> - Ioi >>>> /Claes >>>> >>>>> >>>>> E.g. >>>>> rbt --jvm-args "-Xlog:gc*" >>>>> should always have the intended effect regardless test type. >>>>> >>>>> /Robbin >>>>> >>>>>> >>>>>> Coleen >>>>>> >>>>>> On 8/30/16 10:04 AM, Robbin Ehn wrote: >>>>>>> Hi Dmitry, >>>>>>> >>>>>>> On 08/30/2016 03:56 PM, Dmitry Samersoff wrote: >>>>>>>> Robbin, >>>>>>>> >>>>>>>> As far as I understand + is used as a delimiter >>>>>>>> e.g. -Xlog:class+load >>>>>>>> >>>>>>>> It might be better to use % for wildcard. >>>>>>> >>>>>>> Yes it might, I thought trailing plus was best. >>>>>>> >>>>>>> E.g. >>>>>>> "-Xlog:class+load*" would in this case be "-Xlog:class+load+" >>>>>>> >>>>>>> But if people prefer e.g. "-Xlog:class+load%" then we can go with >>>>>>> that. >>>>>>> >>>>>>> Thanks for input! >>>>>>> >>>>>>> /Robbin >>>>>>> >>>>>>>> >>>>>>>> -Dmitry >>>>>>>> >>>>>>>> On 2016-08-30 15:29, Robbin Ehn wrote: >>>>>>>>> Hi all, >>>>>>>>> >>>>>>>>> The current syntax for -Xlog uses '*' for wildcard matching of >>>>>>>>> tags. >>>>>>>>> In shell scripts where * is unintended expanded causes problems. >>>>>>>>> This patch adds the possibility to use trailing '+' instead. >>>>>>>>> >>>>>>>>> E.g. >>>>>>>>> "-Xlog:gc+" is thus equivalent to "-Xlog:gc*" >>>>>>>>> >>>>>>>>> Tested with modified gtest test. >>>>>>>>> >>>>>>>>> Webrev: >>>>>>>>> http://cr.openjdk.java.net/~rehn/8151600/hotspot.01/webrev/ >>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8151600 >>>>>>>>> >>>>>>>>> Thanks! >>>>>>>>> >>>>>>>>> /Robbin >>>>>>>> >>>>>>>> >>>>>> >>> >> > From robbin.ehn at oracle.com Tue Sep 6 14:33:32 2016 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Tue, 6 Sep 2016 16:33:32 +0200 Subject: RFR: 8151600: Syntax for -Xlog complicates shell scripts In-Reply-To: References: Message-ID: Hi all, thanks for all input, My intention was not to change the current syntax, only to add another way to write '*'. I'll squash this patch since no one seemed to want it and Dmitry had objections. If you really don't want '*' you need to pick up the boll! Thanks! /Robbin PS: Chris, in current patch you can write "-Xlog:gc+*" (not really intend) On 08/30/2016 02:29 PM, Robbin Ehn wrote: > Hi all, > > The current syntax for -Xlog uses '*' for wildcard matching of tags. > In shell scripts where * is unintended expanded causes problems. > This patch adds the possibility to use trailing '+' instead. > > E.g. > "-Xlog:gc+" is thus equivalent to "-Xlog:gc*" > > Tested with modified gtest test. > > Webrev: http://cr.openjdk.java.net/~rehn/8151600/hotspot.01/webrev/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8151600 > > Thanks! > > /Robbin From jon.masamitsu at oracle.com Tue Sep 6 16:31:53 2016 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Tue, 6 Sep 2016 09:31:53 -0700 Subject: Request for review (s) - 8155948: Add message for CMS deprecation for some internal builds In-Reply-To: References: <74eccd9a-1525-a9ef-1608-294de99d4483@oracle.com> Message-ID: Kim, Thanks for the review. On 9/1/2016 7:36 AM, Kim Barrett wrote: >> On Aug 31, 2016, at 2:06 PM, Jon Masamitsu wrote: >> >> Adds the support to emit a deprecation message in some builds. >> >> https://bugs.openjdk.java.net/browse/JDK-8155948 >> >> A data structure is moved to a header file to allow >> sharing and a stub function is added which can be >> specialized for different builds. >> >> http://cr.openjdk.java.net/~jmasa/8155948/webrev.00/ > Mostly looks ok. A style issue: > > ------------------------------------------------------------------------------ > src/share/vm/runtime/arguments.hpp > 45 typedef struct { > ... > 50 } SpecialFlag; > > This is C++ code, not C. Rather than using typedef of an anonymous > struct, we should be using a named struct, e.g. > > struct SpecialFlag { ... }; > > ------------------------------------------------------------------------------ Fixed per your suggestion. Delta webrev http://cr.openjdk.java.net/~jmasa/8155948/webrev_delta.00_01/ Full webrev http://cr.openjdk.java.net/~jmasa/8155948/webrev.01/ Jon > > There are some other similar typedefs in arguments.cpp that aren?t touched by this change. > From jon.masamitsu at oracle.com Tue Sep 6 16:51:06 2016 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Tue, 6 Sep 2016 09:51:06 -0700 Subject: Request for review (s) - 8155948: Add message for CMS deprecation for some internal builds In-Reply-To: References: <74eccd9a-1525-a9ef-1608-294de99d4483@oracle.com> Message-ID: <52db37a5-efa4-79dd-771f-45e1fc1ccaf5@oracle.com> Please ignore this request for the moment. My webrevs are not what I thought they were. Jon On 9/6/2016 9:31 AM, Jon Masamitsu wrote: > Kim, > > Thanks for the review. > > On 9/1/2016 7:36 AM, Kim Barrett wrote: >>> On Aug 31, 2016, at 2:06 PM, Jon Masamitsu >>> wrote: >>> >>> Adds the support to emit a deprecation message in some builds. >>> >>> https://bugs.openjdk.java.net/browse/JDK-8155948 >>> >>> A data structure is moved to a header file to allow >>> sharing and a stub function is added which can be >>> specialized for different builds. >>> >>> http://cr.openjdk.java.net/~jmasa/8155948/webrev.00/ >> Mostly looks ok. A style issue: >> >> ------------------------------------------------------------------------------ >> >> src/share/vm/runtime/arguments.hpp >> 45 typedef struct { >> ... >> 50 } SpecialFlag; >> >> This is C++ code, not C. Rather than using typedef of an anonymous >> struct, we should be using a named struct, e.g. >> >> struct SpecialFlag { ... }; >> >> ------------------------------------------------------------------------------ >> > > Fixed per your suggestion. > > Delta webrev > http://cr.openjdk.java.net/~jmasa/8155948/webrev_delta.00_01/ > > Full webrev > http://cr.openjdk.java.net/~jmasa/8155948/webrev.01/ > > Jon > >> >> There are some other similar typedefs in arguments.cpp that aren?t >> touched by this change. > > > > > > >> > From mikael.gerdin at oracle.com Wed Sep 7 08:22:36 2016 From: mikael.gerdin at oracle.com (Mikael Gerdin) Date: Wed, 7 Sep 2016 10:22:36 +0200 Subject: RFR: 8151600: Syntax for -Xlog complicates shell scripts In-Reply-To: References: Message-ID: <65dca559-721b-b779-e633-e45ffccb740b@oracle.com> Hi, On 2016-09-06 16:33, Robbin Ehn wrote: > Hi all, thanks for all input, > > My intention was not to change the current syntax, only to add another > way to write '*'. > I'll squash this patch since no one seemed to want it and Dmitry had > objections. FWIW I still think it would be useful to have the suggested "+" alias for "*" to avoid having to mess around with shell wildcard problems. /Mikael > > If you really don't want '*' you need to pick up the boll! > > Thanks! > > /Robbin > > PS: > Chris, in current patch you can write "-Xlog:gc+*" (not really intend) > > > On 08/30/2016 02:29 PM, Robbin Ehn wrote: >> Hi all, >> >> The current syntax for -Xlog uses '*' for wildcard matching of tags. >> In shell scripts where * is unintended expanded causes problems. >> This patch adds the possibility to use trailing '+' instead. >> >> E.g. >> "-Xlog:gc+" is thus equivalent to "-Xlog:gc*" >> >> Tested with modified gtest test. >> >> Webrev: http://cr.openjdk.java.net/~rehn/8151600/hotspot.01/webrev/ >> Bug: https://bugs.openjdk.java.net/browse/JDK-8151600 >> >> Thanks! >> >> /Robbin From jon.masamitsu at oracle.com Wed Sep 7 17:12:31 2016 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Wed, 7 Sep 2016 10:12:31 -0700 Subject: Request for review (s) - 8155948: Add message for CMS deprecation for some internal builds In-Reply-To: <52db37a5-efa4-79dd-771f-45e1fc1ccaf5@oracle.com> References: <74eccd9a-1525-a9ef-1608-294de99d4483@oracle.com> <52db37a5-efa4-79dd-771f-45e1fc1ccaf5@oracle.com> Message-ID: Webrevs are ready now and is somewhat larger than before since (as was pointed out to me, thanks Kim) I had not covered all the arguments that I should have. When CMS is turned on as a result of UseAutoGCSelectPolicy, I also turn on ParNew to avoid the message (and exit) > It is not possible to combine the DefNew young collector with the CMS > collector. > Error: Could not create the Java Virtual Machine. > Error: A fatal exception has occurred. Program will exit. Delta webrev http://cr.openjdk.java.net/~jmasa/8155948/webrev_delta.00_01/ Full webrev http://cr.openjdk.java.net/~jmasa/8155948/webrev.01/ Thanks. Jon On 09/06/2016 09:51 AM, Jon Masamitsu wrote: > Please ignore this request for the moment. My webrevs are not what > I thought they were. > > Jon > > On 9/6/2016 9:31 AM, Jon Masamitsu wrote: >> Kim, >> >> Thanks for the review. >> >> On 9/1/2016 7:36 AM, Kim Barrett wrote: >>>> On Aug 31, 2016, at 2:06 PM, Jon Masamitsu >>>> wrote: >>>> >>>> Adds the support to emit a deprecation message in some builds. >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8155948 >>>> >>>> A data structure is moved to a header file to allow >>>> sharing and a stub function is added which can be >>>> specialized for different builds. >>>> >>>> http://cr.openjdk.java.net/~jmasa/8155948/webrev.00/ >>> Mostly looks ok. A style issue: >>> >>> ------------------------------------------------------------------------------ >>> >>> src/share/vm/runtime/arguments.hpp >>> 45 typedef struct { >>> ... >>> 50 } SpecialFlag; >>> >>> This is C++ code, not C. Rather than using typedef of an anonymous >>> struct, we should be using a named struct, e.g. >>> >>> struct SpecialFlag { ... }; >>> >>> ------------------------------------------------------------------------------ >>> >> >> Fixed per your suggestion. >> >> Delta webrev >> http://cr.openjdk.java.net/~jmasa/8155948/webrev_delta.00_01/ >> >> Full webrev >> http://cr.openjdk.java.net/~jmasa/8155948/webrev.01/ >> >> Jon >> >>> >>> There are some other similar typedefs in arguments.cpp that aren?t >>> touched by this change. >> >> >> >> >> >> >>> >> > From kim.barrett at oracle.com Wed Sep 7 22:12:58 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 7 Sep 2016 18:12:58 -0400 Subject: Request for review (s) - 8155948: Add message for CMS deprecation for some internal builds In-Reply-To: References: <74eccd9a-1525-a9ef-1608-294de99d4483@oracle.com> <52db37a5-efa4-79dd-771f-45e1fc1ccaf5@oracle.com> Message-ID: > On Sep 7, 2016, at 1:12 PM, Jon Masamitsu wrote: > > Webrevs are ready now and is somewhat larger than before since > (as was pointed out to me, thanks Kim) I had not covered > all the arguments that I should have. > > When CMS is turned on as a result of UseAutoGCSelectPolicy, I also turn > on ParNew to avoid the message (and exit) > >> It is not possible to combine the DefNew young collector with the CMS collector. >> Error: Could not create the Java Virtual Machine. >> Error: A fatal exception has occurred. Program will exit. > > Delta webrev > http://cr.openjdk.java.net/~jmasa/8155948/webrev_delta.00_01/ > > Full webrev > http://cr.openjdk.java.net/~jmasa/8155948/webrev.01/ ------------------------------------------------------------------------------ src/share/vm/runtime/arguments.cpp 449 static bool lookup_special_flag(const char *flag_name, SpecialFlag& flag) { 450 for (size_t i = 0; special_jvm_flags[i].name != NULL; i++) { ... 456 // Secondary list 457 return lookup_special_flag_ext(flag_name, flag); Perhaps the ext lookup should be done first, to allow ext to override open behavior. For example, consider an option that is being deprecated in openjdk, but a commercial vendor is going to provide continued support. ------------------------------------------------------------------------------ src/share/vm/runtime/arguments.cpp 4064 void Arguments::handle_concgc_flags() { 4065 SpecialFlag flag; 4066 const char *flag_name = "UseConcMarkSweepGC"; 4067 if (log_is_enabled(Warning, gc) && 4068 lookup_special_flag(flag_name, flag)) { 4069 handle_aliases_and_deprecation(flag_name, /* print warning */ true); 4070 log_warning(gc)("-Xconcgc/-Xnoconcgc uses UseConcMarkSweepGC"); 4071 } 4072 } This non-trivial little dance is done both here and for UseAutoGCSelectPolicy. I think handle_concgc_flags could deal with both if it took an argument of the "offending" flag. ------------------------------------------------------------------------------ From sybersnake at gmail.com Thu Sep 8 08:54:25 2016 From: sybersnake at gmail.com (Jon V.) Date: Thu, 8 Sep 2016 04:54:25 -0400 Subject: Does the G1 Garbage Collector move memory during compaction? Message-ID: I found some information that says that G1 does have compaction phases but there isn?t any mention of it actually moves data during compaction or not or if it just operates on lists? Thanks, J From ashipile at redhat.com Thu Sep 8 09:02:36 2016 From: ashipile at redhat.com (Aleksey Shipilev) Date: Thu, 8 Sep 2016 12:02:36 +0300 Subject: Does the G1 Garbage Collector move memory during compaction? In-Reply-To: References: Message-ID: <2b47bdb4-7bb4-c1c2-8087-a6a32e5f230a@redhat.com> On 09/08/2016 11:54 AM, Jon V. wrote: > I found some information that says that G1 does have compaction phases but > there isn?t any mention of it actually moves data during compaction or not > or if it just operates on lists? I can't figure what is compaction if it does not actually moves the data. Collectors move data a lot. See e.g. JOL [1] examples for object promotion [2], compaction [3], and defragmentation [4]. Thanks, -Aleksey [1] http://openjdk.java.net/projects/code-tools/jol/ [2] http://hg.openjdk.java.net/code-tools/jol/file/07087260ce41/jol-samples/src/main/java/org/openjdk/jol/samples/JOLSample_19_Promotion.java [3] http://hg.openjdk.java.net/code-tools/jol/file/07087260ce41/jol-samples/src/main/java/org/openjdk/jol/samples/JOLSample_22_Compaction.java [4] http://hg.openjdk.java.net/code-tools/jol/file/07087260ce41/jol-samples/src/main/java/org/openjdk/jol/samples/JOLSample_23_Defragmentation.java From sybersnake at gmail.com Thu Sep 8 16:30:47 2016 From: sybersnake at gmail.com (Jon V.) Date: Thu, 8 Sep 2016 12:30:47 -0400 Subject: Clarification on what the JNI spec means by nonmovable Message-ID: Hello everyone! I?m trying to document the exact behavior of direct pointer access of passed arguments into JNI in HotSpot. I have a huge post on StackOverflow for posterity purposes as it seems to be a commonly misunderstood behavior. http://stackoverflow.com/questions/39381339/understanding-safe-access-of-jni-arguments from https://docs.oracle.com/javase/7/docs/technotes/guides/jni/spec/design.html#wp16789 The JNI spec says, ?To implement local references, the Java VM creates a registry for each transition of control from Java to a native method. A registry maps nonmovable local references to Java objects, and keeps the objects from being garbage collected.? I?m trying to understand if ?nonmovable? actually means that the objects in the map WILL NOT be compacted/moved (heap memory) or if it just means they won?t be garbage-collected. The answer dictates if the use of critical sections are actually necessary. What are the effects for each garbage collector? I found an email on the archives that says that JNI critical sections do not effect operation of CMS and I?m interested in understanding the behavior for G1 as well. This all boils down to the ability to use unsafe raw direct pointers in JNI without critical sections in HotSpot. Thank you, J From jon.masamitsu at oracle.com Thu Sep 8 16:35:50 2016 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Thu, 8 Sep 2016 09:35:50 -0700 Subject: Request for review (s) - 8155948: Add message for CMS deprecation for some internal builds In-Reply-To: References: <74eccd9a-1525-a9ef-1608-294de99d4483@oracle.com> <52db37a5-efa4-79dd-771f-45e1fc1ccaf5@oracle.com> Message-ID: <02968df4-b53d-c1d1-65e4-a612ca393bae@oracle.com> Kim, Thanks for looking at this. Comments below. On 09/07/2016 03:12 PM, Kim Barrett wrote: >> On Sep 7, 2016, at 1:12 PM, Jon Masamitsu wrote: >> >> Webrevs are ready now and is somewhat larger than before since >> (as was pointed out to me, thanks Kim) I had not covered >> all the arguments that I should have. >> >> When CMS is turned on as a result of UseAutoGCSelectPolicy, I also turn >> on ParNew to avoid the message (and exit) >> >>> It is not possible to combine the DefNew young collector with the CMS collector. >>> Error: Could not create the Java Virtual Machine. >>> Error: A fatal exception has occurred. Program will exit. >> Delta webrev >> http://cr.openjdk.java.net/~jmasa/8155948/webrev_delta.00_01/ >> >> Full webrev >> http://cr.openjdk.java.net/~jmasa/8155948/webrev.01/ > ------------------------------------------------------------------------------ > src/share/vm/runtime/arguments.cpp > 449 static bool lookup_special_flag(const char *flag_name, SpecialFlag& flag) { > 450 for (size_t i = 0; special_jvm_flags[i].name != NULL; i++) { > ... > 456 // Secondary list > 457 return lookup_special_flag_ext(flag_name, flag); > > Perhaps the ext lookup should be done first, to allow ext to override > open behavior. For example, consider an option that is being > deprecated in openjdk, but a commercial vendor is going to provide > continued support. Good point. I will move the _ext check to be done first. > > ------------------------------------------------------------------------------ > src/share/vm/runtime/arguments.cpp > 4064 void Arguments::handle_concgc_flags() { > 4065 SpecialFlag flag; > 4066 const char *flag_name = "UseConcMarkSweepGC"; > 4067 if (log_is_enabled(Warning, gc) && > 4068 lookup_special_flag(flag_name, flag)) { > 4069 handle_aliases_and_deprecation(flag_name, /* print warning */ true); > 4070 log_warning(gc)("-Xconcgc/-Xnoconcgc uses UseConcMarkSweepGC"); > 4071 } > 4072 } > > This non-trivial little dance is done both here and for > UseAutoGCSelectPolicy. I think handle_concgc_flags could deal with > both if it took an argument of the "offending" flag. > > ------------------------------------------------------------------------------ > -Xconcgc does not have a SpecialFlag associated with it currently. Do you mean add one for "concgc" (even through it is not a -XX flags)? Or do you mean that "offending" flag is just a parameter that chooses between the two cases -Xconcgc/-Xnoconcgc and -XX:+UseAutoGCSelectPolicy? Jon From rednaxelafx at gmail.com Thu Sep 8 17:30:18 2016 From: rednaxelafx at gmail.com (Krystal Mok) Date: Thu, 8 Sep 2016 10:30:18 -0700 Subject: Clarification on what the JNI spec means by nonmovable In-Reply-To: References: Message-ID: Hi Jon, On Thu, Sep 8, 2016 at 9:30 AM, Jon V. wrote: > Hello everyone! > > I?m trying to document the exact behavior of direct pointer access of > passed arguments into JNI in HotSpot. I have a huge post on StackOverflow > for posterity purposes as it seems to be a commonly misunderstood behavior. > > http://stackoverflow.com/questions/39381339/understanding-safe-access-of- > jni-arguments > > from > https://docs.oracle.com/javase/7/docs/technotes/ > guides/jni/spec/design.html#wp16789 > > The JNI spec says, ?To implement local references, the Java VM creates a > registry for each transition of control from Java to a native method. A > registry maps nonmovable local references to Java objects, and keeps the > objects from being garbage collected.? > > "Nonmovable" refers to the "local JNI reference", not the Java object instance itself. What that gurantees is that, given a jobject (which is typedef'd as an opaque pointer), you can safely say that as long as this jobject is still in scope, the value of the jobject won't change. But that doesn't imply anything related whether or not the Java object it's referring to is pinned. The "registry map" here is just an abstract notion that doesn't necessarily materialize into any real data structures in some certain JVM implementations. > I?m trying to understand if ?nonmovable? actually means that the objects in > the map WILL NOT be compacted/moved (heap memory) or if it just means they > won?t be garbage-collected. The answer dictates if the use of critical > sections are actually necessary. > > Rule of thumb: don't try to bypass the JNI abstractions. It'll bite back hard if you switch between different JVM implementations. Specifically in HotSpot, jobject and friends are referred to as "JNI handles". That's because they're implemented as handles, i.e. double-indirection pointers. The underlying type that implements jobject is "oop*", where an "oop" is typedef'd from "oopDesc*", and oopDesc is the root type of garbage collected objects. jobject JNIHandle Java object [ oop* ] -> [ oop ] -> [ oopDesc ] Following the spec, what it guarantees is that for a given jobject, the JNIHandle representing the referent object will not be moved, but the actual referent object is free to move. Don't try to access raw oops from outside the VM by accessing the underlying handle internals. It's very likely to get dangling pointers after GCs. What are the effects for each garbage collector? > > All garbage collectors in HotSpot, except for CMS (mostly concurrent mark-sweep), are moving collectors. As such, after each completed collection, Java objects will move. If you're holding raw oops that the GC isn't aware of, they won't be updated during GC, and will be left dangling afterwards. JNI critical sections are implemented in HotSpot via temporarily disabling the GCs, and as soon as all JNI critical sections are completed, a full GC (by default) will be triggered to perform the delayed collection if needed. This mechanism is called the "GCLocker" in HotSpot. (GCs may be able to expand the heap to fulfill allocation requests, if the current capacity of the GC heap is smaller than the maximum capacity configured. So disabling the GC isn't always as bad as it sounds.) > I found an email on the archives that says that JNI critical sections do > not effect operation of CMS and I?m interested in understanding the > behavior for G1 as well. > > CMS is a mark-sweep collector, so it doesn't move objects. Thus, it is possible to perform CMS old gen collections even when the GCLocker is active (meaning there's at least one Java thread in a JNI critical section). G1, on the other hand, is a moving collector (an incremental copying collector with optional concurrent global marking; sometimes simply referred to as a concurrent mark-compact collector, but it's not the usual mark-compact notion). G1 GC cannot be performed when the GCLocker is active. > This all boils down to the ability to use unsafe raw direct pointers in JNI > without critical sections in HotSpot. > > Nope. Don't try that on HotSpot. There are certain code patterns that might be able to access raw oops safely without going into JNI critical sections. But that involves very heavy VM internals knowledge, which is also subject to change without notice, so practically don't try that. Hope that helps, Kris (OpenJDK username: kmo) > Thank you, > J > From sybersnake at gmail.com Thu Sep 8 17:42:31 2016 From: sybersnake at gmail.com (Jon V.) Date: Thu, 8 Sep 2016 13:42:31 -0400 Subject: Clarification on what the JNI spec means by nonmovable In-Reply-To: References: Message-ID: Thank you Kris. That confirms my understanding that the indirect pointers can be moved at anytime. I?m accessing possible multiple heap objects inside of a single JNI call and you aren?t allowed to call critical sections twice. I?ve created a workaround of using getCrtiicalString as a way to stop GC then directly access the heap memory of the other objects. This seems like a safe way to do this. I really wish JNI had another wrapper/macro we could use such as JNI_QUICK_ENTRY that would prevent GC without having to use GCLocker. Some way my JNI call could be considered ?inVM? instead of ?inNative? to keep GC at bay. On Thu, Sep 8, 2016 at 1:30 PM, Krystal Mok wrote: > Hi Jon, > > On Thu, Sep 8, 2016 at 9:30 AM, Jon V. wrote: > >> Hello everyone! >> >> I?m trying to document the exact behavior of direct pointer access of >> passed arguments into JNI in HotSpot. I have a huge post on StackOverflow >> for posterity purposes as it seems to be a commonly misunderstood >> behavior. >> >> http://stackoverflow.com/questions/39381339/understanding- >> safe-access-of-jni-arguments >> >> from >> https://docs.oracle.com/javase/7/docs/technotes/guides/jni/ >> spec/design.html#wp16789 >> >> The JNI spec says, ?To implement local references, the Java VM creates a >> registry for each transition of control from Java to a native method. A >> registry maps nonmovable local references to Java objects, and keeps the >> objects from being garbage collected.? >> >> "Nonmovable" refers to the "local JNI reference", not the Java object > instance itself. What that gurantees is that, given a jobject (which is > typedef'd as an opaque pointer), you can safely say that as long as this > jobject is still in scope, the value of the jobject won't change. But that > doesn't imply anything related whether or not the Java object it's > referring to is pinned. > > The "registry map" here is just an abstract notion that doesn't > necessarily materialize into any real data structures in some certain JVM > implementations. > > >> I?m trying to understand if ?nonmovable? actually means that the objects >> in >> the map WILL NOT be compacted/moved (heap memory) or if it just means they >> won?t be garbage-collected. The answer dictates if the use of critical >> sections are actually necessary. >> >> Rule of thumb: don't try to bypass the JNI abstractions. It'll bite back > hard if you switch between different JVM implementations. > > Specifically in HotSpot, jobject and friends are referred to as "JNI > handles". That's because they're implemented as handles, i.e. > double-indirection pointers. The underlying type that implements jobject is > "oop*", where an "oop" is typedef'd from "oopDesc*", and oopDesc is the > root type of garbage collected objects. > > jobject JNIHandle Java object > [ oop* ] -> [ oop ] -> [ oopDesc ] > > Following the spec, what it guarantees is that for a given jobject, the > JNIHandle representing the referent object will not be moved, but the > actual referent object is free to move. > > Don't try to access raw oops from outside the VM by accessing the > underlying handle internals. It's very likely to get dangling pointers > after GCs. > > What are the effects for each garbage collector? >> >> All garbage collectors in HotSpot, except for CMS (mostly concurrent > mark-sweep), are moving collectors. As such, after each completed > collection, Java objects will move. If you're holding raw oops that the GC > isn't aware of, they won't be updated during GC, and will be left dangling > afterwards. > > JNI critical sections are implemented in HotSpot via temporarily disabling > the GCs, and as soon as all JNI critical sections are completed, a full GC > (by default) will be triggered to perform the delayed collection if needed. > This mechanism is called the "GCLocker" in HotSpot. > (GCs may be able to expand the heap to fulfill allocation requests, if the > current capacity of the GC heap is smaller than the maximum capacity > configured. So disabling the GC isn't always as bad as it sounds.) > > >> I found an email on the archives that says that JNI critical sections do >> not effect operation of CMS and I?m interested in understanding the >> behavior for G1 as well. >> >> CMS is a mark-sweep collector, so it doesn't move objects. Thus, it is > possible to perform CMS old gen collections even when the GCLocker is > active (meaning there's at least one Java thread in a JNI critical section). > > G1, on the other hand, is a moving collector (an incremental copying > collector with optional concurrent global marking; sometimes simply > referred to as a concurrent mark-compact collector, but it's not the usual > mark-compact notion). G1 GC cannot be performed when the GCLocker is active. > > >> This all boils down to the ability to use unsafe raw direct pointers in >> JNI >> without critical sections in HotSpot. >> >> Nope. Don't try that on HotSpot. > There are certain code patterns that might be able to access raw oops > safely without going into JNI critical sections. But that involves very > heavy VM internals knowledge, which is also subject to change without > notice, so practically don't try that. > > Hope that helps, > Kris (OpenJDK username: kmo) > > >> Thank you, >> J >> > > From kim.barrett at oracle.com Thu Sep 8 17:44:52 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 8 Sep 2016 13:44:52 -0400 Subject: Request for review (s) - 8155948: Add message for CMS deprecation for some internal builds In-Reply-To: <02968df4-b53d-c1d1-65e4-a612ca393bae@oracle.com> References: <74eccd9a-1525-a9ef-1608-294de99d4483@oracle.com> <52db37a5-efa4-79dd-771f-45e1fc1ccaf5@oracle.com> <02968df4-b53d-c1d1-65e4-a612ca393bae@oracle.com> Message-ID: <5CB1B554-D733-40D6-881F-6237F9CE9E11@oracle.com> > On Sep 8, 2016, at 12:35 PM, Jon Masamitsu wrote: > > Kim, > > Thanks for looking at this. Comments below. > > On 09/07/2016 03:12 PM, Kim Barrett wrote: >> src/share/vm/runtime/arguments.cpp >> 4064 void Arguments::handle_concgc_flags() { >> 4065 SpecialFlag flag; >> 4066 const char *flag_name = "UseConcMarkSweepGC"; >> 4067 if (log_is_enabled(Warning, gc) && >> 4068 lookup_special_flag(flag_name, flag)) { >> 4069 handle_aliases_and_deprecation(flag_name, /* print warning */ true); >> 4070 log_warning(gc)("-Xconcgc/-Xnoconcgc uses UseConcMarkSweepGC"); >> 4071 } >> 4072 } >> >> This non-trivial little dance is done both here and for >> UseAutoGCSelectPolicy. I think handle_concgc_flags could deal with >> both if it took an argument of the "offending" flag. >> >> ------------------------------------------------------------------------------ >> > > -Xconcgc does not have a SpecialFlag associated with it currently. Do you mean > add one for "concgc" (even through it is not a -XX flags)? Or do you mean that > "offending" flag is just a parameter that chooses between the two cases > -Xconcgc/-Xnoconcgc and -XX:+UseAutoGCSelectPolicy? > > Jon A parameter that is used to tailor the warning, e.g. a const char* that is %s?ed into the log_warning message. From jon.masamitsu at oracle.com Thu Sep 8 17:28:10 2016 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Thu, 8 Sep 2016 10:28:10 -0700 Subject: Request for review (s) - 8155948: Add message for CMS deprecation for some internal builds In-Reply-To: <5CB1B554-D733-40D6-881F-6237F9CE9E11@oracle.com> References: <74eccd9a-1525-a9ef-1608-294de99d4483@oracle.com> <52db37a5-efa4-79dd-771f-45e1fc1ccaf5@oracle.com> <02968df4-b53d-c1d1-65e4-a612ca393bae@oracle.com> <5CB1B554-D733-40D6-881F-6237F9CE9E11@oracle.com> Message-ID: On 09/08/2016 10:44 AM, Kim Barrett wrote: >> On Sep 8, 2016, at 12:35 PM, Jon Masamitsu wrote: >> >> Kim, >> >> Thanks for looking at this. Comments below. >> >> On 09/07/2016 03:12 PM, Kim Barrett wrote: >>> src/share/vm/runtime/arguments.cpp >>> 4064 void Arguments::handle_concgc_flags() { >>> 4065 SpecialFlag flag; >>> 4066 const char *flag_name = "UseConcMarkSweepGC"; >>> 4067 if (log_is_enabled(Warning, gc) && >>> 4068 lookup_special_flag(flag_name, flag)) { >>> 4069 handle_aliases_and_deprecation(flag_name, /* print warning */ true); >>> 4070 log_warning(gc)("-Xconcgc/-Xnoconcgc uses UseConcMarkSweepGC"); >>> 4071 } >>> 4072 } >>> >>> This non-trivial little dance is done both here and for >>> UseAutoGCSelectPolicy. I think handle_concgc_flags could deal with >>> both if it took an argument of the "offending" flag. >>> >>> ------------------------------------------------------------------------------ >>> >> -Xconcgc does not have a SpecialFlag associated with it currently. Do you mean >> add one for "concgc" (even through it is not a -XX flags)? Or do you mean that >> "offending" flag is just a parameter that chooses between the two cases >> -Xconcgc/-Xnoconcgc and -XX:+UseAutoGCSelectPolicy? >> >> Jon > A parameter that is used to tailor the warning, e.g. a const char* that is %s?ed into the log_warning message. > Will do. Thanks again. Jon From Peter.B.Kessler at Oracle.COM Thu Sep 8 18:27:19 2016 From: Peter.B.Kessler at Oracle.COM (Peter B. Kessler) Date: Thu, 8 Sep 2016 11:27:19 -0700 Subject: Clarification on what the JNI spec means by nonmovable In-Reply-To: References: Message-ID: <2b691701-f43d-0965-f306-a648d02a7c1f@Oracle.COM> A CMS young generation collection will move objects. I don't think that changes the way you should use JNI handles. But I didn't want people switching to CMS because they thought it was a non-moving collector and they could cheat. ... peter On 09/ 8/16 10:30 AM, Krystal Mok wrote: > Hi Jon, > > On Thu, Sep 8, 2016 at 9:30 AM, Jon V. wrote: > >> Hello everyone! >> >> I?m trying to document the exact behavior of direct pointer access of >> passed arguments into JNI in HotSpot. I have a huge post on StackOverflow >> for posterity purposes as it seems to be a commonly misunderstood behavior. >> >> http://stackoverflow.com/questions/39381339/understanding-safe-access-of- >> jni-arguments >> >> from >> https://docs.oracle.com/javase/7/docs/technotes/ >> guides/jni/spec/design.html#wp16789 >> >> The JNI spec says, ?To implement local references, the Java VM creates a >> registry for each transition of control from Java to a native method. A >> registry maps nonmovable local references to Java objects, and keeps the >> objects from being garbage collected.? >> >> "Nonmovable" refers to the "local JNI reference", not the Java object > instance itself. What that gurantees is that, given a jobject (which is > typedef'd as an opaque pointer), you can safely say that as long as this > jobject is still in scope, the value of the jobject won't change. But that > doesn't imply anything related whether or not the Java object it's > referring to is pinned. > > The "registry map" here is just an abstract notion that doesn't necessarily > materialize into any real data structures in some certain JVM > implementations. > > >> I?m trying to understand if ?nonmovable? actually means that the objects in >> the map WILL NOT be compacted/moved (heap memory) or if it just means they >> won?t be garbage-collected. The answer dictates if the use of critical >> sections are actually necessary. >> >> Rule of thumb: don't try to bypass the JNI abstractions. It'll bite back > hard if you switch between different JVM implementations. > > Specifically in HotSpot, jobject and friends are referred to as "JNI > handles". That's because they're implemented as handles, i.e. > double-indirection pointers. The underlying type that implements jobject is > "oop*", where an "oop" is typedef'd from "oopDesc*", and oopDesc is the > root type of garbage collected objects. > > jobject JNIHandle Java object > [ oop* ] -> [ oop ] -> [ oopDesc ] > > Following the spec, what it guarantees is that for a given jobject, the > JNIHandle representing the referent object will not be moved, but the > actual referent object is free to move. > > Don't try to access raw oops from outside the VM by accessing the > underlying handle internals. It's very likely to get dangling pointers > after GCs. > > What are the effects for each garbage collector? >> >> All garbage collectors in HotSpot, except for CMS (mostly concurrent > mark-sweep), are moving collectors. As such, after each completed > collection, Java objects will move. If you're holding raw oops that the GC > isn't aware of, they won't be updated during GC, and will be left dangling > afterwards. > > JNI critical sections are implemented in HotSpot via temporarily disabling > the GCs, and as soon as all JNI critical sections are completed, a full GC > (by default) will be triggered to perform the delayed collection if needed. > This mechanism is called the "GCLocker" in HotSpot. > (GCs may be able to expand the heap to fulfill allocation requests, if the > current capacity of the GC heap is smaller than the maximum capacity > configured. So disabling the GC isn't always as bad as it sounds.) > > >> I found an email on the archives that says that JNI critical sections do >> not effect operation of CMS and I?m interested in understanding the >> behavior for G1 as well. >> >> CMS is a mark-sweep collector, so it doesn't move objects. Thus, it is > possible to perform CMS old gen collections even when the GCLocker is > active (meaning there's at least one Java thread in a JNI critical section). > > G1, on the other hand, is a moving collector (an incremental copying > collector with optional concurrent global marking; sometimes simply > referred to as a concurrent mark-compact collector, but it's not the usual > mark-compact notion). G1 GC cannot be performed when the GCLocker is active. > > >> This all boils down to the ability to use unsafe raw direct pointers in JNI >> without critical sections in HotSpot. >> >> Nope. Don't try that on HotSpot. > There are certain code patterns that might be able to access raw oops > safely without going into JNI critical sections. But that involves very > heavy VM internals knowledge, which is also subject to change without > notice, so practically don't try that. > > Hope that helps, > Kris (OpenJDK username: kmo) > > >> Thank you, >> J >> From rednaxelafx at gmail.com Thu Sep 8 18:40:23 2016 From: rednaxelafx at gmail.com (Krystal Mok) Date: Thu, 8 Sep 2016 11:40:23 -0700 Subject: Clarification on what the JNI spec means by nonmovable In-Reply-To: <2b691701-f43d-0965-f306-a648d02a7c1f@Oracle.COM> References: <2b691701-f43d-0965-f306-a648d02a7c1f@Oracle.COM> Message-ID: Hi Peter, Thanks for the follow-up! I did mean to say "CMS the old gen collector" though. I should have made it clear that when "CMS the old gen collector" is chosen, ParNew is automatically chosen to be the young gen collector, and a mark-compact collector ("Serial Old" in current OpenJDK HotSpot, parallelized in some other variants of HotSpot) is chosen as the fallback full GC collector. All of the above, except for "CMS the old gen collector", are moving collectors. It certainly doesn't change anything as to how JNI handles should be used by external code. To paraphrase Peter's words: there's no reliable way from the outside to control whether a young/minor GC, a concurrent old gen GC, or a fallback full GC is going to run. Thanks, Kris On Thu, Sep 8, 2016 at 11:27 AM, Peter B. Kessler < Peter.B.Kessler at oracle.com> wrote: > A CMS young generation collection will move objects. I don't think that > changes the way you should use JNI handles. But I didn't want people > switching to CMS because they thought it was a non-moving collector and > they could cheat. > > ... peter > > > On 09/ 8/16 10:30 AM, Krystal Mok wrote: > >> Hi Jon, >> >> On Thu, Sep 8, 2016 at 9:30 AM, Jon V. wrote: >> >> Hello everyone! >>> >>> I?m trying to document the exact behavior of direct pointer access of >>> passed arguments into JNI in HotSpot. I have a huge post on >>> StackOverflow >>> for posterity purposes as it seems to be a commonly misunderstood >>> behavior. >>> >>> http://stackoverflow.com/questions/39381339/understanding- >>> safe-access-of- >>> jni-arguments >>> >>> from >>> https://docs.oracle.com/javase/7/docs/technotes/ >>> guides/jni/spec/design.html#wp16789 >>> >>> The JNI spec says, ?To implement local references, the Java VM creates a >>> registry for each transition of control from Java to a native method. A >>> registry maps nonmovable local references to Java objects, and keeps the >>> objects from being garbage collected.? >>> >>> "Nonmovable" refers to the "local JNI reference", not the Java object >>> >> instance itself. What that gurantees is that, given a jobject (which is >> typedef'd as an opaque pointer), you can safely say that as long as this >> jobject is still in scope, the value of the jobject won't change. But that >> doesn't imply anything related whether or not the Java object it's >> referring to is pinned. >> >> The "registry map" here is just an abstract notion that doesn't >> necessarily >> materialize into any real data structures in some certain JVM >> implementations. >> >> >> I?m trying to understand if ?nonmovable? actually means that the objects >>> in >>> the map WILL NOT be compacted/moved (heap memory) or if it just means >>> they >>> won?t be garbage-collected. The answer dictates if the use of critical >>> sections are actually necessary. >>> >>> Rule of thumb: don't try to bypass the JNI abstractions. It'll bite back >>> >> hard if you switch between different JVM implementations. >> >> Specifically in HotSpot, jobject and friends are referred to as "JNI >> handles". That's because they're implemented as handles, i.e. >> double-indirection pointers. The underlying type that implements jobject >> is >> "oop*", where an "oop" is typedef'd from "oopDesc*", and oopDesc is the >> root type of garbage collected objects. >> >> jobject JNIHandle Java object >> [ oop* ] -> [ oop ] -> [ oopDesc ] >> >> Following the spec, what it guarantees is that for a given jobject, the >> JNIHandle representing the referent object will not be moved, but the >> actual referent object is free to move. >> >> Don't try to access raw oops from outside the VM by accessing the >> underlying handle internals. It's very likely to get dangling pointers >> after GCs. >> >> What are the effects for each garbage collector? >> >>> >>> All garbage collectors in HotSpot, except for CMS (mostly concurrent >>> >> mark-sweep), are moving collectors. As such, after each completed >> collection, Java objects will move. If you're holding raw oops that the GC >> isn't aware of, they won't be updated during GC, and will be left dangling >> afterwards. >> >> JNI critical sections are implemented in HotSpot via temporarily disabling >> the GCs, and as soon as all JNI critical sections are completed, a full GC >> (by default) will be triggered to perform the delayed collection if >> needed. >> This mechanism is called the "GCLocker" in HotSpot. >> (GCs may be able to expand the heap to fulfill allocation requests, if the >> current capacity of the GC heap is smaller than the maximum capacity >> configured. So disabling the GC isn't always as bad as it sounds.) >> >> >> I found an email on the archives that says that JNI critical sections do >>> not effect operation of CMS and I?m interested in understanding the >>> behavior for G1 as well. >>> >>> CMS is a mark-sweep collector, so it doesn't move objects. Thus, it is >>> >> possible to perform CMS old gen collections even when the GCLocker is >> active (meaning there's at least one Java thread in a JNI critical >> section). >> >> G1, on the other hand, is a moving collector (an incremental copying >> collector with optional concurrent global marking; sometimes simply >> referred to as a concurrent mark-compact collector, but it's not the usual >> mark-compact notion). G1 GC cannot be performed when the GCLocker is >> active. >> >> >> This all boils down to the ability to use unsafe raw direct pointers in >>> JNI >>> without critical sections in HotSpot. >>> >>> Nope. Don't try that on HotSpot. >>> >> There are certain code patterns that might be able to access raw oops >> safely without going into JNI critical sections. But that involves very >> heavy VM internals knowledge, which is also subject to change without >> notice, so practically don't try that. >> >> Hope that helps, >> Kris (OpenJDK username: kmo) >> >> >> Thank you, >>> J >>> >>> From rednaxelafx at gmail.com Thu Sep 8 18:53:53 2016 From: rednaxelafx at gmail.com (Krystal Mok) Date: Thu, 8 Sep 2016 11:53:53 -0700 Subject: Clarification on what the JNI spec means by nonmovable In-Reply-To: References: Message-ID: On Thu, Sep 8, 2016 at 10:42 AM, Jon V. wrote: > Thank you Kris. > > That confirms my understanding that the indirect pointers can be moved at > anytime. I?m accessing possible multiple heap objects inside of a single > JNI call and you aren?t allowed to call critical sections twice. > Wait... a JNI critical section isn't like a OS thread-synchronization critical section. It does support nesting. What is the access pattern you're looking at? c.f. http://docs.oracle.com/javase/7/docs/technotes/guides/jni/spec/functions.html#GetPrimitiveArrayCritical > Multiple pairs of GetPrimtiveArrayCritical and ReleasePrimitiveArrayCritical may be nested. > I?ve created a workaround of using getCrtiicalString as a way to stop GC > then directly access the heap memory of the other objects. This seems like > a safe way to do this. > Nope, the JNI API does not guarantee this to be safe. Doing this is essentially cheating on JVMs such as HotSpot that implements JNI critical sections via temporarily disabling the GC. JNI critical section only guarantees "critical access" to the specified object, and that doesn't necessarily extends to other objects. JNI critical section may be implemented via "object pinning" support in the VM. Pinning may happen at different granularities in different JVMs: * on object level: only the specified object is pinned; * on page / region level: all objects in the page or region in which the specified object resides is pinned as a whole; * on heap level: all objects in the whole GC heap is pinned. Disabling moving GCs from running is effectively pinning on the heap level. That's what HotSpot currently implements. But there's no guarantee that it'll stay that way in the future. JRockit, for example, supports object-level pinning. So your GetStringCritical trick is likely to break if you ever try that on JRockit. > I really wish JNI had another wrapper/macro we could use such as > JNI_QUICK_ENTRY that would prevent GC without having to use GCLocker. Some > way my JNI call could be considered ?inVM? instead of ?inNative? to keep GC > at bay. > > I'll have to leave this one to the HotSpot GC folks... - Kris > On Thu, Sep 8, 2016 at 1:30 PM, Krystal Mok wrote: > >> Hi Jon, >> >> On Thu, Sep 8, 2016 at 9:30 AM, Jon V. wrote: >> >>> Hello everyone! >>> >>> I?m trying to document the exact behavior of direct pointer access of >>> passed arguments into JNI in HotSpot. I have a huge post on >>> StackOverflow >>> for posterity purposes as it seems to be a commonly misunderstood >>> behavior. >>> >>> http://stackoverflow.com/questions/39381339/understanding-sa >>> fe-access-of-jni-arguments >>> >>> from >>> https://docs.oracle.com/javase/7/docs/technotes/guides/jni/s >>> pec/design.html#wp16789 >>> >>> The JNI spec says, ?To implement local references, the Java VM creates a >>> registry for each transition of control from Java to a native method. A >>> registry maps nonmovable local references to Java objects, and keeps the >>> objects from being garbage collected.? >>> >>> "Nonmovable" refers to the "local JNI reference", not the Java object >> instance itself. What that gurantees is that, given a jobject (which is >> typedef'd as an opaque pointer), you can safely say that as long as this >> jobject is still in scope, the value of the jobject won't change. But that >> doesn't imply anything related whether or not the Java object it's >> referring to is pinned. >> >> The "registry map" here is just an abstract notion that doesn't >> necessarily materialize into any real data structures in some certain JVM >> implementations. >> >> >>> I?m trying to understand if ?nonmovable? actually means that the objects >>> in >>> the map WILL NOT be compacted/moved (heap memory) or if it just means >>> they >>> won?t be garbage-collected. The answer dictates if the use of critical >>> sections are actually necessary. >>> >>> Rule of thumb: don't try to bypass the JNI abstractions. It'll bite back >> hard if you switch between different JVM implementations. >> >> Specifically in HotSpot, jobject and friends are referred to as "JNI >> handles". That's because they're implemented as handles, i.e. >> double-indirection pointers. The underlying type that implements jobject is >> "oop*", where an "oop" is typedef'd from "oopDesc*", and oopDesc is the >> root type of garbage collected objects. >> >> jobject JNIHandle Java object >> [ oop* ] -> [ oop ] -> [ oopDesc ] >> >> Following the spec, what it guarantees is that for a given jobject, the >> JNIHandle representing the referent object will not be moved, but the >> actual referent object is free to move. >> >> Don't try to access raw oops from outside the VM by accessing the >> underlying handle internals. It's very likely to get dangling pointers >> after GCs. >> >> What are the effects for each garbage collector? >>> >>> All garbage collectors in HotSpot, except for CMS (mostly concurrent >> mark-sweep), are moving collectors. As such, after each completed >> collection, Java objects will move. If you're holding raw oops that the GC >> isn't aware of, they won't be updated during GC, and will be left dangling >> afterwards. >> >> JNI critical sections are implemented in HotSpot via temporarily >> disabling the GCs, and as soon as all JNI critical sections are completed, >> a full GC (by default) will be triggered to perform the delayed collection >> if needed. This mechanism is called the "GCLocker" in HotSpot. >> (GCs may be able to expand the heap to fulfill allocation requests, if >> the current capacity of the GC heap is smaller than the maximum capacity >> configured. So disabling the GC isn't always as bad as it sounds.) >> >> >>> I found an email on the archives that says that JNI critical sections do >>> not effect operation of CMS and I?m interested in understanding the >>> behavior for G1 as well. >>> >>> CMS is a mark-sweep collector, so it doesn't move objects. Thus, it is >> possible to perform CMS old gen collections even when the GCLocker is >> active (meaning there's at least one Java thread in a JNI critical section). >> >> G1, on the other hand, is a moving collector (an incremental copying >> collector with optional concurrent global marking; sometimes simply >> referred to as a concurrent mark-compact collector, but it's not the usual >> mark-compact notion). G1 GC cannot be performed when the GCLocker is active. >> >> >>> This all boils down to the ability to use unsafe raw direct pointers in >>> JNI >>> without critical sections in HotSpot. >>> >>> Nope. Don't try that on HotSpot. >> There are certain code patterns that might be able to access raw oops >> safely without going into JNI critical sections. But that involves very >> heavy VM internals knowledge, which is also subject to change without >> notice, so practically don't try that. >> >> Hope that helps, >> Kris (OpenJDK username: kmo) >> >> >>> Thank you, >>> J >>> >> >> > From jon.masamitsu at oracle.com Thu Sep 8 19:38:22 2016 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Thu, 8 Sep 2016 12:38:22 -0700 Subject: Request for review (s) - 8155948: Add message for CMS deprecation for some internal builds In-Reply-To: <5CB1B554-D733-40D6-881F-6237F9CE9E11@oracle.com> References: <74eccd9a-1525-a9ef-1608-294de99d4483@oracle.com> <52db37a5-efa4-79dd-771f-45e1fc1ccaf5@oracle.com> <02968df4-b53d-c1d1-65e4-a612ca393bae@oracle.com> <5CB1B554-D733-40D6-881F-6237F9CE9E11@oracle.com> Message-ID: <50bac55e-b92c-f5d5-cc29-f0a85fdc6923@oracle.com> Latest delta http://cr.openjdk.java.net/~jmasa/8155948/webrev_delta.01_02/ and full http://cr.openjdk.java.net/~jmasa/8155948/webrev.02/ Thanks. Jon On 09/08/2016 10:44 AM, Kim Barrett wrote: >> On Sep 8, 2016, at 12:35 PM, Jon Masamitsu wrote: >> >> Kim, >> >> Thanks for looking at this. Comments below. >> >> On 09/07/2016 03:12 PM, Kim Barrett wrote: >>> src/share/vm/runtime/arguments.cpp >>> 4064 void Arguments::handle_concgc_flags() { >>> 4065 SpecialFlag flag; >>> 4066 const char *flag_name = "UseConcMarkSweepGC"; >>> 4067 if (log_is_enabled(Warning, gc) && >>> 4068 lookup_special_flag(flag_name, flag)) { >>> 4069 handle_aliases_and_deprecation(flag_name, /* print warning */ true); >>> 4070 log_warning(gc)("-Xconcgc/-Xnoconcgc uses UseConcMarkSweepGC"); >>> 4071 } >>> 4072 } >>> >>> This non-trivial little dance is done both here and for >>> UseAutoGCSelectPolicy. I think handle_concgc_flags could deal with >>> both if it took an argument of the "offending" flag. >>> >>> ------------------------------------------------------------------------------ >>> >> -Xconcgc does not have a SpecialFlag associated with it currently. Do you mean >> add one for "concgc" (even through it is not a -XX flags)? Or do you mean that >> "offending" flag is just a parameter that chooses between the two cases >> -Xconcgc/-Xnoconcgc and -XX:+UseAutoGCSelectPolicy? >> >> Jon > A parameter that is used to tailor the warning, e.g. a const char* that is %s?ed into the log_warning message. > From coleen.phillimore at oracle.com Thu Sep 8 20:18:59 2016 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Thu, 8 Sep 2016 16:18:59 -0400 Subject: Request for review (s) - 8155948: Add message for CMS deprecation for some internal builds In-Reply-To: <50bac55e-b92c-f5d5-cc29-f0a85fdc6923@oracle.com> References: <74eccd9a-1525-a9ef-1608-294de99d4483@oracle.com> <52db37a5-efa4-79dd-771f-45e1fc1ccaf5@oracle.com> <02968df4-b53d-c1d1-65e4-a612ca393bae@oracle.com> <5CB1B554-D733-40D6-881F-6237F9CE9E11@oracle.com> <50bac55e-b92c-f5d5-cc29-f0a85fdc6923@oracle.com> Message-ID: <624934c3-6277-4022-4094-f56205cf6d94@oracle.com> This looks great! Coleen On 9/8/16 3:38 PM, Jon Masamitsu wrote: > Latest delta > > http://cr.openjdk.java.net/~jmasa/8155948/webrev_delta.01_02/ > > and full > > http://cr.openjdk.java.net/~jmasa/8155948/webrev.02/ > > Thanks. > > Jon > > On 09/08/2016 10:44 AM, Kim Barrett wrote: >>> On Sep 8, 2016, at 12:35 PM, Jon Masamitsu >>> wrote: >>> >>> Kim, >>> >>> Thanks for looking at this. Comments below. >>> >>> On 09/07/2016 03:12 PM, Kim Barrett wrote: >>>> src/share/vm/runtime/arguments.cpp >>>> 4064 void Arguments::handle_concgc_flags() { >>>> 4065 SpecialFlag flag; >>>> 4066 const char *flag_name = "UseConcMarkSweepGC"; >>>> 4067 if (log_is_enabled(Warning, gc) && >>>> 4068 lookup_special_flag(flag_name, flag)) { >>>> 4069 handle_aliases_and_deprecation(flag_name, /* print warning >>>> */ true); >>>> 4070 log_warning(gc)("-Xconcgc/-Xnoconcgc uses >>>> UseConcMarkSweepGC"); >>>> 4071 } >>>> 4072 } >>>> >>>> This non-trivial little dance is done both here and for >>>> UseAutoGCSelectPolicy. I think handle_concgc_flags could deal with >>>> both if it took an argument of the "offending" flag. >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> >>> -Xconcgc does not have a SpecialFlag associated with it currently. >>> Do you mean >>> add one for "concgc" (even through it is not a -XX flags)? Or do >>> you mean that >>> "offending" flag is just a parameter that chooses between the two cases >>> -Xconcgc/-Xnoconcgc and -XX:+UseAutoGCSelectPolicy? >>> >>> Jon >> A parameter that is used to tailor the warning, e.g. a const char* >> that is %s?ed into the log_warning message. >> > From jon.masamitsu at oracle.com Thu Sep 8 20:14:12 2016 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Thu, 8 Sep 2016 13:14:12 -0700 Subject: Request for review (s) - 8155948: Add message for CMS deprecation for some internal builds In-Reply-To: <624934c3-6277-4022-4094-f56205cf6d94@oracle.com> References: <74eccd9a-1525-a9ef-1608-294de99d4483@oracle.com> <52db37a5-efa4-79dd-771f-45e1fc1ccaf5@oracle.com> <02968df4-b53d-c1d1-65e4-a612ca393bae@oracle.com> <5CB1B554-D733-40D6-881F-6237F9CE9E11@oracle.com> <50bac55e-b92c-f5d5-cc29-f0a85fdc6923@oracle.com> <624934c3-6277-4022-4094-f56205cf6d94@oracle.com> Message-ID: <9bf88347-a904-0653-4375-5b801eb5562e@oracle.com> Thanks, Coleen. Jon On 09/08/2016 01:18 PM, Coleen Phillimore wrote: > > This looks great! > Coleen > > On 9/8/16 3:38 PM, Jon Masamitsu wrote: >> Latest delta >> >> http://cr.openjdk.java.net/~jmasa/8155948/webrev_delta.01_02/ >> >> and full >> >> http://cr.openjdk.java.net/~jmasa/8155948/webrev.02/ >> >> Thanks. >> >> Jon >> >> On 09/08/2016 10:44 AM, Kim Barrett wrote: >>>> On Sep 8, 2016, at 12:35 PM, Jon Masamitsu >>>> wrote: >>>> >>>> Kim, >>>> >>>> Thanks for looking at this. Comments below. >>>> >>>> On 09/07/2016 03:12 PM, Kim Barrett wrote: >>>>> src/share/vm/runtime/arguments.cpp >>>>> 4064 void Arguments::handle_concgc_flags() { >>>>> 4065 SpecialFlag flag; >>>>> 4066 const char *flag_name = "UseConcMarkSweepGC"; >>>>> 4067 if (log_is_enabled(Warning, gc) && >>>>> 4068 lookup_special_flag(flag_name, flag)) { >>>>> 4069 handle_aliases_and_deprecation(flag_name, /* print >>>>> warning */ true); >>>>> 4070 log_warning(gc)("-Xconcgc/-Xnoconcgc uses >>>>> UseConcMarkSweepGC"); >>>>> 4071 } >>>>> 4072 } >>>>> >>>>> This non-trivial little dance is done both here and for >>>>> UseAutoGCSelectPolicy. I think handle_concgc_flags could deal with >>>>> both if it took an argument of the "offending" flag. >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> >>>> -Xconcgc does not have a SpecialFlag associated with it currently. >>>> Do you mean >>>> add one for "concgc" (even through it is not a -XX flags)? Or do >>>> you mean that >>>> "offending" flag is just a parameter that chooses between the two >>>> cases >>>> -Xconcgc/-Xnoconcgc and -XX:+UseAutoGCSelectPolicy? >>>> >>>> Jon >>> A parameter that is used to tailor the warning, e.g. a const char* >>> that is %s?ed into the log_warning message. >>> >> > From sybersnake at gmail.com Thu Sep 8 21:21:49 2016 From: sybersnake at gmail.com (Jon V.) Date: Thu, 8 Sep 2016 17:21:49 -0400 Subject: Clarification on what the JNI spec means by nonmovable In-Reply-To: References: Message-ID: Krystal, re http://docs.oracle.com/javase/7/docs/technotes/guides/jni/spec/functions.html#GetPrimitiveArrayCritical Thanks for clearing up that Critical sections can be nested. ?We must treat the code inside this pair of functions as running in a "critical region." Inside a critical region, native code must not call other JNI functions,? wasn?t terribly clear and the code example doesn?t appear everywhere. The code example here: jint len = (*env)->GetArrayLength(env, arr1); jbyte *a1 = (*env)->GetPrimitiveArrayCritical(env, arr1, 0); jbyte *a2 = (*env)->GetPrimitiveArrayCritical(env, arr2, 0); /* We need to check in case the VM tried to make a copy. */ if (a1 == NULL || a2 == NULL) { ... /* out of memory exception thrown */ } memcpy(a1, a2, len); (*env)->ReleasePrimitiveArrayCritical(env, arr2, a2, 0); (*env)->ReleasePrimitiveArrayCritical(env, arr1, a1, 0); Looks unsafe because it would be possible to acquire a1 then not acquire a2 whereas throwing an exception and exiting the method would keep the lock on a1. On Thu, Sep 8, 2016 at 2:53 PM, Krystal Mok wrote: > > > On Thu, Sep 8, 2016 at 10:42 AM, Jon V. wrote: > >> Thank you Kris. >> >> That confirms my understanding that the indirect pointers can be moved at >> anytime. I?m accessing possible multiple heap objects inside of a single >> JNI call and you aren?t allowed to call critical sections twice. >> > > Wait... a JNI critical section isn't like a OS thread-synchronization > critical section. It does support nesting. What is the access pattern > you're looking at? > > c.f. http://docs.oracle.com/javase/7/docs/technotes/ > guides/jni/spec/functions.html#GetPrimitiveArrayCritical > > Multiple pairs of GetPrimtiveArrayCritical and > ReleasePrimitiveArrayCritical may be nested. > > >> I?ve created a workaround of using getCrtiicalString as a way to stop GC >> then directly access the heap memory of the other objects. This seems like >> a safe way to do this. >> > > Nope, the JNI API does not guarantee this to be safe. Doing this is > essentially cheating on JVMs such as HotSpot that implements JNI critical > sections via temporarily disabling the GC. > > JNI critical section only guarantees "critical access" to the specified > object, and that doesn't necessarily extends to other objects. > > JNI critical section may be implemented via "object pinning" support in > the VM. Pinning may happen at different granularities in different JVMs: > * on object level: only the specified object is pinned; > * on page / region level: all objects in the page or region in which the > specified object resides is pinned as a whole; > * on heap level: all objects in the whole GC heap is pinned. > > Disabling moving GCs from running is effectively pinning on the heap > level. That's what HotSpot currently implements. But there's no guarantee > that it'll stay that way in the future. > JRockit, for example, supports object-level pinning. So > your GetStringCritical trick is likely to break if you ever try that on > JRockit. > > > >> I really wish JNI had another wrapper/macro we could use such as >> JNI_QUICK_ENTRY that would prevent GC without having to use GCLocker. Some >> way my JNI call could be considered ?inVM? instead of ?inNative? to keep GC >> at bay. >> >> I'll have to leave this one to the HotSpot GC folks... > > - Kris > > >> On Thu, Sep 8, 2016 at 1:30 PM, Krystal Mok >> wrote: >> >>> Hi Jon, >>> >>> On Thu, Sep 8, 2016 at 9:30 AM, Jon V. wrote: >>> >>>> Hello everyone! >>>> >>>> I?m trying to document the exact behavior of direct pointer access of >>>> passed arguments into JNI in HotSpot. I have a huge post on >>>> StackOverflow >>>> for posterity purposes as it seems to be a commonly misunderstood >>>> behavior. >>>> >>>> http://stackoverflow.com/questions/39381339/understanding-sa >>>> fe-access-of-jni-arguments >>>> >>>> from >>>> https://docs.oracle.com/javase/7/docs/technotes/guides/jni/s >>>> pec/design.html#wp16789 >>>> >>>> The JNI spec says, ?To implement local references, the Java VM creates a >>>> registry for each transition of control from Java to a native method. A >>>> registry maps nonmovable local references to Java objects, and keeps the >>>> objects from being garbage collected.? >>>> >>>> "Nonmovable" refers to the "local JNI reference", not the Java object >>> instance itself. What that gurantees is that, given a jobject (which is >>> typedef'd as an opaque pointer), you can safely say that as long as this >>> jobject is still in scope, the value of the jobject won't change. But that >>> doesn't imply anything related whether or not the Java object it's >>> referring to is pinned. >>> >>> The "registry map" here is just an abstract notion that doesn't >>> necessarily materialize into any real data structures in some certain JVM >>> implementations. >>> >>> >>>> I?m trying to understand if ?nonmovable? actually means that the >>>> objects in >>>> the map WILL NOT be compacted/moved (heap memory) or if it just means >>>> they >>>> won?t be garbage-collected. The answer dictates if the use of critical >>>> sections are actually necessary. >>>> >>>> Rule of thumb: don't try to bypass the JNI abstractions. It'll bite >>> back hard if you switch between different JVM implementations. >>> >>> Specifically in HotSpot, jobject and friends are referred to as "JNI >>> handles". That's because they're implemented as handles, i.e. >>> double-indirection pointers. The underlying type that implements jobject is >>> "oop*", where an "oop" is typedef'd from "oopDesc*", and oopDesc is the >>> root type of garbage collected objects. >>> >>> jobject JNIHandle Java object >>> [ oop* ] -> [ oop ] -> [ oopDesc ] >>> >>> Following the spec, what it guarantees is that for a given jobject, the >>> JNIHandle representing the referent object will not be moved, but the >>> actual referent object is free to move. >>> >>> Don't try to access raw oops from outside the VM by accessing the >>> underlying handle internals. It's very likely to get dangling pointers >>> after GCs. >>> >>> What are the effects for each garbage collector? >>>> >>>> All garbage collectors in HotSpot, except for CMS (mostly concurrent >>> mark-sweep), are moving collectors. As such, after each completed >>> collection, Java objects will move. If you're holding raw oops that the GC >>> isn't aware of, they won't be updated during GC, and will be left dangling >>> afterwards. >>> >>> JNI critical sections are implemented in HotSpot via temporarily >>> disabling the GCs, and as soon as all JNI critical sections are completed, >>> a full GC (by default) will be triggered to perform the delayed collection >>> if needed. This mechanism is called the "GCLocker" in HotSpot. >>> (GCs may be able to expand the heap to fulfill allocation requests, if >>> the current capacity of the GC heap is smaller than the maximum capacity >>> configured. So disabling the GC isn't always as bad as it sounds.) >>> >>> >>>> I found an email on the archives that says that JNI critical sections do >>>> not effect operation of CMS and I?m interested in understanding the >>>> behavior for G1 as well. >>>> >>>> CMS is a mark-sweep collector, so it doesn't move objects. Thus, it is >>> possible to perform CMS old gen collections even when the GCLocker is >>> active (meaning there's at least one Java thread in a JNI critical section). >>> >>> G1, on the other hand, is a moving collector (an incremental copying >>> collector with optional concurrent global marking; sometimes simply >>> referred to as a concurrent mark-compact collector, but it's not the usual >>> mark-compact notion). G1 GC cannot be performed when the GCLocker is active. >>> >>> >>>> This all boils down to the ability to use unsafe raw direct pointers in >>>> JNI >>>> without critical sections in HotSpot. >>>> >>>> Nope. Don't try that on HotSpot. >>> There are certain code patterns that might be able to access raw oops >>> safely without going into JNI critical sections. But that involves very >>> heavy VM internals knowledge, which is also subject to change without >>> notice, so practically don't try that. >>> >>> Hope that helps, >>> Kris (OpenJDK username: kmo) >>> >>> >>>> Thank you, >>>> J >>>> >>> >>> >> > From kim.barrett at oracle.com Thu Sep 8 21:58:35 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 8 Sep 2016 17:58:35 -0400 Subject: Request for review (s) - 8155948: Add message for CMS deprecation for some internal builds In-Reply-To: <50bac55e-b92c-f5d5-cc29-f0a85fdc6923@oracle.com> References: <74eccd9a-1525-a9ef-1608-294de99d4483@oracle.com> <52db37a5-efa4-79dd-771f-45e1fc1ccaf5@oracle.com> <02968df4-b53d-c1d1-65e4-a612ca393bae@oracle.com> <5CB1B554-D733-40D6-881F-6237F9CE9E11@oracle.com> <50bac55e-b92c-f5d5-cc29-f0a85fdc6923@oracle.com> Message-ID: <1CF143FD-1C06-4E72-8C3B-A3BAC69955BB@oracle.com> > On Sep 8, 2016, at 3:38 PM, Jon Masamitsu wrote: > > Latest delta > > http://cr.openjdk.java.net/~jmasa/8155948/webrev_delta.01_02/ > > and full > > http://cr.openjdk.java.net/~jmasa/8155948/webrev.02/ Looks good. From david.holmes at oracle.com Thu Sep 8 22:11:28 2016 From: david.holmes at oracle.com (David Holmes) Date: Fri, 9 Sep 2016 08:11:28 +1000 Subject: Clarification on what the JNI spec means by nonmovable In-Reply-To: References: Message-ID: <1c9e8765-5515-9cad-4bc7-bdbb84e11862@oracle.com> On 9/09/2016 7:21 AM, Jon V. wrote: > Krystal, > > re > http://docs.oracle.com/javase/7/docs/technotes/guides/jni/spec/functions.html#GetPrimitiveArrayCritical > > Thanks for clearing up that Critical sections can be nested. ?We must > treat the code inside this pair of functions as running in a "critical > region." Inside a critical region, native code must not call other JNI > functions,? wasn?t terribly clear and the code example doesn?t appear > everywhere. > > The code example here: > > jint len = (*env)->GetArrayLength(env, arr1); > jbyte *a1 = (*env)->GetPrimitiveArrayCritical(env, arr1, 0); > jbyte *a2 = (*env)->GetPrimitiveArrayCritical(env, arr2, 0); > /* We need to check in case the VM tried to make a copy. */ > if (a1 == NULL || a2 == NULL) { > ... /* out of memory exception thrown */ > } > memcpy(a1, a2, len); > (*env)->ReleasePrimitiveArrayCritical(env, arr2, a2, 0); > (*env)->ReleasePrimitiveArrayCritical(env, arr1, a1, 0); > > > Looks unsafe because it would be possible to acquire a1 then not acquire a2 > whereas throwing an exception and exiting the method would keep the lock on > a1. "throwing an exception" here means to establish a pending exception that will be thrown when you return to Java code. Unless you code it into the if-block there is no transfer of control that will skip the Release* calls. The example is still not great though - not sure what happens if you try to release something you didn't acquire. David > On Thu, Sep 8, 2016 at 2:53 PM, Krystal Mok wrote: > >> >> >> On Thu, Sep 8, 2016 at 10:42 AM, Jon V. wrote: >> >>> Thank you Kris. >>> >>> That confirms my understanding that the indirect pointers can be moved at >>> anytime. I?m accessing possible multiple heap objects inside of a single >>> JNI call and you aren?t allowed to call critical sections twice. >>> >> >> Wait... a JNI critical section isn't like a OS thread-synchronization >> critical section. It does support nesting. What is the access pattern >> you're looking at? >> >> c.f. http://docs.oracle.com/javase/7/docs/technotes/ >> guides/jni/spec/functions.html#GetPrimitiveArrayCritical >>> Multiple pairs of GetPrimtiveArrayCritical and >> ReleasePrimitiveArrayCritical may be nested. >> >> >>> I?ve created a workaround of using getCrtiicalString as a way to stop GC >>> then directly access the heap memory of the other objects. This seems like >>> a safe way to do this. >>> >> >> Nope, the JNI API does not guarantee this to be safe. Doing this is >> essentially cheating on JVMs such as HotSpot that implements JNI critical >> sections via temporarily disabling the GC. >> >> JNI critical section only guarantees "critical access" to the specified >> object, and that doesn't necessarily extends to other objects. >> >> JNI critical section may be implemented via "object pinning" support in >> the VM. Pinning may happen at different granularities in different JVMs: >> * on object level: only the specified object is pinned; >> * on page / region level: all objects in the page or region in which the >> specified object resides is pinned as a whole; >> * on heap level: all objects in the whole GC heap is pinned. >> >> Disabling moving GCs from running is effectively pinning on the heap >> level. That's what HotSpot currently implements. But there's no guarantee >> that it'll stay that way in the future. >> JRockit, for example, supports object-level pinning. So >> your GetStringCritical trick is likely to break if you ever try that on >> JRockit. >> >> >> >>> I really wish JNI had another wrapper/macro we could use such as >>> JNI_QUICK_ENTRY that would prevent GC without having to use GCLocker. Some >>> way my JNI call could be considered ?inVM? instead of ?inNative? to keep GC >>> at bay. >>> >>> I'll have to leave this one to the HotSpot GC folks... >> >> - Kris >> >> >>> On Thu, Sep 8, 2016 at 1:30 PM, Krystal Mok >>> wrote: >>> >>>> Hi Jon, >>>> >>>> On Thu, Sep 8, 2016 at 9:30 AM, Jon V. wrote: >>>> >>>>> Hello everyone! >>>>> >>>>> I?m trying to document the exact behavior of direct pointer access of >>>>> passed arguments into JNI in HotSpot. I have a huge post on >>>>> StackOverflow >>>>> for posterity purposes as it seems to be a commonly misunderstood >>>>> behavior. >>>>> >>>>> http://stackoverflow.com/questions/39381339/understanding-sa >>>>> fe-access-of-jni-arguments >>>>> >>>>> from >>>>> https://docs.oracle.com/javase/7/docs/technotes/guides/jni/s >>>>> pec/design.html#wp16789 >>>>> >>>>> The JNI spec says, ?To implement local references, the Java VM creates a >>>>> registry for each transition of control from Java to a native method. A >>>>> registry maps nonmovable local references to Java objects, and keeps the >>>>> objects from being garbage collected.? >>>>> >>>>> "Nonmovable" refers to the "local JNI reference", not the Java object >>>> instance itself. What that gurantees is that, given a jobject (which is >>>> typedef'd as an opaque pointer), you can safely say that as long as this >>>> jobject is still in scope, the value of the jobject won't change. But that >>>> doesn't imply anything related whether or not the Java object it's >>>> referring to is pinned. >>>> >>>> The "registry map" here is just an abstract notion that doesn't >>>> necessarily materialize into any real data structures in some certain JVM >>>> implementations. >>>> >>>> >>>>> I?m trying to understand if ?nonmovable? actually means that the >>>>> objects in >>>>> the map WILL NOT be compacted/moved (heap memory) or if it just means >>>>> they >>>>> won?t be garbage-collected. The answer dictates if the use of critical >>>>> sections are actually necessary. >>>>> >>>>> Rule of thumb: don't try to bypass the JNI abstractions. It'll bite >>>> back hard if you switch between different JVM implementations. >>>> >>>> Specifically in HotSpot, jobject and friends are referred to as "JNI >>>> handles". That's because they're implemented as handles, i.e. >>>> double-indirection pointers. The underlying type that implements jobject is >>>> "oop*", where an "oop" is typedef'd from "oopDesc*", and oopDesc is the >>>> root type of garbage collected objects. >>>> >>>> jobject JNIHandle Java object >>>> [ oop* ] -> [ oop ] -> [ oopDesc ] >>>> >>>> Following the spec, what it guarantees is that for a given jobject, the >>>> JNIHandle representing the referent object will not be moved, but the >>>> actual referent object is free to move. >>>> >>>> Don't try to access raw oops from outside the VM by accessing the >>>> underlying handle internals. It's very likely to get dangling pointers >>>> after GCs. >>>> >>>> What are the effects for each garbage collector? >>>>> >>>>> All garbage collectors in HotSpot, except for CMS (mostly concurrent >>>> mark-sweep), are moving collectors. As such, after each completed >>>> collection, Java objects will move. If you're holding raw oops that the GC >>>> isn't aware of, they won't be updated during GC, and will be left dangling >>>> afterwards. >>>> >>>> JNI critical sections are implemented in HotSpot via temporarily >>>> disabling the GCs, and as soon as all JNI critical sections are completed, >>>> a full GC (by default) will be triggered to perform the delayed collection >>>> if needed. This mechanism is called the "GCLocker" in HotSpot. >>>> (GCs may be able to expand the heap to fulfill allocation requests, if >>>> the current capacity of the GC heap is smaller than the maximum capacity >>>> configured. So disabling the GC isn't always as bad as it sounds.) >>>> >>>> >>>>> I found an email on the archives that says that JNI critical sections do >>>>> not effect operation of CMS and I?m interested in understanding the >>>>> behavior for G1 as well. >>>>> >>>>> CMS is a mark-sweep collector, so it doesn't move objects. Thus, it is >>>> possible to perform CMS old gen collections even when the GCLocker is >>>> active (meaning there's at least one Java thread in a JNI critical section). >>>> >>>> G1, on the other hand, is a moving collector (an incremental copying >>>> collector with optional concurrent global marking; sometimes simply >>>> referred to as a concurrent mark-compact collector, but it's not the usual >>>> mark-compact notion). G1 GC cannot be performed when the GCLocker is active. >>>> >>>> >>>>> This all boils down to the ability to use unsafe raw direct pointers in >>>>> JNI >>>>> without critical sections in HotSpot. >>>>> >>>>> Nope. Don't try that on HotSpot. >>>> There are certain code patterns that might be able to access raw oops >>>> safely without going into JNI critical sections. But that involves very >>>> heavy VM internals knowledge, which is also subject to change without >>>> notice, so practically don't try that. >>>> >>>> Hope that helps, >>>> Kris (OpenJDK username: kmo) >>>> >>>> >>>>> Thank you, >>>>> J >>>>> >>>> >>>> >>> >> From jon.masamitsu at oracle.com Thu Sep 8 22:22:21 2016 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Thu, 8 Sep 2016 15:22:21 -0700 Subject: Request for review (s) - 8155948: Add message for CMS deprecation for some internal builds In-Reply-To: <1CF143FD-1C06-4E72-8C3B-A3BAC69955BB@oracle.com> References: <74eccd9a-1525-a9ef-1608-294de99d4483@oracle.com> <52db37a5-efa4-79dd-771f-45e1fc1ccaf5@oracle.com> <02968df4-b53d-c1d1-65e4-a612ca393bae@oracle.com> <5CB1B554-D733-40D6-881F-6237F9CE9E11@oracle.com> <50bac55e-b92c-f5d5-cc29-f0a85fdc6923@oracle.com> <1CF143FD-1C06-4E72-8C3B-A3BAC69955BB@oracle.com> Message-ID: Kim, Thanks. Jon On 09/08/2016 02:58 PM, Kim Barrett wrote: >> On Sep 8, 2016, at 3:38 PM, Jon Masamitsu wrote: >> >> Latest delta >> >> http://cr.openjdk.java.net/~jmasa/8155948/webrev_delta.01_02/ >> >> and full >> >> http://cr.openjdk.java.net/~jmasa/8155948/webrev.02/ > Looks good. > From gromero at linux.vnet.ibm.com Thu Sep 8 22:58:54 2016 From: gromero at linux.vnet.ibm.com (Gustavo Romero) Date: Thu, 8 Sep 2016 19:58:54 -0300 Subject: RFR(XXS): 8164987: RTM jtreg tests failing due to unnamed module unable to access class jdk.internal.misc.Unsafe Message-ID: <57D1ED2E.5020408@linux.vnet.ibm.com> Hi, Could the following change be reviewed, please? bug: https://bugs.openjdk.java.net/browse/JDK-8164987 webrev: http://cr.openjdk.java.net/~gromero/8164987/01/ Thank you. Regards, Gustavo From shafi.s.ahmad at oracle.com Fri Sep 9 05:46:34 2016 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Thu, 8 Sep 2016 22:46:34 -0700 (PDT) Subject: [8u] RFR for JDK-8154324: Request to backport JDK-6515172 to 8u Message-ID: <60e87114-fcc2-4a0f-a31b-8440faf91555@default> Hi, Please review the backport [modified] of bug: "JDK-6515172 Runtime.availableProcessors() ignores Linux taskset command" to jdk8u. Please note that the backport is not clean and we can't do as it is. Please note 1. The changes are made by 'Andreas Eriksson' who left Oracle. 2. There is difference in logging mechanism in jdk9 and jdk8 is different and file logTag.hpp doesn't exists in jdk8. 3. Newly added test pass after this change. It fails without this change. Webrev link: http://cr.openjdk.java.net/~shshahma/8154324/webrev.00/ Jdk9 bug: https://bugs.openjdk.java.net/browse/JDK-6515172 Jdk8 bug: https://bugs.openjdk.java.net/browse/JDK-8154324 Original patch pushed to jdk9: http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/c5480d4abfe4 Testing: jprt and running jtreg. Regards, Shafi From david.holmes at oracle.com Fri Sep 9 05:58:54 2016 From: david.holmes at oracle.com (David Holmes) Date: Fri, 9 Sep 2016 15:58:54 +1000 Subject: [8u] RFR for JDK-8154324: Request to backport JDK-6515172 to 8u In-Reply-To: <60e87114-fcc2-4a0f-a31b-8440faf91555@default> References: <60e87114-fcc2-4a0f-a31b-8440faf91555@default> Message-ID: <3695d333-eb30-3a84-9b8e-b0264a9087c9@oracle.com> Hi Shafi, On 9/09/2016 3:46 PM, Shafi Ahmad wrote: > Hi, > > Please review the backport [modified] of bug: "JDK-6515172 Runtime.availableProcessors() ignores Linux taskset command" to jdk8u. > > Please note that the backport is not clean and we can't do as it is. Please note > 1. The changes are made by 'Andreas Eriksson' who left Oracle. > 2. There is difference in logging mechanism in jdk9 and jdk8 is different and file logTag.hpp doesn't exists in jdk8. > 3. Newly added test pass after this change. It fails without this change. > > Webrev link: http://cr.openjdk.java.net/~shshahma/8154324/webrev.00/ > Jdk9 bug: https://bugs.openjdk.java.net/browse/JDK-6515172 > Jdk8 bug: https://bugs.openjdk.java.net/browse/JDK-8154324 > Original patch pushed to jdk9: http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/c5480d4abfe4 I worked extensively with Andreas on this as there were a number of issues. I'll have to try and find those discussions to see where we ended up. The backport as stands is not quite appropriate. For example it adds: diagnostic(bool, UseCpuAllocPath, false, but that does not exist in the actual code for 8. Also this comment: + // If it appears there may be more than 1024 processors then we do a + // dynamic check - see 6515172 for details. is wrong as there is no dynamic check in this version of the code. The last I recall with this is that there were issues caused by building with one version of glibc and running (or trying to) on later versions of glibc. But as I said I will have to see if I have the discussion from that effort. David ---- > Testing: jprt and running jtreg. > > Regards, > Shafi > From tobias.hartmann at oracle.com Fri Sep 9 12:42:23 2016 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Fri, 9 Sep 2016 14:42:23 +0200 Subject: [9] RFR(S): Crash with assert: symbol conversion failure in java_lang_String::create_from_symbol() Message-ID: <57D2AE2F.1080607@oracle.com> Hi, please review the following patch: https://bugs.openjdk.java.net/browse/JDK-8164561 http://cr.openjdk.java.net/~thartmann/8164561/webrev.00/ The verification code in java_lang_String::create_from_symbol() that was added by Compact Strings fails because the input symbol does not contain valid UTF8. The problem is that a JCK JNI test passes an invalid UTF8 string as class name to the JNI method "FindClass". In fact, the string contains garbage from reading past array boundaries because of a bug in the test [1]. The JNI spec [2] states that 'name' should be "a fully-qualified class name (that is, a package name, delimited by ?/?, followed by the class name). If the name begins with ?[? (the array signature character), it returns an array class. The string is encoded in modified UTF-8". I nevertheless think that we should not crash in the case of an invalid UTF8 string and therefore disabled the verification code with a comment. We did the same for java_lang_String::create_from_str() [3]. Tested with failing JCK test and JPRT (running). Thanks, Tobias [1] https://bugs.openjdk.java.net/browse/JCK-7307244 [2] https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/functions.html#FindClass [3] http://hg.openjdk.java.net/jdk9/hs/hotspot/file/d060826d0911/src/share/vm/classfile/javaClasses.cpp#l274 From jon.masamitsu at oracle.com Fri Sep 9 14:47:01 2016 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Fri, 9 Sep 2016 07:47:01 -0700 Subject: Request for review (s) - 8155948: Add message for CMS deprecation for some internal builds In-Reply-To: <50bac55e-b92c-f5d5-cc29-f0a85fdc6923@oracle.com> References: <74eccd9a-1525-a9ef-1608-294de99d4483@oracle.com> <52db37a5-efa4-79dd-771f-45e1fc1ccaf5@oracle.com> <02968df4-b53d-c1d1-65e4-a612ca393bae@oracle.com> <5CB1B554-D733-40D6-881F-6237F9CE9E11@oracle.com> <50bac55e-b92c-f5d5-cc29-f0a85fdc6923@oracle.com> Message-ID: <7618399f-8de9-2faf-00ad-8eb17aae1016@oracle.com> Sorry for dragging this out but I needed to make 1 change. I should have been using warning() and not log_warning(gc). Diff is diff --git a/src/share/vm/runtime/arguments.cpp b/src/share/vm/runtime/arguments.cpp --- a/src/share/vm/runtime/arguments.cpp +++ b/src/share/vm/runtime/arguments.cpp @@ -4065,7 +4065,7 @@ if (log_is_enabled(Warning, gc) && lookup_special_flag(flag_name, flag)) { handle_aliases_and_deprecation(flag_name, /* print warning */ true); - log_warning(gc)("%s", msg); + warning("%s", msg); } } The difference is that warning() honors PrintWarnings. Thanks. Jon On 9/8/2016 12:38 PM, Jon Masamitsu wrote: > Latest delta > > http://cr.openjdk.java.net/~jmasa/8155948/webrev_delta.01_02/ > > and full > > http://cr.openjdk.java.net/~jmasa/8155948/webrev.02/ > > Thanks. > > Jon > > On 09/08/2016 10:44 AM, Kim Barrett wrote: >>> On Sep 8, 2016, at 12:35 PM, Jon Masamitsu >>> wrote: >>> >>> Kim, >>> >>> Thanks for looking at this. Comments below. >>> >>> On 09/07/2016 03:12 PM, Kim Barrett wrote: >>>> src/share/vm/runtime/arguments.cpp >>>> 4064 void Arguments::handle_concgc_flags() { >>>> 4065 SpecialFlag flag; >>>> 4066 const char *flag_name = "UseConcMarkSweepGC"; >>>> 4067 if (log_is_enabled(Warning, gc) && >>>> 4068 lookup_special_flag(flag_name, flag)) { >>>> 4069 handle_aliases_and_deprecation(flag_name, /* print warning >>>> */ true); >>>> 4070 log_warning(gc)("-Xconcgc/-Xnoconcgc uses >>>> UseConcMarkSweepGC"); >>>> 4071 } >>>> 4072 } >>>> >>>> This non-trivial little dance is done both here and for >>>> UseAutoGCSelectPolicy. I think handle_concgc_flags could deal with >>>> both if it took an argument of the "offending" flag. >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> >>> -Xconcgc does not have a SpecialFlag associated with it currently. >>> Do you mean >>> add one for "concgc" (even through it is not a -XX flags)? Or do >>> you mean that >>> "offending" flag is just a parameter that chooses between the two cases >>> -Xconcgc/-Xnoconcgc and -XX:+UseAutoGCSelectPolicy? >>> >>> Jon >> A parameter that is used to tailor the warning, e.g. a const char* >> that is %s?ed into the log_warning message. >> > From doug.simon at oracle.com Fri Sep 9 18:33:40 2016 From: doug.simon at oracle.com (Doug Simon) Date: Fri, 9 Sep 2016 20:33:40 +0200 Subject: RFR: 8165755: [JVMCI] replace use of vm_abort with vm_exit In-Reply-To: <6391B00B-AFBF-410C-A6A1-2ED95B35EBEB@twitter.com> References: <799CA13D-6BDC-4BF5-9241-515A684191F4@oracle.com> <6391B00B-AFBF-410C-A6A1-2ED95B35EBEB@twitter.com> Message-ID: <2F360221-7E6B-43BB-B69C-69A17777E5F2@oracle.com> Can someone from the runtime team confirm that using vm_exit (instead of vm_abort) is the best way to stop the VM when JVMCI initialization fails (e.g., when invalid JVMCI options are provided on the command line). Thanks! -Doug > On 09 Sep 2016, at 19:48, Christian Thalinger wrote: > > I think this looks fine but maybe we should ask the runtime folks. > >> On Sep 8, 2016, at 11:01 PM, Doug Simon wrote: >> >> Calling vm_abort from multiple threads can cause nasty crashes such as double free errors. We've seen this in Graal during JVMCI initialization when an unknown Graal option is encountered. Multiple compiler threads try to initialize JVMCI which fails with an exception indicating the bad option: >> >> Uncaught exception at /scratch/graaluser/buildslave/buildlog/ci_executor/main/graal-jvmci-8/src/share/vm/jvmci/jvmciCompiler.cpp:127 >> java.lang.ExceptionInInitializerError >> at jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.runtime(HotSpotJVMCIRuntime.java:85) >> at jdk.vm.ci.runtime.JVMCI.initializeRuntime(Native Method) >> at jdk.vm.ci.runtime.JVMCI.(JVMCI.java:58) >> Caused by: java.lang.IllegalArgumentException: Could not find option OptSomethingThatDoesNotExcist >> at com.oracle.graal.options.OptionsParser.parseOption(OptionsParser.java:134) >> at com.oracle.graal.options.OptionsParser.parseOptions(OptionsParser.java:62) >> at com.oracle.graal.hotspot.HotSpotGraalCompilerFactory.initializeOptions(HotSpotGraalCompilerFactory.java:156) >> at com.oracle.graal.hotspot.HotSpotGraalCompilerFactory.onSelection(HotSpotGraalCompilerFactory.java:86) >> at jdk.vm.ci.hotspot.HotSpotJVMCICompilerConfig.getCompilerFactory(HotSpotJVMCICompilerConfig.java:96) >> at jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.(HotSpotJVMCIRuntime.java:277) >> at jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.(HotSpotJVMCIRuntime.java:67) >> at jdk.vm.ci.hotspot.HotSpotJVMCIRuntime$DelayedInit.(HotSpotJVMCIRuntime.java:75) >> at jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.runtime(HotSpotJVMCIRuntime.java:85) >> at jdk.vm.ci.runtime.JVMCI.initializeRuntime(Native Method) >> at jdk.vm.ci.runtime.JVMCI.(JVMCI.java:58) >> >> The native JVMCI code then tries to exit the VM by calling vm_abort. If multiple compiler threads do this concurrently, certain destructors can be called twice as shown by these thread dumps: >> >> thread #26: tid = 0x0019, 0x00007fff84280124 libsystem_malloc.dylib`szone_size + 227, stop reason = signal SIGSTOP >> frame #0: 0x00007fff84280124 libsystem_malloc.dylib`szone_size + 227 >> frame #1: 0x00007fff8427fed5 libsystem_malloc.dylib`free + 61 >> frame #2: 0x000000010ac95963 libjvm.dylib`os::free(memblock=0x00007fedc86226e0, memflags=mtInternal) + 307 at os.cpp:711 >> frame #3: 0x000000010a2afc54 libjvm.dylib`FreeHeap(p=0x00007fedc86226e0, memflags=mtInternal) + 52 at allocation.inline.hpp:93 >> frame #4: 0x000000010acf0a9f libjvm.dylib`PerfData::~PerfData(this=0x00007fedc8622650) + 63 at perfData.cpp:116 >> frame #5: 0x000000010acf0ae5 libjvm.dylib`PerfData::~PerfData(this=0x00007fedc8622650) + 21 at perfData.cpp:114 >> frame #6: 0x000000010acf163d libjvm.dylib`PerfDataManager::destroy() + 109 at perfData.cpp:287 >> frame #7: 0x000000010acf3f4d libjvm.dylib`perfMemory_exit() + 61 at perfMemory.cpp:74 >> frame #8: 0x000000010ac9bb0d libjvm.dylib`os::shutdown() + 13 at os_bsd.cpp:1130 >> frame #9: 0x000000010ac9bb55 libjvm.dylib`os::abort(dump_core=false) + 21 at os_bsd.cpp:1150 >> frame #10: 0x000000010a9188e7 libjvm.dylib`vm_abort(dump_core=false) + 39 at java.cpp:666 >> frame #11: 0x000000010aa4f1e7 libjvm.dylib`JVMCIRuntime::abort_on_pending_exception(exception=Handle @ 0x000070000175b208, message="Uncaught exception at /Users/dsimon/graal/graal-jvmci-8/src/share/vm/jvmci/jvmciCompiler.cpp:127", dump_core=false) + 167 at jvmciRuntime.cpp:992 >> frame #12: 0x000000010aa17017 libjvm.dylib`JVMCICompiler::compile_method(this=0x00007fedcb203050, method=0x000070000175b8d8, entry_bci=-1, env=0x000070000175b8f0) + 311 at jvmciCompiler.cpp:127 >> frame #13: 0x000000010a656cd2 libjvm.dylib`CompileBroker::invoke_compiler_on_method(task=0x00007fedc853ca30) + 1314 at compileBroker.cpp:2207 >> >> thread #23: tid = 0x0016, 0x00007fff91fcb122 libsystem_kernel.dylib`__semwait_signal_nocancel + 10, stop reason = signal SIGSTOP >> frame #0: 0x00007fff91fcb122 libsystem_kernel.dylib`__semwait_signal_nocancel + 10 >> frame #1: 0x00007fff9578c318 libsystem_c.dylib`nanosleep$NOCANCEL + 188 >> frame #2: 0x00007fff957b62ce libsystem_c.dylib`usleep$NOCANCEL + 54 >> frame #3: 0x00007fff957e46e9 libsystem_c.dylib`abort + 139 >> frame #4: 0x00007fff8428c396 libsystem_malloc.dylib`szone_error + 626 >> frame #5: 0x000000010ac95963 libjvm.dylib`os::free(memblock=0x00007fedc8601cd0, memflags=mtInternal) + 307 at os.cpp:711 >> frame #6: 0x000000010a2afc54 libjvm.dylib`FreeHeap(p=0x00007fedc8601cd0, memflags=mtInternal) + 52 at allocation.inline.hpp:93 >> frame #7: 0x000000010acf0a9f libjvm.dylib`PerfData::~PerfData(this=0x00007fedc8601c60) + 63 at perfData.cpp:116 >> frame #8: 0x000000010acf0ae5 libjvm.dylib`PerfData::~PerfData(this=0x00007fedc8601c60) + 21 at perfData.cpp:114 >> frame #9: 0x000000010acf163d libjvm.dylib`PerfDataManager::destroy() + 109 at perfData.cpp:287 >> frame #10: 0x000000010acf3f4d libjvm.dylib`perfMemory_exit() + 61 at perfMemory.cpp:74 >> frame #11: 0x000000010ac9bb0d libjvm.dylib`os::shutdown() + 13 at os_bsd.cpp:1130 >> frame #12: 0x000000010ac9bb55 libjvm.dylib`os::abort(dump_core=false) + 21 at os_bsd.cpp:1150 >> frame #13: 0x000000010a9188e7 libjvm.dylib`vm_abort(dump_core=false) + 39 at java.cpp:666 >> frame #14: 0x000000010aa4f1e7 libjvm.dylib`JVMCIRuntime::abort_on_pending_exception(exception=Handle @ 0x0000700001452208, message="Uncaught exception at /Users/dsimon/graal/graal-jvmci-8/src/share/vm/jvmci/jvmciCompiler.cpp:127", dump_core=false) + 167 at jvmciRuntime.cpp:992 >> frame #15: 0x000000010aa17017 libjvm.dylib`JVMCICompiler::compile_method(this=0x00007fedcb203050, method=0x00007000014528d8, entry_bci=-1, env=0x00007000014528f0) + 311 at jvmciCompiler.cpp:127 >> frame #16: 0x000000010a656cd2 libjvm.dylib`CompileBroker::invoke_compiler_on_method(task=0x00007fedc862a320) + 1314 at compileBroker.cpp:2207 >> >> >> This webrev replaces calls to vm_abort() with before_exit() + vm_exit(). The latter is thread safe. >> >> https://bugs.openjdk.java.net/browse/JDK-8165755 >> http://cr.openjdk.java.net/~dnsimon/8165755/ >> >> -Doug > From coleen.phillimore at oracle.com Fri Sep 9 19:23:54 2016 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 9 Sep 2016 15:23:54 -0400 Subject: RFR: 8165755: [JVMCI] replace use of vm_abort with vm_exit In-Reply-To: <2F360221-7E6B-43BB-B69C-69A17777E5F2@oracle.com> References: <799CA13D-6BDC-4BF5-9241-515A684191F4@oracle.com> <6391B00B-AFBF-410C-A6A1-2ED95B35EBEB@twitter.com> <2F360221-7E6B-43BB-B69C-69A17777E5F2@oracle.com> Message-ID: I think you want vm_exit_during_initialization() for that. Definitely not vm_abort, cause then it looks like an internal error/crash. Coleen On 9/9/16 2:33 PM, Doug Simon wrote: > Can someone from the runtime team confirm that using vm_exit (instead of vm_abort) is the best way to stop the VM when JVMCI initialization fails (e.g., when invalid JVMCI options are provided on the command line). Thanks! > > -Doug > > >> On 09 Sep 2016, at 19:48, Christian Thalinger wrote: >> >> I think this looks fine but maybe we should ask the runtime folks. >> >>> On Sep 8, 2016, at 11:01 PM, Doug Simon wrote: >>> >>> Calling vm_abort from multiple threads can cause nasty crashes such as double free errors. We've seen this in Graal during JVMCI initialization when an unknown Graal option is encountered. Multiple compiler threads try to initialize JVMCI which fails with an exception indicating the bad option: >>> >>> Uncaught exception at /scratch/graaluser/buildslave/buildlog/ci_executor/main/graal-jvmci-8/src/share/vm/jvmci/jvmciCompiler.cpp:127 >>> java.lang.ExceptionInInitializerError >>> at jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.runtime(HotSpotJVMCIRuntime.java:85) >>> at jdk.vm.ci.runtime.JVMCI.initializeRuntime(Native Method) >>> at jdk.vm.ci.runtime.JVMCI.(JVMCI.java:58) >>> Caused by: java.lang.IllegalArgumentException: Could not find option OptSomethingThatDoesNotExcist >>> at com.oracle.graal.options.OptionsParser.parseOption(OptionsParser.java:134) >>> at com.oracle.graal.options.OptionsParser.parseOptions(OptionsParser.java:62) >>> at com.oracle.graal.hotspot.HotSpotGraalCompilerFactory.initializeOptions(HotSpotGraalCompilerFactory.java:156) >>> at com.oracle.graal.hotspot.HotSpotGraalCompilerFactory.onSelection(HotSpotGraalCompilerFactory.java:86) >>> at jdk.vm.ci.hotspot.HotSpotJVMCICompilerConfig.getCompilerFactory(HotSpotJVMCICompilerConfig.java:96) >>> at jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.(HotSpotJVMCIRuntime.java:277) >>> at jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.(HotSpotJVMCIRuntime.java:67) >>> at jdk.vm.ci.hotspot.HotSpotJVMCIRuntime$DelayedInit.(HotSpotJVMCIRuntime.java:75) >>> at jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.runtime(HotSpotJVMCIRuntime.java:85) >>> at jdk.vm.ci.runtime.JVMCI.initializeRuntime(Native Method) >>> at jdk.vm.ci.runtime.JVMCI.(JVMCI.java:58) >>> >>> The native JVMCI code then tries to exit the VM by calling vm_abort. If multiple compiler threads do this concurrently, certain destructors can be called twice as shown by these thread dumps: >>> >>> thread #26: tid = 0x0019, 0x00007fff84280124 libsystem_malloc.dylib`szone_size + 227, stop reason = signal SIGSTOP >>> frame #0: 0x00007fff84280124 libsystem_malloc.dylib`szone_size + 227 >>> frame #1: 0x00007fff8427fed5 libsystem_malloc.dylib`free + 61 >>> frame #2: 0x000000010ac95963 libjvm.dylib`os::free(memblock=0x00007fedc86226e0, memflags=mtInternal) + 307 at os.cpp:711 >>> frame #3: 0x000000010a2afc54 libjvm.dylib`FreeHeap(p=0x00007fedc86226e0, memflags=mtInternal) + 52 at allocation.inline.hpp:93 >>> frame #4: 0x000000010acf0a9f libjvm.dylib`PerfData::~PerfData(this=0x00007fedc8622650) + 63 at perfData.cpp:116 >>> frame #5: 0x000000010acf0ae5 libjvm.dylib`PerfData::~PerfData(this=0x00007fedc8622650) + 21 at perfData.cpp:114 >>> frame #6: 0x000000010acf163d libjvm.dylib`PerfDataManager::destroy() + 109 at perfData.cpp:287 >>> frame #7: 0x000000010acf3f4d libjvm.dylib`perfMemory_exit() + 61 at perfMemory.cpp:74 >>> frame #8: 0x000000010ac9bb0d libjvm.dylib`os::shutdown() + 13 at os_bsd.cpp:1130 >>> frame #9: 0x000000010ac9bb55 libjvm.dylib`os::abort(dump_core=false) + 21 at os_bsd.cpp:1150 >>> frame #10: 0x000000010a9188e7 libjvm.dylib`vm_abort(dump_core=false) + 39 at java.cpp:666 >>> frame #11: 0x000000010aa4f1e7 libjvm.dylib`JVMCIRuntime::abort_on_pending_exception(exception=Handle @ 0x000070000175b208, message="Uncaught exception at /Users/dsimon/graal/graal-jvmci-8/src/share/vm/jvmci/jvmciCompiler.cpp:127", dump_core=false) + 167 at jvmciRuntime.cpp:992 >>> frame #12: 0x000000010aa17017 libjvm.dylib`JVMCICompiler::compile_method(this=0x00007fedcb203050, method=0x000070000175b8d8, entry_bci=-1, env=0x000070000175b8f0) + 311 at jvmciCompiler.cpp:127 >>> frame #13: 0x000000010a656cd2 libjvm.dylib`CompileBroker::invoke_compiler_on_method(task=0x00007fedc853ca30) + 1314 at compileBroker.cpp:2207 >>> >>> thread #23: tid = 0x0016, 0x00007fff91fcb122 libsystem_kernel.dylib`__semwait_signal_nocancel + 10, stop reason = signal SIGSTOP >>> frame #0: 0x00007fff91fcb122 libsystem_kernel.dylib`__semwait_signal_nocancel + 10 >>> frame #1: 0x00007fff9578c318 libsystem_c.dylib`nanosleep$NOCANCEL + 188 >>> frame #2: 0x00007fff957b62ce libsystem_c.dylib`usleep$NOCANCEL + 54 >>> frame #3: 0x00007fff957e46e9 libsystem_c.dylib`abort + 139 >>> frame #4: 0x00007fff8428c396 libsystem_malloc.dylib`szone_error + 626 >>> frame #5: 0x000000010ac95963 libjvm.dylib`os::free(memblock=0x00007fedc8601cd0, memflags=mtInternal) + 307 at os.cpp:711 >>> frame #6: 0x000000010a2afc54 libjvm.dylib`FreeHeap(p=0x00007fedc8601cd0, memflags=mtInternal) + 52 at allocation.inline.hpp:93 >>> frame #7: 0x000000010acf0a9f libjvm.dylib`PerfData::~PerfData(this=0x00007fedc8601c60) + 63 at perfData.cpp:116 >>> frame #8: 0x000000010acf0ae5 libjvm.dylib`PerfData::~PerfData(this=0x00007fedc8601c60) + 21 at perfData.cpp:114 >>> frame #9: 0x000000010acf163d libjvm.dylib`PerfDataManager::destroy() + 109 at perfData.cpp:287 >>> frame #10: 0x000000010acf3f4d libjvm.dylib`perfMemory_exit() + 61 at perfMemory.cpp:74 >>> frame #11: 0x000000010ac9bb0d libjvm.dylib`os::shutdown() + 13 at os_bsd.cpp:1130 >>> frame #12: 0x000000010ac9bb55 libjvm.dylib`os::abort(dump_core=false) + 21 at os_bsd.cpp:1150 >>> frame #13: 0x000000010a9188e7 libjvm.dylib`vm_abort(dump_core=false) + 39 at java.cpp:666 >>> frame #14: 0x000000010aa4f1e7 libjvm.dylib`JVMCIRuntime::abort_on_pending_exception(exception=Handle @ 0x0000700001452208, message="Uncaught exception at /Users/dsimon/graal/graal-jvmci-8/src/share/vm/jvmci/jvmciCompiler.cpp:127", dump_core=false) + 167 at jvmciRuntime.cpp:992 >>> frame #15: 0x000000010aa17017 libjvm.dylib`JVMCICompiler::compile_method(this=0x00007fedcb203050, method=0x00007000014528d8, entry_bci=-1, env=0x00007000014528f0) + 311 at jvmciCompiler.cpp:127 >>> frame #16: 0x000000010a656cd2 libjvm.dylib`CompileBroker::invoke_compiler_on_method(task=0x00007fedc862a320) + 1314 at compileBroker.cpp:2207 >>> >>> >>> This webrev replaces calls to vm_abort() with before_exit() + vm_exit(). The latter is thread safe. >>> >>> https://bugs.openjdk.java.net/browse/JDK-8165755 >>> http://cr.openjdk.java.net/~dnsimon/8165755/ >>> >>> -Doug From doug.simon at oracle.com Fri Sep 9 19:31:44 2016 From: doug.simon at oracle.com (Doug Simon) Date: Fri, 9 Sep 2016 21:31:44 +0200 Subject: RFR: 8165755: [JVMCI] replace use of vm_abort with vm_exit In-Reply-To: References: <799CA13D-6BDC-4BF5-9241-515A684191F4@oracle.com> <6391B00B-AFBF-410C-A6A1-2ED95B35EBEB@twitter.com> <2F360221-7E6B-43BB-B69C-69A17777E5F2@oracle.com> Message-ID: Is vm_exit_during_initialization still the right choice when this could be after VM initialization? JVMCI initialization is lazy and can happen after the application has started. Sent from my iPhone > On Sep 9, 2016, at 9:23 PM, Coleen Phillimore wrote: > > > I think you want vm_exit_during_initialization() for that. Definitely not vm_abort, cause then it looks like an internal error/crash. > > Coleen > >> On 9/9/16 2:33 PM, Doug Simon wrote: >> Can someone from the runtime team confirm that using vm_exit (instead of vm_abort) is the best way to stop the VM when JVMCI initialization fails (e.g., when invalid JVMCI options are provided on the command line). Thanks! >> >> -Doug >> >> >>> On 09 Sep 2016, at 19:48, Christian Thalinger wrote: >>> >>> I think this looks fine but maybe we should ask the runtime folks. >>> >>>> On Sep 8, 2016, at 11:01 PM, Doug Simon wrote: >>>> >>>> Calling vm_abort from multiple threads can cause nasty crashes such as double free errors. We've seen this in Graal during JVMCI initialization when an unknown Graal option is encountered. Multiple compiler threads try to initialize JVMCI which fails with an exception indicating the bad option: >>>> >>>> Uncaught exception at /scratch/graaluser/buildslave/buildlog/ci_executor/main/graal-jvmci-8/src/share/vm/jvmci/jvmciCompiler.cpp:127 >>>> java.lang.ExceptionInInitializerError >>>> at jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.runtime(HotSpotJVMCIRuntime.java:85) >>>> at jdk.vm.ci.runtime.JVMCI.initializeRuntime(Native Method) >>>> at jdk.vm.ci.runtime.JVMCI.(JVMCI.java:58) >>>> Caused by: java.lang.IllegalArgumentException: Could not find option OptSomethingThatDoesNotExcist >>>> at com.oracle.graal.options.OptionsParser.parseOption(OptionsParser.java:134) >>>> at com.oracle.graal.options.OptionsParser.parseOptions(OptionsParser.java:62) >>>> at com.oracle.graal.hotspot.HotSpotGraalCompilerFactory.initializeOptions(HotSpotGraalCompilerFactory.java:156) >>>> at com.oracle.graal.hotspot.HotSpotGraalCompilerFactory.onSelection(HotSpotGraalCompilerFactory.java:86) >>>> at jdk.vm.ci.hotspot.HotSpotJVMCICompilerConfig.getCompilerFactory(HotSpotJVMCICompilerConfig.java:96) >>>> at jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.(HotSpotJVMCIRuntime.java:277) >>>> at jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.(HotSpotJVMCIRuntime.java:67) >>>> at jdk.vm.ci.hotspot.HotSpotJVMCIRuntime$DelayedInit.(HotSpotJVMCIRuntime.java:75) >>>> at jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.runtime(HotSpotJVMCIRuntime.java:85) >>>> at jdk.vm.ci.runtime.JVMCI.initializeRuntime(Native Method) >>>> at jdk.vm.ci.runtime.JVMCI.(JVMCI.java:58) >>>> >>>> The native JVMCI code then tries to exit the VM by calling vm_abort. If multiple compiler threads do this concurrently, certain destructors can be called twice as shown by these thread dumps: >>>> >>>> thread #26: tid = 0x0019, 0x00007fff84280124 libsystem_malloc.dylib`szone_size + 227, stop reason = signal SIGSTOP >>>> frame #0: 0x00007fff84280124 libsystem_malloc.dylib`szone_size + 227 >>>> frame #1: 0x00007fff8427fed5 libsystem_malloc.dylib`free + 61 >>>> frame #2: 0x000000010ac95963 libjvm.dylib`os::free(memblock=0x00007fedc86226e0, memflags=mtInternal) + 307 at os.cpp:711 >>>> frame #3: 0x000000010a2afc54 libjvm.dylib`FreeHeap(p=0x00007fedc86226e0, memflags=mtInternal) + 52 at allocation.inline.hpp:93 >>>> frame #4: 0x000000010acf0a9f libjvm.dylib`PerfData::~PerfData(this=0x00007fedc8622650) + 63 at perfData.cpp:116 >>>> frame #5: 0x000000010acf0ae5 libjvm.dylib`PerfData::~PerfData(this=0x00007fedc8622650) + 21 at perfData.cpp:114 >>>> frame #6: 0x000000010acf163d libjvm.dylib`PerfDataManager::destroy() + 109 at perfData.cpp:287 >>>> frame #7: 0x000000010acf3f4d libjvm.dylib`perfMemory_exit() + 61 at perfMemory.cpp:74 >>>> frame #8: 0x000000010ac9bb0d libjvm.dylib`os::shutdown() + 13 at os_bsd.cpp:1130 >>>> frame #9: 0x000000010ac9bb55 libjvm.dylib`os::abort(dump_core=false) + 21 at os_bsd.cpp:1150 >>>> frame #10: 0x000000010a9188e7 libjvm.dylib`vm_abort(dump_core=false) + 39 at java.cpp:666 >>>> frame #11: 0x000000010aa4f1e7 libjvm.dylib`JVMCIRuntime::abort_on_pending_exception(exception=Handle @ 0x000070000175b208, message="Uncaught exception at /Users/dsimon/graal/graal-jvmci-8/src/share/vm/jvmci/jvmciCompiler.cpp:127", dump_core=false) + 167 at jvmciRuntime.cpp:992 >>>> frame #12: 0x000000010aa17017 libjvm.dylib`JVMCICompiler::compile_method(this=0x00007fedcb203050, method=0x000070000175b8d8, entry_bci=-1, env=0x000070000175b8f0) + 311 at jvmciCompiler.cpp:127 >>>> frame #13: 0x000000010a656cd2 libjvm.dylib`CompileBroker::invoke_compiler_on_method(task=0x00007fedc853ca30) + 1314 at compileBroker.cpp:2207 >>>> >>>> thread #23: tid = 0x0016, 0x00007fff91fcb122 libsystem_kernel.dylib`__semwait_signal_nocancel + 10, stop reason = signal SIGSTOP >>>> frame #0: 0x00007fff91fcb122 libsystem_kernel.dylib`__semwait_signal_nocancel + 10 >>>> frame #1: 0x00007fff9578c318 libsystem_c.dylib`nanosleep$NOCANCEL + 188 >>>> frame #2: 0x00007fff957b62ce libsystem_c.dylib`usleep$NOCANCEL + 54 >>>> frame #3: 0x00007fff957e46e9 libsystem_c.dylib`abort + 139 >>>> frame #4: 0x00007fff8428c396 libsystem_malloc.dylib`szone_error + 626 >>>> frame #5: 0x000000010ac95963 libjvm.dylib`os::free(memblock=0x00007fedc8601cd0, memflags=mtInternal) + 307 at os.cpp:711 >>>> frame #6: 0x000000010a2afc54 libjvm.dylib`FreeHeap(p=0x00007fedc8601cd0, memflags=mtInternal) + 52 at allocation.inline.hpp:93 >>>> frame #7: 0x000000010acf0a9f libjvm.dylib`PerfData::~PerfData(this=0x00007fedc8601c60) + 63 at perfData.cpp:116 >>>> frame #8: 0x000000010acf0ae5 libjvm.dylib`PerfData::~PerfData(this=0x00007fedc8601c60) + 21 at perfData.cpp:114 >>>> frame #9: 0x000000010acf163d libjvm.dylib`PerfDataManager::destroy() + 109 at perfData.cpp:287 >>>> frame #10: 0x000000010acf3f4d libjvm.dylib`perfMemory_exit() + 61 at perfMemory.cpp:74 >>>> frame #11: 0x000000010ac9bb0d libjvm.dylib`os::shutdown() + 13 at os_bsd.cpp:1130 >>>> frame #12: 0x000000010ac9bb55 libjvm.dylib`os::abort(dump_core=false) + 21 at os_bsd.cpp:1150 >>>> frame #13: 0x000000010a9188e7 libjvm.dylib`vm_abort(dump_core=false) + 39 at java.cpp:666 >>>> frame #14: 0x000000010aa4f1e7 libjvm.dylib`JVMCIRuntime::abort_on_pending_exception(exception=Handle @ 0x0000700001452208, message="Uncaught exception at /Users/dsimon/graal/graal-jvmci-8/src/share/vm/jvmci/jvmciCompiler.cpp:127", dump_core=false) + 167 at jvmciRuntime.cpp:992 >>>> frame #15: 0x000000010aa17017 libjvm.dylib`JVMCICompiler::compile_method(this=0x00007fedcb203050, method=0x00007000014528d8, entry_bci=-1, env=0x00007000014528f0) + 311 at jvmciCompiler.cpp:127 >>>> frame #16: 0x000000010a656cd2 libjvm.dylib`CompileBroker::invoke_compiler_on_method(task=0x00007fedc862a320) + 1314 at compileBroker.cpp:2207 >>>> >>>> >>>> This webrev replaces calls to vm_abort() with before_exit() + vm_exit(). The latter is thread safe. >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8165755 >>>> http://cr.openjdk.java.net/~dnsimon/8165755/ >>>> >>>> -Doug > From coleen.phillimore at oracle.com Fri Sep 9 20:29:06 2016 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 9 Sep 2016 16:29:06 -0400 Subject: RFR: 8165755: [JVMCI] replace use of vm_abort with vm_exit In-Reply-To: References: <799CA13D-6BDC-4BF5-9241-515A684191F4@oracle.com> <6391B00B-AFBF-410C-A6A1-2ED95B35EBEB@twitter.com> <2F360221-7E6B-43BB-B69C-69A17777E5F2@oracle.com> Message-ID: <1c9b4d32-52a2-17a8-bfc5-f3cc7b494ba0@oracle.com> On 9/9/16 3:31 PM, Doug Simon wrote: > Is vm_exit_during_initialization still the right choice when this could be after VM initialization? JVMCI initialization is lazy and can happen after the application has started. I don't actually know then. vm_exit tries to get threads to a safepoint first, but vm_abort(false) just shuts down the jvm. There aren't a lot of places we terminate the jvm. Maybe David Holmes knows. Coleen > > Sent from my iPhone > >> On Sep 9, 2016, at 9:23 PM, Coleen Phillimore wrote: >> >> >> I think you want vm_exit_during_initialization() for that. Definitely not vm_abort, cause then it looks like an internal error/crash. >> >> Coleen >> >>> On 9/9/16 2:33 PM, Doug Simon wrote: >>> Can someone from the runtime team confirm that using vm_exit (instead of vm_abort) is the best way to stop the VM when JVMCI initialization fails (e.g., when invalid JVMCI options are provided on the command line). Thanks! >>> >>> -Doug >>> >>> >>>> On 09 Sep 2016, at 19:48, Christian Thalinger wrote: >>>> >>>> I think this looks fine but maybe we should ask the runtime folks. >>>> >>>>> On Sep 8, 2016, at 11:01 PM, Doug Simon wrote: >>>>> >>>>> Calling vm_abort from multiple threads can cause nasty crashes such as double free errors. We've seen this in Graal during JVMCI initialization when an unknown Graal option is encountered. Multiple compiler threads try to initialize JVMCI which fails with an exception indicating the bad option: >>>>> >>>>> Uncaught exception at /scratch/graaluser/buildslave/buildlog/ci_executor/main/graal-jvmci-8/src/share/vm/jvmci/jvmciCompiler.cpp:127 >>>>> java.lang.ExceptionInInitializerError >>>>> at jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.runtime(HotSpotJVMCIRuntime.java:85) >>>>> at jdk.vm.ci.runtime.JVMCI.initializeRuntime(Native Method) >>>>> at jdk.vm.ci.runtime.JVMCI.(JVMCI.java:58) >>>>> Caused by: java.lang.IllegalArgumentException: Could not find option OptSomethingThatDoesNotExcist >>>>> at com.oracle.graal.options.OptionsParser.parseOption(OptionsParser.java:134) >>>>> at com.oracle.graal.options.OptionsParser.parseOptions(OptionsParser.java:62) >>>>> at com.oracle.graal.hotspot.HotSpotGraalCompilerFactory.initializeOptions(HotSpotGraalCompilerFactory.java:156) >>>>> at com.oracle.graal.hotspot.HotSpotGraalCompilerFactory.onSelection(HotSpotGraalCompilerFactory.java:86) >>>>> at jdk.vm.ci.hotspot.HotSpotJVMCICompilerConfig.getCompilerFactory(HotSpotJVMCICompilerConfig.java:96) >>>>> at jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.(HotSpotJVMCIRuntime.java:277) >>>>> at jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.(HotSpotJVMCIRuntime.java:67) >>>>> at jdk.vm.ci.hotspot.HotSpotJVMCIRuntime$DelayedInit.(HotSpotJVMCIRuntime.java:75) >>>>> at jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.runtime(HotSpotJVMCIRuntime.java:85) >>>>> at jdk.vm.ci.runtime.JVMCI.initializeRuntime(Native Method) >>>>> at jdk.vm.ci.runtime.JVMCI.(JVMCI.java:58) >>>>> >>>>> The native JVMCI code then tries to exit the VM by calling vm_abort. If multiple compiler threads do this concurrently, certain destructors can be called twice as shown by these thread dumps: >>>>> >>>>> thread #26: tid = 0x0019, 0x00007fff84280124 libsystem_malloc.dylib`szone_size + 227, stop reason = signal SIGSTOP >>>>> frame #0: 0x00007fff84280124 libsystem_malloc.dylib`szone_size + 227 >>>>> frame #1: 0x00007fff8427fed5 libsystem_malloc.dylib`free + 61 >>>>> frame #2: 0x000000010ac95963 libjvm.dylib`os::free(memblock=0x00007fedc86226e0, memflags=mtInternal) + 307 at os.cpp:711 >>>>> frame #3: 0x000000010a2afc54 libjvm.dylib`FreeHeap(p=0x00007fedc86226e0, memflags=mtInternal) + 52 at allocation.inline.hpp:93 >>>>> frame #4: 0x000000010acf0a9f libjvm.dylib`PerfData::~PerfData(this=0x00007fedc8622650) + 63 at perfData.cpp:116 >>>>> frame #5: 0x000000010acf0ae5 libjvm.dylib`PerfData::~PerfData(this=0x00007fedc8622650) + 21 at perfData.cpp:114 >>>>> frame #6: 0x000000010acf163d libjvm.dylib`PerfDataManager::destroy() + 109 at perfData.cpp:287 >>>>> frame #7: 0x000000010acf3f4d libjvm.dylib`perfMemory_exit() + 61 at perfMemory.cpp:74 >>>>> frame #8: 0x000000010ac9bb0d libjvm.dylib`os::shutdown() + 13 at os_bsd.cpp:1130 >>>>> frame #9: 0x000000010ac9bb55 libjvm.dylib`os::abort(dump_core=false) + 21 at os_bsd.cpp:1150 >>>>> frame #10: 0x000000010a9188e7 libjvm.dylib`vm_abort(dump_core=false) + 39 at java.cpp:666 >>>>> frame #11: 0x000000010aa4f1e7 libjvm.dylib`JVMCIRuntime::abort_on_pending_exception(exception=Handle @ 0x000070000175b208, message="Uncaught exception at /Users/dsimon/graal/graal-jvmci-8/src/share/vm/jvmci/jvmciCompiler.cpp:127", dump_core=false) + 167 at jvmciRuntime.cpp:992 >>>>> frame #12: 0x000000010aa17017 libjvm.dylib`JVMCICompiler::compile_method(this=0x00007fedcb203050, method=0x000070000175b8d8, entry_bci=-1, env=0x000070000175b8f0) + 311 at jvmciCompiler.cpp:127 >>>>> frame #13: 0x000000010a656cd2 libjvm.dylib`CompileBroker::invoke_compiler_on_method(task=0x00007fedc853ca30) + 1314 at compileBroker.cpp:2207 >>>>> >>>>> thread #23: tid = 0x0016, 0x00007fff91fcb122 libsystem_kernel.dylib`__semwait_signal_nocancel + 10, stop reason = signal SIGSTOP >>>>> frame #0: 0x00007fff91fcb122 libsystem_kernel.dylib`__semwait_signal_nocancel + 10 >>>>> frame #1: 0x00007fff9578c318 libsystem_c.dylib`nanosleep$NOCANCEL + 188 >>>>> frame #2: 0x00007fff957b62ce libsystem_c.dylib`usleep$NOCANCEL + 54 >>>>> frame #3: 0x00007fff957e46e9 libsystem_c.dylib`abort + 139 >>>>> frame #4: 0x00007fff8428c396 libsystem_malloc.dylib`szone_error + 626 >>>>> frame #5: 0x000000010ac95963 libjvm.dylib`os::free(memblock=0x00007fedc8601cd0, memflags=mtInternal) + 307 at os.cpp:711 >>>>> frame #6: 0x000000010a2afc54 libjvm.dylib`FreeHeap(p=0x00007fedc8601cd0, memflags=mtInternal) + 52 at allocation.inline.hpp:93 >>>>> frame #7: 0x000000010acf0a9f libjvm.dylib`PerfData::~PerfData(this=0x00007fedc8601c60) + 63 at perfData.cpp:116 >>>>> frame #8: 0x000000010acf0ae5 libjvm.dylib`PerfData::~PerfData(this=0x00007fedc8601c60) + 21 at perfData.cpp:114 >>>>> frame #9: 0x000000010acf163d libjvm.dylib`PerfDataManager::destroy() + 109 at perfData.cpp:287 >>>>> frame #10: 0x000000010acf3f4d libjvm.dylib`perfMemory_exit() + 61 at perfMemory.cpp:74 >>>>> frame #11: 0x000000010ac9bb0d libjvm.dylib`os::shutdown() + 13 at os_bsd.cpp:1130 >>>>> frame #12: 0x000000010ac9bb55 libjvm.dylib`os::abort(dump_core=false) + 21 at os_bsd.cpp:1150 >>>>> frame #13: 0x000000010a9188e7 libjvm.dylib`vm_abort(dump_core=false) + 39 at java.cpp:666 >>>>> frame #14: 0x000000010aa4f1e7 libjvm.dylib`JVMCIRuntime::abort_on_pending_exception(exception=Handle @ 0x0000700001452208, message="Uncaught exception at /Users/dsimon/graal/graal-jvmci-8/src/share/vm/jvmci/jvmciCompiler.cpp:127", dump_core=false) + 167 at jvmciRuntime.cpp:992 >>>>> frame #15: 0x000000010aa17017 libjvm.dylib`JVMCICompiler::compile_method(this=0x00007fedcb203050, method=0x00007000014528d8, entry_bci=-1, env=0x00007000014528f0) + 311 at jvmciCompiler.cpp:127 >>>>> frame #16: 0x000000010a656cd2 libjvm.dylib`CompileBroker::invoke_compiler_on_method(task=0x00007fedc862a320) + 1314 at compileBroker.cpp:2207 >>>>> >>>>> >>>>> This webrev replaces calls to vm_abort() with before_exit() + vm_exit(). The latter is thread safe. >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8165755 >>>>> http://cr.openjdk.java.net/~dnsimon/8165755/ >>>>> >>>>> -Doug From coleen.phillimore at oracle.com Fri Sep 9 20:55:08 2016 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 9 Sep 2016 16:55:08 -0400 Subject: [9] RFR(S): Crash with assert: symbol conversion failure in java_lang_String::create_from_symbol() In-Reply-To: <57D2AE2F.1080607@oracle.com> References: <57D2AE2F.1080607@oracle.com> Message-ID: <1203086c-f59f-40ce-f963-e958e570caf8@oracle.com> This change is fine because it matches the commented out assert in create_from_str(). We should probably figure out what it would take to check the characters coming in from JNI and decide whether we should do this. If not, it doesn't make sense to have commented out asserts. But this is okay for jdk9. Thanks, Coleen On 9/9/16 8:42 AM, Tobias Hartmann wrote: > Hi, > > please review the following patch: > https://bugs.openjdk.java.net/browse/JDK-8164561 > http://cr.openjdk.java.net/~thartmann/8164561/webrev.00/ > > The verification code in java_lang_String::create_from_symbol() that was added by Compact Strings fails because the input symbol does not contain valid UTF8. The problem is that a JCK JNI test passes an invalid UTF8 string as class name to the JNI method "FindClass". In fact, the string contains garbage from reading past array boundaries because of a bug in the test [1]. The JNI spec [2] states that 'name' should be "a fully-qualified class name (that is, a package name, delimited by ?/?, followed by the class name). If the name begins with ?[? (the array signature character), it returns an array class. The string is encoded in modified UTF-8". > > I nevertheless think that we should not crash in the case of an invalid UTF8 string and therefore disabled the verification code with a comment. We did the same for java_lang_String::create_from_str() [3]. > > Tested with failing JCK test and JPRT (running). > > Thanks, > Tobias > > [1] https://bugs.openjdk.java.net/browse/JCK-7307244 > [2] https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/functions.html#FindClass > [3] http://hg.openjdk.java.net/jdk9/hs/hotspot/file/d060826d0911/src/share/vm/classfile/javaClasses.cpp#l274 From kim.barrett at oracle.com Sat Sep 10 22:59:20 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Sat, 10 Sep 2016 18:59:20 -0400 Subject: RFR: 8165808: Add release barriers when allocating objects with concurrent collection Message-ID: <13AE434E-FA26-4658-8A77-013462679EBC@oracle.com> Please review this change to add release barriers when allocating objects that may be visible to a concurrent collector. This change only addresses the allocation side, where the initialization occurs; other subtasks of JDK-8160369 will address the (mostly GC-specific) readers that need to accomodate in-progress allocations. As part of this change, eliminated post_allocation_install_obj_klass, which consisted of a call to [release_]set_klass + assertions that are redundant with those in that called function. CR: https://bugs.openjdk.java.net/browse/JDK-8165808 Webrev: http://cr.openjdk.java.net/~kbarrett/8165808/webrev.00/ Testing: Local jtreg tests. From aph at redhat.com Sun Sep 11 10:08:51 2016 From: aph at redhat.com (Andrew Haley) Date: Sun, 11 Sep 2016 11:08:51 +0100 Subject: Fwd: [aarch64-port-dev ] RFR: AArch64: Fix JNI floating point argument handling In-Reply-To: References: Message-ID: <81c57e70-3777-cc33-b5c2-59176fbdfc91@redhat.com> We need a sponsor for this new test. Thanks, Andrew. -------------- next part -------------- An embedded message was scrubbed... From: Ningsheng Jian Subject: [aarch64-port-dev ] RFR: AArch64: Fix JNI floating point argument handling Date: Thu, 8 Sep 2016 15:57:52 +0800 Size: 5899 URL: From david.holmes at oracle.com Mon Sep 12 00:47:46 2016 From: david.holmes at oracle.com (David Holmes) Date: Mon, 12 Sep 2016 10:47:46 +1000 Subject: [9] RFR(S): Crash with assert: symbol conversion failure in java_lang_String::create_from_symbol() In-Reply-To: <1203086c-f59f-40ce-f963-e958e570caf8@oracle.com> References: <57D2AE2F.1080607@oracle.com> <1203086c-f59f-40ce-f963-e958e570caf8@oracle.com> Message-ID: On 10/09/2016 6:55 AM, Coleen Phillimore wrote: > This change is fine because it matches the commented out assert in > create_from_str(). We should probably figure out what it would take to > check the characters coming in from JNI and decide whether we should do > this. If not, it doesn't make sense to have commented out asserts. > But this is okay for jdk9. Grumble, grumble ... both are bad. If the VM doesn't validate this bad UTF-8 then where does it go? And how does the generator of the bad UTF-8 get informed? An assert may be too drastic but can we throw an exception (InternalError?) ? David > Thanks, > Coleen > > > On 9/9/16 8:42 AM, Tobias Hartmann wrote: >> Hi, >> >> please review the following patch: >> https://bugs.openjdk.java.net/browse/JDK-8164561 >> http://cr.openjdk.java.net/~thartmann/8164561/webrev.00/ >> >> The verification code in java_lang_String::create_from_symbol() that >> was added by Compact Strings fails because the input symbol does not >> contain valid UTF8. The problem is that a JCK JNI test passes an >> invalid UTF8 string as class name to the JNI method "FindClass". In >> fact, the string contains garbage from reading past array boundaries >> because of a bug in the test [1]. The JNI spec [2] states that 'name' >> should be "a fully-qualified class name (that is, a package name, >> delimited by ?/?, followed by the class name). If the name begins with >> ?[? (the array signature character), it returns an array class. The >> string is encoded in modified UTF-8". >> >> I nevertheless think that we should not crash in the case of an >> invalid UTF8 string and therefore disabled the verification code with >> a comment. We did the same for java_lang_String::create_from_str() [3]. >> >> Tested with failing JCK test and JPRT (running). >> >> Thanks, >> Tobias >> >> [1] https://bugs.openjdk.java.net/browse/JCK-7307244 >> [2] >> https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/functions.html#FindClass >> >> [3] >> http://hg.openjdk.java.net/jdk9/hs/hotspot/file/d060826d0911/src/share/vm/classfile/javaClasses.cpp#l274 >> > From serguei.spitsyn at oracle.com Mon Sep 12 01:15:55 2016 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Sun, 11 Sep 2016 18:15:55 -0700 Subject: RFR (XS): 8165681 ClassLoad and ClassPrepare JVMTI events are missed in the start phase Message-ID: <45ce10ea-993f-6a07-3e8f-0222fb479566@oracle.com> Please, review the one-liner fix for: https://bugs.openjdk.java.net/browse/JDK-8165681 Webrev: http://cr.openjdk.java.net/~sspitsyn/webrevs/2016/hotspot/8165681-jvmti-events.hs1/ Summary: The JVMTI spec tells the ClassLoad and ClassPrepare events can be send during start phase. The new test below identified that they are missed because of this bug in JVMTI. The fix is to add the events to the set of the set of the EARLY_EVENT_BITS. Testing: The following new test covers this issue: test/serviceability/jvmti/ModuleAwareAgents/ClassLoadPrepare/VMEarly They are developed as a part of the following test enhancement: 8150758 [TESTBUG] need jvmti tests for module aware agents Additionally, the following tests were run: nsk.jvmti, nsk,jdi, jtreg com/sun/jdi and java/lang/instrument Thanks, Serguei From david.holmes at oracle.com Mon Sep 12 01:24:22 2016 From: david.holmes at oracle.com (David Holmes) Date: Mon, 12 Sep 2016 11:24:22 +1000 Subject: RFR: 8165755: [JVMCI] replace use of vm_abort with vm_exit In-Reply-To: <1c9b4d32-52a2-17a8-bfc5-f3cc7b494ba0@oracle.com> References: <799CA13D-6BDC-4BF5-9241-515A684191F4@oracle.com> <6391B00B-AFBF-410C-A6A1-2ED95B35EBEB@twitter.com> <2F360221-7E6B-43BB-B69C-69A17777E5F2@oracle.com> <1c9b4d32-52a2-17a8-bfc5-f3cc7b494ba0@oracle.com> Message-ID: <5d26a289-a262-5882-8061-3bbc4faa4540@oracle.com> On 10/09/2016 6:29 AM, Coleen Phillimore wrote: > On 9/9/16 3:31 PM, Doug Simon wrote: >> Is vm_exit_during_initialization still the right choice when this >> could be after VM initialization? JVMCI initialization is lazy and can >> happen after the application has started. > > I don't actually know then. vm_exit tries to get threads to a safepoint > first, but vm_abort(false) just shuts down the jvm. There aren't a lot > of places we terminate the jvm. Maybe David Holmes knows. If JVMCI initialization is lazy and can happen after the application is started, why should it be a fatal error to fail to initialize it? How is the initialization triggered? Is there a synchronous call that could throw an exception? How does the user know that initialization failure will result in termination? Can you not at least eagerly validate command-line options during VM initialization, even if you don't actually initialize JVMCI fully? That aside it sounds like you want JVMCI initialization to do the equivalent of either Runtime.exit or Runtime.halt, depending on the exact termination semantics you want. If you call vm_exit directly then you get something like Runtime.halt David ----- > Coleen > >> >> Sent from my iPhone >> >>> On Sep 9, 2016, at 9:23 PM, Coleen Phillimore >>> wrote: >>> >>> >>> I think you want vm_exit_during_initialization() for that. Definitely >>> not vm_abort, cause then it looks like an internal error/crash. >>> >>> Coleen >>> >>>> On 9/9/16 2:33 PM, Doug Simon wrote: >>>> Can someone from the runtime team confirm that using vm_exit >>>> (instead of vm_abort) is the best way to stop the VM when JVMCI >>>> initialization fails (e.g., when invalid JVMCI options are provided >>>> on the command line). Thanks! >>>> >>>> -Doug >>>> >>>> >>>>> On 09 Sep 2016, at 19:48, Christian Thalinger >>>>> wrote: >>>>> >>>>> I think this looks fine but maybe we should ask the runtime folks. >>>>> >>>>>> On Sep 8, 2016, at 11:01 PM, Doug Simon >>>>>> wrote: >>>>>> >>>>>> Calling vm_abort from multiple threads can cause nasty crashes >>>>>> such as double free errors. We've seen this in Graal during JVMCI >>>>>> initialization when an unknown Graal option is encountered. >>>>>> Multiple compiler threads try to initialize JVMCI which fails with >>>>>> an exception indicating the bad option: >>>>>> >>>>>> Uncaught exception at >>>>>> /scratch/graaluser/buildslave/buildlog/ci_executor/main/graal-jvmci-8/src/share/vm/jvmci/jvmciCompiler.cpp:127 >>>>>> >>>>>> java.lang.ExceptionInInitializerError >>>>>> at >>>>>> jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.runtime(HotSpotJVMCIRuntime.java:85) >>>>>> >>>>>> at jdk.vm.ci.runtime.JVMCI.initializeRuntime(Native Method) >>>>>> at jdk.vm.ci.runtime.JVMCI.(JVMCI.java:58) >>>>>> Caused by: java.lang.IllegalArgumentException: Could not find >>>>>> option OptSomethingThatDoesNotExcist >>>>>> at >>>>>> com.oracle.graal.options.OptionsParser.parseOption(OptionsParser.java:134) >>>>>> >>>>>> at >>>>>> com.oracle.graal.options.OptionsParser.parseOptions(OptionsParser.java:62) >>>>>> >>>>>> at >>>>>> com.oracle.graal.hotspot.HotSpotGraalCompilerFactory.initializeOptions(HotSpotGraalCompilerFactory.java:156) >>>>>> >>>>>> at >>>>>> com.oracle.graal.hotspot.HotSpotGraalCompilerFactory.onSelection(HotSpotGraalCompilerFactory.java:86) >>>>>> >>>>>> at >>>>>> jdk.vm.ci.hotspot.HotSpotJVMCICompilerConfig.getCompilerFactory(HotSpotJVMCICompilerConfig.java:96) >>>>>> >>>>>> at >>>>>> jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.(HotSpotJVMCIRuntime.java:277) >>>>>> >>>>>> at >>>>>> jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.(HotSpotJVMCIRuntime.java:67) >>>>>> >>>>>> at >>>>>> jdk.vm.ci.hotspot.HotSpotJVMCIRuntime$DelayedInit.(HotSpotJVMCIRuntime.java:75) >>>>>> >>>>>> at >>>>>> jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.runtime(HotSpotJVMCIRuntime.java:85) >>>>>> >>>>>> at jdk.vm.ci.runtime.JVMCI.initializeRuntime(Native Method) >>>>>> at jdk.vm.ci.runtime.JVMCI.(JVMCI.java:58) >>>>>> >>>>>> The native JVMCI code then tries to exit the VM by calling >>>>>> vm_abort. If multiple compiler threads do this concurrently, >>>>>> certain destructors can be called twice as shown by these thread >>>>>> dumps: >>>>>> >>>>>> thread #26: tid = 0x0019, 0x00007fff84280124 >>>>>> libsystem_malloc.dylib`szone_size + 227, stop reason = signal SIGSTOP >>>>>> frame #0: 0x00007fff84280124 libsystem_malloc.dylib`szone_size + 227 >>>>>> frame #1: 0x00007fff8427fed5 libsystem_malloc.dylib`free + 61 >>>>>> frame #2: 0x000000010ac95963 >>>>>> libjvm.dylib`os::free(memblock=0x00007fedc86226e0, >>>>>> memflags=mtInternal) + 307 at os.cpp:711 >>>>>> frame #3: 0x000000010a2afc54 >>>>>> libjvm.dylib`FreeHeap(p=0x00007fedc86226e0, memflags=mtInternal) + >>>>>> 52 at allocation.inline.hpp:93 >>>>>> frame #4: 0x000000010acf0a9f >>>>>> libjvm.dylib`PerfData::~PerfData(this=0x00007fedc8622650) + 63 at >>>>>> perfData.cpp:116 >>>>>> frame #5: 0x000000010acf0ae5 >>>>>> libjvm.dylib`PerfData::~PerfData(this=0x00007fedc8622650) + 21 at >>>>>> perfData.cpp:114 >>>>>> frame #6: 0x000000010acf163d >>>>>> libjvm.dylib`PerfDataManager::destroy() + 109 at perfData.cpp:287 >>>>>> frame #7: 0x000000010acf3f4d libjvm.dylib`perfMemory_exit() + 61 >>>>>> at perfMemory.cpp:74 >>>>>> frame #8: 0x000000010ac9bb0d libjvm.dylib`os::shutdown() + 13 at >>>>>> os_bsd.cpp:1130 >>>>>> frame #9: 0x000000010ac9bb55 >>>>>> libjvm.dylib`os::abort(dump_core=false) + 21 at os_bsd.cpp:1150 >>>>>> frame #10: 0x000000010a9188e7 >>>>>> libjvm.dylib`vm_abort(dump_core=false) + 39 at java.cpp:666 >>>>>> frame #11: 0x000000010aa4f1e7 >>>>>> libjvm.dylib`JVMCIRuntime::abort_on_pending_exception(exception=Handle >>>>>> @ 0x000070000175b208, message="Uncaught exception at >>>>>> /Users/dsimon/graal/graal-jvmci-8/src/share/vm/jvmci/jvmciCompiler.cpp:127", >>>>>> dump_core=false) + 167 at jvmciRuntime.cpp:992 >>>>>> frame #12: 0x000000010aa17017 >>>>>> libjvm.dylib`JVMCICompiler::compile_method(this=0x00007fedcb203050, method=0x000070000175b8d8, >>>>>> entry_bci=-1, env=0x000070000175b8f0) + 311 at jvmciCompiler.cpp:127 >>>>>> frame #13: 0x000000010a656cd2 >>>>>> libjvm.dylib`CompileBroker::invoke_compiler_on_method(task=0x00007fedc853ca30) >>>>>> + 1314 at compileBroker.cpp:2207 >>>>>> >>>>>> thread #23: tid = 0x0016, 0x00007fff91fcb122 >>>>>> libsystem_kernel.dylib`__semwait_signal_nocancel + 10, stop reason >>>>>> = signal SIGSTOP >>>>>> frame #0: 0x00007fff91fcb122 >>>>>> libsystem_kernel.dylib`__semwait_signal_nocancel + 10 >>>>>> frame #1: 0x00007fff9578c318 libsystem_c.dylib`nanosleep$NOCANCEL >>>>>> + 188 >>>>>> frame #2: 0x00007fff957b62ce libsystem_c.dylib`usleep$NOCANCEL + 54 >>>>>> frame #3: 0x00007fff957e46e9 libsystem_c.dylib`abort + 139 >>>>>> frame #4: 0x00007fff8428c396 libsystem_malloc.dylib`szone_error + 626 >>>>>> frame #5: 0x000000010ac95963 >>>>>> libjvm.dylib`os::free(memblock=0x00007fedc8601cd0, >>>>>> memflags=mtInternal) + 307 at os.cpp:711 >>>>>> frame #6: 0x000000010a2afc54 >>>>>> libjvm.dylib`FreeHeap(p=0x00007fedc8601cd0, memflags=mtInternal) + >>>>>> 52 at allocation.inline.hpp:93 >>>>>> frame #7: 0x000000010acf0a9f >>>>>> libjvm.dylib`PerfData::~PerfData(this=0x00007fedc8601c60) + 63 at >>>>>> perfData.cpp:116 >>>>>> frame #8: 0x000000010acf0ae5 >>>>>> libjvm.dylib`PerfData::~PerfData(this=0x00007fedc8601c60) + 21 at >>>>>> perfData.cpp:114 >>>>>> frame #9: 0x000000010acf163d >>>>>> libjvm.dylib`PerfDataManager::destroy() + 109 at perfData.cpp:287 >>>>>> frame #10: 0x000000010acf3f4d libjvm.dylib`perfMemory_exit() + 61 >>>>>> at perfMemory.cpp:74 >>>>>> frame #11: 0x000000010ac9bb0d libjvm.dylib`os::shutdown() + 13 at >>>>>> os_bsd.cpp:1130 >>>>>> frame #12: 0x000000010ac9bb55 >>>>>> libjvm.dylib`os::abort(dump_core=false) + 21 at os_bsd.cpp:1150 >>>>>> frame #13: 0x000000010a9188e7 >>>>>> libjvm.dylib`vm_abort(dump_core=false) + 39 at java.cpp:666 >>>>>> frame #14: 0x000000010aa4f1e7 >>>>>> libjvm.dylib`JVMCIRuntime::abort_on_pending_exception(exception=Handle >>>>>> @ 0x0000700001452208, message="Uncaught exception at >>>>>> /Users/dsimon/graal/graal-jvmci-8/src/share/vm/jvmci/jvmciCompiler.cpp:127", >>>>>> dump_core=false) + 167 at jvmciRuntime.cpp:992 >>>>>> frame #15: 0x000000010aa17017 >>>>>> libjvm.dylib`JVMCICompiler::compile_method(this=0x00007fedcb203050, method=0x00007000014528d8, >>>>>> entry_bci=-1, env=0x00007000014528f0) + 311 at jvmciCompiler.cpp:127 >>>>>> frame #16: 0x000000010a656cd2 >>>>>> libjvm.dylib`CompileBroker::invoke_compiler_on_method(task=0x00007fedc862a320) >>>>>> + 1314 at compileBroker.cpp:2207 >>>>>> >>>>>> >>>>>> This webrev replaces calls to vm_abort() with before_exit() + >>>>>> vm_exit(). The latter is thread safe. >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8165755 >>>>>> http://cr.openjdk.java.net/~dnsimon/8165755/ >>>>>> >>>>>> -Doug > From david.holmes at oracle.com Mon Sep 12 01:38:26 2016 From: david.holmes at oracle.com (David Holmes) Date: Mon, 12 Sep 2016 11:38:26 +1000 Subject: RFR: 8165808: Add release barriers when allocating objects with concurrent collection In-Reply-To: <13AE434E-FA26-4658-8A77-013462679EBC@oracle.com> References: <13AE434E-FA26-4658-8A77-013462679EBC@oracle.com> Message-ID: <6c991053-7347-cc6b-3b94-fc48cb27a1db@oracle.com> Hi Kim, On 11/09/2016 8:59 AM, Kim Barrett wrote: > Please review this change to add release barriers when allocating > objects that may be visible to a concurrent collector. > > This change only addresses the allocation side, where the > initialization occurs; other subtasks of JDK-8160369 will address the > (mostly GC-specific) readers that need to accomodate in-progress > allocations. > > As part of this change, eliminated post_allocation_install_obj_klass, > which consisted of a call to [release_]set_klass + assertions that are > redundant with those in that called function. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8165808 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8165808/webrev.00/ src/share/vm/gc/shared/collectedHeap.inline.hpp Based on experiences with RMO architectures (ref JDK-8033920 - sorry it is a non-open bug report) I would expect the use of release_set_klass to be unconditional, not dependent on the nature of the GC. Publication of the object reference also has to be safe for other mutator threads. I'm not convinced by the brief argument in JDK-8033920 that thread-state transitions will always ensure visibility and ordering. Thanks, David > Testing: > Local jtreg tests. > From david.holmes at oracle.com Mon Sep 12 01:49:05 2016 From: david.holmes at oracle.com (David Holmes) Date: Mon, 12 Sep 2016 11:49:05 +1000 Subject: RFR (XS): 8165681 ClassLoad and ClassPrepare JVMTI events are missed in the start phase In-Reply-To: <45ce10ea-993f-6a07-3e8f-0222fb479566@oracle.com> References: <45ce10ea-993f-6a07-3e8f-0222fb479566@oracle.com> Message-ID: <4ffd4740-0df4-a103-4653-d085e5a98159@oracle.com> Hi Serguei, On 12/09/2016 11:15 AM, serguei.spitsyn at oracle.com wrote: > Please, review the one-liner fix for: > https://bugs.openjdk.java.net/browse/JDK-8165681 > > Webrev: > http://cr.openjdk.java.net/~sspitsyn/webrevs/2016/hotspot/8165681-jvmti-events.hs1/ > > Summary: > The JVMTI spec tells the ClassLoad and ClassPrepare events can be send > during start phase. > The new test below identified that they are missed because of this bug > in JVMTI. > The fix is to add the events to the set of the set of the EARLY_EVENT_BITS. Seems fine. Good to see the additional test coverage. Thanks, David > Testing: > The following new test covers this issue: > test/serviceability/jvmti/ModuleAwareAgents/ClassLoadPrepare/VMEarly > They are developed as a part of the following test enhancement: > 8150758 [TESTBUG] need jvmti tests for module aware agents > > Additionally, the following tests were run: > nsk.jvmti, nsk,jdi, jtreg com/sun/jdi and java/lang/instrument > > > Thanks, > Serguei From serguei.spitsyn at oracle.com Mon Sep 12 03:58:14 2016 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Sun, 11 Sep 2016 20:58:14 -0700 Subject: RFR (XS): 8165681 ClassLoad and ClassPrepare JVMTI events are missed in the start phase In-Reply-To: <4ffd4740-0df4-a103-4653-d085e5a98159@oracle.com> References: <45ce10ea-993f-6a07-3e8f-0222fb479566@oracle.com> <4ffd4740-0df4-a103-4653-d085e5a98159@oracle.com> Message-ID: <6d94edc0-1b32-f6b7-4499-7d4f55a37840@oracle.com> On 9/11/16 18:49, David Holmes wrote: > Hi Serguei, > > On 12/09/2016 11:15 AM, serguei.spitsyn at oracle.com wrote: >> Please, review the one-liner fix for: >> https://bugs.openjdk.java.net/browse/JDK-8165681 >> >> Webrev: >> http://cr.openjdk.java.net/~sspitsyn/webrevs/2016/hotspot/8165681-jvmti-events.hs1/ >> >> >> Summary: >> The JVMTI spec tells the ClassLoad and ClassPrepare events can be send >> during start phase. >> The new test below identified that they are missed because of this bug >> in JVMTI. >> The fix is to add the events to the set of the set of the >> EARLY_EVENT_BITS. > > Seems fine. Thank you for the review, David. > Good to see the additional test coverage. Thanks to D. Dmitriev for the additional coverage. Thanks, Serguei > > Thanks, > David > >> Testing: >> The following new test covers this issue: >> test/serviceability/jvmti/ModuleAwareAgents/ClassLoadPrepare/VMEarly >> They are developed as a part of the following test enhancement: >> 8150758 [TESTBUG] need jvmti tests for module aware agents >> >> Additionally, the following tests were run: >> nsk.jvmti, nsk,jdi, jtreg com/sun/jdi and java/lang/instrument >> >> >> Thanks, >> Serguei From shafi.s.ahmad at oracle.com Mon Sep 12 06:55:25 2016 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Sun, 11 Sep 2016 23:55:25 -0700 (PDT) Subject: [8u] RFR for JDK-8154324: Request to backport JDK-6515172 to 8u In-Reply-To: References: <60e87114-fcc2-4a0f-a31b-8440faf91555@default> <3695d333-eb30-3a84-9b8e-b0264a9087c9@oracle.com> <16e8bd9c-38bd-4d56-a969-00acc3fda174@default> Message-ID: <75629554-9cbd-4eea-8c00-2cad035f5dc3@default> Thanks David for reviewing it. May I have one more thumps up. Regards, Shafi > -----Original Message----- > From: David Holmes > Sent: Monday, September 12, 2016 12:21 PM > To: Shafi Ahmad > Subject: Re: [8u] RFR for JDK-8154324: Request to backport JDK-6515172 to > 8u > > > > On 12/09/2016 4:17 PM, Shafi Ahmad wrote: > > Hi David, > > > >> -----Original Message----- > >> From: David Holmes > >> Sent: Monday, September 12, 2016 10:52 AM > >> To: Shafi Ahmad > >> Subject: Re: [8u] RFR for JDK-8154324: Request to backport > >> JDK-6515172 to 8u > >> > >> Hi Shafi, > >> > >> On 12/09/2016 2:54 PM, Shafi Ahmad wrote: > >>> Hi David, > >>> > >>> Thanks for looking into it. > >>> From the comment of Andreas in the bugdb bug > >> > https://bug.oraclecorp.com/pls/bug/webbug_print.show?c_rptno=23026050 > >> it appears to me he has resolved the issues but I am not sure. > >> > >> I'm not sure either. I don't know if this reduced form is what was > >> need to avoid the glibc issues, or whether there may still be issues. > > > > From his comment > > Internal notes: > > We had some issues with this backport. We are building with glibc > 2.8, while > > our minimum supported system runs with glibc 2.4. > > The backport will have to be adjusted, I'll upload a patch that should > work, > > but I've not tested it extensively. > > After this comment he had attached the diff [in the bugdb] so it looks like > diff file attached in the bugdb bug is free from libc issue. > > > > In the diff file I am seeing only few libc functions: > > 1. sched_getaffinity > > Versions : The CPU affinity system calls were introduced in Linux kernel > 2.5.8. The system call wrappers were introduced in glibc 2.3. Initially, the glibc > interfaces included a cpusetsize argument, typed as unsigned int. In glibc > 2.3.3, the cpusetsize argument was removed, but was then restored in glibc > 2.3.4, with type size_t. > > > > 2. CPU_ISSET > > Versions: The CPU_ZERO(), CPU_SET(), CPU_CLR(), and CPU_ISSET() > macros were added in glibc 2.3.3. > > All these functions are added in glibc 2.3 and our minimum supported > version is 2.4 so its fine. > > Ok. > > >> > >>> Yes you are right there are some unused code and unrelated comment > >>> in > >> the attached webrev like: > >>> > diagnostic(bool, UseCpuAllocPath, false, > + // If it appears > >>> there may be more than 1024 processors then we do a I am sorry I > >>> should verify it before sending it for review. I will remove these. > >>> > >>> Somehow we have missed this issue for long and customer needs the > >>> fix in > >> Oct release so it will be great if we can resolve it earliest. > >> > >> What form of the fix has the customer tested? > > I am not aware of it whether customer has tested this fix or not. There is no > such update in the bug. > > But the newly added test case fails without this fix and passes with this fix. > > Ok. Let's push ahead then. > > Thanks, > David > > > Regards, > > Shafi > > > >> > >> David > >> > >>> Regards, > >>> Shafi > >>> > >>>> -----Original Message----- > >>>> From: David Holmes > >>>> Sent: Friday, September 09, 2016 11:29 AM > >>>> To: Shafi Ahmad; hotspot-dev at openjdk.java.net > >>>> Subject: Re: [8u] RFR for JDK-8154324: Request to backport > >>>> JDK-6515172 to 8u > >>>> > >>>> Hi Shafi, > >>>> > >>>> On 9/09/2016 3:46 PM, Shafi Ahmad wrote: > >>>>> Hi, > >>>>> > >>>>> Please review the backport [modified] of bug: "JDK-6515172 > >>>> Runtime.availableProcessors() ignores Linux taskset command" to > jdk8u. > >>>>> > >>>>> Please note that the backport is not clean and we can't do as it is. > >>>>> Please > >>>> note > >>>>> 1. The changes are made by 'Andreas Eriksson' who left Oracle. > >>>>> 2. There is difference in logging mechanism in jdk9 and jdk8 is > >>>>> different > >>>> and file logTag.hpp doesn't exists in jdk8. > >>>>> 3. Newly added test pass after this change. It fails without this > change. > >>>>> > >>>>> Webrev link: > >> http://cr.openjdk.java.net/~shshahma/8154324/webrev.00/ > >>>>> Jdk9 bug: https://bugs.openjdk.java.net/browse/JDK-6515172 > >>>>> Jdk8 bug: https://bugs.openjdk.java.net/browse/JDK-8154324 > >>>>> Original patch pushed to jdk9: > >>>>> http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/c5480d4abfe4 > >>>> > >>>> I worked extensively with Andreas on this as there were a number of > >> issues. > >>>> I'll have to try and find those discussions to see where we ended up. > >>>> > >>>> The backport as stands is not quite appropriate. For example it adds: > >>>> > >>>> diagnostic(bool, UseCpuAllocPath, false, > >>>> > >>>> but that does not exist in the actual code for 8. > >>>> > >>>> Also this comment: > >>>> > >>>> + // If it appears there may be more than 1024 processors then we > >>>> + do a // dynamic check - see 6515172 for details. > >>>> > >>>> is wrong as there is no dynamic check in this version of the code. > >>>> > >>>> The last I recall with this is that there were issues caused by > >>>> building with one version of glibc and running (or trying to) on > >>>> later versions of glibc. But as I said I will have to see if I have > >>>> the discussion > >> from that effort. > >>>> > >>>> David > >>>> ---- > >>>> > >>>>> Testing: jprt and running jtreg. > >>>>> > >>>>> Regards, > >>>>> Shafi > >>>>> From doug.simon at oracle.com Mon Sep 12 09:26:29 2016 From: doug.simon at oracle.com (Doug Simon) Date: Mon, 12 Sep 2016 11:26:29 +0200 Subject: RFR: 8165755: [JVMCI] replace use of vm_abort with vm_exit In-Reply-To: <5d26a289-a262-5882-8061-3bbc4faa4540@oracle.com> References: <799CA13D-6BDC-4BF5-9241-515A684191F4@oracle.com> <6391B00B-AFBF-410C-A6A1-2ED95B35EBEB@twitter.com> <2F360221-7E6B-43BB-B69C-69A17777E5F2@oracle.com> <1c9b4d32-52a2-17a8-bfc5-f3cc7b494ba0@oracle.com> <5d26a289-a262-5882-8061-3bbc4faa4540@oracle.com> Message-ID: <52A83790-582E-4F8F-AA49-3F6DB3AB36CB@oracle.com> Hi David, > On 12 Sep 2016, at 03:24, David Holmes wrote: > > On 10/09/2016 6:29 AM, Coleen Phillimore wrote: >> On 9/9/16 3:31 PM, Doug Simon wrote: >>> Is vm_exit_during_initialization still the right choice when this >>> could be after VM initialization? JVMCI initialization is lazy and can >>> happen after the application has started. >> >> I don't actually know then. vm_exit tries to get threads to a safepoint >> first, but vm_abort(false) just shuts down the jvm. There aren't a lot >> of places we terminate the jvm. Maybe David Holmes knows. > > If JVMCI initialization is lazy and can happen after the application is started, why should it be a fatal error to fail to initialize it? How is the initialization triggered? Is there a synchronous call that could throw an exception? How does the user know that initialization failure will result in termination? JVMCI initialization is triggered by the first top tier compilation (when UseJVMCICompiler is true) or by an explicit request via the Java API to the JVMCI system. In the latter case, an exception is propagated to the caller who can decide what to do with it. However, in the former case, initialization is triggered by the CompileBroker on a compilation thread. We could make a failure during JVMCI initialization disable compilation but I?m sure that would be more confusing to a user than a VM exit clearly stating the invalid configuration option given. There?s already lazy configuration checking in other compilers that can cause the VM to exit (e.g. Compile::pd_compiler2_init in c2_init_x86.cpp) so this is not too different. > Can you not at least eagerly validate command-line options during VM initialization, even if you don't actually initialize JVMCI fully? Unfortunately not. The command line options are discovered (via a service loader) during initialization and this kind of activity is what we want to avoid during early VM startup. > > That aside it sounds like you want JVMCI initialization to do the equivalent of either Runtime.exit or Runtime.halt, depending on the exact termination semantics you want. If you call vm_exit directly then you get something like Runtime.halt Yes, we want Runtime.halt semantics in the case where invalid JVMCI options are specified. -Doug > David > ----- > >> Coleen >> >>> >>> Sent from my iPhone >>> >>>> On Sep 9, 2016, at 9:23 PM, Coleen Phillimore >>>> wrote: >>>> >>>> >>>> I think you want vm_exit_during_initialization() for that. Definitely >>>> not vm_abort, cause then it looks like an internal error/crash. >>>> >>>> Coleen >>>> >>>>> On 9/9/16 2:33 PM, Doug Simon wrote: >>>>> Can someone from the runtime team confirm that using vm_exit >>>>> (instead of vm_abort) is the best way to stop the VM when JVMCI >>>>> initialization fails (e.g., when invalid JVMCI options are provided >>>>> on the command line). Thanks! >>>>> >>>>> -Doug >>>>> >>>>> >>>>>> On 09 Sep 2016, at 19:48, Christian Thalinger >>>>>> wrote: >>>>>> >>>>>> I think this looks fine but maybe we should ask the runtime folks. >>>>>> >>>>>>> On Sep 8, 2016, at 11:01 PM, Doug Simon >>>>>>> wrote: >>>>>>> >>>>>>> Calling vm_abort from multiple threads can cause nasty crashes >>>>>>> such as double free errors. We've seen this in Graal during JVMCI >>>>>>> initialization when an unknown Graal option is encountered. >>>>>>> Multiple compiler threads try to initialize JVMCI which fails with >>>>>>> an exception indicating the bad option: >>>>>>> >>>>>>> Uncaught exception at >>>>>>> /scratch/graaluser/buildslave/buildlog/ci_executor/main/graal-jvmci-8/src/share/vm/jvmci/jvmciCompiler.cpp:127 >>>>>>> >>>>>>> java.lang.ExceptionInInitializerError >>>>>>> at >>>>>>> jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.runtime(HotSpotJVMCIRuntime.java:85) >>>>>>> >>>>>>> at jdk.vm.ci.runtime.JVMCI.initializeRuntime(Native Method) >>>>>>> at jdk.vm.ci.runtime.JVMCI.(JVMCI.java:58) >>>>>>> Caused by: java.lang.IllegalArgumentException: Could not find >>>>>>> option OptSomethingThatDoesNotExcist >>>>>>> at >>>>>>> com.oracle.graal.options.OptionsParser.parseOption(OptionsParser.java:134) >>>>>>> >>>>>>> at >>>>>>> com.oracle.graal.options.OptionsParser.parseOptions(OptionsParser.java:62) >>>>>>> >>>>>>> at >>>>>>> com.oracle.graal.hotspot.HotSpotGraalCompilerFactory.initializeOptions(HotSpotGraalCompilerFactory.java:156) >>>>>>> >>>>>>> at >>>>>>> com.oracle.graal.hotspot.HotSpotGraalCompilerFactory.onSelection(HotSpotGraalCompilerFactory.java:86) >>>>>>> >>>>>>> at >>>>>>> jdk.vm.ci.hotspot.HotSpotJVMCICompilerConfig.getCompilerFactory(HotSpotJVMCICompilerConfig.java:96) >>>>>>> >>>>>>> at >>>>>>> jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.(HotSpotJVMCIRuntime.java:277) >>>>>>> >>>>>>> at >>>>>>> jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.(HotSpotJVMCIRuntime.java:67) >>>>>>> >>>>>>> at >>>>>>> jdk.vm.ci.hotspot.HotSpotJVMCIRuntime$DelayedInit.(HotSpotJVMCIRuntime.java:75) >>>>>>> >>>>>>> at >>>>>>> jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.runtime(HotSpotJVMCIRuntime.java:85) >>>>>>> >>>>>>> at jdk.vm.ci.runtime.JVMCI.initializeRuntime(Native Method) >>>>>>> at jdk.vm.ci.runtime.JVMCI.(JVMCI.java:58) >>>>>>> >>>>>>> The native JVMCI code then tries to exit the VM by calling >>>>>>> vm_abort. If multiple compiler threads do this concurrently, >>>>>>> certain destructors can be called twice as shown by these thread >>>>>>> dumps: >>>>>>> >>>>>>> thread #26: tid = 0x0019, 0x00007fff84280124 >>>>>>> libsystem_malloc.dylib`szone_size + 227, stop reason = signal SIGSTOP >>>>>>> frame #0: 0x00007fff84280124 libsystem_malloc.dylib`szone_size + 227 >>>>>>> frame #1: 0x00007fff8427fed5 libsystem_malloc.dylib`free + 61 >>>>>>> frame #2: 0x000000010ac95963 >>>>>>> libjvm.dylib`os::free(memblock=0x00007fedc86226e0, >>>>>>> memflags=mtInternal) + 307 at os.cpp:711 >>>>>>> frame #3: 0x000000010a2afc54 >>>>>>> libjvm.dylib`FreeHeap(p=0x00007fedc86226e0, memflags=mtInternal) + >>>>>>> 52 at allocation.inline.hpp:93 >>>>>>> frame #4: 0x000000010acf0a9f >>>>>>> libjvm.dylib`PerfData::~PerfData(this=0x00007fedc8622650) + 63 at >>>>>>> perfData.cpp:116 >>>>>>> frame #5: 0x000000010acf0ae5 >>>>>>> libjvm.dylib`PerfData::~PerfData(this=0x00007fedc8622650) + 21 at >>>>>>> perfData.cpp:114 >>>>>>> frame #6: 0x000000010acf163d >>>>>>> libjvm.dylib`PerfDataManager::destroy() + 109 at perfData.cpp:287 >>>>>>> frame #7: 0x000000010acf3f4d libjvm.dylib`perfMemory_exit() + 61 >>>>>>> at perfMemory.cpp:74 >>>>>>> frame #8: 0x000000010ac9bb0d libjvm.dylib`os::shutdown() + 13 at >>>>>>> os_bsd.cpp:1130 >>>>>>> frame #9: 0x000000010ac9bb55 >>>>>>> libjvm.dylib`os::abort(dump_core=false) + 21 at os_bsd.cpp:1150 >>>>>>> frame #10: 0x000000010a9188e7 >>>>>>> libjvm.dylib`vm_abort(dump_core=false) + 39 at java.cpp:666 >>>>>>> frame #11: 0x000000010aa4f1e7 >>>>>>> libjvm.dylib`JVMCIRuntime::abort_on_pending_exception(exception=Handle >>>>>>> @ 0x000070000175b208, message="Uncaught exception at >>>>>>> /Users/dsimon/graal/graal-jvmci-8/src/share/vm/jvmci/jvmciCompiler.cpp:127", >>>>>>> dump_core=false) + 167 at jvmciRuntime.cpp:992 >>>>>>> frame #12: 0x000000010aa17017 >>>>>>> libjvm.dylib`JVMCICompiler::compile_method(this=0x00007fedcb203050, method=0x000070000175b8d8, >>>>>>> entry_bci=-1, env=0x000070000175b8f0) + 311 at jvmciCompiler.cpp:127 >>>>>>> frame #13: 0x000000010a656cd2 >>>>>>> libjvm.dylib`CompileBroker::invoke_compiler_on_method(task=0x00007fedc853ca30) >>>>>>> + 1314 at compileBroker.cpp:2207 >>>>>>> >>>>>>> thread #23: tid = 0x0016, 0x00007fff91fcb122 >>>>>>> libsystem_kernel.dylib`__semwait_signal_nocancel + 10, stop reason >>>>>>> = signal SIGSTOP >>>>>>> frame #0: 0x00007fff91fcb122 >>>>>>> libsystem_kernel.dylib`__semwait_signal_nocancel + 10 >>>>>>> frame #1: 0x00007fff9578c318 libsystem_c.dylib`nanosleep$NOCANCEL >>>>>>> + 188 >>>>>>> frame #2: 0x00007fff957b62ce libsystem_c.dylib`usleep$NOCANCEL + 54 >>>>>>> frame #3: 0x00007fff957e46e9 libsystem_c.dylib`abort + 139 >>>>>>> frame #4: 0x00007fff8428c396 libsystem_malloc.dylib`szone_error + 626 >>>>>>> frame #5: 0x000000010ac95963 >>>>>>> libjvm.dylib`os::free(memblock=0x00007fedc8601cd0, >>>>>>> memflags=mtInternal) + 307 at os.cpp:711 >>>>>>> frame #6: 0x000000010a2afc54 >>>>>>> libjvm.dylib`FreeHeap(p=0x00007fedc8601cd0, memflags=mtInternal) + >>>>>>> 52 at allocation.inline.hpp:93 >>>>>>> frame #7: 0x000000010acf0a9f >>>>>>> libjvm.dylib`PerfData::~PerfData(this=0x00007fedc8601c60) + 63 at >>>>>>> perfData.cpp:116 >>>>>>> frame #8: 0x000000010acf0ae5 >>>>>>> libjvm.dylib`PerfData::~PerfData(this=0x00007fedc8601c60) + 21 at >>>>>>> perfData.cpp:114 >>>>>>> frame #9: 0x000000010acf163d >>>>>>> libjvm.dylib`PerfDataManager::destroy() + 109 at perfData.cpp:287 >>>>>>> frame #10: 0x000000010acf3f4d libjvm.dylib`perfMemory_exit() + 61 >>>>>>> at perfMemory.cpp:74 >>>>>>> frame #11: 0x000000010ac9bb0d libjvm.dylib`os::shutdown() + 13 at >>>>>>> os_bsd.cpp:1130 >>>>>>> frame #12: 0x000000010ac9bb55 >>>>>>> libjvm.dylib`os::abort(dump_core=false) + 21 at os_bsd.cpp:1150 >>>>>>> frame #13: 0x000000010a9188e7 >>>>>>> libjvm.dylib`vm_abort(dump_core=false) + 39 at java.cpp:666 >>>>>>> frame #14: 0x000000010aa4f1e7 >>>>>>> libjvm.dylib`JVMCIRuntime::abort_on_pending_exception(exception=Handle >>>>>>> @ 0x0000700001452208, message="Uncaught exception at >>>>>>> /Users/dsimon/graal/graal-jvmci-8/src/share/vm/jvmci/jvmciCompiler.cpp:127", >>>>>>> dump_core=false) + 167 at jvmciRuntime.cpp:992 >>>>>>> frame #15: 0x000000010aa17017 >>>>>>> libjvm.dylib`JVMCICompiler::compile_method(this=0x00007fedcb203050, method=0x00007000014528d8, >>>>>>> entry_bci=-1, env=0x00007000014528f0) + 311 at jvmciCompiler.cpp:127 >>>>>>> frame #16: 0x000000010a656cd2 >>>>>>> libjvm.dylib`CompileBroker::invoke_compiler_on_method(task=0x00007fedc862a320) >>>>>>> + 1314 at compileBroker.cpp:2207 >>>>>>> >>>>>>> >>>>>>> This webrev replaces calls to vm_abort() with before_exit() + >>>>>>> vm_exit(). The latter is thread safe. >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165755 >>>>>>> http://cr.openjdk.java.net/~dnsimon/8165755/ >>>>>>> >>>>>>> -Doug >> From gromero at linux.vnet.ibm.com Mon Sep 12 12:48:21 2016 From: gromero at linux.vnet.ibm.com (Gustavo Romero) Date: Mon, 12 Sep 2016 09:48:21 -0300 Subject: RFR(XXS): 8164987: RTM jtreg tests failing due to unnamed module unable to access class jdk.internal.misc.Unsafe In-Reply-To: <57D1ED2E.5020408@linux.vnet.ibm.com> References: <57D1ED2E.5020408@linux.vnet.ibm.com> Message-ID: <57D6A415.3090804@linux.vnet.ibm.com> Hi Dmitry, Could you be the sponsor for that tiny change? It's related to what we've discussed here: http://mail.openjdk.java.net/pipermail/hotspot-dev/2016-August/024395.html I understand that is ok to get pushed even after rampdown phase 1, as per Mark's Process proposal: "P4-P5 bugs should, in general, be left to future releases unless they only affect documentation, demos, or tests, in which case they should be identified as such with the 'noreg-doc', 'noreg-demo', and 'noreg-self' labels, respectively" Thus I labeled that change as "noreg-self". Is my understanding correct? Thank you and best regards, Gustavo On 08-09-2016 19:58, Gustavo Romero wrote: > Hi, > > Could the following change be reviewed, please? > > bug: https://bugs.openjdk.java.net/browse/JDK-8164987 > webrev: http://cr.openjdk.java.net/~gromero/8164987/01/ > > > Thank you. > > Regards, > Gustavo > From shafi.s.ahmad at oracle.com Mon Sep 12 13:20:38 2016 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Mon, 12 Sep 2016 06:20:38 -0700 (PDT) Subject: [8u] RFR for JDK-8154324: Request to backport JDK-6515172 to 8u In-Reply-To: <3695d333-eb30-3a84-9b8e-b0264a9087c9@oracle.com> References: <60e87114-fcc2-4a0f-a31b-8440faf91555@default> <3695d333-eb30-3a84-9b8e-b0264a9087c9@oracle.com> Message-ID: Hi, Please find updated webrev: http://cr.openjdk.java.net/~shshahma/8154324/webrev.01/ I have incorporated both review comment by David. 1. Removed unrelated comment. 2. Removed unreferenced code ' diagnostic(bool, UseCpuAllocPath, false,' Regards, Shafi > -----Original Message----- > From: David Holmes > Sent: Friday, September 09, 2016 11:29 AM > To: Shafi Ahmad; hotspot-dev at openjdk.java.net > Subject: Re: [8u] RFR for JDK-8154324: Request to backport JDK-6515172 to > 8u > > Hi Shafi, > > On 9/09/2016 3:46 PM, Shafi Ahmad wrote: > > Hi, > > > > Please review the backport [modified] of bug: "JDK-6515172 > Runtime.availableProcessors() ignores Linux taskset command" to jdk8u. > > > > Please note that the backport is not clean and we can't do as it is. Please > note > > 1. The changes are made by 'Andreas Eriksson' who left Oracle. > > 2. There is difference in logging mechanism in jdk9 and jdk8 is different > and file logTag.hpp doesn't exists in jdk8. > > 3. Newly added test pass after this change. It fails without this change. > > > > Webrev link: http://cr.openjdk.java.net/~shshahma/8154324/webrev.00/ > > Jdk9 bug: https://bugs.openjdk.java.net/browse/JDK-6515172 > > Jdk8 bug: https://bugs.openjdk.java.net/browse/JDK-8154324 > > Original patch pushed to jdk9: > > http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/c5480d4abfe4 > > I worked extensively with Andreas on this as there were a number of issues. > I'll have to try and find those discussions to see where we ended up. > > The backport as stands is not quite appropriate. For example it adds: > > diagnostic(bool, UseCpuAllocPath, false, > > but that does not exist in the actual code for 8. > > Also this comment: > > + // If it appears there may be more than 1024 processors then we do a > + // dynamic check - see 6515172 for details. > > is wrong as there is no dynamic check in this version of the code. > > The last I recall with this is that there were issues caused by building with one > version of glibc and running (or trying to) on later versions of glibc. But as I > said I will have to see if I have the discussion from that effort. > > David > ---- > > > Testing: jprt and running jtreg. > > > > Regards, > > Shafi > > From christian.tornqvist at oracle.com Mon Sep 12 17:05:00 2016 From: christian.tornqvist at oracle.com (Christian Tornqvist) Date: Mon, 12 Sep 2016 13:05:00 -0400 Subject: RFR(XS): 8165881 - Backout JDK-8164913 Message-ID: <519601d20d17$cc979220$65c6b660$@oracle.com> Hi everyone, Please review this (clean) backout of the change for JDK-8164913, it failed several tests in the nightly testing. The failures are tracked in: https://bugs.openjdk.java.net/browse/JDK-8165869 Webrev: http://cr.openjdk.java.net/~ctornqvi/webrev/8165881/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8165881 Thanks, Christian From harold.seigel at oracle.com Mon Sep 12 17:11:23 2016 From: harold.seigel at oracle.com (harold seigel) Date: Mon, 12 Sep 2016 13:11:23 -0400 Subject: RFR(XS): 8165881 - Backout JDK-8164913 In-Reply-To: <519601d20d17$cc979220$65c6b660$@oracle.com> References: <519601d20d17$cc979220$65c6b660$@oracle.com> Message-ID: <14399a60-88ca-bcf4-0696-09f530aa2955@oracle.com> Looks good. Harold On 9/12/2016 1:05 PM, Christian Tornqvist wrote: > Hi everyone, > > > > Please review this (clean) backout of the change for JDK-8164913, it failed > several tests in the nightly testing. The failures are tracked in: > https://bugs.openjdk.java.net/browse/JDK-8165869 > > > > Webrev: > > http://cr.openjdk.java.net/~ctornqvi/webrev/8165881/webrev.00/ > > > > Bug: > > https://bugs.openjdk.java.net/browse/JDK-8165881 > > > > Thanks, > > Christian > > > From Derek.White at cavium.com Mon Sep 12 17:25:32 2016 From: Derek.White at cavium.com (White, Derek) Date: Mon, 12 Sep 2016 17:25:32 +0000 Subject: RFR: 8165808: Add release barriers when allocating objects with concurrent collection In-Reply-To: <6c991053-7347-cc6b-3b94-fc48cb27a1db@oracle.com> References: <13AE434E-FA26-4658-8A77-013462679EBC@oracle.com>, <6c991053-7347-cc6b-3b94-fc48cb27a1db@oracle.com> Message-ID: Hi David, The release_set_klass is really only used to create an ordering between the writes to the length field(s) and the klass field. And it's only ordered w-r-t readers that read via load_acquire (as you've been hunting down misuses of load_acquire and store_release!). Only CMS has a dependency on getting that ordering correct. There's a separate requirement: external threads don't see an object in a partially created state, if for example an object reference escapes before the object zeroing (and initialization?) have committed. The compilers and interpreter enforce the 2nd requirement via membars (unless they prove that they are unnecessary). There's a claim that the thread-state transitions will do enough membars to keep things safe. I have no opinion on this. Given that there are a huge number of unordered reads of the klass field, and only a few ordered reads (really only by CMS), I don't think a store_release is the right way to get what you want though. - Derek ________________________________ From: hotspot-dev on behalf of David Holmes Sent: Sunday, September 11, 2016 9:38 PM To: Kim Barrett; hotspot-dev Source Developers Subject: Re: RFR: 8165808: Add release barriers when allocating objects with concurrent collection Hi Kim, On 11/09/2016 8:59 AM, Kim Barrett wrote: > Please review this change to add release barriers when allocating > objects that may be visible to a concurrent collector. > > This change only addresses the allocation side, where the > initialization occurs; other subtasks of JDK-8160369 will address the > (mostly GC-specific) readers that need to accomodate in-progress > allocations. > > As part of this change, eliminated post_allocation_install_obj_klass, > which consisted of a call to [release_]set_klass + assertions that are > redundant with those in that called function. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8165808 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8165808/webrev.00/ src/share/vm/gc/shared/collectedHeap.inline.hpp Based on experiences with RMO architectures (ref JDK-8033920 - sorry it is a non-open bug report) I would expect the use of release_set_klass to be unconditional, not dependent on the nature of the GC. Publication of the object reference also has to be safe for other mutator threads. I'm not convinced by the brief argument in JDK-8033920 that thread-state transitions will always ensure visibility and ordering. Thanks, David > Testing: > Local jtreg tests. > From kim.barrett at oracle.com Mon Sep 12 18:08:28 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 12 Sep 2016 14:08:28 -0400 Subject: RFR: 8165808: Add release barriers when allocating objects with concurrent collection In-Reply-To: <6c991053-7347-cc6b-3b94-fc48cb27a1db@oracle.com> References: <13AE434E-FA26-4658-8A77-013462679EBC@oracle.com> <6c991053-7347-cc6b-3b94-fc48cb27a1db@oracle.com> Message-ID: <98CE5C19-7B72-4530-977C-C371164D439E@oracle.com> > On Sep 11, 2016, at 9:38 PM, David Holmes wrote: > > Hi Kim, > > On 11/09/2016 8:59 AM, Kim Barrett wrote: >> Please review this change to add release barriers when allocating >> objects that may be visible to a concurrent collector. >> >> This change only addresses the allocation side, where the >> initialization occurs; other subtasks of JDK-8160369 will address the >> (mostly GC-specific) readers that need to accomodate in-progress >> allocations. >> >> As part of this change, eliminated post_allocation_install_obj_klass, >> which consisted of a call to [release_]set_klass + assertions that are >> redundant with those in that called function. >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8165808 >> >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8165808/webrev.00/ > > src/share/vm/gc/shared/collectedHeap.inline.hpp > > Based on experiences with RMO architectures (ref JDK-8033920 - sorry it is a non-open bug report) I would expect the use of release_set_klass to be unconditional, not dependent on the nature of the GC. Publication of the object reference also has to be safe for other mutator threads. I'm not convinced by the brief argument in JDK-8033920 that thread-state transitions will always ensure visibility and ordering. There were two different issues discussed in (non-open) JDK-8033920. (1) Ensure setting the klass happens after some other parts of the object initialization, to ensure (some) concurrent GCs can parse the heap. (2) Ensure the klass is visible to other (mutator) threads if the object is accessible to those threads, e.g. if the object escapes from the allocating thread. JDK-8160369 is only about (1), and this change (for JDK-8165808) is part of fixing (1). I'm not trying to address (2) here, even assuming there are problems in that area. (I don't know that there *are* problems; I haven't explored that question carefully. I did find storestore barriers in the appropriate places in the bytecode interpreter, but I don't know how the reader side works, nor have I examined relevant parts of the compilers.) For (1), only concurrent GCs are affected. A purely STW GC can rely on safepointing to ensure the ordering. (Safepointing also immunizes STW GCs from (2).) The parsability constraint for CMS and G1 is a consequence of concurrent processing of dirty card information (CMS precleaning and G1 concurrent refinement), which needs to be able to find object boundaries in the regions covered by marked cards. Hence, for both CMS and G1, only allocations directly in the old generation need special care. And for G1, only humongous objects are directly allocated in the old-gen. [I don't know enough about Shenandoah to know whether it has any similar requirements.] This is the rationale for the conditionalization on INCLUDE_ALL_GCS. It is also the rationale for only doing the release_set_klass in the collectedHeap.inline.hpp code paths, and not the TLAB / Eden allocation paths. It would be possible to add runtime conditionalization, but that could quickly add more cost than just executing the barrier unconditionally. Especially on TSO platforms :-) And these are slow-path allocations, used only when TLAB / Eden allocation is disallowed or fails, so an unnecessary release barrier shouldn't have much overall impact even on non-TSO platforms. Hence keeping it simple. From kim.barrett at oracle.com Mon Sep 12 18:13:34 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 12 Sep 2016 14:13:34 -0400 Subject: Request for review (s) - 8155948: Add message for CMS deprecation for some internal builds In-Reply-To: <7618399f-8de9-2faf-00ad-8eb17aae1016@oracle.com> References: <74eccd9a-1525-a9ef-1608-294de99d4483@oracle.com> <52db37a5-efa4-79dd-771f-45e1fc1ccaf5@oracle.com> <02968df4-b53d-c1d1-65e4-a612ca393bae@oracle.com> <5CB1B554-D733-40D6-881F-6237F9CE9E11@oracle.com> <50bac55e-b92c-f5d5-cc29-f0a85fdc6923@oracle.com> <7618399f-8de9-2faf-00ad-8eb17aae1016@oracle.com> Message-ID: > On Sep 9, 2016, at 10:47 AM, Jon Masamitsu wrote: > > Sorry for dragging this out but I needed to make 1 change. I should > have been using warning() and not log_warning(gc). Diff is > > diff --git a/src/share/vm/runtime/arguments.cpp b/src/share/vm/runtime/arguments.cpp > --- a/src/share/vm/runtime/arguments.cpp > +++ b/src/share/vm/runtime/arguments.cpp > @@ -4065,7 +4065,7 @@ > if (log_is_enabled(Warning, gc) && > lookup_special_flag(flag_name, flag)) { > handle_aliases_and_deprecation(flag_name, /* print warning */ true); > - log_warning(gc)("%s", msg); > + warning("%s", msg); > } > } > > The difference is that warning() honors PrintWarnings. In that case, shouldn?t the ?log_is_enabled(Warning, gc)? instead be ?PrintWarnings?? From jon.masamitsu at oracle.com Mon Sep 12 18:01:21 2016 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Mon, 12 Sep 2016 11:01:21 -0700 Subject: Request for review (s) - 8155948: Add message for CMS deprecation for some internal builds In-Reply-To: References: <74eccd9a-1525-a9ef-1608-294de99d4483@oracle.com> <52db37a5-efa4-79dd-771f-45e1fc1ccaf5@oracle.com> <02968df4-b53d-c1d1-65e4-a612ca393bae@oracle.com> <5CB1B554-D733-40D6-881F-6237F9CE9E11@oracle.com> <50bac55e-b92c-f5d5-cc29-f0a85fdc6923@oracle.com> <7618399f-8de9-2faf-00ad-8eb17aae1016@oracle.com> Message-ID: <84397208-927f-204a-da31-6b48026f96b9@oracle.com> On 09/12/2016 11:13 AM, Kim Barrett wrote: >> On Sep 9, 2016, at 10:47 AM, Jon Masamitsu wrote: >> >> Sorry for dragging this out but I needed to make 1 change. I should >> have been using warning() and not log_warning(gc). Diff is >> >> diff --git a/src/share/vm/runtime/arguments.cpp b/src/share/vm/runtime/arguments.cpp >> --- a/src/share/vm/runtime/arguments.cpp >> +++ b/src/share/vm/runtime/arguments.cpp >> @@ -4065,7 +4065,7 @@ >> if (log_is_enabled(Warning, gc) && >> lookup_special_flag(flag_name, flag)) { >> handle_aliases_and_deprecation(flag_name, /* print warning */ true); >> - log_warning(gc)("%s", msg); >> + warning("%s", msg); >> } >> } >> >> The difference is that warning() honors PrintWarnings. > In that case, shouldn?t the ?log_is_enabled(Warning, gc)? instead be ?PrintWarnings?? > I should probably remove the ?log_is_enabled(Warning, gc)? and rely on the check of PrintWarnings in warning(). Let me try that and report back. Jon From david.holmes at oracle.com Mon Sep 12 21:15:47 2016 From: david.holmes at oracle.com (David Holmes) Date: Tue, 13 Sep 2016 07:15:47 +1000 Subject: RFR(XS): 8165881 - Backout JDK-8164913 In-Reply-To: <14399a60-88ca-bcf4-0696-09f530aa2955@oracle.com> References: <519601d20d17$cc979220$65c6b660$@oracle.com> <14399a60-88ca-bcf4-0696-09f530aa2955@oracle.com> Message-ID: <97d8d6c3-2f4d-1475-f9b0-e608878e5aac@oracle.com> For the record it looks like the tests involved are broken, in that because the VM used to lie about the success of attaching the agent, the tests expected different exceptions to occur. David On 13/09/2016 3:11 AM, harold seigel wrote: > Looks good. > > Harold > > > On 9/12/2016 1:05 PM, Christian Tornqvist wrote: >> Hi everyone, >> >> >> Please review this (clean) backout of the change for JDK-8164913, it >> failed >> several tests in the nightly testing. The failures are tracked in: >> https://bugs.openjdk.java.net/browse/JDK-8165869 >> >> >> Webrev: >> >> http://cr.openjdk.java.net/~ctornqvi/webrev/8165881/webrev.00/ >> >> >> Bug: >> >> https://bugs.openjdk.java.net/browse/JDK-8165881 >> >> >> Thanks, >> >> Christian >> >> > From jon.masamitsu at oracle.com Mon Sep 12 21:05:18 2016 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Mon, 12 Sep 2016 14:05:18 -0700 Subject: Request for review (s) - 8155948: Add message for CMS deprecation for some internal builds In-Reply-To: References: <74eccd9a-1525-a9ef-1608-294de99d4483@oracle.com> <52db37a5-efa4-79dd-771f-45e1fc1ccaf5@oracle.com> <02968df4-b53d-c1d1-65e4-a612ca393bae@oracle.com> <5CB1B554-D733-40D6-881F-6237F9CE9E11@oracle.com> <50bac55e-b92c-f5d5-cc29-f0a85fdc6923@oracle.com> <7618399f-8de9-2faf-00ad-8eb17aae1016@oracle.com> Message-ID: On 09/12/2016 11:13 AM, Kim Barrett wrote: >> On Sep 9, 2016, at 10:47 AM, Jon Masamitsu wrote: >> >> Sorry for dragging this out but I needed to make 1 change. I should >> have been using warning() and not log_warning(gc). Diff is >> >> diff --git a/src/share/vm/runtime/arguments.cpp b/src/share/vm/runtime/arguments.cpp >> --- a/src/share/vm/runtime/arguments.cpp >> +++ b/src/share/vm/runtime/arguments.cpp >> @@ -4065,7 +4065,7 @@ >> if (log_is_enabled(Warning, gc) && >> lookup_special_flag(flag_name, flag)) { >> handle_aliases_and_deprecation(flag_name, /* print warning */ true); >> - log_warning(gc)("%s", msg); >> + warning("%s", msg); >> } >> } >> >> The difference is that warning() honors PrintWarnings. > In that case, shouldn?t the ?log_is_enabled(Warning, gc)? instead be ?PrintWarnings?? > Simply deleting the log_is_enabled() call does result in the PrintWarnings being honored. The latest change is --- a/src/share/vm/runtime/arguments.cpp +++ b/src/share/vm/runtime/arguments.cpp @@ -4167,8 +4167,7 @@ void Arguments::handle_extra_cms_flags(const char* msg) { SpecialFlag flag; const char *flag_name = "UseConcMarkSweepGC"; - if (log_is_enabled(Warning, gc) && - lookup_special_flag(flag_name, flag)) { + if (lookup_special_flag(flag_name, flag)) { handle_aliases_and_deprecation(flag_name, /* print warning */ true); warning("%s", msg); } Complete webrev is http://cr.openjdk.java.net/~jmasa/8155948/webrev.03/index.html Thanks. Jon From kim.barrett at oracle.com Mon Sep 12 21:40:16 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 12 Sep 2016 17:40:16 -0400 Subject: Request for review (s) - 8155948: Add message for CMS deprecation for some internal builds In-Reply-To: References: <74eccd9a-1525-a9ef-1608-294de99d4483@oracle.com> <52db37a5-efa4-79dd-771f-45e1fc1ccaf5@oracle.com> <02968df4-b53d-c1d1-65e4-a612ca393bae@oracle.com> <5CB1B554-D733-40D6-881F-6237F9CE9E11@oracle.com> <50bac55e-b92c-f5d5-cc29-f0a85fdc6923@oracle.com> <7618399f-8de9-2faf-00ad-8eb17aae1016@oracle.com> Message-ID: <5F3DCD18-707F-4C3D-BF29-D26CD84AC3AA@oracle.com> > On Sep 12, 2016, at 5:05 PM, Jon Masamitsu wrote: > Simply deleting the log_is_enabled() call does result in the PrintWarnings being > honored. The latest change is > > --- a/src/share/vm/runtime/arguments.cpp > +++ b/src/share/vm/runtime/arguments.cpp > @@ -4167,8 +4167,7 @@ > void Arguments::handle_extra_cms_flags(const char* msg) { > SpecialFlag flag; > const char *flag_name = "UseConcMarkSweepGC"; > - if (log_is_enabled(Warning, gc) && > - lookup_special_flag(flag_name, flag)) { > + if (lookup_special_flag(flag_name, flag)) { > handle_aliases_and_deprecation(flag_name, /* print warning */ true); > warning("%s", msg); > } > > Complete webrev is > > http://cr.openjdk.java.net/~jmasa/8155948/webrev.03/index.html > > Thanks. > > Jon Looks good. From jon.masamitsu at oracle.com Mon Sep 12 21:53:00 2016 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Mon, 12 Sep 2016 14:53:00 -0700 Subject: RFR: 8165808: Add release barriers when allocating objects with concurrent collection In-Reply-To: <98CE5C19-7B72-4530-977C-C371164D439E@oracle.com> References: <13AE434E-FA26-4658-8A77-013462679EBC@oracle.com> <6c991053-7347-cc6b-3b94-fc48cb27a1db@oracle.com> <98CE5C19-7B72-4530-977C-C371164D439E@oracle.com> Message-ID: On 09/12/2016 11:08 AM, Kim Barrett wrote: >> On Sep 11, 2016, at 9:38 PM, David Holmes wrote: >> >> Hi Kim, >> >> On 11/09/2016 8:59 AM, Kim Barrett wrote: >>> Please review this change to add release barriers when allocating >>> objects that may be visible to a concurrent collector. >>> >>> This change only addresses the allocation side, where the >>> initialization occurs; other subtasks of JDK-8160369 will address the >>> (mostly GC-specific) readers that need to accomodate in-progress >>> allocations. >>> >>> As part of this change, eliminated post_allocation_install_obj_klass, >>> which consisted of a call to [release_]set_klass + assertions that are >>> redundant with those in that called function. >>> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8165808 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~kbarrett/8165808/webrev.00/ >> src/share/vm/gc/shared/collectedHeap.inline.hpp >> >> Based on experiences with RMO architectures (ref JDK-8033920 - sorry it is a non-open bug report) I would expect the use of release_set_klass to be unconditional, not dependent on the nature of the GC. Publication of the object reference also has to be safe for other mutator threads. I'm not convinced by the brief argument in JDK-8033920 that thread-state transitions will always ensure visibility and ordering. > There were two different issues discussed in (non-open) JDK-8033920. > > (1) Ensure setting the klass happens after some other parts of the > object initialization, to ensure (some) concurrent GCs can parse the > heap. > > (2) Ensure the klass is visible to other (mutator) threads if the > object is accessible to those threads, e.g. if the object escapes from > the allocating thread. > > JDK-8160369 is only about (1), and this change (for JDK-8165808) is > part of fixing (1). I'm not trying to address (2) here, even assuming > there are problems in that area. (I don't know that there *are* > problems; I haven't explored that question carefully. I did find > storestore barriers in the appropriate places in the bytecode > interpreter, but I don't know how the reader side works, nor have I > examined relevant parts of the compilers.) In the above context (release store part of 8165808), changes look good. Jon > > For (1), only concurrent GCs are affected. A purely STW GC can rely > on safepointing to ensure the ordering. (Safepointing also immunizes > STW GCs from (2).) > > The parsability constraint for CMS and G1 is a consequence of > concurrent processing of dirty card information (CMS precleaning and > G1 concurrent refinement), which needs to be able to find object > boundaries in the regions covered by marked cards. Hence, for both > CMS and G1, only allocations directly in the old generation need > special care. And for G1, only humongous objects are directly > allocated in the old-gen. > > [I don't know enough about Shenandoah to know whether it has any > similar requirements.] > > This is the rationale for the conditionalization on INCLUDE_ALL_GCS. > > It is also the rationale for only doing the release_set_klass in the > collectedHeap.inline.hpp code paths, and not the TLAB / Eden > allocation paths. > > It would be possible to add runtime conditionalization, but that could > quickly add more cost than just executing the barrier unconditionally. > Especially on TSO platforms :-) And these are slow-path allocations, > used only when TLAB / Eden allocation is disallowed or fails, so an > unnecessary release barrier shouldn't have much overall impact even on > non-TSO platforms. Hence keeping it simple. > From kim.barrett at oracle.com Mon Sep 12 23:06:51 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 12 Sep 2016 19:06:51 -0400 Subject: RFR: 8165808: Add release barriers when allocating objects with concurrent collection In-Reply-To: References: <13AE434E-FA26-4658-8A77-013462679EBC@oracle.com> <6c991053-7347-cc6b-3b94-fc48cb27a1db@oracle.com> <98CE5C19-7B72-4530-977C-C371164D439E@oracle.com> Message-ID: <7909C60E-07FC-428C-ABA0-38C182110F58@oracle.com> > On Sep 12, 2016, at 5:53 PM, Jon Masamitsu wrote: > > > > On 09/12/2016 11:08 AM, Kim Barrett wrote: >>> On Sep 11, 2016, at 9:38 PM, David Holmes wrote: >>> >>> Hi Kim, >>> >>> On 11/09/2016 8:59 AM, Kim Barrett wrote: >>>> Please review this change to add release barriers when allocating >>>> objects that may be visible to a concurrent collector. >>>> >>>> This change only addresses the allocation side, where the >>>> initialization occurs; other subtasks of JDK-8160369 will address the >>>> (mostly GC-specific) readers that need to accomodate in-progress >>>> allocations. >>>> >>>> As part of this change, eliminated post_allocation_install_obj_klass, >>>> which consisted of a call to [release_]set_klass + assertions that are >>>> redundant with those in that called function. >>>> >>>> CR: >>>> https://bugs.openjdk.java.net/browse/JDK-8165808 >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~kbarrett/8165808/webrev.00/ >>> src/share/vm/gc/shared/collectedHeap.inline.hpp >>> >>> Based on experiences with RMO architectures (ref JDK-8033920 - sorry it is a non-open bug report) I would expect the use of release_set_klass to be unconditional, not dependent on the nature of the GC. Publication of the object reference also has to be safe for other mutator threads. I'm not convinced by the brief argument in JDK-8033920 that thread-state transitions will always ensure visibility and ordering. >> There were two different issues discussed in (non-open) JDK-8033920. >> >> (1) Ensure setting the klass happens after some other parts of the >> object initialization, to ensure (some) concurrent GCs can parse the >> heap. >> >> (2) Ensure the klass is visible to other (mutator) threads if the >> object is accessible to those threads, e.g. if the object escapes from >> the allocating thread. >> >> JDK-8160369 is only about (1), and this change (for JDK-8165808) is >> part of fixing (1). I'm not trying to address (2) here, even assuming >> there are problems in that area. (I don't know that there *are* >> problems; I haven't explored that question carefully. I did find >> storestore barriers in the appropriate places in the bytecode >> interpreter, but I don't know how the reader side works, nor have I >> examined relevant parts of the compilers.) > > In the above context (release store part of 8165808), changes look > good. > > Jon Thanks. > >> >> For (1), only concurrent GCs are affected. A purely STW GC can rely >> on safepointing to ensure the ordering. (Safepointing also immunizes >> STW GCs from (2).) >> >> The parsability constraint for CMS and G1 is a consequence of >> concurrent processing of dirty card information (CMS precleaning and >> G1 concurrent refinement), which needs to be able to find object >> boundaries in the regions covered by marked cards. Hence, for both >> CMS and G1, only allocations directly in the old generation need >> special care. And for G1, only humongous objects are directly >> allocated in the old-gen. >> >> [I don't know enough about Shenandoah to know whether it has any >> similar requirements.] >> >> This is the rationale for the conditionalization on INCLUDE_ALL_GCS. >> >> It is also the rationale for only doing the release_set_klass in the >> collectedHeap.inline.hpp code paths, and not the TLAB / Eden >> allocation paths. >> >> It would be possible to add runtime conditionalization, but that could >> quickly add more cost than just executing the barrier unconditionally. >> Especially on TSO platforms :-) And these are slow-path allocations, >> used only when TLAB / Eden allocation is disallowed or fails, so an >> unnecessary release barrier shouldn't have much overall impact even on >> non-TSO platforms. Hence keeping it simple. From david.holmes at oracle.com Mon Sep 12 23:22:38 2016 From: david.holmes at oracle.com (David Holmes) Date: Tue, 13 Sep 2016 09:22:38 +1000 Subject: RFR: 8165808: Add release barriers when allocating objects with concurrent collection In-Reply-To: <98CE5C19-7B72-4530-977C-C371164D439E@oracle.com> References: <13AE434E-FA26-4658-8A77-013462679EBC@oracle.com> <6c991053-7347-cc6b-3b94-fc48cb27a1db@oracle.com> <98CE5C19-7B72-4530-977C-C371164D439E@oracle.com> Message-ID: Hi Kim, On 13/09/2016 4:08 AM, Kim Barrett wrote: >> On Sep 11, 2016, at 9:38 PM, David Holmes wrote: >> >> Hi Kim, >> >> On 11/09/2016 8:59 AM, Kim Barrett wrote: >>> Please review this change to add release barriers when allocating >>> objects that may be visible to a concurrent collector. >>> >>> This change only addresses the allocation side, where the >>> initialization occurs; other subtasks of JDK-8160369 will address the >>> (mostly GC-specific) readers that need to accomodate in-progress >>> allocations. >>> >>> As part of this change, eliminated post_allocation_install_obj_klass, >>> which consisted of a call to [release_]set_klass + assertions that are >>> redundant with those in that called function. >>> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8165808 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~kbarrett/8165808/webrev.00/ >> >> src/share/vm/gc/shared/collectedHeap.inline.hpp >> >> Based on experiences with RMO architectures (ref JDK-8033920 - sorry it is a non-open bug report) I would expect the use of release_set_klass to be unconditional, not dependent on the nature of the GC. Publication of the object reference also has to be safe for other mutator threads. I'm not convinced by the brief argument in JDK-8033920 that thread-state transitions will always ensure visibility and ordering. > > There were two different issues discussed in (non-open) JDK-8033920. > > (1) Ensure setting the klass happens after some other parts of the > object initialization, to ensure (some) concurrent GCs can parse the > heap. > > (2) Ensure the klass is visible to other (mutator) threads if the > object is accessible to those threads, e.g. if the object escapes from > the allocating thread. > > JDK-8160369 is only about (1), and this change (for JDK-8165808) is > part of fixing (1). I'm not trying to address (2) here, even assuming > there are problems in that area. (I don't know that there *are* I have my doubts about treating these two issues separately, but will not press the issue. For the general mutator path the devil is in the detail, and as Derek indicated if there is a problem there the solution would likely need the matching load-acquire. > problems; I haven't explored that question carefully. I did find > storestore barriers in the appropriate places in the bytecode > interpreter, but I don't know how the reader side works, nor have I > examined relevant parts of the compilers.) FWIW the presence of storestore barriers require further examination as their use tends to reflect analysis that didn't account for the reader side. But that's a different issue again. Thanks, David ----- > For (1), only concurrent GCs are affected. A purely STW GC can rely > on safepointing to ensure the ordering. (Safepointing also immunizes > STW GCs from (2).) > > The parsability constraint for CMS and G1 is a consequence of > concurrent processing of dirty card information (CMS precleaning and > G1 concurrent refinement), which needs to be able to find object > boundaries in the regions covered by marked cards. Hence, for both > CMS and G1, only allocations directly in the old generation need > special care. And for G1, only humongous objects are directly > allocated in the old-gen. > > [I don't know enough about Shenandoah to know whether it has any > similar requirements.] > > This is the rationale for the conditionalization on INCLUDE_ALL_GCS. > > It is also the rationale for only doing the release_set_klass in the > collectedHeap.inline.hpp code paths, and not the TLAB / Eden > allocation paths. > > It would be possible to add runtime conditionalization, but that could > quickly add more cost than just executing the barrier unconditionally. > Especially on TSO platforms :-) And these are slow-path allocations, > used only when TLAB / Eden allocation is disallowed or fails, so an > unnecessary release barrier shouldn't have much overall impact even on > non-TSO platforms. Hence keeping it simple. > From yasuenag at gmail.com Mon Sep 12 23:23:28 2016 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Tue, 13 Sep 2016 08:23:28 +0900 Subject: RFR(XS): 8165881 - Backout JDK-8164913 In-Reply-To: <97d8d6c3-2f4d-1475-f9b0-e608878e5aac@oracle.com> References: <519601d20d17$cc979220$65c6b660$@oracle.com> <14399a60-88ca-bcf4-0696-09f530aa2955@oracle.com> <97d8d6c3-2f4d-1475-f9b0-e608878e5aac@oracle.com> Message-ID: <8e7b6d80-7c1c-b432-2ef4-3b9aa27956d3@gmail.com> Hi, Hmm, it has been backouted... I agree to David. This error seems to be a test bug. Can you share the test? I want to evaluate it. I do not agree to fix this bug in JDK 10 because VM might lie when the JVMTI agent could not be attached. IMHO this bug should be fixed in 9 GA, and we should fix testcase(s). Thanks, Yasumasa On 2016/09/13 6:15, David Holmes wrote: > For the record it looks like the tests involved are broken, in that > because the VM used to lie about the success of attaching the agent, the > tests expected different exceptions to occur. > > David > > On 13/09/2016 3:11 AM, harold seigel wrote: >> Looks good. >> >> Harold >> >> >> On 9/12/2016 1:05 PM, Christian Tornqvist wrote: >>> Hi everyone, >>> >>> >>> Please review this (clean) backout of the change for JDK-8164913, it >>> failed >>> several tests in the nightly testing. The failures are tracked in: >>> https://bugs.openjdk.java.net/browse/JDK-8165869 >>> >>> >>> Webrev: >>> >>> http://cr.openjdk.java.net/~ctornqvi/webrev/8165881/webrev.00/ >>> >>> >>> Bug: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8165881 >>> >>> >>> Thanks, >>> >>> Christian >>> >>> >> From david.holmes at oracle.com Mon Sep 12 23:54:49 2016 From: david.holmes at oracle.com (David Holmes) Date: Tue, 13 Sep 2016 09:54:49 +1000 Subject: RFR(XS): 8165881 - Backout JDK-8164913 In-Reply-To: <8e7b6d80-7c1c-b432-2ef4-3b9aa27956d3@gmail.com> References: <519601d20d17$cc979220$65c6b660$@oracle.com> <14399a60-88ca-bcf4-0696-09f530aa2955@oracle.com> <97d8d6c3-2f4d-1475-f9b0-e608878e5aac@oracle.com> <8e7b6d80-7c1c-b432-2ef4-3b9aa27956d3@gmail.com> Message-ID: Hi Yasumasa, On 13/09/2016 9:23 AM, Yasumasa Suenaga wrote: > Hi, > > Hmm, it has been backouted... > > I agree to David. This error seems to be a test bug. > Can you share the test? I want to evaluate it. Sorry we can't share the tests. I took a quick look and it seems it may be related to the result code parsing (that I thought we ended up not touching!). Can you run a test of HotSpotVirtualMachine.loadAgent(null) and see what happens? We see AgentLoadException from the code that internally tries to load the "instrument" agent: Exception in thread "main" Failure: Unexpected exception during test execution: java.lang.InternalError: instrument library is missing in target VM ... Caused by: java.lang.InternalError: instrument library is missing in target VM ... Caused by: com.sun.tools.attach.AgentLoadException: Failed to load agent library > I do not agree to fix this bug in JDK 10 because VM might lie when the > JVMTI agent could not be attached. IMHO this bug should be fixed in 9 > GA, and we should fix testcase(s). I agree. It has to be noted that "we" failed to run all the appropriate tests before pushing this, which would have discovered the problem - assuming it is actually a problem with the change and not an unrelated issue in our testing environment. Unfortunately I have some high priority tasks to handle right now, so I can't go into this in any more depth at the moment. David ----- > > Thanks, > > Yasumasa > > > On 2016/09/13 6:15, David Holmes wrote: >> For the record it looks like the tests involved are broken, in that >> because the VM used to lie about the success of attaching the agent, the >> tests expected different exceptions to occur. >> >> David >> >> On 13/09/2016 3:11 AM, harold seigel wrote: >>> Looks good. >>> >>> Harold >>> >>> >>> On 9/12/2016 1:05 PM, Christian Tornqvist wrote: >>>> Hi everyone, >>>> >>>> >>>> Please review this (clean) backout of the change for JDK-8164913, it >>>> failed >>>> several tests in the nightly testing. The failures are tracked in: >>>> https://bugs.openjdk.java.net/browse/JDK-8165869 >>>> >>>> >>>> Webrev: >>>> >>>> http://cr.openjdk.java.net/~ctornqvi/webrev/8165881/webrev.00/ >>>> >>>> >>>> Bug: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8165881 >>>> >>>> >>>> Thanks, >>>> >>>> Christian >>>> >>>> >>> From david.holmes at oracle.com Tue Sep 13 00:00:56 2016 From: david.holmes at oracle.com (David Holmes) Date: Tue, 13 Sep 2016 10:00:56 +1000 Subject: RFR(XS): 8165881 - Backout JDK-8164913 In-Reply-To: References: <519601d20d17$cc979220$65c6b660$@oracle.com> <14399a60-88ca-bcf4-0696-09f530aa2955@oracle.com> <97d8d6c3-2f4d-1475-f9b0-e608878e5aac@oracle.com> <8e7b6d80-7c1c-b432-2ef4-3b9aa27956d3@gmail.com> Message-ID: <07965d03-1663-8da3-d126-aecbb93a7716@oracle.com> I think I see the problem. The attach framework tries to load the "instrument" agent library. Prior to 8164913 if this fails it actually appears to succeed. Now it fails, and that error propagates through. I'm guessing, at the moment, that the failure is due to a missing module related option. David On 13/09/2016 9:54 AM, David Holmes wrote: > Hi Yasumasa, > > On 13/09/2016 9:23 AM, Yasumasa Suenaga wrote: >> Hi, >> >> Hmm, it has been backouted... >> >> I agree to David. This error seems to be a test bug. >> Can you share the test? I want to evaluate it. > > Sorry we can't share the tests. I took a quick look and it seems it may > be related to the result code parsing (that I thought we ended up not > touching!). > > Can you run a test of HotSpotVirtualMachine.loadAgent(null) and see what > happens? We see AgentLoadException from the code that internally tries > to load the "instrument" agent: > > Exception in thread "main" Failure: Unexpected exception during test > execution: java.lang.InternalError: instrument library is missing in > target VM > ... > Caused by: java.lang.InternalError: instrument library is missing in > target VM > ... > Caused by: com.sun.tools.attach.AgentLoadException: Failed to load agent > library > > >> I do not agree to fix this bug in JDK 10 because VM might lie when the >> JVMTI agent could not be attached. IMHO this bug should be fixed in 9 >> GA, and we should fix testcase(s). > > I agree. It has to be noted that "we" failed to run all the appropriate > tests before pushing this, which would have discovered the problem - > assuming it is actually a problem with the change and not an unrelated > issue in our testing environment. > > Unfortunately I have some high priority tasks to handle right now, so I > can't go into this in any more depth at the moment. > > David > ----- > >> >> Thanks, >> >> Yasumasa >> >> >> On 2016/09/13 6:15, David Holmes wrote: >>> For the record it looks like the tests involved are broken, in that >>> because the VM used to lie about the success of attaching the agent, the >>> tests expected different exceptions to occur. >>> >>> David >>> >>> On 13/09/2016 3:11 AM, harold seigel wrote: >>>> Looks good. >>>> >>>> Harold >>>> >>>> >>>> On 9/12/2016 1:05 PM, Christian Tornqvist wrote: >>>>> Hi everyone, >>>>> >>>>> >>>>> Please review this (clean) backout of the change for JDK-8164913, it >>>>> failed >>>>> several tests in the nightly testing. The failures are tracked in: >>>>> https://bugs.openjdk.java.net/browse/JDK-8165869 >>>>> >>>>> >>>>> Webrev: >>>>> >>>>> http://cr.openjdk.java.net/~ctornqvi/webrev/8165881/webrev.00/ >>>>> >>>>> >>>>> Bug: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8165881 >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Christian >>>>> >>>>> >>>> From yasuenag at gmail.com Tue Sep 13 03:30:34 2016 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Tue, 13 Sep 2016 12:30:34 +0900 Subject: RFR(XS): 8165881 - Backout JDK-8164913 In-Reply-To: <07965d03-1663-8da3-d126-aecbb93a7716@oracle.com> References: <519601d20d17$cc979220$65c6b660$@oracle.com> <14399a60-88ca-bcf4-0696-09f530aa2955@oracle.com> <97d8d6c3-2f4d-1475-f9b0-e608878e5aac@oracle.com> <8e7b6d80-7c1c-b432-2ef4-3b9aa27956d3@gmail.com> <07965d03-1663-8da3-d126-aecbb93a7716@oracle.com> Message-ID: Hi, I could reproduce and I added a comment to JBS: https://bugs.openjdk.java.net/browse/JDK-8165869?focusedCommentId=14000623&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14000623 In case of BasicTests.java, I think it is a test bug. If it is accepted, I will upload webrev for this redo task. However, I cannot access (and fix) Oracle internal test. Can someone help me? (I cannot access JPRT. So I need a sponsor.) Thanks, Yasumasa On 2016/09/13 9:00, David Holmes wrote: > I think I see the problem. The attach framework tries to load the "instrument" agent library. Prior to 8164913 if this fails it actually appears to succeed. Now it fails, and that error propagates through. I'm guessing, at the moment, that the failure is due to a missing module related option. > > David > > On 13/09/2016 9:54 AM, David Holmes wrote: >> Hi Yasumasa, >> >> On 13/09/2016 9:23 AM, Yasumasa Suenaga wrote: >>> Hi, >>> >>> Hmm, it has been backouted... >>> >>> I agree to David. This error seems to be a test bug. >>> Can you share the test? I want to evaluate it. >> >> Sorry we can't share the tests. I took a quick look and it seems it may >> be related to the result code parsing (that I thought we ended up not >> touching!). >> >> Can you run a test of HotSpotVirtualMachine.loadAgent(null) and see what >> happens? We see AgentLoadException from the code that internally tries >> to load the "instrument" agent: >> >> Exception in thread "main" Failure: Unexpected exception during test >> execution: java.lang.InternalError: instrument library is missing in >> target VM >> ... >> Caused by: java.lang.InternalError: instrument library is missing in >> target VM >> ... >> Caused by: com.sun.tools.attach.AgentLoadException: Failed to load agent >> library >> >> >>> I do not agree to fix this bug in JDK 10 because VM might lie when the >>> JVMTI agent could not be attached. IMHO this bug should be fixed in 9 >>> GA, and we should fix testcase(s). >> >> I agree. It has to be noted that "we" failed to run all the appropriate >> tests before pushing this, which would have discovered the problem - >> assuming it is actually a problem with the change and not an unrelated >> issue in our testing environment. >> >> Unfortunately I have some high priority tasks to handle right now, so I >> can't go into this in any more depth at the moment. >> >> David >> ----- >> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> On 2016/09/13 6:15, David Holmes wrote: >>>> For the record it looks like the tests involved are broken, in that >>>> because the VM used to lie about the success of attaching the agent, the >>>> tests expected different exceptions to occur. >>>> >>>> David >>>> >>>> On 13/09/2016 3:11 AM, harold seigel wrote: >>>>> Looks good. >>>>> >>>>> Harold >>>>> >>>>> >>>>> On 9/12/2016 1:05 PM, Christian Tornqvist wrote: >>>>>> Hi everyone, >>>>>> >>>>>> >>>>>> Please review this (clean) backout of the change for JDK-8164913, it >>>>>> failed >>>>>> several tests in the nightly testing. The failures are tracked in: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8165869 >>>>>> >>>>>> >>>>>> Webrev: >>>>>> >>>>>> http://cr.openjdk.java.net/~ctornqvi/webrev/8165881/webrev.00/ >>>>>> >>>>>> >>>>>> Bug: >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8165881 >>>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Christian >>>>>> >>>>>> >>>>> From david.holmes at oracle.com Tue Sep 13 03:53:23 2016 From: david.holmes at oracle.com (David Holmes) Date: Tue, 13 Sep 2016 13:53:23 +1000 Subject: RFR(XS): 8165881 - Backout JDK-8164913 In-Reply-To: References: <519601d20d17$cc979220$65c6b660$@oracle.com> <14399a60-88ca-bcf4-0696-09f530aa2955@oracle.com> <97d8d6c3-2f4d-1475-f9b0-e608878e5aac@oracle.com> <8e7b6d80-7c1c-b432-2ef4-3b9aa27956d3@gmail.com> <07965d03-1663-8da3-d126-aecbb93a7716@oracle.com> Message-ID: <839d84a3-46e8-7c89-a1e7-ba03fa0ef98c@oracle.com> On 13/09/2016 1:30 PM, Yasumasa Suenaga wrote: > Hi, > > I could reproduce and I added a comment to JBS: > https://bugs.openjdk.java.net/browse/JDK-8165869?focusedCommentId=14000623&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14000623 > > > In case of BasicTests.java, I think it is a test bug. I don't agree as yet. I have not been able to isolate the exact difference between what happens with your fix and what happens without. I know it relates to the presence of the error code, but also how we get AgentInitializationException instead of AgentLoadException. I need to be able to run the test outside of jtreg but that is proving to be very difficult to arrange. :( David > If it is accepted, I will upload webrev for this redo task. > However, I cannot access (and fix) Oracle internal test. Can someone > help me? > (I cannot access JPRT. So I need a sponsor.) > > > Thanks, > > Yasumasa > > > On 2016/09/13 9:00, David Holmes wrote: >> I think I see the problem. The attach framework tries to load the >> "instrument" agent library. Prior to 8164913 if this fails it actually >> appears to succeed. Now it fails, and that error propagates through. >> I'm guessing, at the moment, that the failure is due to a missing >> module related option. >> >> David >> >> On 13/09/2016 9:54 AM, David Holmes wrote: >>> Hi Yasumasa, >>> >>> On 13/09/2016 9:23 AM, Yasumasa Suenaga wrote: >>>> Hi, >>>> >>>> Hmm, it has been backouted... >>>> >>>> I agree to David. This error seems to be a test bug. >>>> Can you share the test? I want to evaluate it. >>> >>> Sorry we can't share the tests. I took a quick look and it seems it may >>> be related to the result code parsing (that I thought we ended up not >>> touching!). >>> >>> Can you run a test of HotSpotVirtualMachine.loadAgent(null) and see what >>> happens? We see AgentLoadException from the code that internally tries >>> to load the "instrument" agent: >>> >>> Exception in thread "main" Failure: Unexpected exception during test >>> execution: java.lang.InternalError: instrument library is missing in >>> target VM >>> ... >>> Caused by: java.lang.InternalError: instrument library is missing in >>> target VM >>> ... >>> Caused by: com.sun.tools.attach.AgentLoadException: Failed to load agent >>> library >>> >>> >>>> I do not agree to fix this bug in JDK 10 because VM might lie when the >>>> JVMTI agent could not be attached. IMHO this bug should be fixed in 9 >>>> GA, and we should fix testcase(s). >>> >>> I agree. It has to be noted that "we" failed to run all the appropriate >>> tests before pushing this, which would have discovered the problem - >>> assuming it is actually a problem with the change and not an unrelated >>> issue in our testing environment. >>> >>> Unfortunately I have some high priority tasks to handle right now, so I >>> can't go into this in any more depth at the moment. >>> >>> David >>> ----- >>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> On 2016/09/13 6:15, David Holmes wrote: >>>>> For the record it looks like the tests involved are broken, in that >>>>> because the VM used to lie about the success of attaching the >>>>> agent, the >>>>> tests expected different exceptions to occur. >>>>> >>>>> David >>>>> >>>>> On 13/09/2016 3:11 AM, harold seigel wrote: >>>>>> Looks good. >>>>>> >>>>>> Harold >>>>>> >>>>>> >>>>>> On 9/12/2016 1:05 PM, Christian Tornqvist wrote: >>>>>>> Hi everyone, >>>>>>> >>>>>>> >>>>>>> Please review this (clean) backout of the change for JDK-8164913, it >>>>>>> failed >>>>>>> several tests in the nightly testing. The failures are tracked in: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165869 >>>>>>> >>>>>>> >>>>>>> Webrev: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~ctornqvi/webrev/8165881/webrev.00/ >>>>>>> >>>>>>> >>>>>>> Bug: >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165881 >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Christian >>>>>>> >>>>>>> >>>>>> From david.holmes at oracle.com Tue Sep 13 05:08:19 2016 From: david.holmes at oracle.com (David Holmes) Date: Tue, 13 Sep 2016 15:08:19 +1000 Subject: RFR(XS): 8165881 - Backout JDK-8164913 In-Reply-To: <839d84a3-46e8-7c89-a1e7-ba03fa0ef98c@oracle.com> References: <519601d20d17$cc979220$65c6b660$@oracle.com> <14399a60-88ca-bcf4-0696-09f530aa2955@oracle.com> <97d8d6c3-2f4d-1475-f9b0-e608878e5aac@oracle.com> <8e7b6d80-7c1c-b432-2ef4-3b9aa27956d3@gmail.com> <07965d03-1663-8da3-d126-aecbb93a7716@oracle.com> <839d84a3-46e8-7c89-a1e7-ba03fa0ef98c@oracle.com> Message-ID: On 13/09/2016 1:53 PM, David Holmes wrote: > On 13/09/2016 1:30 PM, Yasumasa Suenaga wrote: >> Hi, >> >> I could reproduce and I added a comment to JBS: >> https://bugs.openjdk.java.net/browse/JDK-8165869?focusedCommentId=14000623&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14000623 >> >> >> >> In case of BasicTests.java, I think it is a test bug. > > I don't agree as yet. I have not been able to isolate the exact > difference between what happens with your fix and what happens without. > I know it relates to the presence of the error code, but also how we get > AgentInitializationException instead of AgentLoadException. I need to be > able to run the test outside of jtreg but that is proving to be very > difficult to arrange. :( I finally managed to connect all the pieces on this. The basic problem is that with the proposed change VirtualMachineImpl.execute() will throw AgentLoadException, which then leads to the InternalError. Without the change we reach this block in HotspotVirtualMachine.loadAgentLibrary: int result = readInt(in); if (result != 0) { throw new AgentInitializationException("Agent_OnAttach failed", result); } and so get the expected AgentInitializationException. Looking at the proposed change in jvmtiExport.cpp if we have the original: st->print_cr("%d", result); result = JNI_OK; then execute() manages to read a zero completion status from the stream, and then loadAgentLibrary is able to read the actual "result" - whether zero or not - from the stream. This is because the AttachListener code calls op->complete(result, &st) where st is the stream where we wrote the result value above in print_cr. Then if we look at, for example, LinuxAttachOperation::complete, we will write "result" to the socket first, followed by the contents of st. Hence on a successful operation we can read 0 for execute, and then 0 for loadAgent. On error we read 0 for execute and the actual error code for loadAgent. With the proposed change execute() sees the real result (instead of JNI_OK) and so throws AgentLoadException. So in summary there are two different error indicators written into the stream, and we need the first to be zero unless some major error with the actual attach functionality occurred - otherwise even if the "load" failed in some other way, it is treated as a secondary error. With that in mind I suspect the real fix for the original issue is to have Dcmd recognize that it also has to read two "results". Though I'm not sure how exactly. David ----- > David > > > >> If it is accepted, I will upload webrev for this redo task. >> However, I cannot access (and fix) Oracle internal test. Can someone >> help me? >> (I cannot access JPRT. So I need a sponsor.) >> >> >> Thanks, >> >> Yasumasa >> >> >> On 2016/09/13 9:00, David Holmes wrote: >>> I think I see the problem. The attach framework tries to load the >>> "instrument" agent library. Prior to 8164913 if this fails it actually >>> appears to succeed. Now it fails, and that error propagates through. >>> I'm guessing, at the moment, that the failure is due to a missing >>> module related option. >>> >>> David >>> >>> On 13/09/2016 9:54 AM, David Holmes wrote: >>>> Hi Yasumasa, >>>> >>>> On 13/09/2016 9:23 AM, Yasumasa Suenaga wrote: >>>>> Hi, >>>>> >>>>> Hmm, it has been backouted... >>>>> >>>>> I agree to David. This error seems to be a test bug. >>>>> Can you share the test? I want to evaluate it. >>>> >>>> Sorry we can't share the tests. I took a quick look and it seems it may >>>> be related to the result code parsing (that I thought we ended up not >>>> touching!). >>>> >>>> Can you run a test of HotSpotVirtualMachine.loadAgent(null) and see >>>> what >>>> happens? We see AgentLoadException from the code that internally tries >>>> to load the "instrument" agent: >>>> >>>> Exception in thread "main" Failure: Unexpected exception during test >>>> execution: java.lang.InternalError: instrument library is missing in >>>> target VM >>>> ... >>>> Caused by: java.lang.InternalError: instrument library is missing in >>>> target VM >>>> ... >>>> Caused by: com.sun.tools.attach.AgentLoadException: Failed to load >>>> agent >>>> library >>>> >>>> >>>>> I do not agree to fix this bug in JDK 10 because VM might lie when the >>>>> JVMTI agent could not be attached. IMHO this bug should be fixed in 9 >>>>> GA, and we should fix testcase(s). >>>> >>>> I agree. It has to be noted that "we" failed to run all the appropriate >>>> tests before pushing this, which would have discovered the problem - >>>> assuming it is actually a problem with the change and not an unrelated >>>> issue in our testing environment. >>>> >>>> Unfortunately I have some high priority tasks to handle right now, so I >>>> can't go into this in any more depth at the moment. >>>> >>>> David >>>> ----- >>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2016/09/13 6:15, David Holmes wrote: >>>>>> For the record it looks like the tests involved are broken, in that >>>>>> because the VM used to lie about the success of attaching the >>>>>> agent, the >>>>>> tests expected different exceptions to occur. >>>>>> >>>>>> David >>>>>> >>>>>> On 13/09/2016 3:11 AM, harold seigel wrote: >>>>>>> Looks good. >>>>>>> >>>>>>> Harold >>>>>>> >>>>>>> >>>>>>> On 9/12/2016 1:05 PM, Christian Tornqvist wrote: >>>>>>>> Hi everyone, >>>>>>>> >>>>>>>> >>>>>>>> Please review this (clean) backout of the change for >>>>>>>> JDK-8164913, it >>>>>>>> failed >>>>>>>> several tests in the nightly testing. The failures are tracked in: >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165869 >>>>>>>> >>>>>>>> >>>>>>>> Webrev: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~ctornqvi/webrev/8165881/webrev.00/ >>>>>>>> >>>>>>>> >>>>>>>> Bug: >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165881 >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Christian >>>>>>>> >>>>>>>> >>>>>>> From dmitry.samersoff at oracle.com Tue Sep 13 08:06:45 2016 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Tue, 13 Sep 2016 11:06:45 +0300 Subject: RFR(XS): 8165881 - Backout JDK-8164913 In-Reply-To: References: <519601d20d17$cc979220$65c6b660$@oracle.com> <14399a60-88ca-bcf4-0696-09f530aa2955@oracle.com> <97d8d6c3-2f4d-1475-f9b0-e608878e5aac@oracle.com> <8e7b6d80-7c1c-b432-2ef4-3b9aa27956d3@gmail.com> <07965d03-1663-8da3-d126-aecbb93a7716@oracle.com> <839d84a3-46e8-7c89-a1e7-ba03fa0ef98c@oracle.com> Message-ID: David, Thank you for the evaluation. > With that in mind I suspect the real fix for the original issue is to > have Dcmd recognize that it also has to read two "results". Though I'm > not sure how exactly. This logic seems completely broken for me. But I don't see anything we can do right now - for jdk 9. It requires careful changes of both - code and tests. I can help with this task but not today. -Dmitry On 2016-09-13 08:08, David Holmes wrote: > On 13/09/2016 1:53 PM, David Holmes wrote: >> On 13/09/2016 1:30 PM, Yasumasa Suenaga wrote: >>> Hi, >>> >>> I could reproduce and I added a comment to JBS: >>> https://bugs.openjdk.java.net/browse/JDK-8165869?focusedCommentId=14000623&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14000623 >>> >>> >>> >>> >>> In case of BasicTests.java, I think it is a test bug. >> >> I don't agree as yet. I have not been able to isolate the exact >> difference between what happens with your fix and what happens without. >> I know it relates to the presence of the error code, but also how we get >> AgentInitializationException instead of AgentLoadException. I need to be >> able to run the test outside of jtreg but that is proving to be very >> difficult to arrange. :( > > I finally managed to connect all the pieces on this. > > The basic problem is that with the proposed change > VirtualMachineImpl.execute() will throw AgentLoadException, which then > leads to the InternalError. Without the change we reach this block in > HotspotVirtualMachine.loadAgentLibrary: > > int result = readInt(in); > if (result != 0) { > throw new AgentInitializationException("Agent_OnAttach failed", result); > } > > and so get the expected AgentInitializationException. > > Looking at the proposed change in jvmtiExport.cpp if we have the original: > > st->print_cr("%d", result); > result = JNI_OK; > > then execute() manages to read a zero completion status from the stream, > and then loadAgentLibrary is able to read the actual "result" - whether > zero or not - from the stream. This is because the AttachListener code > calls op->complete(result, &st) where st is the stream where we wrote > the result value above in print_cr. Then if we look at, for example, > LinuxAttachOperation::complete, we will write "result" to the socket > first, followed by the contents of st. Hence on a successful operation > we can read 0 for execute, and then 0 for loadAgent. On error we read 0 > for execute and the actual error code for loadAgent. With the proposed > change execute() sees the real result (instead of JNI_OK) and so throws > AgentLoadException. > > So in summary there are two different error indicators written into the > stream, and we need the first to be zero unless some major error with > the actual attach functionality occurred - otherwise even if the "load" > failed in some other way, it is treated as a secondary error. > > With that in mind I suspect the real fix for the original issue is to > have Dcmd recognize that it also has to read two "results". Though I'm > not sure how exactly. > > David > ----- > >> David >> >> >> >>> If it is accepted, I will upload webrev for this redo task. >>> However, I cannot access (and fix) Oracle internal test. Can someone >>> help me? >>> (I cannot access JPRT. So I need a sponsor.) >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> On 2016/09/13 9:00, David Holmes wrote: >>>> I think I see the problem. The attach framework tries to load the >>>> "instrument" agent library. Prior to 8164913 if this fails it actually >>>> appears to succeed. Now it fails, and that error propagates through. >>>> I'm guessing, at the moment, that the failure is due to a missing >>>> module related option. >>>> >>>> David >>>> >>>> On 13/09/2016 9:54 AM, David Holmes wrote: >>>>> Hi Yasumasa, >>>>> >>>>> On 13/09/2016 9:23 AM, Yasumasa Suenaga wrote: >>>>>> Hi, >>>>>> >>>>>> Hmm, it has been backouted... >>>>>> >>>>>> I agree to David. This error seems to be a test bug. >>>>>> Can you share the test? I want to evaluate it. >>>>> >>>>> Sorry we can't share the tests. I took a quick look and it seems it >>>>> may >>>>> be related to the result code parsing (that I thought we ended up not >>>>> touching!). >>>>> >>>>> Can you run a test of HotSpotVirtualMachine.loadAgent(null) and see >>>>> what >>>>> happens? We see AgentLoadException from the code that internally tries >>>>> to load the "instrument" agent: >>>>> >>>>> Exception in thread "main" Failure: Unexpected exception during test >>>>> execution: java.lang.InternalError: instrument library is missing in >>>>> target VM >>>>> ... >>>>> Caused by: java.lang.InternalError: instrument library is missing in >>>>> target VM >>>>> ... >>>>> Caused by: com.sun.tools.attach.AgentLoadException: Failed to load >>>>> agent >>>>> library >>>>> >>>>> >>>>>> I do not agree to fix this bug in JDK 10 because VM might lie when >>>>>> the >>>>>> JVMTI agent could not be attached. IMHO this bug should be fixed in 9 >>>>>> GA, and we should fix testcase(s). >>>>> >>>>> I agree. It has to be noted that "we" failed to run all the >>>>> appropriate >>>>> tests before pushing this, which would have discovered the problem - >>>>> assuming it is actually a problem with the change and not an unrelated >>>>> issue in our testing environment. >>>>> >>>>> Unfortunately I have some high priority tasks to handle right now, >>>>> so I >>>>> can't go into this in any more depth at the moment. >>>>> >>>>> David >>>>> ----- >>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2016/09/13 6:15, David Holmes wrote: >>>>>>> For the record it looks like the tests involved are broken, in that >>>>>>> because the VM used to lie about the success of attaching the >>>>>>> agent, the >>>>>>> tests expected different exceptions to occur. >>>>>>> >>>>>>> David >>>>>>> >>>>>>> On 13/09/2016 3:11 AM, harold seigel wrote: >>>>>>>> Looks good. >>>>>>>> >>>>>>>> Harold >>>>>>>> >>>>>>>> >>>>>>>> On 9/12/2016 1:05 PM, Christian Tornqvist wrote: >>>>>>>>> Hi everyone, >>>>>>>>> >>>>>>>>> >>>>>>>>> Please review this (clean) backout of the change for >>>>>>>>> JDK-8164913, it >>>>>>>>> failed >>>>>>>>> several tests in the nightly testing. The failures are tracked in: >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165869 >>>>>>>>> >>>>>>>>> >>>>>>>>> Webrev: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~ctornqvi/webrev/8165881/webrev.00/ >>>>>>>>> >>>>>>>>> >>>>>>>>> Bug: >>>>>>>>> >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165881 >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Christian >>>>>>>>> >>>>>>>>> >>>>>>>> -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From yasuenag at gmail.com Tue Sep 13 12:31:02 2016 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Tue, 13 Sep 2016 21:31:02 +0900 Subject: RFR(XS): 8165881 - Backout JDK-8164913 In-Reply-To: References: <519601d20d17$cc979220$65c6b660$@oracle.com> <14399a60-88ca-bcf4-0696-09f530aa2955@oracle.com> <97d8d6c3-2f4d-1475-f9b0-e608878e5aac@oracle.com> <8e7b6d80-7c1c-b432-2ef4-3b9aa27956d3@gmail.com> <07965d03-1663-8da3-d126-aecbb93a7716@oracle.com> <839d84a3-46e8-7c89-a1e7-ba03fa0ef98c@oracle.com> Message-ID: <29ff139b-6618-6216-ccf3-2fc03af1d6b6@gmail.com> Thanks David! If we should not change the meaning of return code from JvmtiExport::load_agent_library(), we should print return code as below: ------------------- diff -r 0cf03b9d9b1f src/share/vm/prims/jvmtiExport.cpp --- a/src/share/vm/prims/jvmtiExport.cpp Mon Sep 12 18:59:13 2016 +0000 +++ b/src/share/vm/prims/jvmtiExport.cpp Tue Sep 13 21:12:14 2016 +0900 @@ -2412,6 +2412,10 @@ result = JNI_OK; } } + // Print error code if Agent_OnAttach cannot be executed + if (result != JNI_OK) { + st->print_cr("%d", result); + } return result; } ------------------- It can pass com/sun/tools/attach/BasicTests.java, and we can get -1 if we try to attach invalid agent. Yasumasa On 2016/09/13 17:06, Dmitry Samersoff wrote: > David, > > Thank you for the evaluation. > >> With that in mind I suspect the real fix for the original issue is to >> have Dcmd recognize that it also has to read two "results". Though I'm >> not sure how exactly. > > This logic seems completely broken for me. But I don't see anything we > can do right now - for jdk 9. > > It requires careful changes of both - code and tests. > > I can help with this task but not today. > > -Dmitry > > On 2016-09-13 08:08, David Holmes wrote: >> On 13/09/2016 1:53 PM, David Holmes wrote: >>> On 13/09/2016 1:30 PM, Yasumasa Suenaga wrote: >>>> Hi, >>>> >>>> I could reproduce and I added a comment to JBS: >>>> https://bugs.openjdk.java.net/browse/JDK-8165869?focusedCommentId=14000623&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14000623 >>>> >>>> >>>> >>>> >>>> In case of BasicTests.java, I think it is a test bug. >>> >>> I don't agree as yet. I have not been able to isolate the exact >>> difference between what happens with your fix and what happens without. >>> I know it relates to the presence of the error code, but also how we get >>> AgentInitializationException instead of AgentLoadException. I need to be >>> able to run the test outside of jtreg but that is proving to be very >>> difficult to arrange. :( >> >> I finally managed to connect all the pieces on this. >> >> The basic problem is that with the proposed change >> VirtualMachineImpl.execute() will throw AgentLoadException, which then >> leads to the InternalError. Without the change we reach this block in >> HotspotVirtualMachine.loadAgentLibrary: >> >> int result = readInt(in); >> if (result != 0) { >> throw new AgentInitializationException("Agent_OnAttach failed", result); >> } >> >> and so get the expected AgentInitializationException. >> >> Looking at the proposed change in jvmtiExport.cpp if we have the original: >> >> st->print_cr("%d", result); >> result = JNI_OK; >> >> then execute() manages to read a zero completion status from the stream, >> and then loadAgentLibrary is able to read the actual "result" - whether >> zero or not - from the stream. This is because the AttachListener code >> calls op->complete(result, &st) where st is the stream where we wrote >> the result value above in print_cr. Then if we look at, for example, >> LinuxAttachOperation::complete, we will write "result" to the socket >> first, followed by the contents of st. Hence on a successful operation >> we can read 0 for execute, and then 0 for loadAgent. On error we read 0 >> for execute and the actual error code for loadAgent. With the proposed >> change execute() sees the real result (instead of JNI_OK) and so throws >> AgentLoadException. >> >> So in summary there are two different error indicators written into the >> stream, and we need the first to be zero unless some major error with >> the actual attach functionality occurred - otherwise even if the "load" >> failed in some other way, it is treated as a secondary error. >> >> With that in mind I suspect the real fix for the original issue is to >> have Dcmd recognize that it also has to read two "results". Though I'm >> not sure how exactly. >> >> David >> ----- >> >>> David >>> >>> >>> >>>> If it is accepted, I will upload webrev for this redo task. >>>> However, I cannot access (and fix) Oracle internal test. Can someone >>>> help me? >>>> (I cannot access JPRT. So I need a sponsor.) >>>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> On 2016/09/13 9:00, David Holmes wrote: >>>>> I think I see the problem. The attach framework tries to load the >>>>> "instrument" agent library. Prior to 8164913 if this fails it actually >>>>> appears to succeed. Now it fails, and that error propagates through. >>>>> I'm guessing, at the moment, that the failure is due to a missing >>>>> module related option. >>>>> >>>>> David >>>>> >>>>> On 13/09/2016 9:54 AM, David Holmes wrote: >>>>>> Hi Yasumasa, >>>>>> >>>>>> On 13/09/2016 9:23 AM, Yasumasa Suenaga wrote: >>>>>>> Hi, >>>>>>> >>>>>>> Hmm, it has been backouted... >>>>>>> >>>>>>> I agree to David. This error seems to be a test bug. >>>>>>> Can you share the test? I want to evaluate it. >>>>>> >>>>>> Sorry we can't share the tests. I took a quick look and it seems it >>>>>> may >>>>>> be related to the result code parsing (that I thought we ended up not >>>>>> touching!). >>>>>> >>>>>> Can you run a test of HotSpotVirtualMachine.loadAgent(null) and see >>>>>> what >>>>>> happens? We see AgentLoadException from the code that internally tries >>>>>> to load the "instrument" agent: >>>>>> >>>>>> Exception in thread "main" Failure: Unexpected exception during test >>>>>> execution: java.lang.InternalError: instrument library is missing in >>>>>> target VM >>>>>> ... >>>>>> Caused by: java.lang.InternalError: instrument library is missing in >>>>>> target VM >>>>>> ... >>>>>> Caused by: com.sun.tools.attach.AgentLoadException: Failed to load >>>>>> agent >>>>>> library >>>>>> >>>>>> >>>>>>> I do not agree to fix this bug in JDK 10 because VM might lie when >>>>>>> the >>>>>>> JVMTI agent could not be attached. IMHO this bug should be fixed in 9 >>>>>>> GA, and we should fix testcase(s). >>>>>> >>>>>> I agree. It has to be noted that "we" failed to run all the >>>>>> appropriate >>>>>> tests before pushing this, which would have discovered the problem - >>>>>> assuming it is actually a problem with the change and not an unrelated >>>>>> issue in our testing environment. >>>>>> >>>>>> Unfortunately I have some high priority tasks to handle right now, >>>>>> so I >>>>>> can't go into this in any more depth at the moment. >>>>>> >>>>>> David >>>>>> ----- >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2016/09/13 6:15, David Holmes wrote: >>>>>>>> For the record it looks like the tests involved are broken, in that >>>>>>>> because the VM used to lie about the success of attaching the >>>>>>>> agent, the >>>>>>>> tests expected different exceptions to occur. >>>>>>>> >>>>>>>> David >>>>>>>> >>>>>>>> On 13/09/2016 3:11 AM, harold seigel wrote: >>>>>>>>> Looks good. >>>>>>>>> >>>>>>>>> Harold >>>>>>>>> >>>>>>>>> >>>>>>>>> On 9/12/2016 1:05 PM, Christian Tornqvist wrote: >>>>>>>>>> Hi everyone, >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Please review this (clean) backout of the change for >>>>>>>>>> JDK-8164913, it >>>>>>>>>> failed >>>>>>>>>> several tests in the nightly testing. The failures are tracked in: >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165869 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Webrev: >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~ctornqvi/webrev/8165881/webrev.00/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Bug: >>>>>>>>>> >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165881 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Christian >>>>>>>>>> >>>>>>>>>> >>>>>>>>> > > From george.triantafillou at oracle.com Tue Sep 13 13:27:14 2016 From: george.triantafillou at oracle.com (George Triantafillou) Date: Tue, 13 Sep 2016 09:27:14 -0400 Subject: RFR 8165889: Remove jdk.test.lib.unsafe.UnsafeHelper Message-ID: Please review the following fix: JBS: https://bugs.openjdk.java.net/browse/JDK-8165889 Open webrevs: http://cr.openjdk.java.net/~gtriantafill/8165889/jdk.patch/webrev/ http://cr.openjdk.java.net/~gtriantafill/8165889/jdk-test.patch/webrev/ http://cr.openjdk.java.net/~gtriantafill/8165889/hotspot-test.patch/webrev/ Summary: jdk.test.lib.unsafe.UnsafeHelper has been replaced by jdk.internal.misc.Unsafe. Tests using jdk.test.lib.unsafe.UnsafeHelper need to be updated to use jdk.internal.misc.Unsafe. - Removed test/lib/jdk/test/lib/unsafe/UnsafeHelper.java - Replaced uses of UnsafeHelper.getUnsafe() with Unsafe.getUnsafe() - Removed unnecessary calls to setAccessible(true) Tested locally on Linux. Thanks. -George From shade at redhat.com Tue Sep 13 13:38:49 2016 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 13 Sep 2016 15:38:49 +0200 Subject: RFR 8165889: Remove jdk.test.lib.unsafe.UnsafeHelper In-Reply-To: References: Message-ID: <7425a509-0c0b-13b2-79d7-96fc21c2dd43@redhat.com> On 09/13/2016 03:27 PM, George Triantafillou wrote: > Open webrevs: > http://cr.openjdk.java.net/~gtriantafill/8165889/jdk.patch/webrev/ > http://cr.openjdk.java.net/~gtriantafill/8165889/jdk-test.patch/webrev/ > > http://cr.openjdk.java.net/~gtriantafill/8165889/hotspot-test.patch/webrev/ > These look good. Thanks, -Aleksey From george.triantafillou at oracle.com Tue Sep 13 13:40:49 2016 From: george.triantafillou at oracle.com (George Triantafillou) Date: Tue, 13 Sep 2016 09:40:49 -0400 Subject: RFR 8165889: Remove jdk.test.lib.unsafe.UnsafeHelper In-Reply-To: <7425a509-0c0b-13b2-79d7-96fc21c2dd43@redhat.com> References: <7425a509-0c0b-13b2-79d7-96fc21c2dd43@redhat.com> Message-ID: Thanks Aleksey. -George On 9/13/2016 9:38 AM, Aleksey Shipilev wrote: > On 09/13/2016 03:27 PM, George Triantafillou wrote: >> Open webrevs: >> http://cr.openjdk.java.net/~gtriantafill/8165889/jdk.patch/webrev/ >> http://cr.openjdk.java.net/~gtriantafill/8165889/jdk-test.patch/webrev/ >> >> http://cr.openjdk.java.net/~gtriantafill/8165889/hotspot-test.patch/webrev/ >> > These look good. > > Thanks, > -Aleksey From lois.foltan at oracle.com Tue Sep 13 13:43:38 2016 From: lois.foltan at oracle.com (Lois Foltan) Date: Tue, 13 Sep 2016 09:43:38 -0400 Subject: RFR 8165889: Remove jdk.test.lib.unsafe.UnsafeHelper In-Reply-To: References: Message-ID: <57D8028A.9010409@oracle.com> Looks good. Thank you for making these changes! Lois On 9/13/2016 9:27 AM, George Triantafillou wrote: > Please review the following fix: > > JBS: https://bugs.openjdk.java.net/browse/JDK-8165889 > > Open webrevs: > http://cr.openjdk.java.net/~gtriantafill/8165889/jdk.patch/webrev/ > http://cr.openjdk.java.net/~gtriantafill/8165889/jdk-test.patch/webrev/ > > > http://cr.openjdk.java.net/~gtriantafill/8165889/hotspot-test.patch/webrev/ > > > > Summary: > jdk.test.lib.unsafe.UnsafeHelper has been replaced by > jdk.internal.misc.Unsafe. Tests using jdk.test.lib.unsafe.UnsafeHelper > need to be updated to use jdk.internal.misc.Unsafe. > > - Removed test/lib/jdk/test/lib/unsafe/UnsafeHelper.java > - Replaced uses of UnsafeHelper.getUnsafe() with Unsafe.getUnsafe() > - Removed unnecessary calls to setAccessible(true) > > Tested locally on Linux. > > Thanks. > > -George > > From george.triantafillou at oracle.com Tue Sep 13 13:49:18 2016 From: george.triantafillou at oracle.com (George Triantafillou) Date: Tue, 13 Sep 2016 09:49:18 -0400 Subject: RFR 8165889: Remove jdk.test.lib.unsafe.UnsafeHelper In-Reply-To: <57D8028A.9010409@oracle.com> References: <57D8028A.9010409@oracle.com> Message-ID: <43996299-42c9-c65e-5b88-7a29e2925709@oracle.com> Thanks Lois. -George On 9/13/2016 9:43 AM, Lois Foltan wrote: > Looks good. Thank you for making these changes! > Lois > > On 9/13/2016 9:27 AM, George Triantafillou wrote: >> Please review the following fix: >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8165889 >> >> Open webrevs: >> http://cr.openjdk.java.net/~gtriantafill/8165889/jdk.patch/webrev/ >> http://cr.openjdk.java.net/~gtriantafill/8165889/jdk-test.patch/webrev/ >> >> >> http://cr.openjdk.java.net/~gtriantafill/8165889/hotspot-test.patch/webrev/ >> >> >> >> Summary: >> jdk.test.lib.unsafe.UnsafeHelper has been replaced by >> jdk.internal.misc.Unsafe. Tests using >> jdk.test.lib.unsafe.UnsafeHelper need to be updated to use >> jdk.internal.misc.Unsafe. >> >> - Removed test/lib/jdk/test/lib/unsafe/UnsafeHelper.java >> - Replaced uses of UnsafeHelper.getUnsafe() with Unsafe.getUnsafe() >> - Removed unnecessary calls to setAccessible(true) >> >> Tested locally on Linux. >> >> Thanks. >> >> -George >> >> > From doug.simon at oracle.com Tue Sep 13 14:41:27 2016 From: doug.simon at oracle.com (Doug Simon) Date: Tue, 13 Sep 2016 16:41:27 +0200 Subject: RFR: 8165755: [JVMCI] replace use of vm_abort with vm_exit In-Reply-To: <52A83790-582E-4F8F-AA49-3F6DB3AB36CB@oracle.com> References: <799CA13D-6BDC-4BF5-9241-515A684191F4@oracle.com> <6391B00B-AFBF-410C-A6A1-2ED95B35EBEB@twitter.com> <2F360221-7E6B-43BB-B69C-69A17777E5F2@oracle.com> <1c9b4d32-52a2-17a8-bfc5-f3cc7b494ba0@oracle.com> <5d26a289-a262-5882-8061-3bbc4faa4540@oracle.com> <52A83790-582E-4F8F-AA49-3F6DB3AB36CB@oracle.com> Message-ID: <42F9D0A5-3017-45DA-BBB6-92D2B76FBD55@oracle.com> Hi David, Do you still have concerns with the webrev in its current form? -Doug > On 12 Sep 2016, at 11:26, Doug Simon wrote: > > Hi David, >> On 12 Sep 2016, at 03:24, David Holmes wrote: >> >> On 10/09/2016 6:29 AM, Coleen Phillimore wrote: >>> On 9/9/16 3:31 PM, Doug Simon wrote: >>>> Is vm_exit_during_initialization still the right choice when this >>>> could be after VM initialization? JVMCI initialization is lazy and can >>>> happen after the application has started. >>> >>> I don't actually know then. vm_exit tries to get threads to a safepoint >>> first, but vm_abort(false) just shuts down the jvm. There aren't a lot >>> of places we terminate the jvm. Maybe David Holmes knows. >> >> If JVMCI initialization is lazy and can happen after the application is started, why should it be a fatal error to fail to initialize it? How is the initialization triggered? Is there a synchronous call that could throw an exception? How does the user know that initialization failure will result in termination? > > JVMCI initialization is triggered by the first top tier compilation (when UseJVMCICompiler is true) or by an explicit request via the Java API to the JVMCI system. In the latter case, an exception is propagated to the caller who can decide what to do with it. However, in the former case, initialization is triggered by the CompileBroker on a compilation thread. We could make a failure during JVMCI initialization disable compilation but I?m sure that would be more confusing to a user than a VM exit clearly stating the invalid configuration option given. There?s already lazy configuration checking in other compilers that can cause the VM to exit (e.g. Compile::pd_compiler2_init in c2_init_x86.cpp) so this is not too different. > >> Can you not at least eagerly validate command-line options during VM initialization, even if you don't actually initialize JVMCI fully? > > Unfortunately not. The command line options are discovered (via a service loader) during initialization and this kind of activity is what we want to avoid during early VM startup. > >> >> That aside it sounds like you want JVMCI initialization to do the equivalent of either Runtime.exit or Runtime.halt, depending on the exact termination semantics you want. If you call vm_exit directly then you get something like Runtime.halt > > Yes, we want Runtime.halt semantics in the case where invalid JVMCI options are specified. > > -Doug > >> David >> ----- >> >>> Coleen >>> >>>> >>>> Sent from my iPhone >>>> >>>>> On Sep 9, 2016, at 9:23 PM, Coleen Phillimore >>>>> wrote: >>>>> >>>>> >>>>> I think you want vm_exit_during_initialization() for that. Definitely >>>>> not vm_abort, cause then it looks like an internal error/crash. >>>>> >>>>> Coleen >>>>> >>>>>> On 9/9/16 2:33 PM, Doug Simon wrote: >>>>>> Can someone from the runtime team confirm that using vm_exit >>>>>> (instead of vm_abort) is the best way to stop the VM when JVMCI >>>>>> initialization fails (e.g., when invalid JVMCI options are provided >>>>>> on the command line). Thanks! >>>>>> >>>>>> -Doug >>>>>> >>>>>> >>>>>>> On 09 Sep 2016, at 19:48, Christian Thalinger >>>>>>> wrote: >>>>>>> >>>>>>> I think this looks fine but maybe we should ask the runtime folks. >>>>>>> >>>>>>>> On Sep 8, 2016, at 11:01 PM, Doug Simon >>>>>>>> wrote: >>>>>>>> >>>>>>>> Calling vm_abort from multiple threads can cause nasty crashes >>>>>>>> such as double free errors. We've seen this in Graal during JVMCI >>>>>>>> initialization when an unknown Graal option is encountered. >>>>>>>> Multiple compiler threads try to initialize JVMCI which fails with >>>>>>>> an exception indicating the bad option: >>>>>>>> >>>>>>>> Uncaught exception at >>>>>>>> /scratch/graaluser/buildslave/buildlog/ci_executor/main/graal-jvmci-8/src/share/vm/jvmci/jvmciCompiler.cpp:127 >>>>>>>> >>>>>>>> java.lang.ExceptionInInitializerError >>>>>>>> at >>>>>>>> jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.runtime(HotSpotJVMCIRuntime.java:85) >>>>>>>> >>>>>>>> at jdk.vm.ci.runtime.JVMCI.initializeRuntime(Native Method) >>>>>>>> at jdk.vm.ci.runtime.JVMCI.(JVMCI.java:58) >>>>>>>> Caused by: java.lang.IllegalArgumentException: Could not find >>>>>>>> option OptSomethingThatDoesNotExcist >>>>>>>> at >>>>>>>> com.oracle.graal.options.OptionsParser.parseOption(OptionsParser.java:134) >>>>>>>> >>>>>>>> at >>>>>>>> com.oracle.graal.options.OptionsParser.parseOptions(OptionsParser.java:62) >>>>>>>> >>>>>>>> at >>>>>>>> com.oracle.graal.hotspot.HotSpotGraalCompilerFactory.initializeOptions(HotSpotGraalCompilerFactory.java:156) >>>>>>>> >>>>>>>> at >>>>>>>> com.oracle.graal.hotspot.HotSpotGraalCompilerFactory.onSelection(HotSpotGraalCompilerFactory.java:86) >>>>>>>> >>>>>>>> at >>>>>>>> jdk.vm.ci.hotspot.HotSpotJVMCICompilerConfig.getCompilerFactory(HotSpotJVMCICompilerConfig.java:96) >>>>>>>> >>>>>>>> at >>>>>>>> jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.(HotSpotJVMCIRuntime.java:277) >>>>>>>> >>>>>>>> at >>>>>>>> jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.(HotSpotJVMCIRuntime.java:67) >>>>>>>> >>>>>>>> at >>>>>>>> jdk.vm.ci.hotspot.HotSpotJVMCIRuntime$DelayedInit.(HotSpotJVMCIRuntime.java:75) >>>>>>>> >>>>>>>> at >>>>>>>> jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.runtime(HotSpotJVMCIRuntime.java:85) >>>>>>>> >>>>>>>> at jdk.vm.ci.runtime.JVMCI.initializeRuntime(Native Method) >>>>>>>> at jdk.vm.ci.runtime.JVMCI.(JVMCI.java:58) >>>>>>>> >>>>>>>> The native JVMCI code then tries to exit the VM by calling >>>>>>>> vm_abort. If multiple compiler threads do this concurrently, >>>>>>>> certain destructors can be called twice as shown by these thread >>>>>>>> dumps: >>>>>>>> >>>>>>>> thread #26: tid = 0x0019, 0x00007fff84280124 >>>>>>>> libsystem_malloc.dylib`szone_size + 227, stop reason = signal SIGSTOP >>>>>>>> frame #0: 0x00007fff84280124 libsystem_malloc.dylib`szone_size + 227 >>>>>>>> frame #1: 0x00007fff8427fed5 libsystem_malloc.dylib`free + 61 >>>>>>>> frame #2: 0x000000010ac95963 >>>>>>>> libjvm.dylib`os::free(memblock=0x00007fedc86226e0, >>>>>>>> memflags=mtInternal) + 307 at os.cpp:711 >>>>>>>> frame #3: 0x000000010a2afc54 >>>>>>>> libjvm.dylib`FreeHeap(p=0x00007fedc86226e0, memflags=mtInternal) + >>>>>>>> 52 at allocation.inline.hpp:93 >>>>>>>> frame #4: 0x000000010acf0a9f >>>>>>>> libjvm.dylib`PerfData::~PerfData(this=0x00007fedc8622650) + 63 at >>>>>>>> perfData.cpp:116 >>>>>>>> frame #5: 0x000000010acf0ae5 >>>>>>>> libjvm.dylib`PerfData::~PerfData(this=0x00007fedc8622650) + 21 at >>>>>>>> perfData.cpp:114 >>>>>>>> frame #6: 0x000000010acf163d >>>>>>>> libjvm.dylib`PerfDataManager::destroy() + 109 at perfData.cpp:287 >>>>>>>> frame #7: 0x000000010acf3f4d libjvm.dylib`perfMemory_exit() + 61 >>>>>>>> at perfMemory.cpp:74 >>>>>>>> frame #8: 0x000000010ac9bb0d libjvm.dylib`os::shutdown() + 13 at >>>>>>>> os_bsd.cpp:1130 >>>>>>>> frame #9: 0x000000010ac9bb55 >>>>>>>> libjvm.dylib`os::abort(dump_core=false) + 21 at os_bsd.cpp:1150 >>>>>>>> frame #10: 0x000000010a9188e7 >>>>>>>> libjvm.dylib`vm_abort(dump_core=false) + 39 at java.cpp:666 >>>>>>>> frame #11: 0x000000010aa4f1e7 >>>>>>>> libjvm.dylib`JVMCIRuntime::abort_on_pending_exception(exception=Handle >>>>>>>> @ 0x000070000175b208, message="Uncaught exception at >>>>>>>> /Users/dsimon/graal/graal-jvmci-8/src/share/vm/jvmci/jvmciCompiler.cpp:127", >>>>>>>> dump_core=false) + 167 at jvmciRuntime.cpp:992 >>>>>>>> frame #12: 0x000000010aa17017 >>>>>>>> libjvm.dylib`JVMCICompiler::compile_method(this=0x00007fedcb203050, method=0x000070000175b8d8, >>>>>>>> entry_bci=-1, env=0x000070000175b8f0) + 311 at jvmciCompiler.cpp:127 >>>>>>>> frame #13: 0x000000010a656cd2 >>>>>>>> libjvm.dylib`CompileBroker::invoke_compiler_on_method(task=0x00007fedc853ca30) >>>>>>>> + 1314 at compileBroker.cpp:2207 >>>>>>>> >>>>>>>> thread #23: tid = 0x0016, 0x00007fff91fcb122 >>>>>>>> libsystem_kernel.dylib`__semwait_signal_nocancel + 10, stop reason >>>>>>>> = signal SIGSTOP >>>>>>>> frame #0: 0x00007fff91fcb122 >>>>>>>> libsystem_kernel.dylib`__semwait_signal_nocancel + 10 >>>>>>>> frame #1: 0x00007fff9578c318 libsystem_c.dylib`nanosleep$NOCANCEL >>>>>>>> + 188 >>>>>>>> frame #2: 0x00007fff957b62ce libsystem_c.dylib`usleep$NOCANCEL + 54 >>>>>>>> frame #3: 0x00007fff957e46e9 libsystem_c.dylib`abort + 139 >>>>>>>> frame #4: 0x00007fff8428c396 libsystem_malloc.dylib`szone_error + 626 >>>>>>>> frame #5: 0x000000010ac95963 >>>>>>>> libjvm.dylib`os::free(memblock=0x00007fedc8601cd0, >>>>>>>> memflags=mtInternal) + 307 at os.cpp:711 >>>>>>>> frame #6: 0x000000010a2afc54 >>>>>>>> libjvm.dylib`FreeHeap(p=0x00007fedc8601cd0, memflags=mtInternal) + >>>>>>>> 52 at allocation.inline.hpp:93 >>>>>>>> frame #7: 0x000000010acf0a9f >>>>>>>> libjvm.dylib`PerfData::~PerfData(this=0x00007fedc8601c60) + 63 at >>>>>>>> perfData.cpp:116 >>>>>>>> frame #8: 0x000000010acf0ae5 >>>>>>>> libjvm.dylib`PerfData::~PerfData(this=0x00007fedc8601c60) + 21 at >>>>>>>> perfData.cpp:114 >>>>>>>> frame #9: 0x000000010acf163d >>>>>>>> libjvm.dylib`PerfDataManager::destroy() + 109 at perfData.cpp:287 >>>>>>>> frame #10: 0x000000010acf3f4d libjvm.dylib`perfMemory_exit() + 61 >>>>>>>> at perfMemory.cpp:74 >>>>>>>> frame #11: 0x000000010ac9bb0d libjvm.dylib`os::shutdown() + 13 at >>>>>>>> os_bsd.cpp:1130 >>>>>>>> frame #12: 0x000000010ac9bb55 >>>>>>>> libjvm.dylib`os::abort(dump_core=false) + 21 at os_bsd.cpp:1150 >>>>>>>> frame #13: 0x000000010a9188e7 >>>>>>>> libjvm.dylib`vm_abort(dump_core=false) + 39 at java.cpp:666 >>>>>>>> frame #14: 0x000000010aa4f1e7 >>>>>>>> libjvm.dylib`JVMCIRuntime::abort_on_pending_exception(exception=Handle >>>>>>>> @ 0x0000700001452208, message="Uncaught exception at >>>>>>>> /Users/dsimon/graal/graal-jvmci-8/src/share/vm/jvmci/jvmciCompiler.cpp:127", >>>>>>>> dump_core=false) + 167 at jvmciRuntime.cpp:992 >>>>>>>> frame #15: 0x000000010aa17017 >>>>>>>> libjvm.dylib`JVMCICompiler::compile_method(this=0x00007fedcb203050, method=0x00007000014528d8, >>>>>>>> entry_bci=-1, env=0x00007000014528f0) + 311 at jvmciCompiler.cpp:127 >>>>>>>> frame #16: 0x000000010a656cd2 >>>>>>>> libjvm.dylib`CompileBroker::invoke_compiler_on_method(task=0x00007fedc862a320) >>>>>>>> + 1314 at compileBroker.cpp:2207 >>>>>>>> >>>>>>>> >>>>>>>> This webrev replaces calls to vm_abort() with before_exit() + >>>>>>>> vm_exit(). The latter is thread safe. >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165755 >>>>>>>> http://cr.openjdk.java.net/~dnsimon/8165755/ >>>>>>>> >>>>>>>> -Doug >>> > From vladimir.kozlov at oracle.com Tue Sep 13 17:25:16 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 13 Sep 2016 10:25:16 -0700 Subject: RFR(XXS): 8164987: RTM jtreg tests failing due to unnamed module unable to access class jdk.internal.misc.Unsafe In-Reply-To: <57D6A415.3090804@linux.vnet.ibm.com> References: <57D1ED2E.5020408@linux.vnet.ibm.com> <57D6A415.3090804@linux.vnet.ibm.com> Message-ID: <57D8367C.2020107@oracle.com> Hi Gustavo, Changes look fine and there is no problem to push it now. I will sponsor it. I assume you tested it on machines with RTM. Thanks, Vladimir On 9/12/16 5:48 AM, Gustavo Romero wrote: > Hi Dmitry, > > Could you be the sponsor for that tiny change? > > It's related to what we've discussed here: > http://mail.openjdk.java.net/pipermail/hotspot-dev/2016-August/024395.html > > I understand that is ok to get pushed even after rampdown phase 1, as per > Mark's Process proposal: > > "P4-P5 bugs should, in general, be left to future releases unless they only > affect documentation, demos, or tests, in which case they should be identified > as such with the 'noreg-doc', 'noreg-demo', and 'noreg-self' labels, > respectively" > > Thus I labeled that change as "noreg-self". > > Is my understanding correct? > > > Thank you and best regards, > Gustavo > > On 08-09-2016 19:58, Gustavo Romero wrote: >> Hi, >> >> Could the following change be reviewed, please? >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8164987 >> webrev: http://cr.openjdk.java.net/~gromero/8164987/01/ >> >> >> Thank you. >> >> Regards, >> Gustavo >> > From gromero at linux.vnet.ibm.com Tue Sep 13 18:59:50 2016 From: gromero at linux.vnet.ibm.com (Gustavo Romero) Date: Tue, 13 Sep 2016 15:59:50 -0300 Subject: RFR(XXS): 8164987: RTM jtreg tests failing due to unnamed module unable to access class jdk.internal.misc.Unsafe In-Reply-To: <57D8367C.2020107@oracle.com> References: <57D1ED2E.5020408@linux.vnet.ibm.com> <57D6A415.3090804@linux.vnet.ibm.com> <57D8367C.2020107@oracle.com> Message-ID: <57D84CA6.4000203@linux.vnet.ibm.com> Hi Vladimir, On 13-09-2016 14:25, Vladimir Kozlov wrote: > I assume you tested it on machines with RTM. Yes, I tested both on PPC64 with HTM support and on x64 with RTM support. Thank you very much for reviewing and sponsoring it. Regards, Gustavo From dmitry.samersoff at oracle.com Tue Sep 13 19:49:20 2016 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Tue, 13 Sep 2016 22:49:20 +0300 Subject: RFR (XS): 8165681 ClassLoad and ClassPrepare JVMTI events are missed in the start phase In-Reply-To: <6d94edc0-1b32-f6b7-4499-7d4f55a37840@oracle.com> References: <45ce10ea-993f-6a07-3e8f-0222fb479566@oracle.com> <4ffd4740-0df4-a103-4653-d085e5a98159@oracle.com> <6d94edc0-1b32-f6b7-4499-7d4f55a37840@oracle.com> Message-ID: <23a35c46-70ce-411c-3d71-5ccab721a837@oracle.com> Serguei, Looks good for me! -Dmitry On 2016-09-12 06:58, serguei.spitsyn at oracle.com wrote: > On 9/11/16 18:49, David Holmes wrote: >> Hi Serguei, >> >> On 12/09/2016 11:15 AM, serguei.spitsyn at oracle.com wrote: >>> Please, review the one-liner fix for: >>> https://bugs.openjdk.java.net/browse/JDK-8165681 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2016/hotspot/8165681-jvmti-events.hs1/ >>> >>> >>> Summary: >>> The JVMTI spec tells the ClassLoad and ClassPrepare events can be send >>> during start phase. >>> The new test below identified that they are missed because of this bug >>> in JVMTI. >>> The fix is to add the events to the set of the set of the >>> EARLY_EVENT_BITS. >> >> Seems fine. > > Thank you for the review, David. > >> Good to see the additional test coverage. > > Thanks to D. Dmitriev for the additional coverage. > > Thanks, > Serguei > > >> >> Thanks, >> David >> >>> Testing: >>> The following new test covers this issue: >>> test/serviceability/jvmti/ModuleAwareAgents/ClassLoadPrepare/VMEarly >>> They are developed as a part of the following test enhancement: >>> 8150758 [TESTBUG] need jvmti tests for module aware agents >>> >>> Additionally, the following tests were run: >>> nsk.jvmti, nsk,jdi, jtreg com/sun/jdi and java/lang/instrument >>> >>> >>> Thanks, >>> Serguei > > -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From serguei.spitsyn at oracle.com Tue Sep 13 19:59:42 2016 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 13 Sep 2016 12:59:42 -0700 Subject: RFR (XS): 8165681 ClassLoad and ClassPrepare JVMTI events are missed in the start phase In-Reply-To: <23a35c46-70ce-411c-3d71-5ccab721a837@oracle.com> References: <45ce10ea-993f-6a07-3e8f-0222fb479566@oracle.com> <4ffd4740-0df4-a103-4653-d085e5a98159@oracle.com> <6d94edc0-1b32-f6b7-4499-7d4f55a37840@oracle.com> <23a35c46-70ce-411c-3d71-5ccab721a837@oracle.com> Message-ID: Thanks, Dmitry! Serguei On 9/13/16 12:49, Dmitry Samersoff wrote: > Serguei, > > Looks good for me! > > -Dmitry > > > On 2016-09-12 06:58, serguei.spitsyn at oracle.com wrote: >> On 9/11/16 18:49, David Holmes wrote: >>> Hi Serguei, >>> >>> On 12/09/2016 11:15 AM, serguei.spitsyn at oracle.com wrote: >>>> Please, review the one-liner fix for: >>>> https://bugs.openjdk.java.net/browse/JDK-8165681 >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2016/hotspot/8165681-jvmti-events.hs1/ >>>> >>>> >>>> Summary: >>>> The JVMTI spec tells the ClassLoad and ClassPrepare events can be send >>>> during start phase. >>>> The new test below identified that they are missed because of this bug >>>> in JVMTI. >>>> The fix is to add the events to the set of the set of the >>>> EARLY_EVENT_BITS. >>> Seems fine. >> Thank you for the review, David. >> >>> Good to see the additional test coverage. >> Thanks to D. Dmitriev for the additional coverage. >> >> Thanks, >> Serguei >> >> >>> Thanks, >>> David >>> >>>> Testing: >>>> The following new test covers this issue: >>>> test/serviceability/jvmti/ModuleAwareAgents/ClassLoadPrepare/VMEarly >>>> They are developed as a part of the following test enhancement: >>>> 8150758 [TESTBUG] need jvmti tests for module aware agents >>>> >>>> Additionally, the following tests were run: >>>> nsk.jvmti, nsk,jdi, jtreg com/sun/jdi and java/lang/instrument >>>> >>>> >>>> Thanks, >>>> Serguei >> > From david.holmes at oracle.com Tue Sep 13 21:50:33 2016 From: david.holmes at oracle.com (David Holmes) Date: Wed, 14 Sep 2016 07:50:33 +1000 Subject: [8u] RFR for JDK-8154324: Request to backport JDK-6515172 to 8u In-Reply-To: References: <60e87114-fcc2-4a0f-a31b-8440faf91555@default> <3695d333-eb30-3a84-9b8e-b0264a9087c9@oracle.com> Message-ID: <26e8f72e-0a8f-6449-d824-48b5d49635dd@oracle.com> Hi Shafi, On 12/09/2016 11:20 PM, Shafi Ahmad wrote: > Hi, > > Please find updated webrev: http://cr.openjdk.java.net/~shshahma/8154324/webrev.01/ > I have incorporated both review comment by David. > 1. Removed unrelated comment. > 2. Removed unreferenced code ' diagnostic(bool, UseCpuAllocPath, false,' Incorrect comment here in globals_linux.hpp: ! diagnostic(bool, PrintActiveCpus, false, \ ! "Use CPU_ALLOC code path in os::active_processor_count ") should be: "Print the number of CPUs detected in os::active_processor_count" --- os_linux.cpp: if(PrintActiveCpus) { Need space after if. -- Otherwise okay. No need to see updated webrev if above are fixed. Thanks, David > > Regards, > Shafi > >> -----Original Message----- >> From: David Holmes >> Sent: Friday, September 09, 2016 11:29 AM >> To: Shafi Ahmad; hotspot-dev at openjdk.java.net >> Subject: Re: [8u] RFR for JDK-8154324: Request to backport JDK-6515172 to >> 8u >> >> Hi Shafi, >> >> On 9/09/2016 3:46 PM, Shafi Ahmad wrote: >>> Hi, >>> >>> Please review the backport [modified] of bug: "JDK-6515172 >> Runtime.availableProcessors() ignores Linux taskset command" to jdk8u. >>> >>> Please note that the backport is not clean and we can't do as it is. Please >> note >>> 1. The changes are made by 'Andreas Eriksson' who left Oracle. >>> 2. There is difference in logging mechanism in jdk9 and jdk8 is different >> and file logTag.hpp doesn't exists in jdk8. >>> 3. Newly added test pass after this change. It fails without this change. >>> >>> Webrev link: http://cr.openjdk.java.net/~shshahma/8154324/webrev.00/ >>> Jdk9 bug: https://bugs.openjdk.java.net/browse/JDK-6515172 >>> Jdk8 bug: https://bugs.openjdk.java.net/browse/JDK-8154324 >>> Original patch pushed to jdk9: >>> http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/c5480d4abfe4 >> >> I worked extensively with Andreas on this as there were a number of issues. >> I'll have to try and find those discussions to see where we ended up. >> >> The backport as stands is not quite appropriate. For example it adds: >> >> diagnostic(bool, UseCpuAllocPath, false, >> >> but that does not exist in the actual code for 8. >> >> Also this comment: >> >> + // If it appears there may be more than 1024 processors then we do a >> + // dynamic check - see 6515172 for details. >> >> is wrong as there is no dynamic check in this version of the code. >> >> The last I recall with this is that there were issues caused by building with one >> version of glibc and running (or trying to) on later versions of glibc. But as I >> said I will have to see if I have the discussion from that effort. >> >> David >> ---- >> >>> Testing: jprt and running jtreg. >>> >>> Regards, >>> Shafi >>> From david.holmes at oracle.com Tue Sep 13 22:03:27 2016 From: david.holmes at oracle.com (David Holmes) Date: Wed, 14 Sep 2016 08:03:27 +1000 Subject: RFR(XS): 8165881 - Backout JDK-8164913 In-Reply-To: <29ff139b-6618-6216-ccf3-2fc03af1d6b6@gmail.com> References: <519601d20d17$cc979220$65c6b660$@oracle.com> <14399a60-88ca-bcf4-0696-09f530aa2955@oracle.com> <97d8d6c3-2f4d-1475-f9b0-e608878e5aac@oracle.com> <8e7b6d80-7c1c-b432-2ef4-3b9aa27956d3@gmail.com> <07965d03-1663-8da3-d126-aecbb93a7716@oracle.com> <839d84a3-46e8-7c89-a1e7-ba03fa0ef98c@oracle.com> <29ff139b-6618-6216-ccf3-2fc03af1d6b6@gmail.com> Message-ID: On 13/09/2016 10:31 PM, Yasumasa Suenaga wrote: > Thanks David! > If we should not change the meaning of return code from > JvmtiExport::load_agent_library(), we should print return code as below: > ------------------- > diff -r 0cf03b9d9b1f src/share/vm/prims/jvmtiExport.cpp > --- a/src/share/vm/prims/jvmtiExport.cpp Mon Sep 12 18:59:13 2016 +0000 > +++ b/src/share/vm/prims/jvmtiExport.cpp Tue Sep 13 21:12:14 2016 +0900 > @@ -2412,6 +2412,10 @@ > result = JNI_OK; > } > } > + // Print error code if Agent_OnAttach cannot be executed > + if (result != JNI_OK) { > + st->print_cr("%d", result); > + } > return result; > } > ------------------- Not sure I see the point. "return result" will put the error code into the socket stream and that error will be seen and responded to. I don't expect anything to then read further into the stream to see the "result" repeated a second time. In other words if execute() doesn't see a zero result then you go no further; if execute sees a zero result then the actual action may have had an issue so you proceed to read the "action's result". > It can pass com/sun/tools/attach/BasicTests.java, and we can get -1 if > we try to attach invalid agent. Can you please outline your test case for this again. If this is done through jcmd then jcmd needs to know how to "unwrap" the information it gets back from the Dcmd. Thanks, David > > Yasumasa > > > On 2016/09/13 17:06, Dmitry Samersoff wrote: >> David, >> >> Thank you for the evaluation. >> >>> With that in mind I suspect the real fix for the original issue is to >>> have Dcmd recognize that it also has to read two "results". Though I'm >>> not sure how exactly. >> >> This logic seems completely broken for me. But I don't see anything we >> can do right now - for jdk 9. >> >> It requires careful changes of both - code and tests. >> >> I can help with this task but not today. >> >> -Dmitry >> >> On 2016-09-13 08:08, David Holmes wrote: >>> On 13/09/2016 1:53 PM, David Holmes wrote: >>>> On 13/09/2016 1:30 PM, Yasumasa Suenaga wrote: >>>>> Hi, >>>>> >>>>> I could reproduce and I added a comment to JBS: >>>>> https://bugs.openjdk.java.net/browse/JDK-8165869?focusedCommentId=14000623&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14000623 >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> In case of BasicTests.java, I think it is a test bug. >>>> >>>> I don't agree as yet. I have not been able to isolate the exact >>>> difference between what happens with your fix and what happens without. >>>> I know it relates to the presence of the error code, but also how we >>>> get >>>> AgentInitializationException instead of AgentLoadException. I need >>>> to be >>>> able to run the test outside of jtreg but that is proving to be very >>>> difficult to arrange. :( >>> >>> I finally managed to connect all the pieces on this. >>> >>> The basic problem is that with the proposed change >>> VirtualMachineImpl.execute() will throw AgentLoadException, which then >>> leads to the InternalError. Without the change we reach this block in >>> HotspotVirtualMachine.loadAgentLibrary: >>> >>> int result = readInt(in); >>> if (result != 0) { >>> throw new AgentInitializationException("Agent_OnAttach failed", >>> result); >>> } >>> >>> and so get the expected AgentInitializationException. >>> >>> Looking at the proposed change in jvmtiExport.cpp if we have the >>> original: >>> >>> st->print_cr("%d", result); >>> result = JNI_OK; >>> >>> then execute() manages to read a zero completion status from the stream, >>> and then loadAgentLibrary is able to read the actual "result" - whether >>> zero or not - from the stream. This is because the AttachListener code >>> calls op->complete(result, &st) where st is the stream where we wrote >>> the result value above in print_cr. Then if we look at, for example, >>> LinuxAttachOperation::complete, we will write "result" to the socket >>> first, followed by the contents of st. Hence on a successful operation >>> we can read 0 for execute, and then 0 for loadAgent. On error we read 0 >>> for execute and the actual error code for loadAgent. With the proposed >>> change execute() sees the real result (instead of JNI_OK) and so throws >>> AgentLoadException. >>> >>> So in summary there are two different error indicators written into the >>> stream, and we need the first to be zero unless some major error with >>> the actual attach functionality occurred - otherwise even if the "load" >>> failed in some other way, it is treated as a secondary error. >>> >>> With that in mind I suspect the real fix for the original issue is to >>> have Dcmd recognize that it also has to read two "results". Though I'm >>> not sure how exactly. >>> >>> David >>> ----- >>> >>>> David >>>> >>>> >>>> >>>>> If it is accepted, I will upload webrev for this redo task. >>>>> However, I cannot access (and fix) Oracle internal test. Can someone >>>>> help me? >>>>> (I cannot access JPRT. So I need a sponsor.) >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2016/09/13 9:00, David Holmes wrote: >>>>>> I think I see the problem. The attach framework tries to load the >>>>>> "instrument" agent library. Prior to 8164913 if this fails it >>>>>> actually >>>>>> appears to succeed. Now it fails, and that error propagates through. >>>>>> I'm guessing, at the moment, that the failure is due to a missing >>>>>> module related option. >>>>>> >>>>>> David >>>>>> >>>>>> On 13/09/2016 9:54 AM, David Holmes wrote: >>>>>>> Hi Yasumasa, >>>>>>> >>>>>>> On 13/09/2016 9:23 AM, Yasumasa Suenaga wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> Hmm, it has been backouted... >>>>>>>> >>>>>>>> I agree to David. This error seems to be a test bug. >>>>>>>> Can you share the test? I want to evaluate it. >>>>>>> >>>>>>> Sorry we can't share the tests. I took a quick look and it seems it >>>>>>> may >>>>>>> be related to the result code parsing (that I thought we ended up >>>>>>> not >>>>>>> touching!). >>>>>>> >>>>>>> Can you run a test of HotSpotVirtualMachine.loadAgent(null) and see >>>>>>> what >>>>>>> happens? We see AgentLoadException from the code that internally >>>>>>> tries >>>>>>> to load the "instrument" agent: >>>>>>> >>>>>>> Exception in thread "main" Failure: Unexpected exception during test >>>>>>> execution: java.lang.InternalError: instrument library is missing in >>>>>>> target VM >>>>>>> ... >>>>>>> Caused by: java.lang.InternalError: instrument library is missing in >>>>>>> target VM >>>>>>> ... >>>>>>> Caused by: com.sun.tools.attach.AgentLoadException: Failed to load >>>>>>> agent >>>>>>> library >>>>>>> >>>>>>> >>>>>>>> I do not agree to fix this bug in JDK 10 because VM might lie when >>>>>>>> the >>>>>>>> JVMTI agent could not be attached. IMHO this bug should be fixed >>>>>>>> in 9 >>>>>>>> GA, and we should fix testcase(s). >>>>>>> >>>>>>> I agree. It has to be noted that "we" failed to run all the >>>>>>> appropriate >>>>>>> tests before pushing this, which would have discovered the problem - >>>>>>> assuming it is actually a problem with the change and not an >>>>>>> unrelated >>>>>>> issue in our testing environment. >>>>>>> >>>>>>> Unfortunately I have some high priority tasks to handle right now, >>>>>>> so I >>>>>>> can't go into this in any more depth at the moment. >>>>>>> >>>>>>> David >>>>>>> ----- >>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> On 2016/09/13 6:15, David Holmes wrote: >>>>>>>>> For the record it looks like the tests involved are broken, in >>>>>>>>> that >>>>>>>>> because the VM used to lie about the success of attaching the >>>>>>>>> agent, the >>>>>>>>> tests expected different exceptions to occur. >>>>>>>>> >>>>>>>>> David >>>>>>>>> >>>>>>>>> On 13/09/2016 3:11 AM, harold seigel wrote: >>>>>>>>>> Looks good. >>>>>>>>>> >>>>>>>>>> Harold >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 9/12/2016 1:05 PM, Christian Tornqvist wrote: >>>>>>>>>>> Hi everyone, >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Please review this (clean) backout of the change for >>>>>>>>>>> JDK-8164913, it >>>>>>>>>>> failed >>>>>>>>>>> several tests in the nightly testing. The failures are >>>>>>>>>>> tracked in: >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165869 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Webrev: >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~ctornqvi/webrev/8165881/webrev.00/ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Bug: >>>>>>>>>>> >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165881 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Christian >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >> >> From david.holmes at oracle.com Wed Sep 14 01:42:17 2016 From: david.holmes at oracle.com (David Holmes) Date: Wed, 14 Sep 2016 11:42:17 +1000 Subject: RFR: 8165755: [JVMCI] replace use of vm_abort with vm_exit In-Reply-To: <42F9D0A5-3017-45DA-BBB6-92D2B76FBD55@oracle.com> References: <799CA13D-6BDC-4BF5-9241-515A684191F4@oracle.com> <6391B00B-AFBF-410C-A6A1-2ED95B35EBEB@twitter.com> <2F360221-7E6B-43BB-B69C-69A17777E5F2@oracle.com> <1c9b4d32-52a2-17a8-bfc5-f3cc7b494ba0@oracle.com> <5d26a289-a262-5882-8061-3bbc4faa4540@oracle.com> <52A83790-582E-4F8F-AA49-3F6DB3AB36CB@oracle.com> <42F9D0A5-3017-45DA-BBB6-92D2B76FBD55@oracle.com> Message-ID: Hi Doug, On 14/09/2016 12:41 AM, Doug Simon wrote: > Hi David, > > Do you still have concerns with the webrev in its current form? Sorry, yesterday was "one of those days". :) tl;dr: Your actual final exit code matches what JVM_Halt does, so that is fine. Your overall initialization and termination approach raises several general concerns. From a usability perspective the idea that JVMCI can terminate the VM at some arbitrary point after application startup is quite alarming! Looking at the initialization code I'm unclear why it isn't being serialized in some way at a level higher than the SystemDictionary - if it was then you wouldn't need the termination logic to deal with racing threads. Cheers, David > -Doug > >> On 12 Sep 2016, at 11:26, Doug Simon wrote: >> >> Hi David, >>> On 12 Sep 2016, at 03:24, David Holmes wrote: >>> >>> On 10/09/2016 6:29 AM, Coleen Phillimore wrote: >>>> On 9/9/16 3:31 PM, Doug Simon wrote: >>>>> Is vm_exit_during_initialization still the right choice when this >>>>> could be after VM initialization? JVMCI initialization is lazy and can >>>>> happen after the application has started. >>>> >>>> I don't actually know then. vm_exit tries to get threads to a safepoint >>>> first, but vm_abort(false) just shuts down the jvm. There aren't a lot >>>> of places we terminate the jvm. Maybe David Holmes knows. >>> >>> If JVMCI initialization is lazy and can happen after the application is started, why should it be a fatal error to fail to initialize it? How is the initialization triggered? Is there a synchronous call that could throw an exception? How does the user know that initialization failure will result in termination? >> >> JVMCI initialization is triggered by the first top tier compilation (when UseJVMCICompiler is true) or by an explicit request via the Java API to the JVMCI system. In the latter case, an exception is propagated to the caller who can decide what to do with it. However, in the former case, initialization is triggered by the CompileBroker on a compilation thread. We could make a failure during JVMCI initialization disable compilation but I?m sure that would be more confusing to a user than a VM exit clearly stating the invalid configuration option given. There?s already lazy configuration checking in other compilers that can cause the VM to exit (e.g. Compile::pd_compiler2_init in c2_init_x86.cpp) so this is not too different. >> >>> Can you not at least eagerly validate command-line options during VM initialization, even if you don't actually initialize JVMCI fully? >> >> Unfortunately not. The command line options are discovered (via a service loader) during initialization and this kind of activity is what we want to avoid during early VM startup. >> >>> >>> That aside it sounds like you want JVMCI initialization to do the equivalent of either Runtime.exit or Runtime.halt, depending on the exact termination semantics you want. If you call vm_exit directly then you get something like Runtime.halt >> >> Yes, we want Runtime.halt semantics in the case where invalid JVMCI options are specified. >> >> -Doug >> >>> David >>> ----- >>> >>>> Coleen >>>> >>>>> >>>>> Sent from my iPhone >>>>> >>>>>> On Sep 9, 2016, at 9:23 PM, Coleen Phillimore >>>>>> wrote: >>>>>> >>>>>> >>>>>> I think you want vm_exit_during_initialization() for that. Definitely >>>>>> not vm_abort, cause then it looks like an internal error/crash. >>>>>> >>>>>> Coleen >>>>>> >>>>>>> On 9/9/16 2:33 PM, Doug Simon wrote: >>>>>>> Can someone from the runtime team confirm that using vm_exit >>>>>>> (instead of vm_abort) is the best way to stop the VM when JVMCI >>>>>>> initialization fails (e.g., when invalid JVMCI options are provided >>>>>>> on the command line). Thanks! >>>>>>> >>>>>>> -Doug >>>>>>> >>>>>>> >>>>>>>> On 09 Sep 2016, at 19:48, Christian Thalinger >>>>>>>> wrote: >>>>>>>> >>>>>>>> I think this looks fine but maybe we should ask the runtime folks. >>>>>>>> >>>>>>>>> On Sep 8, 2016, at 11:01 PM, Doug Simon >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>> Calling vm_abort from multiple threads can cause nasty crashes >>>>>>>>> such as double free errors. We've seen this in Graal during JVMCI >>>>>>>>> initialization when an unknown Graal option is encountered. >>>>>>>>> Multiple compiler threads try to initialize JVMCI which fails with >>>>>>>>> an exception indicating the bad option: >>>>>>>>> >>>>>>>>> Uncaught exception at >>>>>>>>> /scratch/graaluser/buildslave/buildlog/ci_executor/main/graal-jvmci-8/src/share/vm/jvmci/jvmciCompiler.cpp:127 >>>>>>>>> >>>>>>>>> java.lang.ExceptionInInitializerError >>>>>>>>> at >>>>>>>>> jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.runtime(HotSpotJVMCIRuntime.java:85) >>>>>>>>> >>>>>>>>> at jdk.vm.ci.runtime.JVMCI.initializeRuntime(Native Method) >>>>>>>>> at jdk.vm.ci.runtime.JVMCI.(JVMCI.java:58) >>>>>>>>> Caused by: java.lang.IllegalArgumentException: Could not find >>>>>>>>> option OptSomethingThatDoesNotExcist >>>>>>>>> at >>>>>>>>> com.oracle.graal.options.OptionsParser.parseOption(OptionsParser.java:134) >>>>>>>>> >>>>>>>>> at >>>>>>>>> com.oracle.graal.options.OptionsParser.parseOptions(OptionsParser.java:62) >>>>>>>>> >>>>>>>>> at >>>>>>>>> com.oracle.graal.hotspot.HotSpotGraalCompilerFactory.initializeOptions(HotSpotGraalCompilerFactory.java:156) >>>>>>>>> >>>>>>>>> at >>>>>>>>> com.oracle.graal.hotspot.HotSpotGraalCompilerFactory.onSelection(HotSpotGraalCompilerFactory.java:86) >>>>>>>>> >>>>>>>>> at >>>>>>>>> jdk.vm.ci.hotspot.HotSpotJVMCICompilerConfig.getCompilerFactory(HotSpotJVMCICompilerConfig.java:96) >>>>>>>>> >>>>>>>>> at >>>>>>>>> jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.(HotSpotJVMCIRuntime.java:277) >>>>>>>>> >>>>>>>>> at >>>>>>>>> jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.(HotSpotJVMCIRuntime.java:67) >>>>>>>>> >>>>>>>>> at >>>>>>>>> jdk.vm.ci.hotspot.HotSpotJVMCIRuntime$DelayedInit.(HotSpotJVMCIRuntime.java:75) >>>>>>>>> >>>>>>>>> at >>>>>>>>> jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.runtime(HotSpotJVMCIRuntime.java:85) >>>>>>>>> >>>>>>>>> at jdk.vm.ci.runtime.JVMCI.initializeRuntime(Native Method) >>>>>>>>> at jdk.vm.ci.runtime.JVMCI.(JVMCI.java:58) >>>>>>>>> >>>>>>>>> The native JVMCI code then tries to exit the VM by calling >>>>>>>>> vm_abort. If multiple compiler threads do this concurrently, >>>>>>>>> certain destructors can be called twice as shown by these thread >>>>>>>>> dumps: >>>>>>>>> >>>>>>>>> thread #26: tid = 0x0019, 0x00007fff84280124 >>>>>>>>> libsystem_malloc.dylib`szone_size + 227, stop reason = signal SIGSTOP >>>>>>>>> frame #0: 0x00007fff84280124 libsystem_malloc.dylib`szone_size + 227 >>>>>>>>> frame #1: 0x00007fff8427fed5 libsystem_malloc.dylib`free + 61 >>>>>>>>> frame #2: 0x000000010ac95963 >>>>>>>>> libjvm.dylib`os::free(memblock=0x00007fedc86226e0, >>>>>>>>> memflags=mtInternal) + 307 at os.cpp:711 >>>>>>>>> frame #3: 0x000000010a2afc54 >>>>>>>>> libjvm.dylib`FreeHeap(p=0x00007fedc86226e0, memflags=mtInternal) + >>>>>>>>> 52 at allocation.inline.hpp:93 >>>>>>>>> frame #4: 0x000000010acf0a9f >>>>>>>>> libjvm.dylib`PerfData::~PerfData(this=0x00007fedc8622650) + 63 at >>>>>>>>> perfData.cpp:116 >>>>>>>>> frame #5: 0x000000010acf0ae5 >>>>>>>>> libjvm.dylib`PerfData::~PerfData(this=0x00007fedc8622650) + 21 at >>>>>>>>> perfData.cpp:114 >>>>>>>>> frame #6: 0x000000010acf163d >>>>>>>>> libjvm.dylib`PerfDataManager::destroy() + 109 at perfData.cpp:287 >>>>>>>>> frame #7: 0x000000010acf3f4d libjvm.dylib`perfMemory_exit() + 61 >>>>>>>>> at perfMemory.cpp:74 >>>>>>>>> frame #8: 0x000000010ac9bb0d libjvm.dylib`os::shutdown() + 13 at >>>>>>>>> os_bsd.cpp:1130 >>>>>>>>> frame #9: 0x000000010ac9bb55 >>>>>>>>> libjvm.dylib`os::abort(dump_core=false) + 21 at os_bsd.cpp:1150 >>>>>>>>> frame #10: 0x000000010a9188e7 >>>>>>>>> libjvm.dylib`vm_abort(dump_core=false) + 39 at java.cpp:666 >>>>>>>>> frame #11: 0x000000010aa4f1e7 >>>>>>>>> libjvm.dylib`JVMCIRuntime::abort_on_pending_exception(exception=Handle >>>>>>>>> @ 0x000070000175b208, message="Uncaught exception at >>>>>>>>> /Users/dsimon/graal/graal-jvmci-8/src/share/vm/jvmci/jvmciCompiler.cpp:127", >>>>>>>>> dump_core=false) + 167 at jvmciRuntime.cpp:992 >>>>>>>>> frame #12: 0x000000010aa17017 >>>>>>>>> libjvm.dylib`JVMCICompiler::compile_method(this=0x00007fedcb203050, method=0x000070000175b8d8, >>>>>>>>> entry_bci=-1, env=0x000070000175b8f0) + 311 at jvmciCompiler.cpp:127 >>>>>>>>> frame #13: 0x000000010a656cd2 >>>>>>>>> libjvm.dylib`CompileBroker::invoke_compiler_on_method(task=0x00007fedc853ca30) >>>>>>>>> + 1314 at compileBroker.cpp:2207 >>>>>>>>> >>>>>>>>> thread #23: tid = 0x0016, 0x00007fff91fcb122 >>>>>>>>> libsystem_kernel.dylib`__semwait_signal_nocancel + 10, stop reason >>>>>>>>> = signal SIGSTOP >>>>>>>>> frame #0: 0x00007fff91fcb122 >>>>>>>>> libsystem_kernel.dylib`__semwait_signal_nocancel + 10 >>>>>>>>> frame #1: 0x00007fff9578c318 libsystem_c.dylib`nanosleep$NOCANCEL >>>>>>>>> + 188 >>>>>>>>> frame #2: 0x00007fff957b62ce libsystem_c.dylib`usleep$NOCANCEL + 54 >>>>>>>>> frame #3: 0x00007fff957e46e9 libsystem_c.dylib`abort + 139 >>>>>>>>> frame #4: 0x00007fff8428c396 libsystem_malloc.dylib`szone_error + 626 >>>>>>>>> frame #5: 0x000000010ac95963 >>>>>>>>> libjvm.dylib`os::free(memblock=0x00007fedc8601cd0, >>>>>>>>> memflags=mtInternal) + 307 at os.cpp:711 >>>>>>>>> frame #6: 0x000000010a2afc54 >>>>>>>>> libjvm.dylib`FreeHeap(p=0x00007fedc8601cd0, memflags=mtInternal) + >>>>>>>>> 52 at allocation.inline.hpp:93 >>>>>>>>> frame #7: 0x000000010acf0a9f >>>>>>>>> libjvm.dylib`PerfData::~PerfData(this=0x00007fedc8601c60) + 63 at >>>>>>>>> perfData.cpp:116 >>>>>>>>> frame #8: 0x000000010acf0ae5 >>>>>>>>> libjvm.dylib`PerfData::~PerfData(this=0x00007fedc8601c60) + 21 at >>>>>>>>> perfData.cpp:114 >>>>>>>>> frame #9: 0x000000010acf163d >>>>>>>>> libjvm.dylib`PerfDataManager::destroy() + 109 at perfData.cpp:287 >>>>>>>>> frame #10: 0x000000010acf3f4d libjvm.dylib`perfMemory_exit() + 61 >>>>>>>>> at perfMemory.cpp:74 >>>>>>>>> frame #11: 0x000000010ac9bb0d libjvm.dylib`os::shutdown() + 13 at >>>>>>>>> os_bsd.cpp:1130 >>>>>>>>> frame #12: 0x000000010ac9bb55 >>>>>>>>> libjvm.dylib`os::abort(dump_core=false) + 21 at os_bsd.cpp:1150 >>>>>>>>> frame #13: 0x000000010a9188e7 >>>>>>>>> libjvm.dylib`vm_abort(dump_core=false) + 39 at java.cpp:666 >>>>>>>>> frame #14: 0x000000010aa4f1e7 >>>>>>>>> libjvm.dylib`JVMCIRuntime::abort_on_pending_exception(exception=Handle >>>>>>>>> @ 0x0000700001452208, message="Uncaught exception at >>>>>>>>> /Users/dsimon/graal/graal-jvmci-8/src/share/vm/jvmci/jvmciCompiler.cpp:127", >>>>>>>>> dump_core=false) + 167 at jvmciRuntime.cpp:992 >>>>>>>>> frame #15: 0x000000010aa17017 >>>>>>>>> libjvm.dylib`JVMCICompiler::compile_method(this=0x00007fedcb203050, method=0x00007000014528d8, >>>>>>>>> entry_bci=-1, env=0x00007000014528f0) + 311 at jvmciCompiler.cpp:127 >>>>>>>>> frame #16: 0x000000010a656cd2 >>>>>>>>> libjvm.dylib`CompileBroker::invoke_compiler_on_method(task=0x00007fedc862a320) >>>>>>>>> + 1314 at compileBroker.cpp:2207 >>>>>>>>> >>>>>>>>> >>>>>>>>> This webrev replaces calls to vm_abort() with before_exit() + >>>>>>>>> vm_exit(). The latter is thread safe. >>>>>>>>> >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165755 >>>>>>>>> http://cr.openjdk.java.net/~dnsimon/8165755/ >>>>>>>>> >>>>>>>>> -Doug >>>> >> > From shafi.s.ahmad at oracle.com Wed Sep 14 02:29:36 2016 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Tue, 13 Sep 2016 19:29:36 -0700 (PDT) Subject: [8u] RFR for JDK-8154324: Request to backport JDK-6515172 to 8u In-Reply-To: <26e8f72e-0a8f-6449-d824-48b5d49635dd@oracle.com> References: <60e87114-fcc2-4a0f-a31b-8440faf91555@default> <3695d333-eb30-3a84-9b8e-b0264a9087c9@oracle.com> <26e8f72e-0a8f-6449-d824-48b5d49635dd@oracle.com> Message-ID: <8c2c1492-4c21-4066-af25-1516e797e399@default> Thanks David for review. Regards, Shafi > -----Original Message----- > From: David Holmes > Sent: Wednesday, September 14, 2016 3:21 AM > To: Shafi Ahmad; hotspot-dev at openjdk.java.net > Subject: Re: [8u] RFR for JDK-8154324: Request to backport JDK-6515172 to > 8u > > Hi Shafi, > > On 12/09/2016 11:20 PM, Shafi Ahmad wrote: > > Hi, > > > > Please find updated webrev: > > http://cr.openjdk.java.net/~shshahma/8154324/webrev.01/ > > I have incorporated both review comment by David. > > 1. Removed unrelated comment. > > 2. Removed unreferenced code ' diagnostic(bool, UseCpuAllocPath, false,' > > Incorrect comment here in globals_linux.hpp: > > ! diagnostic(bool, PrintActiveCpus, false, \ > ! "Use CPU_ALLOC code path in os::active_processor_count ") > > should be: > > "Print the number of CPUs detected in os::active_processor_count" > > --- > > os_linux.cpp: > > if(PrintActiveCpus) { > > Need space after if. > > -- > > Otherwise okay. No need to see updated webrev if above are fixed. > > Thanks, > David > > > > > Regards, > > Shafi > > > >> -----Original Message----- > >> From: David Holmes > >> Sent: Friday, September 09, 2016 11:29 AM > >> To: Shafi Ahmad; hotspot-dev at openjdk.java.net > >> Subject: Re: [8u] RFR for JDK-8154324: Request to backport > >> JDK-6515172 to 8u > >> > >> Hi Shafi, > >> > >> On 9/09/2016 3:46 PM, Shafi Ahmad wrote: > >>> Hi, > >>> > >>> Please review the backport [modified] of bug: "JDK-6515172 > >> Runtime.availableProcessors() ignores Linux taskset command" to jdk8u. > >>> > >>> Please note that the backport is not clean and we can't do as it is. > >>> Please > >> note > >>> 1. The changes are made by 'Andreas Eriksson' who left Oracle. > >>> 2. There is difference in logging mechanism in jdk9 and jdk8 is > >>> different > >> and file logTag.hpp doesn't exists in jdk8. > >>> 3. Newly added test pass after this change. It fails without this change. > >>> > >>> Webrev link: > http://cr.openjdk.java.net/~shshahma/8154324/webrev.00/ > >>> Jdk9 bug: https://bugs.openjdk.java.net/browse/JDK-6515172 > >>> Jdk8 bug: https://bugs.openjdk.java.net/browse/JDK-8154324 > >>> Original patch pushed to jdk9: > >>> http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/c5480d4abfe4 > >> > >> I worked extensively with Andreas on this as there were a number of > issues. > >> I'll have to try and find those discussions to see where we ended up. > >> > >> The backport as stands is not quite appropriate. For example it adds: > >> > >> diagnostic(bool, UseCpuAllocPath, false, > >> > >> but that does not exist in the actual code for 8. > >> > >> Also this comment: > >> > >> + // If it appears there may be more than 1024 processors then we do > >> + a // dynamic check - see 6515172 for details. > >> > >> is wrong as there is no dynamic check in this version of the code. > >> > >> The last I recall with this is that there were issues caused by > >> building with one version of glibc and running (or trying to) on > >> later versions of glibc. But as I said I will have to see if I have the discussion > from that effort. > >> > >> David > >> ---- > >> > >>> Testing: jprt and running jtreg. > >>> > >>> Regards, > >>> Shafi > >>> From shafi.s.ahmad at oracle.com Wed Sep 14 02:46:07 2016 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Tue, 13 Sep 2016 19:46:07 -0700 (PDT) Subject: [8u] RFR for JDK-8154324: Request to backport JDK-6515172 to 8u In-Reply-To: <26e8f72e-0a8f-6449-d824-48b5d49635dd@oracle.com> References: <60e87114-fcc2-4a0f-a31b-8440faf91555@default> <3695d333-eb30-3a84-9b8e-b0264a9087c9@oracle.com> <26e8f72e-0a8f-6449-d824-48b5d49635dd@oracle.com> Message-ID: May I get second review for this. Updated webrev: http://cr.openjdk.java.net/~shshahma/8154324/webrev.02/ Regards, Shafi > -----Original Message----- > From: David Holmes > Sent: Wednesday, September 14, 2016 3:21 AM > To: Shafi Ahmad; hotspot-dev at openjdk.java.net > Subject: Re: [8u] RFR for JDK-8154324: Request to backport JDK-6515172 to > 8u > > Hi Shafi, > > On 12/09/2016 11:20 PM, Shafi Ahmad wrote: > > Hi, > > > > Please find updated webrev: > > http://cr.openjdk.java.net/~shshahma/8154324/webrev.01/ > > I have incorporated both review comment by David. > > 1. Removed unrelated comment. > > 2. Removed unreferenced code ' diagnostic(bool, UseCpuAllocPath, false,' > > Incorrect comment here in globals_linux.hpp: > > ! diagnostic(bool, PrintActiveCpus, false, \ > ! "Use CPU_ALLOC code path in os::active_processor_count ") > > should be: > > "Print the number of CPUs detected in os::active_processor_count" > > --- > > os_linux.cpp: > > if(PrintActiveCpus) { > > Need space after if. > > -- > > Otherwise okay. No need to see updated webrev if above are fixed. > > Thanks, > David > > > > > Regards, > > Shafi > > > >> -----Original Message----- > >> From: David Holmes > >> Sent: Friday, September 09, 2016 11:29 AM > >> To: Shafi Ahmad; hotspot-dev at openjdk.java.net > >> Subject: Re: [8u] RFR for JDK-8154324: Request to backport > >> JDK-6515172 to 8u > >> > >> Hi Shafi, > >> > >> On 9/09/2016 3:46 PM, Shafi Ahmad wrote: > >>> Hi, > >>> > >>> Please review the backport [modified] of bug: "JDK-6515172 > >> Runtime.availableProcessors() ignores Linux taskset command" to jdk8u. > >>> > >>> Please note that the backport is not clean and we can't do as it is. > >>> Please > >> note > >>> 1. The changes are made by 'Andreas Eriksson' who left Oracle. > >>> 2. There is difference in logging mechanism in jdk9 and jdk8 is > >>> different > >> and file logTag.hpp doesn't exists in jdk8. > >>> 3. Newly added test pass after this change. It fails without this change. > >>> > >>> Webrev link: > http://cr.openjdk.java.net/~shshahma/8154324/webrev.00/ > >>> Jdk9 bug: https://bugs.openjdk.java.net/browse/JDK-6515172 > >>> Jdk8 bug: https://bugs.openjdk.java.net/browse/JDK-8154324 > >>> Original patch pushed to jdk9: > >>> http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/c5480d4abfe4 > >> > >> I worked extensively with Andreas on this as there were a number of > issues. > >> I'll have to try and find those discussions to see where we ended up. > >> > >> The backport as stands is not quite appropriate. For example it adds: > >> > >> diagnostic(bool, UseCpuAllocPath, false, > >> > >> but that does not exist in the actual code for 8. > >> > >> Also this comment: > >> > >> + // If it appears there may be more than 1024 processors then we do > >> + a // dynamic check - see 6515172 for details. > >> > >> is wrong as there is no dynamic check in this version of the code. > >> > >> The last I recall with this is that there were issues caused by > >> building with one version of glibc and running (or trying to) on > >> later versions of glibc. But as I said I will have to see if I have the discussion > from that effort. > >> > >> David > >> ---- > >> > >>> Testing: jprt and running jtreg. > >>> > >>> Regards, > >>> Shafi > >>> From doug.simon at oracle.com Wed Sep 14 07:53:34 2016 From: doug.simon at oracle.com (Doug Simon) Date: Wed, 14 Sep 2016 09:53:34 +0200 Subject: RFR: 8165755: [JVMCI] replace use of vm_abort with vm_exit In-Reply-To: References: <799CA13D-6BDC-4BF5-9241-515A684191F4@oracle.com> <6391B00B-AFBF-410C-A6A1-2ED95B35EBEB@twitter.com> <2F360221-7E6B-43BB-B69C-69A17777E5F2@oracle.com> <1c9b4d32-52a2-17a8-bfc5-f3cc7b494ba0@oracle.com> <5d26a289-a262-5882-8061-3bbc4faa4540@oracle.com> <52A83790-582E-4F8F-AA49-3F6DB3AB36CB@oracle.com> <42F9D0A5-3017-45DA-BBB6-92D2B76FBD55@oracle.com> Message-ID: <416C33DC-A44B-4D0F-AB4F-C83BC5C52265@oracle.com> > On 14 Sep 2016, at 03:42, David Holmes wrote: > > Hi Doug, > > On 14/09/2016 12:41 AM, Doug Simon wrote: >> Hi David, >> >> Do you still have concerns with the webrev in its current form? > > Sorry, yesterday was "one of those days". :) No problem, we have those around here sometimes as well ;-) > tl;dr: Your actual final exit code matches what JVM_Halt does, so that is fine. > > Your overall initialization and termination approach raises several general concerns. From a usability perspective the idea that JVMCI can terminate the VM at some arbitrary point after application startup is quite alarming! It?s not completely arbitrary. Top tier compilation kicks in very soon after VM startup for all but HelloWorld type applications. Also, termination happens only due to incorrect command line in which case I?d say termination is what?s normal/expected. > > Looking at the initialization code I'm unclear why it isn't being serialized in some way at a level higher than the SystemDictionary - if it was then you wouldn't need the termination logic to deal with racing threads. I?m not sure I follow. JVMCI initialization is serialized on the initialization of the jdk.vm.ci.runtime.JVMCI class. Since class initialization errors are preserved, multiple compiler threads can see the same error so will each try to go down the termination path. It?s the latter race that needs serialization which is done now in JVMCICompiler::abort_on_pending_exception. -Doug > > Cheers, > David > >> -Doug >> >>> On 12 Sep 2016, at 11:26, Doug Simon wrote: >>> >>> Hi David, >>>> On 12 Sep 2016, at 03:24, David Holmes wrote: >>>> >>>> On 10/09/2016 6:29 AM, Coleen Phillimore wrote: >>>>> On 9/9/16 3:31 PM, Doug Simon wrote: >>>>>> Is vm_exit_during_initialization still the right choice when this >>>>>> could be after VM initialization? JVMCI initialization is lazy and can >>>>>> happen after the application has started. >>>>> >>>>> I don't actually know then. vm_exit tries to get threads to a safepoint >>>>> first, but vm_abort(false) just shuts down the jvm. There aren't a lot >>>>> of places we terminate the jvm. Maybe David Holmes knows. >>>> >>>> If JVMCI initialization is lazy and can happen after the application is started, why should it be a fatal error to fail to initialize it? How is the initialization triggered? Is there a synchronous call that could throw an exception? How does the user know that initialization failure will result in termination? >>> >>> JVMCI initialization is triggered by the first top tier compilation (when UseJVMCICompiler is true) or by an explicit request via the Java API to the JVMCI system. In the latter case, an exception is propagated to the caller who can decide what to do with it. However, in the former case, initialization is triggered by the CompileBroker on a compilation thread. We could make a failure during JVMCI initialization disable compilation but I?m sure that would be more confusing to a user than a VM exit clearly stating the invalid configuration option given. There?s already lazy configuration checking in other compilers that can cause the VM to exit (e.g. Compile::pd_compiler2_init in c2_init_x86.cpp) so this is not too different. >>> >>>> Can you not at least eagerly validate command-line options during VM initialization, even if you don't actually initialize JVMCI fully? >>> >>> Unfortunately not. The command line options are discovered (via a service loader) during initialization and this kind of activity is what we want to avoid during early VM startup. >>> >>>> >>>> That aside it sounds like you want JVMCI initialization to do the equivalent of either Runtime.exit or Runtime.halt, depending on the exact termination semantics you want. If you call vm_exit directly then you get something like Runtime.halt >>> >>> Yes, we want Runtime.halt semantics in the case where invalid JVMCI options are specified. >>> >>> -Doug >>> >>>> David >>>> ----- >>>> >>>>> Coleen >>>>> >>>>>> >>>>>> Sent from my iPhone >>>>>> >>>>>>> On Sep 9, 2016, at 9:23 PM, Coleen Phillimore >>>>>>> wrote: >>>>>>> >>>>>>> >>>>>>> I think you want vm_exit_during_initialization() for that. Definitely >>>>>>> not vm_abort, cause then it looks like an internal error/crash. >>>>>>> >>>>>>> Coleen >>>>>>> >>>>>>>> On 9/9/16 2:33 PM, Doug Simon wrote: >>>>>>>> Can someone from the runtime team confirm that using vm_exit >>>>>>>> (instead of vm_abort) is the best way to stop the VM when JVMCI >>>>>>>> initialization fails (e.g., when invalid JVMCI options are provided >>>>>>>> on the command line). Thanks! >>>>>>>> >>>>>>>> -Doug >>>>>>>> >>>>>>>> >>>>>>>>> On 09 Sep 2016, at 19:48, Christian Thalinger >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>> I think this looks fine but maybe we should ask the runtime folks. >>>>>>>>> >>>>>>>>>> On Sep 8, 2016, at 11:01 PM, Doug Simon >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> Calling vm_abort from multiple threads can cause nasty crashes >>>>>>>>>> such as double free errors. We've seen this in Graal during JVMCI >>>>>>>>>> initialization when an unknown Graal option is encountered. >>>>>>>>>> Multiple compiler threads try to initialize JVMCI which fails with >>>>>>>>>> an exception indicating the bad option: >>>>>>>>>> >>>>>>>>>> Uncaught exception at >>>>>>>>>> /scratch/graaluser/buildslave/buildlog/ci_executor/main/graal-jvmci-8/src/share/vm/jvmci/jvmciCompiler.cpp:127 >>>>>>>>>> >>>>>>>>>> java.lang.ExceptionInInitializerError >>>>>>>>>> at >>>>>>>>>> jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.runtime(HotSpotJVMCIRuntime.java:85) >>>>>>>>>> >>>>>>>>>> at jdk.vm.ci.runtime.JVMCI.initializeRuntime(Native Method) >>>>>>>>>> at jdk.vm.ci.runtime.JVMCI.(JVMCI.java:58) >>>>>>>>>> Caused by: java.lang.IllegalArgumentException: Could not find >>>>>>>>>> option OptSomethingThatDoesNotExcist >>>>>>>>>> at >>>>>>>>>> com.oracle.graal.options.OptionsParser.parseOption(OptionsParser.java:134) >>>>>>>>>> >>>>>>>>>> at >>>>>>>>>> com.oracle.graal.options.OptionsParser.parseOptions(OptionsParser.java:62) >>>>>>>>>> >>>>>>>>>> at >>>>>>>>>> com.oracle.graal.hotspot.HotSpotGraalCompilerFactory.initializeOptions(HotSpotGraalCompilerFactory.java:156) >>>>>>>>>> >>>>>>>>>> at >>>>>>>>>> com.oracle.graal.hotspot.HotSpotGraalCompilerFactory.onSelection(HotSpotGraalCompilerFactory.java:86) >>>>>>>>>> >>>>>>>>>> at >>>>>>>>>> jdk.vm.ci.hotspot.HotSpotJVMCICompilerConfig.getCompilerFactory(HotSpotJVMCICompilerConfig.java:96) >>>>>>>>>> >>>>>>>>>> at >>>>>>>>>> jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.(HotSpotJVMCIRuntime.java:277) >>>>>>>>>> >>>>>>>>>> at >>>>>>>>>> jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.(HotSpotJVMCIRuntime.java:67) >>>>>>>>>> >>>>>>>>>> at >>>>>>>>>> jdk.vm.ci.hotspot.HotSpotJVMCIRuntime$DelayedInit.(HotSpotJVMCIRuntime.java:75) >>>>>>>>>> >>>>>>>>>> at >>>>>>>>>> jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.runtime(HotSpotJVMCIRuntime.java:85) >>>>>>>>>> >>>>>>>>>> at jdk.vm.ci.runtime.JVMCI.initializeRuntime(Native Method) >>>>>>>>>> at jdk.vm.ci.runtime.JVMCI.(JVMCI.java:58) >>>>>>>>>> >>>>>>>>>> The native JVMCI code then tries to exit the VM by calling >>>>>>>>>> vm_abort. If multiple compiler threads do this concurrently, >>>>>>>>>> certain destructors can be called twice as shown by these thread >>>>>>>>>> dumps: >>>>>>>>>> >>>>>>>>>> thread #26: tid = 0x0019, 0x00007fff84280124 >>>>>>>>>> libsystem_malloc.dylib`szone_size + 227, stop reason = signal SIGSTOP >>>>>>>>>> frame #0: 0x00007fff84280124 libsystem_malloc.dylib`szone_size + 227 >>>>>>>>>> frame #1: 0x00007fff8427fed5 libsystem_malloc.dylib`free + 61 >>>>>>>>>> frame #2: 0x000000010ac95963 >>>>>>>>>> libjvm.dylib`os::free(memblock=0x00007fedc86226e0, >>>>>>>>>> memflags=mtInternal) + 307 at os.cpp:711 >>>>>>>>>> frame #3: 0x000000010a2afc54 >>>>>>>>>> libjvm.dylib`FreeHeap(p=0x00007fedc86226e0, memflags=mtInternal) + >>>>>>>>>> 52 at allocation.inline.hpp:93 >>>>>>>>>> frame #4: 0x000000010acf0a9f >>>>>>>>>> libjvm.dylib`PerfData::~PerfData(this=0x00007fedc8622650) + 63 at >>>>>>>>>> perfData.cpp:116 >>>>>>>>>> frame #5: 0x000000010acf0ae5 >>>>>>>>>> libjvm.dylib`PerfData::~PerfData(this=0x00007fedc8622650) + 21 at >>>>>>>>>> perfData.cpp:114 >>>>>>>>>> frame #6: 0x000000010acf163d >>>>>>>>>> libjvm.dylib`PerfDataManager::destroy() + 109 at perfData.cpp:287 >>>>>>>>>> frame #7: 0x000000010acf3f4d libjvm.dylib`perfMemory_exit() + 61 >>>>>>>>>> at perfMemory.cpp:74 >>>>>>>>>> frame #8: 0x000000010ac9bb0d libjvm.dylib`os::shutdown() + 13 at >>>>>>>>>> os_bsd.cpp:1130 >>>>>>>>>> frame #9: 0x000000010ac9bb55 >>>>>>>>>> libjvm.dylib`os::abort(dump_core=false) + 21 at os_bsd.cpp:1150 >>>>>>>>>> frame #10: 0x000000010a9188e7 >>>>>>>>>> libjvm.dylib`vm_abort(dump_core=false) + 39 at java.cpp:666 >>>>>>>>>> frame #11: 0x000000010aa4f1e7 >>>>>>>>>> libjvm.dylib`JVMCIRuntime::abort_on_pending_exception(exception=Handle >>>>>>>>>> @ 0x000070000175b208, message="Uncaught exception at >>>>>>>>>> /Users/dsimon/graal/graal-jvmci-8/src/share/vm/jvmci/jvmciCompiler.cpp:127", >>>>>>>>>> dump_core=false) + 167 at jvmciRuntime.cpp:992 >>>>>>>>>> frame #12: 0x000000010aa17017 >>>>>>>>>> libjvm.dylib`JVMCICompiler::compile_method(this=0x00007fedcb203050, method=0x000070000175b8d8, >>>>>>>>>> entry_bci=-1, env=0x000070000175b8f0) + 311 at jvmciCompiler.cpp:127 >>>>>>>>>> frame #13: 0x000000010a656cd2 >>>>>>>>>> libjvm.dylib`CompileBroker::invoke_compiler_on_method(task=0x00007fedc853ca30) >>>>>>>>>> + 1314 at compileBroker.cpp:2207 >>>>>>>>>> >>>>>>>>>> thread #23: tid = 0x0016, 0x00007fff91fcb122 >>>>>>>>>> libsystem_kernel.dylib`__semwait_signal_nocancel + 10, stop reason >>>>>>>>>> = signal SIGSTOP >>>>>>>>>> frame #0: 0x00007fff91fcb122 >>>>>>>>>> libsystem_kernel.dylib`__semwait_signal_nocancel + 10 >>>>>>>>>> frame #1: 0x00007fff9578c318 libsystem_c.dylib`nanosleep$NOCANCEL >>>>>>>>>> + 188 >>>>>>>>>> frame #2: 0x00007fff957b62ce libsystem_c.dylib`usleep$NOCANCEL + 54 >>>>>>>>>> frame #3: 0x00007fff957e46e9 libsystem_c.dylib`abort + 139 >>>>>>>>>> frame #4: 0x00007fff8428c396 libsystem_malloc.dylib`szone_error + 626 >>>>>>>>>> frame #5: 0x000000010ac95963 >>>>>>>>>> libjvm.dylib`os::free(memblock=0x00007fedc8601cd0, >>>>>>>>>> memflags=mtInternal) + 307 at os.cpp:711 >>>>>>>>>> frame #6: 0x000000010a2afc54 >>>>>>>>>> libjvm.dylib`FreeHeap(p=0x00007fedc8601cd0, memflags=mtInternal) + >>>>>>>>>> 52 at allocation.inline.hpp:93 >>>>>>>>>> frame #7: 0x000000010acf0a9f >>>>>>>>>> libjvm.dylib`PerfData::~PerfData(this=0x00007fedc8601c60) + 63 at >>>>>>>>>> perfData.cpp:116 >>>>>>>>>> frame #8: 0x000000010acf0ae5 >>>>>>>>>> libjvm.dylib`PerfData::~PerfData(this=0x00007fedc8601c60) + 21 at >>>>>>>>>> perfData.cpp:114 >>>>>>>>>> frame #9: 0x000000010acf163d >>>>>>>>>> libjvm.dylib`PerfDataManager::destroy() + 109 at perfData.cpp:287 >>>>>>>>>> frame #10: 0x000000010acf3f4d libjvm.dylib`perfMemory_exit() + 61 >>>>>>>>>> at perfMemory.cpp:74 >>>>>>>>>> frame #11: 0x000000010ac9bb0d libjvm.dylib`os::shutdown() + 13 at >>>>>>>>>> os_bsd.cpp:1130 >>>>>>>>>> frame #12: 0x000000010ac9bb55 >>>>>>>>>> libjvm.dylib`os::abort(dump_core=false) + 21 at os_bsd.cpp:1150 >>>>>>>>>> frame #13: 0x000000010a9188e7 >>>>>>>>>> libjvm.dylib`vm_abort(dump_core=false) + 39 at java.cpp:666 >>>>>>>>>> frame #14: 0x000000010aa4f1e7 >>>>>>>>>> libjvm.dylib`JVMCIRuntime::abort_on_pending_exception(exception=Handle >>>>>>>>>> @ 0x0000700001452208, message="Uncaught exception at >>>>>>>>>> /Users/dsimon/graal/graal-jvmci-8/src/share/vm/jvmci/jvmciCompiler.cpp:127", >>>>>>>>>> dump_core=false) + 167 at jvmciRuntime.cpp:992 >>>>>>>>>> frame #15: 0x000000010aa17017 >>>>>>>>>> libjvm.dylib`JVMCICompiler::compile_method(this=0x00007fedcb203050, method=0x00007000014528d8, >>>>>>>>>> entry_bci=-1, env=0x00007000014528f0) + 311 at jvmciCompiler.cpp:127 >>>>>>>>>> frame #16: 0x000000010a656cd2 >>>>>>>>>> libjvm.dylib`CompileBroker::invoke_compiler_on_method(task=0x00007fedc862a320) >>>>>>>>>> + 1314 at compileBroker.cpp:2207 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> This webrev replaces calls to vm_abort() with before_exit() + >>>>>>>>>> vm_exit(). The latter is thread safe. >>>>>>>>>> >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165755 >>>>>>>>>> http://cr.openjdk.java.net/~dnsimon/8165755/ >>>>>>>>>> >>>>>>>>>> -Doug >>>>> >>> >> From david.holmes at oracle.com Wed Sep 14 09:25:23 2016 From: david.holmes at oracle.com (David Holmes) Date: Wed, 14 Sep 2016 19:25:23 +1000 Subject: RFR: 8165755: [JVMCI] replace use of vm_abort with vm_exit In-Reply-To: <416C33DC-A44B-4D0F-AB4F-C83BC5C52265@oracle.com> References: <799CA13D-6BDC-4BF5-9241-515A684191F4@oracle.com> <6391B00B-AFBF-410C-A6A1-2ED95B35EBEB@twitter.com> <2F360221-7E6B-43BB-B69C-69A17777E5F2@oracle.com> <1c9b4d32-52a2-17a8-bfc5-f3cc7b494ba0@oracle.com> <5d26a289-a262-5882-8061-3bbc4faa4540@oracle.com> <52A83790-582E-4F8F-AA49-3F6DB3AB36CB@oracle.com> <42F9D0A5-3017-45DA-BBB6-92D2B76FBD55@oracle.com> <416C33DC-A44B-4D0F-AB4F-C83BC5C52265@oracle.com> Message-ID: On 14/09/2016 5:53 PM, Doug Simon wrote: > >> On 14 Sep 2016, at 03:42, David Holmes wrote: >> >> Hi Doug, >> >> On 14/09/2016 12:41 AM, Doug Simon wrote: >>> Hi David, >>> >>> Do you still have concerns with the webrev in its current form? >> >> Sorry, yesterday was "one of those days". :) > > No problem, we have those around here sometimes as well ;-) > >> tl;dr: Your actual final exit code matches what JVM_Halt does, so that is fine. >> >> Your overall initialization and termination approach raises several general concerns. From a usability perspective the idea that JVMCI can terminate the VM at some arbitrary point after application startup is quite alarming! > > It?s not completely arbitrary. Top tier compilation kicks in very soon after VM startup for all but HelloWorld type applications. Also, termination happens only due to incorrect command line in which case I?d say termination is what?s normal/expected. Yes but people also expect bad command line options to be detected before anything else happens that is part of the application logic. > >> >> Looking at the initialization code I'm unclear why it isn't being serialized in some way at a level higher than the SystemDictionary - if it was then you wouldn't need the termination logic to deal with racing threads. > > I?m not sure I follow. JVMCI initialization is serialized on the initialization of the jdk.vm.ci.runtime.JVMCI class. Since class initialization errors are preserved, multiple compiler threads can see the same error so will each try to go down the termination path. It?s the latter race that needs serialization which is done now in JVMCICompiler::abort_on_pending_exception. The code I looked at has the initialization logic called from JVMCICompiler::compile_method. I can't say that isn't called from init of the JVMCI class, but it sure doesn't sound like it. :) If the thread that encounters the initial error is holding an initialization "lock" (Java monitor in JVMCI's case I expect) when it takes the termination path then there's no race in taking that path. Anyway, just musing. Cheers, David > -Doug > >> >> Cheers, >> David >> >>> -Doug >>> >>>> On 12 Sep 2016, at 11:26, Doug Simon wrote: >>>> >>>> Hi David, >>>>> On 12 Sep 2016, at 03:24, David Holmes wrote: >>>>> >>>>> On 10/09/2016 6:29 AM, Coleen Phillimore wrote: >>>>>> On 9/9/16 3:31 PM, Doug Simon wrote: >>>>>>> Is vm_exit_during_initialization still the right choice when this >>>>>>> could be after VM initialization? JVMCI initialization is lazy and can >>>>>>> happen after the application has started. >>>>>> >>>>>> I don't actually know then. vm_exit tries to get threads to a safepoint >>>>>> first, but vm_abort(false) just shuts down the jvm. There aren't a lot >>>>>> of places we terminate the jvm. Maybe David Holmes knows. >>>>> >>>>> If JVMCI initialization is lazy and can happen after the application is started, why should it be a fatal error to fail to initialize it? How is the initialization triggered? Is there a synchronous call that could throw an exception? How does the user know that initialization failure will result in termination? >>>> >>>> JVMCI initialization is triggered by the first top tier compilation (when UseJVMCICompiler is true) or by an explicit request via the Java API to the JVMCI system. In the latter case, an exception is propagated to the caller who can decide what to do with it. However, in the former case, initialization is triggered by the CompileBroker on a compilation thread. We could make a failure during JVMCI initialization disable compilation but I?m sure that would be more confusing to a user than a VM exit clearly stating the invalid configuration option given. There?s already lazy configuration checking in other compilers that can cause the VM to exit (e.g. Compile::pd_compiler2_init in c2_init_x86.cpp) so this is not too different. >>>> >>>>> Can you not at least eagerly validate command-line options during VM initialization, even if you don't actually initialize JVMCI fully? >>>> >>>> Unfortunately not. The command line options are discovered (via a service loader) during initialization and this kind of activity is what we want to avoid during early VM startup. >>>> >>>>> >>>>> That aside it sounds like you want JVMCI initialization to do the equivalent of either Runtime.exit or Runtime.halt, depending on the exact termination semantics you want. If you call vm_exit directly then you get something like Runtime.halt >>>> >>>> Yes, we want Runtime.halt semantics in the case where invalid JVMCI options are specified. >>>> >>>> -Doug >>>> >>>>> David >>>>> ----- >>>>> >>>>>> Coleen >>>>>> >>>>>>> >>>>>>> Sent from my iPhone >>>>>>> >>>>>>>> On Sep 9, 2016, at 9:23 PM, Coleen Phillimore >>>>>>>> wrote: >>>>>>>> >>>>>>>> >>>>>>>> I think you want vm_exit_during_initialization() for that. Definitely >>>>>>>> not vm_abort, cause then it looks like an internal error/crash. >>>>>>>> >>>>>>>> Coleen >>>>>>>> >>>>>>>>> On 9/9/16 2:33 PM, Doug Simon wrote: >>>>>>>>> Can someone from the runtime team confirm that using vm_exit >>>>>>>>> (instead of vm_abort) is the best way to stop the VM when JVMCI >>>>>>>>> initialization fails (e.g., when invalid JVMCI options are provided >>>>>>>>> on the command line). Thanks! >>>>>>>>> >>>>>>>>> -Doug >>>>>>>>> >>>>>>>>> >>>>>>>>>> On 09 Sep 2016, at 19:48, Christian Thalinger >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> I think this looks fine but maybe we should ask the runtime folks. >>>>>>>>>> >>>>>>>>>>> On Sep 8, 2016, at 11:01 PM, Doug Simon >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> Calling vm_abort from multiple threads can cause nasty crashes >>>>>>>>>>> such as double free errors. We've seen this in Graal during JVMCI >>>>>>>>>>> initialization when an unknown Graal option is encountered. >>>>>>>>>>> Multiple compiler threads try to initialize JVMCI which fails with >>>>>>>>>>> an exception indicating the bad option: >>>>>>>>>>> >>>>>>>>>>> Uncaught exception at >>>>>>>>>>> /scratch/graaluser/buildslave/buildlog/ci_executor/main/graal-jvmci-8/src/share/vm/jvmci/jvmciCompiler.cpp:127 >>>>>>>>>>> >>>>>>>>>>> java.lang.ExceptionInInitializerError >>>>>>>>>>> at >>>>>>>>>>> jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.runtime(HotSpotJVMCIRuntime.java:85) >>>>>>>>>>> >>>>>>>>>>> at jdk.vm.ci.runtime.JVMCI.initializeRuntime(Native Method) >>>>>>>>>>> at jdk.vm.ci.runtime.JVMCI.(JVMCI.java:58) >>>>>>>>>>> Caused by: java.lang.IllegalArgumentException: Could not find >>>>>>>>>>> option OptSomethingThatDoesNotExcist >>>>>>>>>>> at >>>>>>>>>>> com.oracle.graal.options.OptionsParser.parseOption(OptionsParser.java:134) >>>>>>>>>>> >>>>>>>>>>> at >>>>>>>>>>> com.oracle.graal.options.OptionsParser.parseOptions(OptionsParser.java:62) >>>>>>>>>>> >>>>>>>>>>> at >>>>>>>>>>> com.oracle.graal.hotspot.HotSpotGraalCompilerFactory.initializeOptions(HotSpotGraalCompilerFactory.java:156) >>>>>>>>>>> >>>>>>>>>>> at >>>>>>>>>>> com.oracle.graal.hotspot.HotSpotGraalCompilerFactory.onSelection(HotSpotGraalCompilerFactory.java:86) >>>>>>>>>>> >>>>>>>>>>> at >>>>>>>>>>> jdk.vm.ci.hotspot.HotSpotJVMCICompilerConfig.getCompilerFactory(HotSpotJVMCICompilerConfig.java:96) >>>>>>>>>>> >>>>>>>>>>> at >>>>>>>>>>> jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.(HotSpotJVMCIRuntime.java:277) >>>>>>>>>>> >>>>>>>>>>> at >>>>>>>>>>> jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.(HotSpotJVMCIRuntime.java:67) >>>>>>>>>>> >>>>>>>>>>> at >>>>>>>>>>> jdk.vm.ci.hotspot.HotSpotJVMCIRuntime$DelayedInit.(HotSpotJVMCIRuntime.java:75) >>>>>>>>>>> >>>>>>>>>>> at >>>>>>>>>>> jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.runtime(HotSpotJVMCIRuntime.java:85) >>>>>>>>>>> >>>>>>>>>>> at jdk.vm.ci.runtime.JVMCI.initializeRuntime(Native Method) >>>>>>>>>>> at jdk.vm.ci.runtime.JVMCI.(JVMCI.java:58) >>>>>>>>>>> >>>>>>>>>>> The native JVMCI code then tries to exit the VM by calling >>>>>>>>>>> vm_abort. If multiple compiler threads do this concurrently, >>>>>>>>>>> certain destructors can be called twice as shown by these thread >>>>>>>>>>> dumps: >>>>>>>>>>> >>>>>>>>>>> thread #26: tid = 0x0019, 0x00007fff84280124 >>>>>>>>>>> libsystem_malloc.dylib`szone_size + 227, stop reason = signal SIGSTOP >>>>>>>>>>> frame #0: 0x00007fff84280124 libsystem_malloc.dylib`szone_size + 227 >>>>>>>>>>> frame #1: 0x00007fff8427fed5 libsystem_malloc.dylib`free + 61 >>>>>>>>>>> frame #2: 0x000000010ac95963 >>>>>>>>>>> libjvm.dylib`os::free(memblock=0x00007fedc86226e0, >>>>>>>>>>> memflags=mtInternal) + 307 at os.cpp:711 >>>>>>>>>>> frame #3: 0x000000010a2afc54 >>>>>>>>>>> libjvm.dylib`FreeHeap(p=0x00007fedc86226e0, memflags=mtInternal) + >>>>>>>>>>> 52 at allocation.inline.hpp:93 >>>>>>>>>>> frame #4: 0x000000010acf0a9f >>>>>>>>>>> libjvm.dylib`PerfData::~PerfData(this=0x00007fedc8622650) + 63 at >>>>>>>>>>> perfData.cpp:116 >>>>>>>>>>> frame #5: 0x000000010acf0ae5 >>>>>>>>>>> libjvm.dylib`PerfData::~PerfData(this=0x00007fedc8622650) + 21 at >>>>>>>>>>> perfData.cpp:114 >>>>>>>>>>> frame #6: 0x000000010acf163d >>>>>>>>>>> libjvm.dylib`PerfDataManager::destroy() + 109 at perfData.cpp:287 >>>>>>>>>>> frame #7: 0x000000010acf3f4d libjvm.dylib`perfMemory_exit() + 61 >>>>>>>>>>> at perfMemory.cpp:74 >>>>>>>>>>> frame #8: 0x000000010ac9bb0d libjvm.dylib`os::shutdown() + 13 at >>>>>>>>>>> os_bsd.cpp:1130 >>>>>>>>>>> frame #9: 0x000000010ac9bb55 >>>>>>>>>>> libjvm.dylib`os::abort(dump_core=false) + 21 at os_bsd.cpp:1150 >>>>>>>>>>> frame #10: 0x000000010a9188e7 >>>>>>>>>>> libjvm.dylib`vm_abort(dump_core=false) + 39 at java.cpp:666 >>>>>>>>>>> frame #11: 0x000000010aa4f1e7 >>>>>>>>>>> libjvm.dylib`JVMCIRuntime::abort_on_pending_exception(exception=Handle >>>>>>>>>>> @ 0x000070000175b208, message="Uncaught exception at >>>>>>>>>>> /Users/dsimon/graal/graal-jvmci-8/src/share/vm/jvmci/jvmciCompiler.cpp:127", >>>>>>>>>>> dump_core=false) + 167 at jvmciRuntime.cpp:992 >>>>>>>>>>> frame #12: 0x000000010aa17017 >>>>>>>>>>> libjvm.dylib`JVMCICompiler::compile_method(this=0x00007fedcb203050, method=0x000070000175b8d8, >>>>>>>>>>> entry_bci=-1, env=0x000070000175b8f0) + 311 at jvmciCompiler.cpp:127 >>>>>>>>>>> frame #13: 0x000000010a656cd2 >>>>>>>>>>> libjvm.dylib`CompileBroker::invoke_compiler_on_method(task=0x00007fedc853ca30) >>>>>>>>>>> + 1314 at compileBroker.cpp:2207 >>>>>>>>>>> >>>>>>>>>>> thread #23: tid = 0x0016, 0x00007fff91fcb122 >>>>>>>>>>> libsystem_kernel.dylib`__semwait_signal_nocancel + 10, stop reason >>>>>>>>>>> = signal SIGSTOP >>>>>>>>>>> frame #0: 0x00007fff91fcb122 >>>>>>>>>>> libsystem_kernel.dylib`__semwait_signal_nocancel + 10 >>>>>>>>>>> frame #1: 0x00007fff9578c318 libsystem_c.dylib`nanosleep$NOCANCEL >>>>>>>>>>> + 188 >>>>>>>>>>> frame #2: 0x00007fff957b62ce libsystem_c.dylib`usleep$NOCANCEL + 54 >>>>>>>>>>> frame #3: 0x00007fff957e46e9 libsystem_c.dylib`abort + 139 >>>>>>>>>>> frame #4: 0x00007fff8428c396 libsystem_malloc.dylib`szone_error + 626 >>>>>>>>>>> frame #5: 0x000000010ac95963 >>>>>>>>>>> libjvm.dylib`os::free(memblock=0x00007fedc8601cd0, >>>>>>>>>>> memflags=mtInternal) + 307 at os.cpp:711 >>>>>>>>>>> frame #6: 0x000000010a2afc54 >>>>>>>>>>> libjvm.dylib`FreeHeap(p=0x00007fedc8601cd0, memflags=mtInternal) + >>>>>>>>>>> 52 at allocation.inline.hpp:93 >>>>>>>>>>> frame #7: 0x000000010acf0a9f >>>>>>>>>>> libjvm.dylib`PerfData::~PerfData(this=0x00007fedc8601c60) + 63 at >>>>>>>>>>> perfData.cpp:116 >>>>>>>>>>> frame #8: 0x000000010acf0ae5 >>>>>>>>>>> libjvm.dylib`PerfData::~PerfData(this=0x00007fedc8601c60) + 21 at >>>>>>>>>>> perfData.cpp:114 >>>>>>>>>>> frame #9: 0x000000010acf163d >>>>>>>>>>> libjvm.dylib`PerfDataManager::destroy() + 109 at perfData.cpp:287 >>>>>>>>>>> frame #10: 0x000000010acf3f4d libjvm.dylib`perfMemory_exit() + 61 >>>>>>>>>>> at perfMemory.cpp:74 >>>>>>>>>>> frame #11: 0x000000010ac9bb0d libjvm.dylib`os::shutdown() + 13 at >>>>>>>>>>> os_bsd.cpp:1130 >>>>>>>>>>> frame #12: 0x000000010ac9bb55 >>>>>>>>>>> libjvm.dylib`os::abort(dump_core=false) + 21 at os_bsd.cpp:1150 >>>>>>>>>>> frame #13: 0x000000010a9188e7 >>>>>>>>>>> libjvm.dylib`vm_abort(dump_core=false) + 39 at java.cpp:666 >>>>>>>>>>> frame #14: 0x000000010aa4f1e7 >>>>>>>>>>> libjvm.dylib`JVMCIRuntime::abort_on_pending_exception(exception=Handle >>>>>>>>>>> @ 0x0000700001452208, message="Uncaught exception at >>>>>>>>>>> /Users/dsimon/graal/graal-jvmci-8/src/share/vm/jvmci/jvmciCompiler.cpp:127", >>>>>>>>>>> dump_core=false) + 167 at jvmciRuntime.cpp:992 >>>>>>>>>>> frame #15: 0x000000010aa17017 >>>>>>>>>>> libjvm.dylib`JVMCICompiler::compile_method(this=0x00007fedcb203050, method=0x00007000014528d8, >>>>>>>>>>> entry_bci=-1, env=0x00007000014528f0) + 311 at jvmciCompiler.cpp:127 >>>>>>>>>>> frame #16: 0x000000010a656cd2 >>>>>>>>>>> libjvm.dylib`CompileBroker::invoke_compiler_on_method(task=0x00007fedc862a320) >>>>>>>>>>> + 1314 at compileBroker.cpp:2207 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> This webrev replaces calls to vm_abort() with before_exit() + >>>>>>>>>>> vm_exit(). The latter is thread safe. >>>>>>>>>>> >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165755 >>>>>>>>>>> http://cr.openjdk.java.net/~dnsimon/8165755/ >>>>>>>>>>> >>>>>>>>>>> -Doug >>>>>> >>>> >>> > From doug.simon at oracle.com Wed Sep 14 10:13:18 2016 From: doug.simon at oracle.com (Doug Simon) Date: Wed, 14 Sep 2016 12:13:18 +0200 Subject: RFR: 8165755: [JVMCI] replace use of vm_abort with vm_exit In-Reply-To: References: <799CA13D-6BDC-4BF5-9241-515A684191F4@oracle.com> <6391B00B-AFBF-410C-A6A1-2ED95B35EBEB@twitter.com> <2F360221-7E6B-43BB-B69C-69A17777E5F2@oracle.com> <1c9b4d32-52a2-17a8-bfc5-f3cc7b494ba0@oracle.com> <5d26a289-a262-5882-8061-3bbc4faa4540@oracle.com> <52A83790-582E-4F8F-AA49-3F6DB3AB36CB@oracle.com> <42F9D0A5-3017-45DA-BBB6-92D2B76FBD55@oracle.com> <416C33DC-A44B-4D0F-AB4F-C83BC5C52265@oracle.com> Message-ID: > On 14 Sep 2016, at 11:25, David Holmes wrote: > > On 14/09/2016 5:53 PM, Doug Simon wrote: >> >>> On 14 Sep 2016, at 03:42, David Holmes wrote: >>> >>> Hi Doug, >>> >>> On 14/09/2016 12:41 AM, Doug Simon wrote: >>>> Hi David, >>>> >>>> Do you still have concerns with the webrev in its current form? >>> >>> Sorry, yesterday was "one of those days". :) >> >> No problem, we have those around here sometimes as well ;-) >> >>> tl;dr: Your actual final exit code matches what JVM_Halt does, so that is fine. >>> >>> Your overall initialization and termination approach raises several general concerns. From a usability perspective the idea that JVMCI can terminate the VM at some arbitrary point after application startup is quite alarming! >> >> It?s not completely arbitrary. Top tier compilation kicks in very soon after VM startup for all but HelloWorld type applications. Also, termination happens only due to incorrect command line in which case I?d say termination is what?s normal/expected. > > Yes but people also expect bad command line options to be detected before anything else happens that is part of the application logic. I agree. However, a) I don?t see any way to avoid this without making JVMCI non-lazy (not an option), b) very little application logic can run before top tier compilation kicks in and c) there?s already configuration checking by the native compilers in a compiler thread that could cause the VM to exit after an app thread has started (even though the chances of this are tiny). >>> Looking at the initialization code I'm unclear why it isn't being serialized in some way at a level higher than the SystemDictionary - if it was then you wouldn't need the termination logic to deal with racing threads. >> >> I?m not sure I follow. JVMCI initialization is serialized on the initialization of the jdk.vm.ci.runtime.JVMCI class. Since class initialization errors are preserved, multiple compiler threads can see the same error so will each try to go down the termination path. It?s the latter race that needs serialization which is done now in JVMCICompiler::abort_on_pending_exception. > > The code I looked at has the initialization logic called from JVMCICompiler::compile_method. I can't say that isn't called from init of the JVMCI class, but it sure doesn't sound like it. :) That path is what triggers initialization of the JVMCI class: JVMCICompiler::compile_method -> JVMCIRuntime::get_HotSpotJVMCIRuntime -> JVMCIRuntime::initialize_JVMCI -> JVMCI. > > If the thread that encounters the initial error is holding an initialization "lock" (Java monitor in JVMCI's case I expect) when it takes the termination path then there's no race in taking that path. The lock is released before returning from JVMCI. so once we?re back in JVMCICompiler::compile_method, the compiler threads are no longer synchronized. > Anyway, just musing. Which means you?re ok with the current webrev? -Doug From david.holmes at oracle.com Wed Sep 14 12:28:33 2016 From: david.holmes at oracle.com (David Holmes) Date: Wed, 14 Sep 2016 22:28:33 +1000 Subject: RFR: 8165755: [JVMCI] replace use of vm_abort with vm_exit In-Reply-To: References: <799CA13D-6BDC-4BF5-9241-515A684191F4@oracle.com> <6391B00B-AFBF-410C-A6A1-2ED95B35EBEB@twitter.com> <2F360221-7E6B-43BB-B69C-69A17777E5F2@oracle.com> <1c9b4d32-52a2-17a8-bfc5-f3cc7b494ba0@oracle.com> <5d26a289-a262-5882-8061-3bbc4faa4540@oracle.com> <52A83790-582E-4F8F-AA49-3F6DB3AB36CB@oracle.com> <42F9D0A5-3017-45DA-BBB6-92D2B76FBD55@oracle.com> <416C33DC-A44B-4D0F-AB4F-C83BC5C52265@oracle.com> Message-ID: <49fff00f-0d2d-a57d-9d41-4f70d03935bd@oracle.com> On 14/09/2016 8:13 PM, Doug Simon wrote: > >> On 14 Sep 2016, at 11:25, David Holmes wrote: >> >> On 14/09/2016 5:53 PM, Doug Simon wrote: >>> >>>> On 14 Sep 2016, at 03:42, David Holmes wrote: >>>> >>>> Hi Doug, >>>> >>>> On 14/09/2016 12:41 AM, Doug Simon wrote: >>>>> Hi David, >>>>> >>>>> Do you still have concerns with the webrev in its current form? >>>> >>>> Sorry, yesterday was "one of those days". :) >>> >>> No problem, we have those around here sometimes as well ;-) >>> >>>> tl;dr: Your actual final exit code matches what JVM_Halt does, so that is fine. >>>> >>>> Your overall initialization and termination approach raises several general concerns. From a usability perspective the idea that JVMCI can terminate the VM at some arbitrary point after application startup is quite alarming! >>> >>> It?s not completely arbitrary. Top tier compilation kicks in very soon after VM startup for all but HelloWorld type applications. Also, termination happens only due to incorrect command line in which case I?d say termination is what?s normal/expected. >> >> Yes but people also expect bad command line options to be detected before anything else happens that is part of the application logic. > > I agree. However, > > a) I don?t see any way to avoid this without making JVMCI non-lazy (not an option), > b) very little application logic can run before top tier compilation kicks in and > c) there?s already configuration checking by the native compilers in a compiler thread that could cause the VM to exit after an app thread has started (even though the chances of this are tiny). > >>>> Looking at the initialization code I'm unclear why it isn't being serialized in some way at a level higher than the SystemDictionary - if it was then you wouldn't need the termination logic to deal with racing threads. >>> >>> I?m not sure I follow. JVMCI initialization is serialized on the initialization of the jdk.vm.ci.runtime.JVMCI class. Since class initialization errors are preserved, multiple compiler threads can see the same error so will each try to go down the termination path. It?s the latter race that needs serialization which is done now in JVMCICompiler::abort_on_pending_exception. >> >> The code I looked at has the initialization logic called from JVMCICompiler::compile_method. I can't say that isn't called from init of the JVMCI class, but it sure doesn't sound like it. :) > > That path is what triggers initialization of the JVMCI class: > > JVMCICompiler::compile_method -> JVMCIRuntime::get_HotSpotJVMCIRuntime -> JVMCIRuntime::initialize_JVMCI -> JVMCI. > >> >> If the thread that encounters the initial error is holding an initialization "lock" (Java monitor in JVMCI's case I expect) when it takes the termination path then there's no race in taking that path. > > The lock is released before returning from JVMCI. so once we?re back in JVMCICompiler::compile_method, the compiler threads are no longer synchronized. > >> Anyway, just musing. > > Which means you?re ok with the current webrev? Yes, that is what the tl;dr referred to :) David > > -Doug > From kim.barrett at oracle.com Wed Sep 14 23:45:43 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 14 Sep 2016 19:45:43 -0400 Subject: RFR: 8165808: Add release barriers when allocating objects with concurrent collection In-Reply-To: References: <13AE434E-FA26-4658-8A77-013462679EBC@oracle.com> <6c991053-7347-cc6b-3b94-fc48cb27a1db@oracle.com> <98CE5C19-7B72-4530-977C-C371164D439E@oracle.com> Message-ID: > On Sep 12, 2016, at 7:22 PM, David Holmes wrote: > On 13/09/2016 4:08 AM, Kim Barrett wrote: >>> On Sep 11, 2016, at 9:38 PM, David Holmes wrote: >>>> Webrev: >>>> http://cr.openjdk.java.net/~kbarrett/8165808/webrev.00/ >>> >>> src/share/vm/gc/shared/collectedHeap.inline.hpp >>> >>> Based on experiences with RMO architectures (ref JDK-8033920 - sorry it is a non-open bug report) I would expect the use of release_set_klass to be unconditional, not dependent on the nature of the GC. Publication of the object reference also has to be safe for other mutator threads. I'm not convinced by the brief argument in JDK-8033920 that thread-state transitions will always ensure visibility and ordering. >> >> There were two different issues discussed in (non-open) JDK-8033920. >> >> (1) Ensure setting the klass happens after some other parts of the >> object initialization, to ensure (some) concurrent GCs can parse the >> heap. >> >> (2) Ensure the klass is visible to other (mutator) threads if the >> object is accessible to those threads, e.g. if the object escapes from >> the allocating thread. >> >> JDK-8160369 is only about (1), and this change (for JDK-8165808) is >> part of fixing (1). I'm not trying to address (2) here, even assuming >> there are problems in that area. (I don't know that there *are* > > I have my doubts about treating these two issues separately, but will not press the issue. For the general mutator path the devil is in the detail, and as Derek indicated if there is a problem there the solution would likely need the matching load-acquire. I'm pretty sure they are different, leading to barriers in different places for different reasons. I'm investigating a hypothesis about the general mutator path, but not yet ready to say anything more because I may be talking out of my hat. So can I get a second review of this change? From yasuenag at gmail.com Thu Sep 15 03:56:13 2016 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Thu, 15 Sep 2016 12:56:13 +0900 Subject: RFR(XS): 8165881 - Backout JDK-8164913 In-Reply-To: References: <519601d20d17$cc979220$65c6b660$@oracle.com> <14399a60-88ca-bcf4-0696-09f530aa2955@oracle.com> <97d8d6c3-2f4d-1475-f9b0-e608878e5aac@oracle.com> <8e7b6d80-7c1c-b432-2ef4-3b9aa27956d3@gmail.com> <07965d03-1663-8da3-d126-aecbb93a7716@oracle.com> <839d84a3-46e8-7c89-a1e7-ba03fa0ef98c@oracle.com> <29ff139b-6618-6216-ccf3-2fc03af1d6b6@gmail.com> Message-ID: <1ef93f3b-a9ce-93e7-cb16-d72406a2774a@gmail.com> Hi, > If this is done through jcmd then jcmd needs to know how to "unwrap" the information it gets back from the Dcmd. In case of JVMTI.agent_load dcmd, I think we should be able to get the response as below: 1. Invalid JVMTI agent 2. Agent_OnAttach() did not return JNI_OK 3. Succeeded Currently, JvmtiExport::load_agent_library() returns JNI_OK to caller, and jcmd() in attachListener.cpp returns JNI_ERR if exception is occurred (in Agent_OnAttach()). IMHO, HotSpot should return error code (in top of the stream: written by AttachListener at first). So we should be able to get some error code in case 1. and 2. Then the client (jcmd or other methods) can decide to parse text information in the stream. Sorry for my English. Yasumasa On 2016/09/14 7:03, David Holmes wrote: > On 13/09/2016 10:31 PM, Yasumasa Suenaga wrote: >> Thanks David! >> If we should not change the meaning of return code from >> JvmtiExport::load_agent_library(), we should print return code as below: >> ------------------- >> diff -r 0cf03b9d9b1f src/share/vm/prims/jvmtiExport.cpp >> --- a/src/share/vm/prims/jvmtiExport.cpp Mon Sep 12 18:59:13 2016 +0000 >> +++ b/src/share/vm/prims/jvmtiExport.cpp Tue Sep 13 21:12:14 2016 +0900 >> @@ -2412,6 +2412,10 @@ >> result = JNI_OK; >> } >> } >> + // Print error code if Agent_OnAttach cannot be executed >> + if (result != JNI_OK) { >> + st->print_cr("%d", result); >> + } >> return result; >> } >> ------------------- > > Not sure I see the point. "return result" will put the error code into the socket stream and that error will be seen and responded to. I don't expect anything to then read further into the stream to see the "result" repeated a second time. In other words if execute() doesn't see a zero result then you go no further; if execute sees a zero result then the actual action may have had an issue so you proceed to read the "action's result". > >> It can pass com/sun/tools/attach/BasicTests.java, and we can get -1 if >> we try to attach invalid agent. > > Can you please outline your test case for this again. If this is done through jcmd then jcmd needs to know how to "unwrap" the information it gets back from the Dcmd. > > Thanks, > David > >> >> Yasumasa >> >> >> On 2016/09/13 17:06, Dmitry Samersoff wrote: >>> David, >>> >>> Thank you for the evaluation. >>> >>>> With that in mind I suspect the real fix for the original issue is to >>>> have Dcmd recognize that it also has to read two "results". Though I'm >>>> not sure how exactly. >>> >>> This logic seems completely broken for me. But I don't see anything we >>> can do right now - for jdk 9. >>> >>> It requires careful changes of both - code and tests. >>> >>> I can help with this task but not today. >>> >>> -Dmitry >>> >>> On 2016-09-13 08:08, David Holmes wrote: >>>> On 13/09/2016 1:53 PM, David Holmes wrote: >>>>> On 13/09/2016 1:30 PM, Yasumasa Suenaga wrote: >>>>>> Hi, >>>>>> >>>>>> I could reproduce and I added a comment to JBS: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8165869?focusedCommentId=14000623&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14000623 >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> In case of BasicTests.java, I think it is a test bug. >>>>> >>>>> I don't agree as yet. I have not been able to isolate the exact >>>>> difference between what happens with your fix and what happens without. >>>>> I know it relates to the presence of the error code, but also how we >>>>> get >>>>> AgentInitializationException instead of AgentLoadException. I need >>>>> to be >>>>> able to run the test outside of jtreg but that is proving to be very >>>>> difficult to arrange. :( >>>> >>>> I finally managed to connect all the pieces on this. >>>> >>>> The basic problem is that with the proposed change >>>> VirtualMachineImpl.execute() will throw AgentLoadException, which then >>>> leads to the InternalError. Without the change we reach this block in >>>> HotspotVirtualMachine.loadAgentLibrary: >>>> >>>> int result = readInt(in); >>>> if (result != 0) { >>>> throw new AgentInitializationException("Agent_OnAttach failed", >>>> result); >>>> } >>>> >>>> and so get the expected AgentInitializationException. >>>> >>>> Looking at the proposed change in jvmtiExport.cpp if we have the >>>> original: >>>> >>>> st->print_cr("%d", result); >>>> result = JNI_OK; >>>> >>>> then execute() manages to read a zero completion status from the stream, >>>> and then loadAgentLibrary is able to read the actual "result" - whether >>>> zero or not - from the stream. This is because the AttachListener code >>>> calls op->complete(result, &st) where st is the stream where we wrote >>>> the result value above in print_cr. Then if we look at, for example, >>>> LinuxAttachOperation::complete, we will write "result" to the socket >>>> first, followed by the contents of st. Hence on a successful operation >>>> we can read 0 for execute, and then 0 for loadAgent. On error we read 0 >>>> for execute and the actual error code for loadAgent. With the proposed >>>> change execute() sees the real result (instead of JNI_OK) and so throws >>>> AgentLoadException. >>>> >>>> So in summary there are two different error indicators written into the >>>> stream, and we need the first to be zero unless some major error with >>>> the actual attach functionality occurred - otherwise even if the "load" >>>> failed in some other way, it is treated as a secondary error. >>>> >>>> With that in mind I suspect the real fix for the original issue is to >>>> have Dcmd recognize that it also has to read two "results". Though I'm >>>> not sure how exactly. >>>> >>>> David >>>> ----- >>>> >>>>> David >>>>> >>>>> >>>>> >>>>>> If it is accepted, I will upload webrev for this redo task. >>>>>> However, I cannot access (and fix) Oracle internal test. Can someone >>>>>> help me? >>>>>> (I cannot access JPRT. So I need a sponsor.) >>>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2016/09/13 9:00, David Holmes wrote: >>>>>>> I think I see the problem. The attach framework tries to load the >>>>>>> "instrument" agent library. Prior to 8164913 if this fails it >>>>>>> actually >>>>>>> appears to succeed. Now it fails, and that error propagates through. >>>>>>> I'm guessing, at the moment, that the failure is due to a missing >>>>>>> module related option. >>>>>>> >>>>>>> David >>>>>>> >>>>>>> On 13/09/2016 9:54 AM, David Holmes wrote: >>>>>>>> Hi Yasumasa, >>>>>>>> >>>>>>>> On 13/09/2016 9:23 AM, Yasumasa Suenaga wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> Hmm, it has been backouted... >>>>>>>>> >>>>>>>>> I agree to David. This error seems to be a test bug. >>>>>>>>> Can you share the test? I want to evaluate it. >>>>>>>> >>>>>>>> Sorry we can't share the tests. I took a quick look and it seems it >>>>>>>> may >>>>>>>> be related to the result code parsing (that I thought we ended up >>>>>>>> not >>>>>>>> touching!). >>>>>>>> >>>>>>>> Can you run a test of HotSpotVirtualMachine.loadAgent(null) and see >>>>>>>> what >>>>>>>> happens? We see AgentLoadException from the code that internally >>>>>>>> tries >>>>>>>> to load the "instrument" agent: >>>>>>>> >>>>>>>> Exception in thread "main" Failure: Unexpected exception during test >>>>>>>> execution: java.lang.InternalError: instrument library is missing in >>>>>>>> target VM >>>>>>>> ... >>>>>>>> Caused by: java.lang.InternalError: instrument library is missing in >>>>>>>> target VM >>>>>>>> ... >>>>>>>> Caused by: com.sun.tools.attach.AgentLoadException: Failed to load >>>>>>>> agent >>>>>>>> library >>>>>>>> >>>>>>>> >>>>>>>>> I do not agree to fix this bug in JDK 10 because VM might lie when >>>>>>>>> the >>>>>>>>> JVMTI agent could not be attached. IMHO this bug should be fixed >>>>>>>>> in 9 >>>>>>>>> GA, and we should fix testcase(s). >>>>>>>> >>>>>>>> I agree. It has to be noted that "we" failed to run all the >>>>>>>> appropriate >>>>>>>> tests before pushing this, which would have discovered the problem - >>>>>>>> assuming it is actually a problem with the change and not an >>>>>>>> unrelated >>>>>>>> issue in our testing environment. >>>>>>>> >>>>>>>> Unfortunately I have some high priority tasks to handle right now, >>>>>>>> so I >>>>>>>> can't go into this in any more depth at the moment. >>>>>>>> >>>>>>>> David >>>>>>>> ----- >>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2016/09/13 6:15, David Holmes wrote: >>>>>>>>>> For the record it looks like the tests involved are broken, in >>>>>>>>>> that >>>>>>>>>> because the VM used to lie about the success of attaching the >>>>>>>>>> agent, the >>>>>>>>>> tests expected different exceptions to occur. >>>>>>>>>> >>>>>>>>>> David >>>>>>>>>> >>>>>>>>>> On 13/09/2016 3:11 AM, harold seigel wrote: >>>>>>>>>>> Looks good. >>>>>>>>>>> >>>>>>>>>>> Harold >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 9/12/2016 1:05 PM, Christian Tornqvist wrote: >>>>>>>>>>>> Hi everyone, >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Please review this (clean) backout of the change for >>>>>>>>>>>> JDK-8164913, it >>>>>>>>>>>> failed >>>>>>>>>>>> several tests in the nightly testing. The failures are >>>>>>>>>>>> tracked in: >>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165869 >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Webrev: >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~ctornqvi/webrev/8165881/webrev.00/ >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Bug: >>>>>>>>>>>> >>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165881 >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Christian >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>> >>> From shafi.s.ahmad at oracle.com Thu Sep 15 06:40:37 2016 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Wed, 14 Sep 2016 23:40:37 -0700 (PDT) Subject: [8u] RFR for JDK-6515172: Runtime.availableProcessors() ignores Linux taskset command Message-ID: <4a227590-41cd-4a4e-9b7d-9e8f8ad8f892@default> Hi, Please review the backport [modified] of bug: "JDK-6515172 Runtime.availableProcessors() ignores Linux taskset command" to jdk8u. Please note that the backport is not clean and we can't do as it is. Please note 1. The changes are made by 'Andreas Eriksson' who left Oracle. 2. There is difference in logging mechanism in jdk9 and jdk8 is different and file logTag.hpp doesn't exists in jdk8. 3. Newly added test pass after this change. It fails without this change. Webrev link: http://cr.openjdk.java.net/~shshahma/8154324/webrev.02/ Jdk9 bug: https://bugs.openjdk.java.net/browse/JDK-6515172 Original patch pushed to jdk9: http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/c5480d4abfe4 Testing: jprt and running jtreg. Regards, Shafi From david.holmes at oracle.com Thu Sep 15 09:43:10 2016 From: david.holmes at oracle.com (David Holmes) Date: Thu, 15 Sep 2016 19:43:10 +1000 Subject: RFR(XS): 8165881 - Backout JDK-8164913 In-Reply-To: <1ef93f3b-a9ce-93e7-cb16-d72406a2774a@gmail.com> References: <519601d20d17$cc979220$65c6b660$@oracle.com> <14399a60-88ca-bcf4-0696-09f530aa2955@oracle.com> <97d8d6c3-2f4d-1475-f9b0-e608878e5aac@oracle.com> <8e7b6d80-7c1c-b432-2ef4-3b9aa27956d3@gmail.com> <07965d03-1663-8da3-d126-aecbb93a7716@oracle.com> <839d84a3-46e8-7c89-a1e7-ba03fa0ef98c@oracle.com> <29ff139b-6618-6216-ccf3-2fc03af1d6b6@gmail.com> <1ef93f3b-a9ce-93e7-cb16-d72406a2774a@gmail.com> Message-ID: Hi Yasumasa, On 15/09/2016 1:56 PM, Yasumasa Suenaga wrote: > Hi, > >> If this is done through jcmd then jcmd needs to know how to "unwrap" >> the information it gets back from the Dcmd. > > In case of JVMTI.agent_load dcmd, I think we should be able to get the > response as below: > > 1. Invalid JVMTI agent > 2. Agent_OnAttach() did not return JNI_OK > 3. Succeeded > > Currently, JvmtiExport::load_agent_library() returns JNI_OK to caller, > and jcmd() in attachListener.cpp returns JNI_ERR if exception is > occurred (in Agent_OnAttach()). Can you respond to my comment in the bug report about the chain of calls and explain exactly where you think things are going wrong in this scenario. I'm having a lot of trouble trying to see how the information flows back through the layers. > IMHO, HotSpot should return error code (in top of the stream: written by > AttachListener at first). So we should be able to get some error code in > case 1. and 2. Then the client (jcmd or other methods) can decide to > parse text information in the stream. It returns the high-level response code first "did communication with the target VM succeed", then the actual action response code. That seems the right thing to do to me. It is up to the callers reading the stream to understand how to read it. To me the issue lies somewhere on the jcmd/dcmd side. Thanks, David > > Sorry for my English. > > Yasumasa > > > On 2016/09/14 7:03, David Holmes wrote: >> On 13/09/2016 10:31 PM, Yasumasa Suenaga wrote: >>> Thanks David! >>> If we should not change the meaning of return code from >>> JvmtiExport::load_agent_library(), we should print return code as below: >>> ------------------- >>> diff -r 0cf03b9d9b1f src/share/vm/prims/jvmtiExport.cpp >>> --- a/src/share/vm/prims/jvmtiExport.cpp Mon Sep 12 18:59:13 2016 >>> +0000 >>> +++ b/src/share/vm/prims/jvmtiExport.cpp Tue Sep 13 21:12:14 2016 >>> +0900 >>> @@ -2412,6 +2412,10 @@ >>> result = JNI_OK; >>> } >>> } >>> + // Print error code if Agent_OnAttach cannot be executed >>> + if (result != JNI_OK) { >>> + st->print_cr("%d", result); >>> + } >>> return result; >>> } >>> ------------------- >> >> Not sure I see the point. "return result" will put the error code into >> the socket stream and that error will be seen and responded to. I >> don't expect anything to then read further into the stream to see the >> "result" repeated a second time. In other words if execute() doesn't >> see a zero result then you go no further; if execute sees a zero >> result then the actual action may have had an issue so you proceed to >> read the "action's result". >> >>> It can pass com/sun/tools/attach/BasicTests.java, and we can get -1 if >>> we try to attach invalid agent. >> >> Can you please outline your test case for this again. If this is done >> through jcmd then jcmd needs to know how to "unwrap" the information >> it gets back from the Dcmd. >> >> Thanks, >> David >> >>> >>> Yasumasa >>> >>> >>> On 2016/09/13 17:06, Dmitry Samersoff wrote: >>>> David, >>>> >>>> Thank you for the evaluation. >>>> >>>>> With that in mind I suspect the real fix for the original issue is to >>>>> have Dcmd recognize that it also has to read two "results". Though I'm >>>>> not sure how exactly. >>>> >>>> This logic seems completely broken for me. But I don't see anything we >>>> can do right now - for jdk 9. >>>> >>>> It requires careful changes of both - code and tests. >>>> >>>> I can help with this task but not today. >>>> >>>> -Dmitry >>>> >>>> On 2016-09-13 08:08, David Holmes wrote: >>>>> On 13/09/2016 1:53 PM, David Holmes wrote: >>>>>> On 13/09/2016 1:30 PM, Yasumasa Suenaga wrote: >>>>>>> Hi, >>>>>>> >>>>>>> I could reproduce and I added a comment to JBS: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165869?focusedCommentId=14000623&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14000623 >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> In case of BasicTests.java, I think it is a test bug. >>>>>> >>>>>> I don't agree as yet. I have not been able to isolate the exact >>>>>> difference between what happens with your fix and what happens >>>>>> without. >>>>>> I know it relates to the presence of the error code, but also how we >>>>>> get >>>>>> AgentInitializationException instead of AgentLoadException. I need >>>>>> to be >>>>>> able to run the test outside of jtreg but that is proving to be very >>>>>> difficult to arrange. :( >>>>> >>>>> I finally managed to connect all the pieces on this. >>>>> >>>>> The basic problem is that with the proposed change >>>>> VirtualMachineImpl.execute() will throw AgentLoadException, which then >>>>> leads to the InternalError. Without the change we reach this block in >>>>> HotspotVirtualMachine.loadAgentLibrary: >>>>> >>>>> int result = readInt(in); >>>>> if (result != 0) { >>>>> throw new AgentInitializationException("Agent_OnAttach failed", >>>>> result); >>>>> } >>>>> >>>>> and so get the expected AgentInitializationException. >>>>> >>>>> Looking at the proposed change in jvmtiExport.cpp if we have the >>>>> original: >>>>> >>>>> st->print_cr("%d", result); >>>>> result = JNI_OK; >>>>> >>>>> then execute() manages to read a zero completion status from the >>>>> stream, >>>>> and then loadAgentLibrary is able to read the actual "result" - >>>>> whether >>>>> zero or not - from the stream. This is because the AttachListener code >>>>> calls op->complete(result, &st) where st is the stream where we wrote >>>>> the result value above in print_cr. Then if we look at, for example, >>>>> LinuxAttachOperation::complete, we will write "result" to the socket >>>>> first, followed by the contents of st. Hence on a successful operation >>>>> we can read 0 for execute, and then 0 for loadAgent. On error we >>>>> read 0 >>>>> for execute and the actual error code for loadAgent. With the proposed >>>>> change execute() sees the real result (instead of JNI_OK) and so >>>>> throws >>>>> AgentLoadException. >>>>> >>>>> So in summary there are two different error indicators written into >>>>> the >>>>> stream, and we need the first to be zero unless some major error with >>>>> the actual attach functionality occurred - otherwise even if the >>>>> "load" >>>>> failed in some other way, it is treated as a secondary error. >>>>> >>>>> With that in mind I suspect the real fix for the original issue is to >>>>> have Dcmd recognize that it also has to read two "results". Though I'm >>>>> not sure how exactly. >>>>> >>>>> David >>>>> ----- >>>>> >>>>>> David >>>>>> >>>>>> >>>>>> >>>>>>> If it is accepted, I will upload webrev for this redo task. >>>>>>> However, I cannot access (and fix) Oracle internal test. Can someone >>>>>>> help me? >>>>>>> (I cannot access JPRT. So I need a sponsor.) >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2016/09/13 9:00, David Holmes wrote: >>>>>>>> I think I see the problem. The attach framework tries to load the >>>>>>>> "instrument" agent library. Prior to 8164913 if this fails it >>>>>>>> actually >>>>>>>> appears to succeed. Now it fails, and that error propagates >>>>>>>> through. >>>>>>>> I'm guessing, at the moment, that the failure is due to a missing >>>>>>>> module related option. >>>>>>>> >>>>>>>> David >>>>>>>> >>>>>>>> On 13/09/2016 9:54 AM, David Holmes wrote: >>>>>>>>> Hi Yasumasa, >>>>>>>>> >>>>>>>>> On 13/09/2016 9:23 AM, Yasumasa Suenaga wrote: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> Hmm, it has been backouted... >>>>>>>>>> >>>>>>>>>> I agree to David. This error seems to be a test bug. >>>>>>>>>> Can you share the test? I want to evaluate it. >>>>>>>>> >>>>>>>>> Sorry we can't share the tests. I took a quick look and it >>>>>>>>> seems it >>>>>>>>> may >>>>>>>>> be related to the result code parsing (that I thought we ended up >>>>>>>>> not >>>>>>>>> touching!). >>>>>>>>> >>>>>>>>> Can you run a test of HotSpotVirtualMachine.loadAgent(null) and >>>>>>>>> see >>>>>>>>> what >>>>>>>>> happens? We see AgentLoadException from the code that internally >>>>>>>>> tries >>>>>>>>> to load the "instrument" agent: >>>>>>>>> >>>>>>>>> Exception in thread "main" Failure: Unexpected exception during >>>>>>>>> test >>>>>>>>> execution: java.lang.InternalError: instrument library is >>>>>>>>> missing in >>>>>>>>> target VM >>>>>>>>> ... >>>>>>>>> Caused by: java.lang.InternalError: instrument library is >>>>>>>>> missing in >>>>>>>>> target VM >>>>>>>>> ... >>>>>>>>> Caused by: com.sun.tools.attach.AgentLoadException: Failed to load >>>>>>>>> agent >>>>>>>>> library >>>>>>>>> >>>>>>>>> >>>>>>>>>> I do not agree to fix this bug in JDK 10 because VM might lie >>>>>>>>>> when >>>>>>>>>> the >>>>>>>>>> JVMTI agent could not be attached. IMHO this bug should be fixed >>>>>>>>>> in 9 >>>>>>>>>> GA, and we should fix testcase(s). >>>>>>>>> >>>>>>>>> I agree. It has to be noted that "we" failed to run all the >>>>>>>>> appropriate >>>>>>>>> tests before pushing this, which would have discovered the >>>>>>>>> problem - >>>>>>>>> assuming it is actually a problem with the change and not an >>>>>>>>> unrelated >>>>>>>>> issue in our testing environment. >>>>>>>>> >>>>>>>>> Unfortunately I have some high priority tasks to handle right now, >>>>>>>>> so I >>>>>>>>> can't go into this in any more depth at the moment. >>>>>>>>> >>>>>>>>> David >>>>>>>>> ----- >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 2016/09/13 6:15, David Holmes wrote: >>>>>>>>>>> For the record it looks like the tests involved are broken, in >>>>>>>>>>> that >>>>>>>>>>> because the VM used to lie about the success of attaching the >>>>>>>>>>> agent, the >>>>>>>>>>> tests expected different exceptions to occur. >>>>>>>>>>> >>>>>>>>>>> David >>>>>>>>>>> >>>>>>>>>>> On 13/09/2016 3:11 AM, harold seigel wrote: >>>>>>>>>>>> Looks good. >>>>>>>>>>>> >>>>>>>>>>>> Harold >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 9/12/2016 1:05 PM, Christian Tornqvist wrote: >>>>>>>>>>>>> Hi everyone, >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Please review this (clean) backout of the change for >>>>>>>>>>>>> JDK-8164913, it >>>>>>>>>>>>> failed >>>>>>>>>>>>> several tests in the nightly testing. The failures are >>>>>>>>>>>>> tracked in: >>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165869 >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Webrev: >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~ctornqvi/webrev/8165881/webrev.00/ >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Bug: >>>>>>>>>>>>> >>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165881 >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> >>>>>>>>>>>>> Christian >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>> >>>> From david.holmes at oracle.com Thu Sep 15 09:49:01 2016 From: david.holmes at oracle.com (David Holmes) Date: Thu, 15 Sep 2016 19:49:01 +1000 Subject: RFR: 8165808: Add release barriers when allocating objects with concurrent collection In-Reply-To: References: <13AE434E-FA26-4658-8A77-013462679EBC@oracle.com> <6c991053-7347-cc6b-3b94-fc48cb27a1db@oracle.com> <98CE5C19-7B72-4530-977C-C371164D439E@oracle.com> Message-ID: <64d35b20-c1fd-cfbc-7da9-4cc86b28685e@oracle.com> You can count me a reviewer #2. Thanks, David On 15/09/2016 9:45 AM, Kim Barrett wrote: >> On Sep 12, 2016, at 7:22 PM, David Holmes wrote: >> On 13/09/2016 4:08 AM, Kim Barrett wrote: >>>> On Sep 11, 2016, at 9:38 PM, David Holmes wrote: >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~kbarrett/8165808/webrev.00/ >>>> >>>> src/share/vm/gc/shared/collectedHeap.inline.hpp >>>> >>>> Based on experiences with RMO architectures (ref JDK-8033920 - sorry it is a non-open bug report) I would expect the use of release_set_klass to be unconditional, not dependent on the nature of the GC. Publication of the object reference also has to be safe for other mutator threads. I'm not convinced by the brief argument in JDK-8033920 that thread-state transitions will always ensure visibility and ordering. >>> >>> There were two different issues discussed in (non-open) JDK-8033920. >>> >>> (1) Ensure setting the klass happens after some other parts of the >>> object initialization, to ensure (some) concurrent GCs can parse the >>> heap. >>> >>> (2) Ensure the klass is visible to other (mutator) threads if the >>> object is accessible to those threads, e.g. if the object escapes from >>> the allocating thread. >>> >>> JDK-8160369 is only about (1), and this change (for JDK-8165808) is >>> part of fixing (1). I'm not trying to address (2) here, even assuming >>> there are problems in that area. (I don't know that there *are* >> >> I have my doubts about treating these two issues separately, but will not press the issue. For the general mutator path the devil is in the detail, and as Derek indicated if there is a problem there the solution would likely need the matching load-acquire. > > I'm pretty sure they are different, leading to barriers in different > places for different reasons. > > I'm investigating a hypothesis about the general mutator path, but not > yet ready to say anything more because I may be talking out of my hat. > > So can I get a second review of this change? > From david.holmes at oracle.com Thu Sep 15 09:51:22 2016 From: david.holmes at oracle.com (David Holmes) Date: Thu, 15 Sep 2016 19:51:22 +1000 Subject: [8u] RFR for JDK-6515172: Runtime.availableProcessors() ignores Linux taskset command In-Reply-To: <4a227590-41cd-4a4e-9b7d-9e8f8ad8f892@default> References: <4a227590-41cd-4a4e-9b7d-9e8f8ad8f892@default> Message-ID: <11cf69a4-140b-90c4-e7af-ad0fb983f997@oracle.com> Looks good. Reviewed. Thanks, David On 15/09/2016 4:40 PM, Shafi Ahmad wrote: > Hi, > > Please review the backport [modified] of bug: "JDK-6515172 Runtime.availableProcessors() ignores Linux taskset command" to jdk8u. > > Please note that the backport is not clean and we can't do as it is. Please note > 1. The changes are made by 'Andreas Eriksson' who left Oracle. > 2. There is difference in logging mechanism in jdk9 and jdk8 is different and file logTag.hpp doesn't exists in jdk8. > 3. Newly added test pass after this change. It fails without this change. > > Webrev link: http://cr.openjdk.java.net/~shshahma/8154324/webrev.02/ > Jdk9 bug: https://bugs.openjdk.java.net/browse/JDK-6515172 > Original patch pushed to jdk9: http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/c5480d4abfe4 > > Testing: jprt and running jtreg. > > Regards, > Shafi > From yasuenag at gmail.com Thu Sep 15 11:01:10 2016 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Thu, 15 Sep 2016 20:01:10 +0900 Subject: RFR(XS): 8165881 - Backout JDK-8164913 In-Reply-To: References: <519601d20d17$cc979220$65c6b660$@oracle.com> <14399a60-88ca-bcf4-0696-09f530aa2955@oracle.com> <97d8d6c3-2f4d-1475-f9b0-e608878e5aac@oracle.com> <8e7b6d80-7c1c-b432-2ef4-3b9aa27956d3@gmail.com> <07965d03-1663-8da3-d126-aecbb93a7716@oracle.com> <839d84a3-46e8-7c89-a1e7-ba03fa0ef98c@oracle.com> <29ff139b-6618-6216-ccf3-2fc03af1d6b6@gmail.com> <1ef93f3b-a9ce-93e7-cb16-d72406a2774a@gmail.com> Message-ID: Hi David, I agree the call sequence which you write in the case of "jcmd" operation. However, we should think about "load" operation in AttachListener. We can access JvmtiExport::load_agent_library() through two routes: Route "load": AttachListener -> JvmtiExport::load_agent_library() Route "jcmd": AttachListener -> jcmd() in attachListener.cpp -> DCmd::parse_and_execute() "load" sets error code (it means first data in the stream) when JvmtiExport::load_agent_library() returns JNI_ERR. OTOH "jcmd" sets error code if exception is occurred ONLY. If we try to load invalid JVMTI agent, "load" writes JNI_ERR to stream, however "jcmd" writes JNI_OK because pending exception is not available at this point. Currently, JCmd.java shows "Command executed successfully" when it cannot read the message from the socket. In case of this issue, JVMTIAgentLoadDCmd does not write any message because it cannot attach the agent. If return code which is in the top of stream should mean whether communication with AttachListener is succeeded, I think HotSpot should write error message or error code to the stream for JCmd.java . I'm not sure this email is the answer for you. Sorry for my English. Yasumasa On 2016/09/15 18:43, David Holmes wrote: > Hi Yasumasa, > > On 15/09/2016 1:56 PM, Yasumasa Suenaga wrote: >> Hi, >> >>> If this is done through jcmd then jcmd needs to know how to "unwrap" >>> the information it gets back from the Dcmd. >> >> In case of JVMTI.agent_load dcmd, I think we should be able to get the >> response as below: >> >> 1. Invalid JVMTI agent >> 2. Agent_OnAttach() did not return JNI_OK >> 3. Succeeded >> >> Currently, JvmtiExport::load_agent_library() returns JNI_OK to caller, >> and jcmd() in attachListener.cpp returns JNI_ERR if exception is >> occurred (in Agent_OnAttach()). > > Can you respond to my comment in the bug report about the chain of calls and explain exactly where you think things are going wrong in this scenario. I'm having a lot of trouble trying to see how the information flows back through the layers. > >> IMHO, HotSpot should return error code (in top of the stream: written by >> AttachListener at first). So we should be able to get some error code in >> case 1. and 2. Then the client (jcmd or other methods) can decide to >> parse text information in the stream. > > It returns the high-level response code first "did communication with the target VM succeed", then the actual action response code. That seems the right thing to do to me. It is up to the callers reading the stream to understand how to read it. To me the issue lies somewhere on the jcmd/dcmd side. > > Thanks, > David > >> >> Sorry for my English. >> >> Yasumasa >> >> >> On 2016/09/14 7:03, David Holmes wrote: >>> On 13/09/2016 10:31 PM, Yasumasa Suenaga wrote: >>>> Thanks David! >>>> If we should not change the meaning of return code from >>>> JvmtiExport::load_agent_library(), we should print return code as below: >>>> ------------------- >>>> diff -r 0cf03b9d9b1f src/share/vm/prims/jvmtiExport.cpp >>>> --- a/src/share/vm/prims/jvmtiExport.cpp Mon Sep 12 18:59:13 2016 >>>> +0000 >>>> +++ b/src/share/vm/prims/jvmtiExport.cpp Tue Sep 13 21:12:14 2016 >>>> +0900 >>>> @@ -2412,6 +2412,10 @@ >>>> result = JNI_OK; >>>> } >>>> } >>>> + // Print error code if Agent_OnAttach cannot be executed >>>> + if (result != JNI_OK) { >>>> + st->print_cr("%d", result); >>>> + } >>>> return result; >>>> } >>>> ------------------- >>> >>> Not sure I see the point. "return result" will put the error code into >>> the socket stream and that error will be seen and responded to. I >>> don't expect anything to then read further into the stream to see the >>> "result" repeated a second time. In other words if execute() doesn't >>> see a zero result then you go no further; if execute sees a zero >>> result then the actual action may have had an issue so you proceed to >>> read the "action's result". >>> >>>> It can pass com/sun/tools/attach/BasicTests.java, and we can get -1 if >>>> we try to attach invalid agent. >>> >>> Can you please outline your test case for this again. If this is done >>> through jcmd then jcmd needs to know how to "unwrap" the information >>> it gets back from the Dcmd. >>> >>> Thanks, >>> David >>> >>>> >>>> Yasumasa >>>> >>>> >>>> On 2016/09/13 17:06, Dmitry Samersoff wrote: >>>>> David, >>>>> >>>>> Thank you for the evaluation. >>>>> >>>>>> With that in mind I suspect the real fix for the original issue is to >>>>>> have Dcmd recognize that it also has to read two "results". Though I'm >>>>>> not sure how exactly. >>>>> >>>>> This logic seems completely broken for me. But I don't see anything we >>>>> can do right now - for jdk 9. >>>>> >>>>> It requires careful changes of both - code and tests. >>>>> >>>>> I can help with this task but not today. >>>>> >>>>> -Dmitry >>>>> >>>>> On 2016-09-13 08:08, David Holmes wrote: >>>>>> On 13/09/2016 1:53 PM, David Holmes wrote: >>>>>>> On 13/09/2016 1:30 PM, Yasumasa Suenaga wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> I could reproduce and I added a comment to JBS: >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165869?focusedCommentId=14000623&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14000623 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> In case of BasicTests.java, I think it is a test bug. >>>>>>> >>>>>>> I don't agree as yet. I have not been able to isolate the exact >>>>>>> difference between what happens with your fix and what happens >>>>>>> without. >>>>>>> I know it relates to the presence of the error code, but also how we >>>>>>> get >>>>>>> AgentInitializationException instead of AgentLoadException. I need >>>>>>> to be >>>>>>> able to run the test outside of jtreg but that is proving to be very >>>>>>> difficult to arrange. :( >>>>>> >>>>>> I finally managed to connect all the pieces on this. >>>>>> >>>>>> The basic problem is that with the proposed change >>>>>> VirtualMachineImpl.execute() will throw AgentLoadException, which then >>>>>> leads to the InternalError. Without the change we reach this block in >>>>>> HotspotVirtualMachine.loadAgentLibrary: >>>>>> >>>>>> int result = readInt(in); >>>>>> if (result != 0) { >>>>>> throw new AgentInitializationException("Agent_OnAttach failed", >>>>>> result); >>>>>> } >>>>>> >>>>>> and so get the expected AgentInitializationException. >>>>>> >>>>>> Looking at the proposed change in jvmtiExport.cpp if we have the >>>>>> original: >>>>>> >>>>>> st->print_cr("%d", result); >>>>>> result = JNI_OK; >>>>>> >>>>>> then execute() manages to read a zero completion status from the >>>>>> stream, >>>>>> and then loadAgentLibrary is able to read the actual "result" - >>>>>> whether >>>>>> zero or not - from the stream. This is because the AttachListener code >>>>>> calls op->complete(result, &st) where st is the stream where we wrote >>>>>> the result value above in print_cr. Then if we look at, for example, >>>>>> LinuxAttachOperation::complete, we will write "result" to the socket >>>>>> first, followed by the contents of st. Hence on a successful operation >>>>>> we can read 0 for execute, and then 0 for loadAgent. On error we >>>>>> read 0 >>>>>> for execute and the actual error code for loadAgent. With the proposed >>>>>> change execute() sees the real result (instead of JNI_OK) and so >>>>>> throws >>>>>> AgentLoadException. >>>>>> >>>>>> So in summary there are two different error indicators written into >>>>>> the >>>>>> stream, and we need the first to be zero unless some major error with >>>>>> the actual attach functionality occurred - otherwise even if the >>>>>> "load" >>>>>> failed in some other way, it is treated as a secondary error. >>>>>> >>>>>> With that in mind I suspect the real fix for the original issue is to >>>>>> have Dcmd recognize that it also has to read two "results". Though I'm >>>>>> not sure how exactly. >>>>>> >>>>>> David >>>>>> ----- >>>>>> >>>>>>> David >>>>>>> >>>>>>> >>>>>>> >>>>>>>> If it is accepted, I will upload webrev for this redo task. >>>>>>>> However, I cannot access (and fix) Oracle internal test. Can someone >>>>>>>> help me? >>>>>>>> (I cannot access JPRT. So I need a sponsor.) >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> On 2016/09/13 9:00, David Holmes wrote: >>>>>>>>> I think I see the problem. The attach framework tries to load the >>>>>>>>> "instrument" agent library. Prior to 8164913 if this fails it >>>>>>>>> actually >>>>>>>>> appears to succeed. Now it fails, and that error propagates >>>>>>>>> through. >>>>>>>>> I'm guessing, at the moment, that the failure is due to a missing >>>>>>>>> module related option. >>>>>>>>> >>>>>>>>> David >>>>>>>>> >>>>>>>>> On 13/09/2016 9:54 AM, David Holmes wrote: >>>>>>>>>> Hi Yasumasa, >>>>>>>>>> >>>>>>>>>> On 13/09/2016 9:23 AM, Yasumasa Suenaga wrote: >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> Hmm, it has been backouted... >>>>>>>>>>> >>>>>>>>>>> I agree to David. This error seems to be a test bug. >>>>>>>>>>> Can you share the test? I want to evaluate it. >>>>>>>>>> >>>>>>>>>> Sorry we can't share the tests. I took a quick look and it >>>>>>>>>> seems it >>>>>>>>>> may >>>>>>>>>> be related to the result code parsing (that I thought we ended up >>>>>>>>>> not >>>>>>>>>> touching!). >>>>>>>>>> >>>>>>>>>> Can you run a test of HotSpotVirtualMachine.loadAgent(null) and >>>>>>>>>> see >>>>>>>>>> what >>>>>>>>>> happens? We see AgentLoadException from the code that internally >>>>>>>>>> tries >>>>>>>>>> to load the "instrument" agent: >>>>>>>>>> >>>>>>>>>> Exception in thread "main" Failure: Unexpected exception during >>>>>>>>>> test >>>>>>>>>> execution: java.lang.InternalError: instrument library is >>>>>>>>>> missing in >>>>>>>>>> target VM >>>>>>>>>> ... >>>>>>>>>> Caused by: java.lang.InternalError: instrument library is >>>>>>>>>> missing in >>>>>>>>>> target VM >>>>>>>>>> ... >>>>>>>>>> Caused by: com.sun.tools.attach.AgentLoadException: Failed to load >>>>>>>>>> agent >>>>>>>>>> library >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> I do not agree to fix this bug in JDK 10 because VM might lie >>>>>>>>>>> when >>>>>>>>>>> the >>>>>>>>>>> JVMTI agent could not be attached. IMHO this bug should be fixed >>>>>>>>>>> in 9 >>>>>>>>>>> GA, and we should fix testcase(s). >>>>>>>>>> >>>>>>>>>> I agree. It has to be noted that "we" failed to run all the >>>>>>>>>> appropriate >>>>>>>>>> tests before pushing this, which would have discovered the >>>>>>>>>> problem - >>>>>>>>>> assuming it is actually a problem with the change and not an >>>>>>>>>> unrelated >>>>>>>>>> issue in our testing environment. >>>>>>>>>> >>>>>>>>>> Unfortunately I have some high priority tasks to handle right now, >>>>>>>>>> so I >>>>>>>>>> can't go into this in any more depth at the moment. >>>>>>>>>> >>>>>>>>>> David >>>>>>>>>> ----- >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 2016/09/13 6:15, David Holmes wrote: >>>>>>>>>>>> For the record it looks like the tests involved are broken, in >>>>>>>>>>>> that >>>>>>>>>>>> because the VM used to lie about the success of attaching the >>>>>>>>>>>> agent, the >>>>>>>>>>>> tests expected different exceptions to occur. >>>>>>>>>>>> >>>>>>>>>>>> David >>>>>>>>>>>> >>>>>>>>>>>> On 13/09/2016 3:11 AM, harold seigel wrote: >>>>>>>>>>>>> Looks good. >>>>>>>>>>>>> >>>>>>>>>>>>> Harold >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 9/12/2016 1:05 PM, Christian Tornqvist wrote: >>>>>>>>>>>>>> Hi everyone, >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Please review this (clean) backout of the change for >>>>>>>>>>>>>> JDK-8164913, it >>>>>>>>>>>>>> failed >>>>>>>>>>>>>> several tests in the nightly testing. The failures are >>>>>>>>>>>>>> tracked in: >>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165869 >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Webrev: >>>>>>>>>>>>>> >>>>>>>>>>>>>> http://cr.openjdk.java.net/~ctornqvi/webrev/8165881/webrev.00/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Bug: >>>>>>>>>>>>>> >>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165881 >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Christian >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>> >>>>> From dmitry.samersoff at oracle.com Thu Sep 15 11:08:11 2016 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Thu, 15 Sep 2016 14:08:11 +0300 Subject: RFR(XS): 8165881 - Backout JDK-8164913 In-Reply-To: References: <519601d20d17$cc979220$65c6b660$@oracle.com> <14399a60-88ca-bcf4-0696-09f530aa2955@oracle.com> <97d8d6c3-2f4d-1475-f9b0-e608878e5aac@oracle.com> <8e7b6d80-7c1c-b432-2ef4-3b9aa27956d3@gmail.com> <07965d03-1663-8da3-d126-aecbb93a7716@oracle.com> <839d84a3-46e8-7c89-a1e7-ba03fa0ef98c@oracle.com> <29ff139b-6618-6216-ccf3-2fc03af1d6b6@gmail.com> <1ef93f3b-a9ce-93e7-cb16-d72406a2774a@gmail.com> Message-ID: Yasumasa, David, I'm looking into this issue and come back as soon as I have a clear picture. It takes some time. Sorry! -Dmitry On 2016-09-15 14:01, Yasumasa Suenaga wrote: > Hi David, > > I agree the call sequence which you write in the case of "jcmd" operation. > However, we should think about "load" operation in AttachListener. > > We can access JvmtiExport::load_agent_library() through two routes: > > Route "load": AttachListener -> JvmtiExport::load_agent_library() > Route "jcmd": AttachListener -> jcmd() in attachListener.cpp -> > DCmd::parse_and_execute() > > "load" sets error code (it means first data in the stream) when > JvmtiExport::load_agent_library() returns JNI_ERR. > OTOH "jcmd" sets error code if exception is occurred ONLY. > > If we try to load invalid JVMTI agent, "load" writes JNI_ERR to stream, > however "jcmd" writes JNI_OK because pending exception is not available > at this point. > Currently, JCmd.java shows "Command executed successfully" when it > cannot read the message from the socket. In case of this issue, > JVMTIAgentLoadDCmd does not write any message because it cannot attach > the agent. > If return code which is in the top of stream should mean whether > communication with AttachListener is succeeded, I think HotSpot should > write error message or error code to the stream for JCmd.java . > > I'm not sure this email is the answer for you. > Sorry for my English. > > > Yasumasa > > > On 2016/09/15 18:43, David Holmes wrote: >> Hi Yasumasa, >> >> On 15/09/2016 1:56 PM, Yasumasa Suenaga wrote: >>> Hi, >>> >>>> If this is done through jcmd then jcmd needs to know how to "unwrap" >>>> the information it gets back from the Dcmd. >>> >>> In case of JVMTI.agent_load dcmd, I think we should be able to get the >>> response as below: >>> >>> 1. Invalid JVMTI agent >>> 2. Agent_OnAttach() did not return JNI_OK >>> 3. Succeeded >>> >>> Currently, JvmtiExport::load_agent_library() returns JNI_OK to caller, >>> and jcmd() in attachListener.cpp returns JNI_ERR if exception is >>> occurred (in Agent_OnAttach()). >> >> Can you respond to my comment in the bug report about the chain of >> calls and explain exactly where you think things are going wrong in >> this scenario. I'm having a lot of trouble trying to see how the >> information flows back through the layers. >> >>> IMHO, HotSpot should return error code (in top of the stream: written by >>> AttachListener at first). So we should be able to get some error code in >>> case 1. and 2. Then the client (jcmd or other methods) can decide to >>> parse text information in the stream. >> >> It returns the high-level response code first "did communication with >> the target VM succeed", then the actual action response code. That >> seems the right thing to do to me. It is up to the callers reading the >> stream to understand how to read it. To me the issue lies somewhere on >> the jcmd/dcmd side. >> >> Thanks, >> David >> >>> >>> Sorry for my English. >>> >>> Yasumasa >>> >>> >>> On 2016/09/14 7:03, David Holmes wrote: >>>> On 13/09/2016 10:31 PM, Yasumasa Suenaga wrote: >>>>> Thanks David! >>>>> If we should not change the meaning of return code from >>>>> JvmtiExport::load_agent_library(), we should print return code as >>>>> below: >>>>> ------------------- >>>>> diff -r 0cf03b9d9b1f src/share/vm/prims/jvmtiExport.cpp >>>>> --- a/src/share/vm/prims/jvmtiExport.cpp Mon Sep 12 18:59:13 2016 >>>>> +0000 >>>>> +++ b/src/share/vm/prims/jvmtiExport.cpp Tue Sep 13 21:12:14 2016 >>>>> +0900 >>>>> @@ -2412,6 +2412,10 @@ >>>>> result = JNI_OK; >>>>> } >>>>> } >>>>> + // Print error code if Agent_OnAttach cannot be executed >>>>> + if (result != JNI_OK) { >>>>> + st->print_cr("%d", result); >>>>> + } >>>>> return result; >>>>> } >>>>> ------------------- >>>> >>>> Not sure I see the point. "return result" will put the error code into >>>> the socket stream and that error will be seen and responded to. I >>>> don't expect anything to then read further into the stream to see the >>>> "result" repeated a second time. In other words if execute() doesn't >>>> see a zero result then you go no further; if execute sees a zero >>>> result then the actual action may have had an issue so you proceed to >>>> read the "action's result". >>>> >>>>> It can pass com/sun/tools/attach/BasicTests.java, and we can get -1 if >>>>> we try to attach invalid agent. >>>> >>>> Can you please outline your test case for this again. If this is done >>>> through jcmd then jcmd needs to know how to "unwrap" the information >>>> it gets back from the Dcmd. >>>> >>>> Thanks, >>>> David >>>> >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2016/09/13 17:06, Dmitry Samersoff wrote: >>>>>> David, >>>>>> >>>>>> Thank you for the evaluation. >>>>>> >>>>>>> With that in mind I suspect the real fix for the original issue >>>>>>> is to >>>>>>> have Dcmd recognize that it also has to read two "results". >>>>>>> Though I'm >>>>>>> not sure how exactly. >>>>>> >>>>>> This logic seems completely broken for me. But I don't see >>>>>> anything we >>>>>> can do right now - for jdk 9. >>>>>> >>>>>> It requires careful changes of both - code and tests. >>>>>> >>>>>> I can help with this task but not today. >>>>>> >>>>>> -Dmitry >>>>>> >>>>>> On 2016-09-13 08:08, David Holmes wrote: >>>>>>> On 13/09/2016 1:53 PM, David Holmes wrote: >>>>>>>> On 13/09/2016 1:30 PM, Yasumasa Suenaga wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> I could reproduce and I added a comment to JBS: >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165869?focusedCommentId=14000623&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14000623 >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> In case of BasicTests.java, I think it is a test bug. >>>>>>>> >>>>>>>> I don't agree as yet. I have not been able to isolate the exact >>>>>>>> difference between what happens with your fix and what happens >>>>>>>> without. >>>>>>>> I know it relates to the presence of the error code, but also >>>>>>>> how we >>>>>>>> get >>>>>>>> AgentInitializationException instead of AgentLoadException. I need >>>>>>>> to be >>>>>>>> able to run the test outside of jtreg but that is proving to be >>>>>>>> very >>>>>>>> difficult to arrange. :( >>>>>>> >>>>>>> I finally managed to connect all the pieces on this. >>>>>>> >>>>>>> The basic problem is that with the proposed change >>>>>>> VirtualMachineImpl.execute() will throw AgentLoadException, which >>>>>>> then >>>>>>> leads to the InternalError. Without the change we reach this >>>>>>> block in >>>>>>> HotspotVirtualMachine.loadAgentLibrary: >>>>>>> >>>>>>> int result = readInt(in); >>>>>>> if (result != 0) { >>>>>>> throw new AgentInitializationException("Agent_OnAttach failed", >>>>>>> result); >>>>>>> } >>>>>>> >>>>>>> and so get the expected AgentInitializationException. >>>>>>> >>>>>>> Looking at the proposed change in jvmtiExport.cpp if we have the >>>>>>> original: >>>>>>> >>>>>>> st->print_cr("%d", result); >>>>>>> result = JNI_OK; >>>>>>> >>>>>>> then execute() manages to read a zero completion status from the >>>>>>> stream, >>>>>>> and then loadAgentLibrary is able to read the actual "result" - >>>>>>> whether >>>>>>> zero or not - from the stream. This is because the AttachListener >>>>>>> code >>>>>>> calls op->complete(result, &st) where st is the stream where we >>>>>>> wrote >>>>>>> the result value above in print_cr. Then if we look at, for example, >>>>>>> LinuxAttachOperation::complete, we will write "result" to the socket >>>>>>> first, followed by the contents of st. Hence on a successful >>>>>>> operation >>>>>>> we can read 0 for execute, and then 0 for loadAgent. On error we >>>>>>> read 0 >>>>>>> for execute and the actual error code for loadAgent. With the >>>>>>> proposed >>>>>>> change execute() sees the real result (instead of JNI_OK) and so >>>>>>> throws >>>>>>> AgentLoadException. >>>>>>> >>>>>>> So in summary there are two different error indicators written into >>>>>>> the >>>>>>> stream, and we need the first to be zero unless some major error >>>>>>> with >>>>>>> the actual attach functionality occurred - otherwise even if the >>>>>>> "load" >>>>>>> failed in some other way, it is treated as a secondary error. >>>>>>> >>>>>>> With that in mind I suspect the real fix for the original issue >>>>>>> is to >>>>>>> have Dcmd recognize that it also has to read two "results". >>>>>>> Though I'm >>>>>>> not sure how exactly. >>>>>>> >>>>>>> David >>>>>>> ----- >>>>>>> >>>>>>>> David >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> If it is accepted, I will upload webrev for this redo task. >>>>>>>>> However, I cannot access (and fix) Oracle internal test. Can >>>>>>>>> someone >>>>>>>>> help me? >>>>>>>>> (I cannot access JPRT. So I need a sponsor.) >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2016/09/13 9:00, David Holmes wrote: >>>>>>>>>> I think I see the problem. The attach framework tries to load the >>>>>>>>>> "instrument" agent library. Prior to 8164913 if this fails it >>>>>>>>>> actually >>>>>>>>>> appears to succeed. Now it fails, and that error propagates >>>>>>>>>> through. >>>>>>>>>> I'm guessing, at the moment, that the failure is due to a missing >>>>>>>>>> module related option. >>>>>>>>>> >>>>>>>>>> David >>>>>>>>>> >>>>>>>>>> On 13/09/2016 9:54 AM, David Holmes wrote: >>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>> >>>>>>>>>>> On 13/09/2016 9:23 AM, Yasumasa Suenaga wrote: >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> Hmm, it has been backouted... >>>>>>>>>>>> >>>>>>>>>>>> I agree to David. This error seems to be a test bug. >>>>>>>>>>>> Can you share the test? I want to evaluate it. >>>>>>>>>>> >>>>>>>>>>> Sorry we can't share the tests. I took a quick look and it >>>>>>>>>>> seems it >>>>>>>>>>> may >>>>>>>>>>> be related to the result code parsing (that I thought we >>>>>>>>>>> ended up >>>>>>>>>>> not >>>>>>>>>>> touching!). >>>>>>>>>>> >>>>>>>>>>> Can you run a test of HotSpotVirtualMachine.loadAgent(null) and >>>>>>>>>>> see >>>>>>>>>>> what >>>>>>>>>>> happens? We see AgentLoadException from the code that internally >>>>>>>>>>> tries >>>>>>>>>>> to load the "instrument" agent: >>>>>>>>>>> >>>>>>>>>>> Exception in thread "main" Failure: Unexpected exception during >>>>>>>>>>> test >>>>>>>>>>> execution: java.lang.InternalError: instrument library is >>>>>>>>>>> missing in >>>>>>>>>>> target VM >>>>>>>>>>> ... >>>>>>>>>>> Caused by: java.lang.InternalError: instrument library is >>>>>>>>>>> missing in >>>>>>>>>>> target VM >>>>>>>>>>> ... >>>>>>>>>>> Caused by: com.sun.tools.attach.AgentLoadException: Failed to >>>>>>>>>>> load >>>>>>>>>>> agent >>>>>>>>>>> library >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> I do not agree to fix this bug in JDK 10 because VM might lie >>>>>>>>>>>> when >>>>>>>>>>>> the >>>>>>>>>>>> JVMTI agent could not be attached. IMHO this bug should be >>>>>>>>>>>> fixed >>>>>>>>>>>> in 9 >>>>>>>>>>>> GA, and we should fix testcase(s). >>>>>>>>>>> >>>>>>>>>>> I agree. It has to be noted that "we" failed to run all the >>>>>>>>>>> appropriate >>>>>>>>>>> tests before pushing this, which would have discovered the >>>>>>>>>>> problem - >>>>>>>>>>> assuming it is actually a problem with the change and not an >>>>>>>>>>> unrelated >>>>>>>>>>> issue in our testing environment. >>>>>>>>>>> >>>>>>>>>>> Unfortunately I have some high priority tasks to handle right >>>>>>>>>>> now, >>>>>>>>>>> so I >>>>>>>>>>> can't go into this in any more depth at the moment. >>>>>>>>>>> >>>>>>>>>>> David >>>>>>>>>>> ----- >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Yasumasa >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 2016/09/13 6:15, David Holmes wrote: >>>>>>>>>>>>> For the record it looks like the tests involved are broken, in >>>>>>>>>>>>> that >>>>>>>>>>>>> because the VM used to lie about the success of attaching the >>>>>>>>>>>>> agent, the >>>>>>>>>>>>> tests expected different exceptions to occur. >>>>>>>>>>>>> >>>>>>>>>>>>> David >>>>>>>>>>>>> >>>>>>>>>>>>> On 13/09/2016 3:11 AM, harold seigel wrote: >>>>>>>>>>>>>> Looks good. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Harold >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 9/12/2016 1:05 PM, Christian Tornqvist wrote: >>>>>>>>>>>>>>> Hi everyone, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Please review this (clean) backout of the change for >>>>>>>>>>>>>>> JDK-8164913, it >>>>>>>>>>>>>>> failed >>>>>>>>>>>>>>> several tests in the nightly testing. The failures are >>>>>>>>>>>>>>> tracked in: >>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165869 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Webrev: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ctornqvi/webrev/8165881/webrev.00/ >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Bug: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165881 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Christian >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>> >>>>>> -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From david.holmes at oracle.com Thu Sep 15 13:13:10 2016 From: david.holmes at oracle.com (David Holmes) Date: Thu, 15 Sep 2016 23:13:10 +1000 Subject: RFR(XS): 8165881 - Backout JDK-8164913 In-Reply-To: References: <519601d20d17$cc979220$65c6b660$@oracle.com> <14399a60-88ca-bcf4-0696-09f530aa2955@oracle.com> <97d8d6c3-2f4d-1475-f9b0-e608878e5aac@oracle.com> <8e7b6d80-7c1c-b432-2ef4-3b9aa27956d3@gmail.com> <07965d03-1663-8da3-d126-aecbb93a7716@oracle.com> <839d84a3-46e8-7c89-a1e7-ba03fa0ef98c@oracle.com> <29ff139b-6618-6216-ccf3-2fc03af1d6b6@gmail.com> <1ef93f3b-a9ce-93e7-cb16-d72406a2774a@gmail.com> Message-ID: On 15/09/2016 9:01 PM, Yasumasa Suenaga wrote: > Hi David, > > I agree the call sequence which you write in the case of "jcmd" operation. > However, we should think about "load" operation in AttachListener. > > We can access JvmtiExport::load_agent_library() through two routes: > > Route "load": AttachListener -> JvmtiExport::load_agent_library() > Route "jcmd": AttachListener -> jcmd() in attachListener.cpp -> > DCmd::parse_and_execute() > > "load" sets error code (it means first data in the stream) when > JvmtiExport::load_agent_library() returns JNI_ERR. > OTOH "jcmd" sets error code if exception is occurred ONLY. In attachListener we have: static jint jcmd(AttachOperation* op, outputStream* out) { Thread* THREAD = Thread::current(); // All the supplied jcmd arguments are stored as a single // string (op->arg(0)). This is parsed by the Dcmd framework. DCmd::parse_and_execute(DCmd_Source_AttachAPI, out, op->arg(0), ' ', THREAD); if (HAS_PENDING_EXCEPTION) { java_lang_Throwable::print(PENDING_EXCEPTION, out); out->cr(); CLEAR_PENDING_EXCEPTION; return JNI_ERR; } return JNI_OK; } If there is an exception pending it comes from Dcmd::parse_and_execute itself, not from the operation that was requested. Again we will call op->complete(res, &st) so this error code is first in the stream, and as with the other analysis, it is followed (if 0) by the real operation result code and possibly an error message. The stream is read back in VirtualMachineImpl.execute which reads the completion status first: // Create an input stream to read reply SocketInputStream sis = new SocketInputStream(s); // Read the command completion status int completionStatus; try { completionStatus = readInt(sis); } catch (IOException x) { sis.close(); if (ioe != null) { throw ioe; } else { throw x; } } if (completionStatus != 0) { // read from the stream and use that as the error message String message = readErrorMessage(sis); ... } return sis; The completion status will be 0 because we did manage to communicate with the target VM and process the DCmd request. It passes sis back up to the caller. So looking at JCmd.java I see: try (InputStream in = hvm.executeJCmd(line);) { // read to EOF and just print output byte b[] = new byte[256]; int n; boolean messagePrinted = false; do { n = in.read(b); if (n > 0) { String s = new String(b, 0, n, "UTF-8"); System.out.print(s); messagePrinted = true; } } while (n > 0); if (!messagePrinted) { System.out.println("Command executed successfully"); } } and that logic, AFAICS, is not parsing out a return code followed by an error message! David ----- > If we try to load invalid JVMTI agent, "load" writes JNI_ERR to stream, > however "jcmd" writes JNI_OK because pending exception is not available > at this point. > Currently, JCmd.java shows "Command executed successfully" when it > cannot read the message from the socket. In case of this issue, > JVMTIAgentLoadDCmd does not write any message because it cannot attach > the agent. > If return code which is in the top of stream should mean whether > communication with AttachListener is succeeded, I think HotSpot should > write error message or error code to the stream for JCmd.java . > > I'm not sure this email is the answer for you. > Sorry for my English. > > > Yasumasa > > > On 2016/09/15 18:43, David Holmes wrote: >> Hi Yasumasa, >> >> On 15/09/2016 1:56 PM, Yasumasa Suenaga wrote: >>> Hi, >>> >>>> If this is done through jcmd then jcmd needs to know how to "unwrap" >>>> the information it gets back from the Dcmd. >>> >>> In case of JVMTI.agent_load dcmd, I think we should be able to get the >>> response as below: >>> >>> 1. Invalid JVMTI agent >>> 2. Agent_OnAttach() did not return JNI_OK >>> 3. Succeeded >>> >>> Currently, JvmtiExport::load_agent_library() returns JNI_OK to caller, >>> and jcmd() in attachListener.cpp returns JNI_ERR if exception is >>> occurred (in Agent_OnAttach()). >> >> Can you respond to my comment in the bug report about the chain of >> calls and explain exactly where you think things are going wrong in >> this scenario. I'm having a lot of trouble trying to see how the >> information flows back through the layers. >> >>> IMHO, HotSpot should return error code (in top of the stream: written by >>> AttachListener at first). So we should be able to get some error code in >>> case 1. and 2. Then the client (jcmd or other methods) can decide to >>> parse text information in the stream. >> >> It returns the high-level response code first "did communication with >> the target VM succeed", then the actual action response code. That >> seems the right thing to do to me. It is up to the callers reading the >> stream to understand how to read it. To me the issue lies somewhere on >> the jcmd/dcmd side. >> >> Thanks, >> David >> >>> >>> Sorry for my English. >>> >>> Yasumasa >>> >>> >>> On 2016/09/14 7:03, David Holmes wrote: >>>> On 13/09/2016 10:31 PM, Yasumasa Suenaga wrote: >>>>> Thanks David! >>>>> If we should not change the meaning of return code from >>>>> JvmtiExport::load_agent_library(), we should print return code as >>>>> below: >>>>> ------------------- >>>>> diff -r 0cf03b9d9b1f src/share/vm/prims/jvmtiExport.cpp >>>>> --- a/src/share/vm/prims/jvmtiExport.cpp Mon Sep 12 18:59:13 2016 >>>>> +0000 >>>>> +++ b/src/share/vm/prims/jvmtiExport.cpp Tue Sep 13 21:12:14 2016 >>>>> +0900 >>>>> @@ -2412,6 +2412,10 @@ >>>>> result = JNI_OK; >>>>> } >>>>> } >>>>> + // Print error code if Agent_OnAttach cannot be executed >>>>> + if (result != JNI_OK) { >>>>> + st->print_cr("%d", result); >>>>> + } >>>>> return result; >>>>> } >>>>> ------------------- >>>> >>>> Not sure I see the point. "return result" will put the error code into >>>> the socket stream and that error will be seen and responded to. I >>>> don't expect anything to then read further into the stream to see the >>>> "result" repeated a second time. In other words if execute() doesn't >>>> see a zero result then you go no further; if execute sees a zero >>>> result then the actual action may have had an issue so you proceed to >>>> read the "action's result". >>>> >>>>> It can pass com/sun/tools/attach/BasicTests.java, and we can get -1 if >>>>> we try to attach invalid agent. >>>> >>>> Can you please outline your test case for this again. If this is done >>>> through jcmd then jcmd needs to know how to "unwrap" the information >>>> it gets back from the Dcmd. >>>> >>>> Thanks, >>>> David >>>> >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2016/09/13 17:06, Dmitry Samersoff wrote: >>>>>> David, >>>>>> >>>>>> Thank you for the evaluation. >>>>>> >>>>>>> With that in mind I suspect the real fix for the original issue >>>>>>> is to >>>>>>> have Dcmd recognize that it also has to read two "results". >>>>>>> Though I'm >>>>>>> not sure how exactly. >>>>>> >>>>>> This logic seems completely broken for me. But I don't see >>>>>> anything we >>>>>> can do right now - for jdk 9. >>>>>> >>>>>> It requires careful changes of both - code and tests. >>>>>> >>>>>> I can help with this task but not today. >>>>>> >>>>>> -Dmitry >>>>>> >>>>>> On 2016-09-13 08:08, David Holmes wrote: >>>>>>> On 13/09/2016 1:53 PM, David Holmes wrote: >>>>>>>> On 13/09/2016 1:30 PM, Yasumasa Suenaga wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> I could reproduce and I added a comment to JBS: >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165869?focusedCommentId=14000623&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14000623 >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> In case of BasicTests.java, I think it is a test bug. >>>>>>>> >>>>>>>> I don't agree as yet. I have not been able to isolate the exact >>>>>>>> difference between what happens with your fix and what happens >>>>>>>> without. >>>>>>>> I know it relates to the presence of the error code, but also >>>>>>>> how we >>>>>>>> get >>>>>>>> AgentInitializationException instead of AgentLoadException. I need >>>>>>>> to be >>>>>>>> able to run the test outside of jtreg but that is proving to be >>>>>>>> very >>>>>>>> difficult to arrange. :( >>>>>>> >>>>>>> I finally managed to connect all the pieces on this. >>>>>>> >>>>>>> The basic problem is that with the proposed change >>>>>>> VirtualMachineImpl.execute() will throw AgentLoadException, which >>>>>>> then >>>>>>> leads to the InternalError. Without the change we reach this >>>>>>> block in >>>>>>> HotspotVirtualMachine.loadAgentLibrary: >>>>>>> >>>>>>> int result = readInt(in); >>>>>>> if (result != 0) { >>>>>>> throw new AgentInitializationException("Agent_OnAttach failed", >>>>>>> result); >>>>>>> } >>>>>>> >>>>>>> and so get the expected AgentInitializationException. >>>>>>> >>>>>>> Looking at the proposed change in jvmtiExport.cpp if we have the >>>>>>> original: >>>>>>> >>>>>>> st->print_cr("%d", result); >>>>>>> result = JNI_OK; >>>>>>> >>>>>>> then execute() manages to read a zero completion status from the >>>>>>> stream, >>>>>>> and then loadAgentLibrary is able to read the actual "result" - >>>>>>> whether >>>>>>> zero or not - from the stream. This is because the AttachListener >>>>>>> code >>>>>>> calls op->complete(result, &st) where st is the stream where we >>>>>>> wrote >>>>>>> the result value above in print_cr. Then if we look at, for example, >>>>>>> LinuxAttachOperation::complete, we will write "result" to the socket >>>>>>> first, followed by the contents of st. Hence on a successful >>>>>>> operation >>>>>>> we can read 0 for execute, and then 0 for loadAgent. On error we >>>>>>> read 0 >>>>>>> for execute and the actual error code for loadAgent. With the >>>>>>> proposed >>>>>>> change execute() sees the real result (instead of JNI_OK) and so >>>>>>> throws >>>>>>> AgentLoadException. >>>>>>> >>>>>>> So in summary there are two different error indicators written into >>>>>>> the >>>>>>> stream, and we need the first to be zero unless some major error >>>>>>> with >>>>>>> the actual attach functionality occurred - otherwise even if the >>>>>>> "load" >>>>>>> failed in some other way, it is treated as a secondary error. >>>>>>> >>>>>>> With that in mind I suspect the real fix for the original issue >>>>>>> is to >>>>>>> have Dcmd recognize that it also has to read two "results". >>>>>>> Though I'm >>>>>>> not sure how exactly. >>>>>>> >>>>>>> David >>>>>>> ----- >>>>>>> >>>>>>>> David >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> If it is accepted, I will upload webrev for this redo task. >>>>>>>>> However, I cannot access (and fix) Oracle internal test. Can >>>>>>>>> someone >>>>>>>>> help me? >>>>>>>>> (I cannot access JPRT. So I need a sponsor.) >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2016/09/13 9:00, David Holmes wrote: >>>>>>>>>> I think I see the problem. The attach framework tries to load the >>>>>>>>>> "instrument" agent library. Prior to 8164913 if this fails it >>>>>>>>>> actually >>>>>>>>>> appears to succeed. Now it fails, and that error propagates >>>>>>>>>> through. >>>>>>>>>> I'm guessing, at the moment, that the failure is due to a missing >>>>>>>>>> module related option. >>>>>>>>>> >>>>>>>>>> David >>>>>>>>>> >>>>>>>>>> On 13/09/2016 9:54 AM, David Holmes wrote: >>>>>>>>>>> Hi Yasumasa, >>>>>>>>>>> >>>>>>>>>>> On 13/09/2016 9:23 AM, Yasumasa Suenaga wrote: >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> Hmm, it has been backouted... >>>>>>>>>>>> >>>>>>>>>>>> I agree to David. This error seems to be a test bug. >>>>>>>>>>>> Can you share the test? I want to evaluate it. >>>>>>>>>>> >>>>>>>>>>> Sorry we can't share the tests. I took a quick look and it >>>>>>>>>>> seems it >>>>>>>>>>> may >>>>>>>>>>> be related to the result code parsing (that I thought we >>>>>>>>>>> ended up >>>>>>>>>>> not >>>>>>>>>>> touching!). >>>>>>>>>>> >>>>>>>>>>> Can you run a test of HotSpotVirtualMachine.loadAgent(null) and >>>>>>>>>>> see >>>>>>>>>>> what >>>>>>>>>>> happens? We see AgentLoadException from the code that internally >>>>>>>>>>> tries >>>>>>>>>>> to load the "instrument" agent: >>>>>>>>>>> >>>>>>>>>>> Exception in thread "main" Failure: Unexpected exception during >>>>>>>>>>> test >>>>>>>>>>> execution: java.lang.InternalError: instrument library is >>>>>>>>>>> missing in >>>>>>>>>>> target VM >>>>>>>>>>> ... >>>>>>>>>>> Caused by: java.lang.InternalError: instrument library is >>>>>>>>>>> missing in >>>>>>>>>>> target VM >>>>>>>>>>> ... >>>>>>>>>>> Caused by: com.sun.tools.attach.AgentLoadException: Failed to >>>>>>>>>>> load >>>>>>>>>>> agent >>>>>>>>>>> library >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> I do not agree to fix this bug in JDK 10 because VM might lie >>>>>>>>>>>> when >>>>>>>>>>>> the >>>>>>>>>>>> JVMTI agent could not be attached. IMHO this bug should be >>>>>>>>>>>> fixed >>>>>>>>>>>> in 9 >>>>>>>>>>>> GA, and we should fix testcase(s). >>>>>>>>>>> >>>>>>>>>>> I agree. It has to be noted that "we" failed to run all the >>>>>>>>>>> appropriate >>>>>>>>>>> tests before pushing this, which would have discovered the >>>>>>>>>>> problem - >>>>>>>>>>> assuming it is actually a problem with the change and not an >>>>>>>>>>> unrelated >>>>>>>>>>> issue in our testing environment. >>>>>>>>>>> >>>>>>>>>>> Unfortunately I have some high priority tasks to handle right >>>>>>>>>>> now, >>>>>>>>>>> so I >>>>>>>>>>> can't go into this in any more depth at the moment. >>>>>>>>>>> >>>>>>>>>>> David >>>>>>>>>>> ----- >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Yasumasa >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 2016/09/13 6:15, David Holmes wrote: >>>>>>>>>>>>> For the record it looks like the tests involved are broken, in >>>>>>>>>>>>> that >>>>>>>>>>>>> because the VM used to lie about the success of attaching the >>>>>>>>>>>>> agent, the >>>>>>>>>>>>> tests expected different exceptions to occur. >>>>>>>>>>>>> >>>>>>>>>>>>> David >>>>>>>>>>>>> >>>>>>>>>>>>> On 13/09/2016 3:11 AM, harold seigel wrote: >>>>>>>>>>>>>> Looks good. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Harold >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 9/12/2016 1:05 PM, Christian Tornqvist wrote: >>>>>>>>>>>>>>> Hi everyone, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Please review this (clean) backout of the change for >>>>>>>>>>>>>>> JDK-8164913, it >>>>>>>>>>>>>>> failed >>>>>>>>>>>>>>> several tests in the nightly testing. The failures are >>>>>>>>>>>>>>> tracked in: >>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165869 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Webrev: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ctornqvi/webrev/8165881/webrev.00/ >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Bug: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8165881 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Christian >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>> >>>>>> From kim.barrett at oracle.com Thu Sep 15 15:32:06 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 15 Sep 2016 11:32:06 -0400 Subject: RFR: 8165808: Add release barriers when allocating objects with concurrent collection In-Reply-To: <64d35b20-c1fd-cfbc-7da9-4cc86b28685e@oracle.com> References: <13AE434E-FA26-4658-8A77-013462679EBC@oracle.com> <6c991053-7347-cc6b-3b94-fc48cb27a1db@oracle.com> <98CE5C19-7B72-4530-977C-C371164D439E@oracle.com> <64d35b20-c1fd-cfbc-7da9-4cc86b28685e@oracle.com> Message-ID: <6C22E368-923F-406E-AA5F-1C09F14E8F0D@oracle.com> > On Sep 15, 2016, at 5:49 AM, David Holmes wrote: > > You can count me a reviewer #2. Thanks! From coleen.phillimore at oracle.com Thu Sep 15 16:09:43 2016 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Thu, 15 Sep 2016 12:09:43 -0400 Subject: RFR(S): 8165602: Convert TestChunkedList_test to GTest In-Reply-To: References: Message-ID: The GC group added this test, so copying them. Coleen On 9/15/16 11:25 AM, Kirill Zhaldybin wrote: > Dear all, > > Could you please review this fix for 8165602? > > WebRev: > http://cr.openjdk.java.net/~kzhaldyb/webrevs/JDK-8165602/webrev.01/ > CR: https://bugs.openjdk.java.net/browse/JDK-8165602 > > Thank you. > > Regards, Kirill From stevenschlansker at gmail.com Thu Sep 15 17:17:56 2016 From: stevenschlansker at gmail.com (Steven Schlansker) Date: Thu, 15 Sep 2016 10:17:56 -0700 Subject: Understanding "class" native memory usage Message-ID: Hi hotspot-dev, Hopefully I found an appropriate mailing list. Let me know if I should be asking elsewhere. We run OpenJDK 8u91 inside of Linux containers. One of the challenges we've faced is ensuring that the container memory limits don't kill our Java processes unexpectedly -- heap sizing is relatively easy, but there's a number of other regions the JVM uses that aren't as easy to account for. We use Native Memory Tracking and export the statistics. Plotting the "class committed" NMT metric: You'll notice what looks like a very slow memory leak. We've confirmed with "-verbose:class" that we are are not loading many new classes - this graph starts a day after launch, so the application should have long reached a relatively steady state. The jump at 9/15 08:00 was due to classloading relating to attaching JMX monitoring. The long slow rise though we can't account for. Eventually the application exceeds its container memory bound and is SIGKILLed by the kernel. We are in the process of iteratively raising the limit, but it's unclear how large this class space could grow. We've observed some evidence that it can be GCed eventually, but it's not clear what prompts it or how we'd encourage it to happen more often. Here are our relevant JVM options: -XX:+AlwaysPreTouch -XX:MaxMetaspaceSize=64m -XX:CompressedClassSpaceSize=32m -XX:ReservedCodeCacheSize=64m -XX:ParallelGCThreads=4 -XX:+UseConcMarkSweepGC -XX:+DisableExplicitGC -XX:NativeMemoryTracking=summary -Xmx2100m -Xms2100m Notice that we set a 32m limit on "compressed class space size" -- which is apparently not the same as "class" usage in NMT? Or the limit isn't effective? The questions I am trying to answer: * What causes this long slow rise of "class" usage? It almost looks like a leak. * How do we limit this native memory region? We're trying to set absolute limits; we'd much prefer a Java OutOfMemoryError than a visit from the kernel OOM killer * It'd be nice to signal to the JVM "You have this much memory total, and not a byte more" and have the other tuneables set sensibly based on that value. Maybe I'm dreaming. This also isn't a question. Any other tips on diagnosing this sort of issue also appreciated. Thanks in advance, Steven From chris.plummer at oracle.com Thu Sep 15 18:17:51 2016 From: chris.plummer at oracle.com (Chris Plummer) Date: Thu, 15 Sep 2016 11:17:51 -0700 Subject: [8u] RFR for JDK-6515172: Runtime.availableProcessors() ignores Linux taskset command In-Reply-To: <11cf69a4-140b-90c4-e7af-ad0fb983f997@oracle.com> References: <4a227590-41cd-4a4e-9b7d-9e8f8ad8f892@default> <11cf69a4-140b-90c4-e7af-ad0fb983f997@oracle.com> Message-ID: You should probably include the backport of JDK-8165153 since it was introduced by JDK-6515172. cheers, Chris On 9/15/16 2:51 AM, David Holmes wrote: > Looks good. Reviewed. > > Thanks, > David > > On 15/09/2016 4:40 PM, Shafi Ahmad wrote: >> Hi, >> >> Please review the backport [modified] of bug: "JDK-6515172 >> Runtime.availableProcessors() ignores Linux taskset command" to jdk8u. >> >> Please note that the backport is not clean and we can't do as it is. >> Please note >> 1. The changes are made by 'Andreas Eriksson' who left Oracle. >> 2. There is difference in logging mechanism in jdk9 and jdk8 is >> different and file logTag.hpp doesn't exists in jdk8. >> 3. Newly added test pass after this change. It fails without this >> change. >> >> Webrev link: http://cr.openjdk.java.net/~shshahma/8154324/webrev.02/ >> Jdk9 bug: https://bugs.openjdk.java.net/browse/JDK-6515172 >> Original patch pushed to jdk9: >> http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/c5480d4abfe4 >> >> Testing: jprt and running jtreg. >> >> Regards, >> Shafi >> From zgu at redhat.com Thu Sep 15 18:31:56 2016 From: zgu at redhat.com (Zhengyu Gu) Date: Thu, 15 Sep 2016 14:31:56 -0400 Subject: Understanding "class" native memory usage In-Reply-To: References: Message-ID: Hi Steven, On 09/15/2016 01:17 PM, Steven Schlansker wrote: > Hi hotspot-dev, > > Hopefully I found an appropriate mailing list. Let me know if I should be asking elsewhere. > > We run OpenJDK 8u91 inside of Linux containers. One of the challenges we've faced is > ensuring that the container memory limits don't kill our Java processes unexpectedly -- > heap sizing is relatively easy, but there's a number of other regions the JVM uses that > aren't as easy to account for. > > We use Native Memory Tracking and export the statistics. Plotting the "class committed" NMT > metric: > > You'll notice what looks like a very slow memory leak. We've confirmed with "-verbose:class" that we are > are not loading many new classes - this graph starts a day after launch, so the application should have long > reached a relatively steady state. The jump at 9/15 08:00 was due to classloading relating to attaching > JMX monitoring. The long slow rise though we can't account for. > > Eventually the application exceeds its container memory bound and is SIGKILLed by the kernel. > We are in the process of iteratively raising the limit, but it's unclear how large this class > space could grow. We've observed some evidence that it can be GCed eventually, but it's not > clear what prompts it or how we'd encourage it to happen more often. > > Here are our relevant JVM options: > -XX:+AlwaysPreTouch > -XX:MaxMetaspaceSize=64m > -XX:CompressedClassSpaceSize=32m > -XX:ReservedCodeCacheSize=64m > -XX:ParallelGCThreads=4 > -XX:+UseConcMarkSweepGC > -XX:+DisableExplicitGC > -XX:NativeMemoryTracking=summary > -Xmx2100m > -Xms2100m > > Notice that we set a 32m limit on "compressed class space size" -- which is apparently not the same as > "class" usage in NMT? Or the limit isn't effective? NMT counts many runtime data structures that associate with classes in "class" category, such as hashtable for class lookup, class loaders and etc. > The questions I am trying to answer: > > * What causes this long slow rise of "class" usage? It almost looks like a leak. > * How do we limit this native memory region? We're trying to set absolute limits; we'd much prefer a > Java OutOfMemoryError than a visit from the kernel OOM killer They are different type of OOM. Java OutOfMemoryError is due to running out of Java heap space, which is 2100m in your case. I think the memory leak that you are talking about here, is native memory, which is outside of Java heap. > * It'd be nice to signal to the JVM "You have this much memory total, and not a byte more" and have > the other tuneables set sensibly based on that value. Maybe I'm dreaming. This also isn't a question. > > Any other tips on diagnosing this sort of issue also appreciated. To track down exactly where leaks memory, you can use detail tracking option (-XX:NativeMemoryTracking=detail) After application starts, use NMT "baseline" command to establish an early memory baseline, then you can issue detail.diff command periodically to see native memory activities over time. If there are memory leaks, you should be able to see that some call sites have increasing memory allocations. Hope this helps. Thanks, -Zhengyu > Thanks in advance, > Steven > From jon.masamitsu at oracle.com Thu Sep 15 19:02:27 2016 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Thu, 15 Sep 2016 12:02:27 -0700 Subject: Understanding "class" native memory usage In-Reply-To: References: Message-ID: In addition to Zhengyu's advice, turn on -XX:+PrintGCDetails (if not already turned on) so we can see what GC is doing. Jon On 9/15/2016 11:31 AM, Zhengyu Gu wrote: > Hi Steven, > > On 09/15/2016 01:17 PM, Steven Schlansker wrote: >> Hi hotspot-dev, >> >> Hopefully I found an appropriate mailing list. Let me know if I >> should be asking elsewhere. >> >> We run OpenJDK 8u91 inside of Linux containers. One of the >> challenges we've faced is >> ensuring that the container memory limits don't kill our Java >> processes unexpectedly -- >> heap sizing is relatively easy, but there's a number of other regions >> the JVM uses that >> aren't as easy to account for. >> >> We use Native Memory Tracking and export the statistics. Plotting the >> "class committed" NMT >> metric: >> >> You'll notice what looks like a very slow memory leak. We've >> confirmed with "-verbose:class" that we are >> are not loading many new classes - this graph starts a day after >> launch, so the application should have long >> reached a relatively steady state. The jump at 9/15 08:00 was due to >> classloading relating to attaching >> JMX monitoring. The long slow rise though we can't account for. >> >> Eventually the application exceeds its container memory bound and is >> SIGKILLed by the kernel. >> We are in the process of iteratively raising the limit, but it's >> unclear how large this class >> space could grow. We've observed some evidence that it can be GCed >> eventually, but it's not >> clear what prompts it or how we'd encourage it to happen more often. >> >> Here are our relevant JVM options: >> -XX:+AlwaysPreTouch >> -XX:MaxMetaspaceSize=64m >> -XX:CompressedClassSpaceSize=32m >> -XX:ReservedCodeCacheSize=64m >> -XX:ParallelGCThreads=4 >> -XX:+UseConcMarkSweepGC >> -XX:+DisableExplicitGC >> -XX:NativeMemoryTracking=summary >> -Xmx2100m >> -Xms2100m >> >> Notice that we set a 32m limit on "compressed class space size" -- >> which is apparently not the same as >> "class" usage in NMT? Or the limit isn't effective? > > NMT counts many runtime data structures that associate with classes in > "class" category, > such as hashtable for class lookup, class loaders and etc. > >> The questions I am trying to answer: >> >> * What causes this long slow rise of "class" usage? It almost looks >> like a leak. >> * How do we limit this native memory region? We're trying to set >> absolute limits; we'd much prefer a >> Java OutOfMemoryError than a visit from the kernel OOM killer > > They are different type of OOM. > > Java OutOfMemoryError is due to running out of Java heap space, which > is 2100m in your case. > I think the memory leak that you are talking about here, is native > memory, which is outside of Java heap. > >> * It'd be nice to signal to the JVM "You have this much memory total, >> and not a byte more" and have >> the other tuneables set sensibly based on that value. Maybe I'm >> dreaming. This also isn't a question. >> >> Any other tips on diagnosing this sort of issue also appreciated. > > To track down exactly where leaks memory, you can use detail tracking > option (-XX:NativeMemoryTracking=detail) > After application starts, use NMT "baseline" command to establish an > early memory baseline, then you can issue > detail.diff command periodically to see native memory activities over > time. > > If there are memory leaks, you should be able to see that some call > sites have increasing memory allocations. > > Hope this helps. > > Thanks, > > -Zhengyu > > >> Thanks in advance, >> Steven >> > From david.holmes at oracle.com Fri Sep 16 01:45:32 2016 From: david.holmes at oracle.com (David Holmes) Date: Fri, 16 Sep 2016 11:45:32 +1000 Subject: [8u] RFR for JDK-6515172: Runtime.availableProcessors() ignores Linux taskset command In-Reply-To: References: <4a227590-41cd-4a4e-9b7d-9e8f8ad8f892@default> <11cf69a4-140b-90c4-e7af-ad0fb983f997@oracle.com> Message-ID: On 16/09/2016 4:17 AM, Chris Plummer wrote: > You should probably include the backport of JDK-8165153 since it was > introduced by JDK-6515172. This is a simplified backport of 6515172 suitable for Java 8. It does end up being a combination of that fix and 8165153, but you can't be a backport of two bugs, and there are no distinct parts that might map to each bug. That was why originally this was flagged under a separate bug id. But I agreed that this could be considered a simpified backport of 6515172. Thanks, David > cheers, > > Chris > > On 9/15/16 2:51 AM, David Holmes wrote: >> Looks good. Reviewed. >> >> Thanks, >> David >> >> On 15/09/2016 4:40 PM, Shafi Ahmad wrote: >>> Hi, >>> >>> Please review the backport [modified] of bug: "JDK-6515172 >>> Runtime.availableProcessors() ignores Linux taskset command" to jdk8u. >>> >>> Please note that the backport is not clean and we can't do as it is. >>> Please note >>> 1. The changes are made by 'Andreas Eriksson' who left Oracle. >>> 2. There is difference in logging mechanism in jdk9 and jdk8 is >>> different and file logTag.hpp doesn't exists in jdk8. >>> 3. Newly added test pass after this change. It fails without this >>> change. >>> >>> Webrev link: http://cr.openjdk.java.net/~shshahma/8154324/webrev.02/ >>> Jdk9 bug: https://bugs.openjdk.java.net/browse/JDK-6515172 >>> Original patch pushed to jdk9: >>> http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/c5480d4abfe4 >>> >>> Testing: jprt and running jtreg. >>> >>> Regards, >>> Shafi >>> > From shafi.s.ahmad at oracle.com Mon Sep 19 04:54:58 2016 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Sun, 18 Sep 2016 21:54:58 -0700 (PDT) Subject: [8u] RFR for JDK-6515172: Runtime.availableProcessors() ignores Linux taskset command In-Reply-To: References: <4a227590-41cd-4a4e-9b7d-9e8f8ad8f892@default> <11cf69a4-140b-90c4-e7af-ad0fb983f997@oracle.com> Message-ID: May I get the second review for this change. Regards, Shafi > -----Original Message----- > From: David Holmes > Sent: Friday, September 16, 2016 7:16 AM > To: Chris Plummer; Shafi Ahmad; hotspot-dev at openjdk.java.net > Subject: Re: [8u] RFR for JDK-6515172: Runtime.availableProcessors() ignores > Linux taskset command > > On 16/09/2016 4:17 AM, Chris Plummer wrote: > > You should probably include the backport of JDK-8165153 since it was > > introduced by JDK-6515172. > > This is a simplified backport of 6515172 suitable for Java 8. It does end up > being a combination of that fix and 8165153, but you can't be a backport of > two bugs, and there are no distinct parts that might map to each bug. That > was why originally this was flagged under a separate bug id. But I agreed that > this could be considered a simpified backport of 6515172. > > Thanks, > David > > > cheers, > > > > Chris > > > > On 9/15/16 2:51 AM, David Holmes wrote: > >> Looks good. Reviewed. > >> > >> Thanks, > >> David > >> > >> On 15/09/2016 4:40 PM, Shafi Ahmad wrote: > >>> Hi, > >>> > >>> Please review the backport [modified] of bug: "JDK-6515172 > >>> Runtime.availableProcessors() ignores Linux taskset command" to jdk8u. > >>> > >>> Please note that the backport is not clean and we can't do as it is. > >>> Please note > >>> 1. The changes are made by 'Andreas Eriksson' who left Oracle. > >>> 2. There is difference in logging mechanism in jdk9 and jdk8 is > >>> different and file logTag.hpp doesn't exists in jdk8. > >>> 3. Newly added test pass after this change. It fails without this > >>> change. > >>> > >>> Webrev link: > http://cr.openjdk.java.net/~shshahma/8154324/webrev.02/ > >>> Jdk9 bug: https://bugs.openjdk.java.net/browse/JDK-6515172 > >>> Original patch pushed to jdk9: > >>> http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/c5480d4abfe4 > >>> > >>> Testing: jprt and running jtreg. > >>> > >>> Regards, > >>> Shafi > >>> > > From shafi.s.ahmad at oracle.com Mon Sep 19 05:14:28 2016 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Sun, 18 Sep 2016 22:14:28 -0700 (PDT) Subject: [8u] RFR for JDK-8157548: JVM crashes sometimes while starting Message-ID: Hi, Please review the small code change for bug: "JDK-8157548: JVM crashes sometimes while starting" on jdk8u-dev Summary: int strncmp(const char *s1, const char *s2, size_t n); s1 = "abcdefgh" // Assume this is not null terminated string. s2 = "abcdefghijk" n = 10 In case if s1 is not null terminated then for above input strncmp may crash. In expression marked as (B) parsed_name->bytes() returns base address of non-null terminated string buffer. + size_t pkglen = strlen(pkg); if (!HAS_PENDING_EXCEPTION && !class_loader.is_null() && parsed_name != NULL && - !strncmp((const char*)parsed_name->bytes(), pkg, strlen(pkg))) { + parsed_name->utf8_length() >= (int)pkglen && // ------------------------------ (A) + !strncmp((const char*)parsed_name->bytes(), pkg, pkglen)) { //------------------------------ (B) Adding expression marked as (A) avoid the above similar input scenario. Webrev: http://cr.openjdk.java.net/~shshahma/8157548/webrev.00/ Jdk8 bug: https://bugs.openjdk.java.net/browse/JDK-8157548 Test: Run jprt Note: Thanks to Ioi for providing the code change. Regards, Shafi From david.holmes at oracle.com Mon Sep 19 07:03:12 2016 From: david.holmes at oracle.com (David Holmes) Date: Mon, 19 Sep 2016 17:03:12 +1000 Subject: [8u] RFR for JDK-8157548: JVM crashes sometimes while starting In-Reply-To: References: Message-ID: <3fa82060-9a3a-f43b-efbf-d196ac473c41@oracle.com> Hi Shafi, This looks okay to me. Thanks, David On 19/09/2016 3:14 PM, Shafi Ahmad wrote: > Hi, > > Please review the small code change for bug: "JDK-8157548: JVM crashes sometimes while starting" on jdk8u-dev > > Summary: > int strncmp(const char *s1, const char *s2, size_t n); > > s1 = "abcdefgh" // Assume this is not null terminated string. > s2 = "abcdefghijk" > n = 10 > > In case if s1 is not null terminated then for above input strncmp may crash. > > In expression marked as (B) parsed_name->bytes() returns base address of non-null terminated string buffer. > > + size_t pkglen = strlen(pkg); > if (!HAS_PENDING_EXCEPTION && > !class_loader.is_null() && > parsed_name != NULL && > - !strncmp((const char*)parsed_name->bytes(), pkg, strlen(pkg))) { > + parsed_name->utf8_length() >= (int)pkglen && // ------------------------------ (A) > + !strncmp((const char*)parsed_name->bytes(), pkg, pkglen)) { //------------------------------ (B) > > Adding expression marked as (A) avoid the above similar input scenario. > > Webrev: http://cr.openjdk.java.net/~shshahma/8157548/webrev.00/ > Jdk8 bug: https://bugs.openjdk.java.net/browse/JDK-8157548 > > Test: Run jprt > > Note: Thanks to Ioi for providing the code change. > > Regards, > Shafi > From tobias.hartmann at oracle.com Mon Sep 19 08:09:30 2016 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Mon, 19 Sep 2016 10:09:30 +0200 Subject: [9] RFR(S): Crash with assert: symbol conversion failure in java_lang_String::create_from_symbol() In-Reply-To: References: <57D2AE2F.1080607@oracle.com> <1203086c-f59f-40ce-f963-e958e570caf8@oracle.com> Message-ID: <57DF9D3A.3000802@oracle.com> Coleen, David, thanks for the reviews and sorry for the delay! I agree that we should validate the Strings we get through JNI, but the code in create_from_str() and create_from_symbol() was originally added to check the result of String compression with the Compact Strings feature and not to check for valid UTF-8. I think UTF-8 validity should be checked earlier (before or during Symbol creation) and this affects other JNI methods that take C-Strings as well (jni_NewString, jni_NewStringUTF, jni_DefineClass, ...). This is a general problem with JNI not validating input Strings. Are you fine with pushing the proposed fix (webrev.00) and open a follow up bug to fix JNI (and potentially re-enable the Compact Strings asserts because their existence is still justified)? Thanks, Tobias On 12.09.2016 02:47, David Holmes wrote: > On 10/09/2016 6:55 AM, Coleen Phillimore wrote: >> This change is fine because it matches the commented out assert in >> create_from_str(). We should probably figure out what it would take to >> check the characters coming in from JNI and decide whether we should do >> this. If not, it doesn't make sense to have commented out asserts. >> But this is okay for jdk9. > > Grumble, grumble ... both are bad. If the VM doesn't validate this bad UTF-8 then where does it go? And how does the generator of the bad UTF-8 get informed? An assert may be too drastic but can we throw an exception (InternalError?) ? > > David > >> Thanks, >> Coleen >> >> >> On 9/9/16 8:42 AM, Tobias Hartmann wrote: >>> Hi, >>> >>> please review the following patch: >>> https://bugs.openjdk.java.net/browse/JDK-8164561 >>> http://cr.openjdk.java.net/~thartmann/8164561/webrev.00/ >>> >>> The verification code in java_lang_String::create_from_symbol() that >>> was added by Compact Strings fails because the input symbol does not >>> contain valid UTF8. The problem is that a JCK JNI test passes an >>> invalid UTF8 string as class name to the JNI method "FindClass". In >>> fact, the string contains garbage from reading past array boundaries >>> because of a bug in the test [1]. The JNI spec [2] states that 'name' >>> should be "a fully-qualified class name (that is, a package name, >>> delimited by ?/?, followed by the class name). If the name begins with >>> ?[? (the array signature character), it returns an array class. The >>> string is encoded in modified UTF-8". >>> >>> I nevertheless think that we should not crash in the case of an >>> invalid UTF8 string and therefore disabled the verification code with >>> a comment. We did the same for java_lang_String::create_from_str() [3]. >>> >>> Tested with failing JCK test and JPRT (running). >>> >>> Thanks, >>> Tobias >>> >>> [1] https://bugs.openjdk.java.net/browse/JCK-7307244 >>> [2] >>> https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/functions.html#FindClass >>> >>> [3] >>> http://hg.openjdk.java.net/jdk9/hs/hotspot/file/d060826d0911/src/share/vm/classfile/javaClasses.cpp#l274 >>> >> From david.holmes at oracle.com Mon Sep 19 08:30:06 2016 From: david.holmes at oracle.com (David Holmes) Date: Mon, 19 Sep 2016 18:30:06 +1000 Subject: [9] RFR(S): Crash with assert: symbol conversion failure in java_lang_String::create_from_symbol() In-Reply-To: <57DF9D3A.3000802@oracle.com> References: <57D2AE2F.1080607@oracle.com> <1203086c-f59f-40ce-f963-e958e570caf8@oracle.com> <57DF9D3A.3000802@oracle.com> Message-ID: On 19/09/2016 6:09 PM, Tobias Hartmann wrote: > Coleen, David, thanks for the reviews and sorry for the delay! > > I agree that we should validate the Strings we get through JNI, but the code in create_from_str() and create_from_symbol() was originally added to check the result of String compression with the Compact Strings feature and not to check for valid UTF-8. > > I think UTF-8 validity should be checked earlier (before or during Symbol creation) and this affects other JNI methods that take C-Strings as well (jni_NewString, jni_NewStringUTF, jni_DefineClass, ...). This is a general problem with JNI not validating input Strings. > > Are you fine with pushing the proposed fix (webrev.00) and open a follow up bug to fix JNI (and potentially re-enable the Compact Strings asserts because their existence is still justified)? As I wrote in the bug report: "I don't think I agree with the removed checks in either case. JNI doesn't do argument checking and so doesn't detect the invalid UTF-8 string. But the VM expects to work with valid strings. It seems quite reasonable to me that the VM should validate the UTF-8 and report some kind of failure to make these invalid inputs visible. Otherwise the bad JNI codes goes undetected. Maybe an assertion is too strong, perhaps we should throw InternalError? " If the VM does not detect this and respond to it exactly what happens with the bad UTF-8 string? JNI doesn't do input validation - it is a "feature" of JNI so not to penalize correct code. If the test that triggered this is sending in bad UTF-8 because it expects JNI to do validation then it is an invalid test. I think the asserts serve their purpose exactly - to show where invalid inputs came from. If this was bad user code then it would be up to them to fix it. If it is a bad test then the test should be fixed. Sorry. David > Thanks, > Tobias > > On 12.09.2016 02:47, David Holmes wrote: >> On 10/09/2016 6:55 AM, Coleen Phillimore wrote: >>> This change is fine because it matches the commented out assert in >>> create_from_str(). We should probably figure out what it would take to >>> check the characters coming in from JNI and decide whether we should do >>> this. If not, it doesn't make sense to have commented out asserts. >>> But this is okay for jdk9. >> >> Grumble, grumble ... both are bad. If the VM doesn't validate this bad UTF-8 then where does it go? And how does the generator of the bad UTF-8 get informed? An assert may be too drastic but can we throw an exception (InternalError?) ? >> >> David >> >>> Thanks, >>> Coleen >>> >>> >>> On 9/9/16 8:42 AM, Tobias Hartmann wrote: >>>> Hi, >>>> >>>> please review the following patch: >>>> https://bugs.openjdk.java.net/browse/JDK-8164561 >>>> http://cr.openjdk.java.net/~thartmann/8164561/webrev.00/ >>>> >>>> The verification code in java_lang_String::create_from_symbol() that >>>> was added by Compact Strings fails because the input symbol does not >>>> contain valid UTF8. The problem is that a JCK JNI test passes an >>>> invalid UTF8 string as class name to the JNI method "FindClass". In >>>> fact, the string contains garbage from reading past array boundaries >>>> because of a bug in the test [1]. The JNI spec [2] states that 'name' >>>> should be "a fully-qualified class name (that is, a package name, >>>> delimited by ?/?, followed by the class name). If the name begins with >>>> ?[? (the array signature character), it returns an array class. The >>>> string is encoded in modified UTF-8". >>>> >>>> I nevertheless think that we should not crash in the case of an >>>> invalid UTF8 string and therefore disabled the verification code with >>>> a comment. We did the same for java_lang_String::create_from_str() [3]. >>>> >>>> Tested with failing JCK test and JPRT (running). >>>> >>>> Thanks, >>>> Tobias >>>> >>>> [1] https://bugs.openjdk.java.net/browse/JCK-7307244 >>>> [2] >>>> https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/functions.html#FindClass >>>> >>>> [3] >>>> http://hg.openjdk.java.net/jdk9/hs/hotspot/file/d060826d0911/src/share/vm/classfile/javaClasses.cpp#l274 >>>> >>> From shafi.s.ahmad at oracle.com Mon Sep 19 08:47:46 2016 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Mon, 19 Sep 2016 01:47:46 -0700 (PDT) Subject: [8u] RFR for JDK-8157548: JVM crashes sometimes while starting In-Reply-To: <3fa82060-9a3a-f43b-efbf-d196ac473c41@oracle.com> References: <3fa82060-9a3a-f43b-efbf-d196ac473c41@oracle.com> Message-ID: Thanks David for the review. Regards, Shafi > -----Original Message----- > From: David Holmes > Sent: Monday, September 19, 2016 12:33 PM > To: Shafi Ahmad; hotspot-dev at openjdk.java.net > Subject: Re: [8u] RFR for JDK-8157548: JVM crashes sometimes while starting > > Hi Shafi, > > This looks okay to me. > > Thanks, > David > > On 19/09/2016 3:14 PM, Shafi Ahmad wrote: > > Hi, > > > > Please review the small code change for bug: "JDK-8157548: JVM crashes > sometimes while starting" on jdk8u-dev > > > > Summary: > > int strncmp(const char *s1, const char *s2, size_t n); > > > > s1 = "abcdefgh" // Assume this is not null terminated string. > > s2 = "abcdefghijk" > > n = 10 > > > > In case if s1 is not null terminated then for above input strncmp may crash. > > > > In expression marked as (B) parsed_name->bytes() returns base address > of non-null terminated string buffer. > > > > + size_t pkglen = strlen(pkg); > > if (!HAS_PENDING_EXCEPTION && > > !class_loader.is_null() && > > parsed_name != NULL && > > - !strncmp((const char*)parsed_name->bytes(), pkg, strlen(pkg))) { > > + parsed_name->utf8_length() >= (int)pkglen && // ------- > ----------------------- (A) > > + !strncmp((const char*)parsed_name->bytes(), pkg, pkglen)) { //-------- > ---------------------- (B) > > > > Adding expression marked as (A) avoid the above similar input scenario. > > > > Webrev: http://cr.openjdk.java.net/~shshahma/8157548/webrev.00/ > > Jdk8 bug: https://bugs.openjdk.java.net/browse/JDK-8157548 > > > > Test: Run jprt > > > > Note: Thanks to Ioi for providing the code change. > > > > Regards, > > Shafi > > From igor.ignatyev at oracle.com Mon Sep 19 09:17:20 2016 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Mon, 19 Sep 2016 12:17:20 +0300 Subject: RFR(S) : 8166262 : failurehandler should not use jtreg internal API Message-ID: <52CCFEE2-1F82-4A2C-909F-43EA26FD163B@oracle.com> http://cr.openjdk.java.net/~iignatyev/8166262/webrev.00/ > 62 lines changed: 56 ins; 2 del; 4 mod; Hi all, could you please review this small patch which removes usage of jtreg internal API from failurehandler lib? JBS: https://bugs.openjdk.java.net/browse/JDK-8166262 webrev: http://cr.openjdk.java.net/~iignatyev/8166262/webrev.00/ Thanks, ? Igor From tobias.hartmann at oracle.com Mon Sep 19 09:21:14 2016 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Mon, 19 Sep 2016 11:21:14 +0200 Subject: [9] RFR(S): Crash with assert: symbol conversion failure in java_lang_String::create_from_symbol() In-Reply-To: References: <57D2AE2F.1080607@oracle.com> <1203086c-f59f-40ce-f963-e958e570caf8@oracle.com> <57DF9D3A.3000802@oracle.com> Message-ID: <57DFAE0A.2090104@oracle.com> Hi David, On 19.09.2016 10:30, David Holmes wrote: > On 19/09/2016 6:09 PM, Tobias Hartmann wrote: >> Coleen, David, thanks for the reviews and sorry for the delay! >> >> I agree that we should validate the Strings we get through JNI, but the code in create_from_str() and create_from_symbol() was originally added to check the result of String compression with the Compact Strings feature and not to check for valid UTF-8. >> >> I think UTF-8 validity should be checked earlier (before or during Symbol creation) and this affects other JNI methods that take C-Strings as well (jni_NewString, jni_NewStringUTF, jni_DefineClass, ...). This is a general problem with JNI not validating input Strings. >> >> Are you fine with pushing the proposed fix (webrev.00) and open a follow up bug to fix JNI (and potentially re-enable the Compact Strings asserts because their existence is still justified)? > > As I wrote in the bug report: > > "I don't think I agree with the removed checks in either case. JNI doesn't do argument checking and so doesn't detect the invalid UTF-8 string. But the VM expects to work with valid strings. It seems quite reasonable to me that the VM should validate the UTF-8 and report some kind of failure to make these invalid inputs visible. Otherwise the bad JNI codes goes undetected. Maybe an assertion is too strong, perhaps we should throw InternalError? " Yes, I've seen that and, as I said, I agree that we should validate the Strings. > If the VM does not detect this and respond to it exactly what happens with the bad UTF-8 string? In the FindClass case, we just return NULL. I don't know if there are potential problems with invalid UTF-8 strings at other places in the code. > JNI doesn't do input validation - it is a "feature" of JNI so not to penalize correct code. Right, but this check only affects debug builds. > If the test that triggered this is sending in bad UTF-8 because it expects JNI to do validation then it is an invalid test. I think the asserts serve their purpose exactly - to show where invalid inputs came from. If this was bad user code then it would be up to them to fix it. If it is a bad test then the test should be fixed. Sure, the test is broken and will be fixed. So you're suggesting to leave the check in create_from_symbol() as it is and re-enable the check in create_from_str()? I'm fine with that but as I remember, re-enabling the check in create_from_str() causes other test failures. Best regards, Tobias > Sorry. > > David > >> Thanks, >> Tobias >> >> On 12.09.2016 02:47, David Holmes wrote: >>> On 10/09/2016 6:55 AM, Coleen Phillimore wrote: >>>> This change is fine because it matches the commented out assert in >>>> create_from_str(). We should probably figure out what it would take to >>>> check the characters coming in from JNI and decide whether we should do >>>> this. If not, it doesn't make sense to have commented out asserts. >>>> But this is okay for jdk9. >>> >>> Grumble, grumble ... both are bad. If the VM doesn't validate this bad UTF-8 then where does it go? And how does the generator of the bad UTF-8 get informed? An assert may be too drastic but can we throw an exception (InternalError?) ? >>> >>> David >>> >>>> Thanks, >>>> Coleen >>>> >>>> >>>> On 9/9/16 8:42 AM, Tobias Hartmann wrote: >>>>> Hi, >>>>> >>>>> please review the following patch: >>>>> https://bugs.openjdk.java.net/browse/JDK-8164561 >>>>> http://cr.openjdk.java.net/~thartmann/8164561/webrev.00/ >>>>> >>>>> The verification code in java_lang_String::create_from_symbol() that >>>>> was added by Compact Strings fails because the input symbol does not >>>>> contain valid UTF8. The problem is that a JCK JNI test passes an >>>>> invalid UTF8 string as class name to the JNI method "FindClass". In >>>>> fact, the string contains garbage from reading past array boundaries >>>>> because of a bug in the test [1]. The JNI spec [2] states that 'name' >>>>> should be "a fully-qualified class name (that is, a package name, >>>>> delimited by ?/?, followed by the class name). If the name begins with >>>>> ?[? (the array signature character), it returns an array class. The >>>>> string is encoded in modified UTF-8". >>>>> >>>>> I nevertheless think that we should not crash in the case of an >>>>> invalid UTF8 string and therefore disabled the verification code with >>>>> a comment. We did the same for java_lang_String::create_from_str() [3]. >>>>> >>>>> Tested with failing JCK test and JPRT (running). >>>>> >>>>> Thanks, >>>>> Tobias >>>>> >>>>> [1] https://bugs.openjdk.java.net/browse/JCK-7307244 >>>>> [2] >>>>> https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/functions.html#FindClass >>>>> >>>>> [3] >>>>> http://hg.openjdk.java.net/jdk9/hs/hotspot/file/d060826d0911/src/share/vm/classfile/javaClasses.cpp#l274 >>>>> >>>> From staffan.larsen at oracle.com Mon Sep 19 09:25:50 2016 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Mon, 19 Sep 2016 11:25:50 +0200 Subject: RFR(S) : 8166262 : failurehandler should not use jtreg internal API In-Reply-To: <52CCFEE2-1F82-4A2C-909F-43EA26FD163B@oracle.com> References: <52CCFEE2-1F82-4A2C-909F-43EA26FD163B@oracle.com> Message-ID: <00F2839C-A3DD-44AB-B7EA-C6C7D482FE69@oracle.com> Looks good! Thanks, /Staffan > On 19 sep. 2016, at 11:17, Igor Ignatyev wrote: > > http://cr.openjdk.java.net/~iignatyev/8166262/webrev.00/ >> 62 lines changed: 56 ins; 2 del; 4 mod; > > Hi all, > > could you please review this small patch which removes usage of jtreg internal API from failurehandler lib? > > JBS: https://bugs.openjdk.java.net/browse/JDK-8166262 > webrev: http://cr.openjdk.java.net/~iignatyev/8166262/webrev.00/ > > Thanks, > ? Igor From dmitry.samersoff at oracle.com Mon Sep 19 10:40:27 2016 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Mon, 19 Sep 2016 13:40:27 +0300 Subject: RFR(S) : 8166262 : failurehandler should not use jtreg internal API In-Reply-To: <52CCFEE2-1F82-4A2C-909F-43EA26FD163B@oracle.com> References: <52CCFEE2-1F82-4A2C-909F-43EA26FD163B@oracle.com> Message-ID: <6bae7fe5-9ba6-0428-d0e0-8589c756d449@oracle.com> Igor, Looks good for me. -Dmitry On 2016-09-19 12:17, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev/8166262/webrev.00/ >> 62 lines changed: 56 ins; 2 del; 4 mod; > > Hi all, > > could you please review this small patch which removes usage of jtreg internal API from failurehandler lib? > > JBS: https://bugs.openjdk.java.net/browse/JDK-8166262 > webrev: http://cr.openjdk.java.net/~iignatyev/8166262/webrev.00/ > > Thanks, > ? Igor > -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From igor.ignatyev at oracle.com Mon Sep 19 10:42:56 2016 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Mon, 19 Sep 2016 13:42:56 +0300 Subject: RFR(S) : 8166262 : failurehandler should not use jtreg internal API In-Reply-To: <6bae7fe5-9ba6-0428-d0e0-8589c756d449@oracle.com> References: <52CCFEE2-1F82-4A2C-909F-43EA26FD163B@oracle.com> <6bae7fe5-9ba6-0428-d0e0-8589c756d449@oracle.com> Message-ID: Dmitry, Thank you for review. ? Igor > On Sep 19, 2016, at 1:40 PM, Dmitry Samersoff wrote: > > Igor, > > Looks good for me. > > -Dmitry > > On 2016-09-19 12:17, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev/8166262/webrev.00/ >>> 62 lines changed: 56 ins; 2 del; 4 mod; >> >> Hi all, >> >> could you please review this small patch which removes usage of jtreg internal API from failurehandler lib? >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8166262 >> webrev: http://cr.openjdk.java.net/~iignatyev/8166262/webrev.00/ >> >> Thanks, >> ? Igor >> > > > -- > Dmitry Samersoff > Oracle Java development team, Saint Petersburg, Russia > * I would love to change the world, but they won't give me the sources. From igor.ignatyev at oracle.com Mon Sep 19 10:43:12 2016 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Mon, 19 Sep 2016 13:43:12 +0300 Subject: RFR(S) : 8166262 : failurehandler should not use jtreg internal API In-Reply-To: <00F2839C-A3DD-44AB-B7EA-C6C7D482FE69@oracle.com> References: <52CCFEE2-1F82-4A2C-909F-43EA26FD163B@oracle.com> <00F2839C-A3DD-44AB-B7EA-C6C7D482FE69@oracle.com> Message-ID: Staffan, Thank you for review! ? Igor > On Sep 19, 2016, at 12:25 PM, Staffan Larsen wrote: > > Looks good! > > Thanks, > /Staffan > >> On 19 sep. 2016, at 11:17, Igor Ignatyev wrote: >> >> http://cr.openjdk.java.net/~iignatyev/8166262/webrev.00/ >>> 62 lines changed: 56 ins; 2 del; 4 mod; >> >> Hi all, >> >> could you please review this small patch which removes usage of jtreg internal API from failurehandler lib? >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8166262 >> webrev: http://cr.openjdk.java.net/~iignatyev/8166262/webrev.00/ >> >> Thanks, >> ? Igor > From dmitry.dmitriev at oracle.com Mon Sep 19 12:47:16 2016 From: dmitry.dmitriev at oracle.com (Dmitry Dmitriev) Date: Mon, 19 Sep 2016 15:47:16 +0300 Subject: RFR: 8150758: [TESTBUG] need jvmti tests for module aware agents Message-ID: <93940e3d-48ae-9f04-045f-04618dc80d16@oracle.com> Hello, Please review new tests for module aware agents. There are 3 tests: 1) MAAClassFileLoadHook.java - verifies ClassFileLoadHook event with different combinations of can_generate_early_vmstart and can_generate_early_class_hook_events JVMTI capabilities. Expects to find(or not) class from java.base module in the right VM phase. 2) MAAClassLoadPrepare.java - verifies ClassLoad and ClassPrepare events with and without can_generate_early_vmstart JVMTI capability. Expects to find(or not) class from java.base module in the VM start phase. 3) MAAThreadStart.java - verifies ThreadStart event with can_generate_early_vmstart JVMTI capability. Expect to find events in the VM start phase. JBS: https://bugs.openjdk.java.net/browse/JDK-8150758 webrev.00: http://cr.openjdk.java.net/~ddmitriev/8150758/webrev.00/ Testing: RBT on all platforms Thanks, Dmitry From martin.walsh at oracle.com Mon Sep 19 14:16:11 2016 From: martin.walsh at oracle.com (Martin Walsh) Date: Mon, 19 Sep 2016 15:16:11 +0100 Subject: RFR for JDK-8165482 java in ldoms, with cpu-arch=generic has problems Message-ID: <8046b263-8826-aa34-083a-9a83e105770f@oracle.com> Could I get a code review for the following bug: JDK-8165482 java in ldoms, with cpu-arch=generic has problems Webrev of the changes is available here: http://cr.openjdk.java.net/~mwalsh/JDK-8165482/ Thanks, Martin From ioi.lam at oracle.com Mon Sep 19 18:38:36 2016 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 19 Sep 2016 11:38:36 -0700 Subject: [8u] RFR for JDK-8157548: JVM crashes sometimes while starting In-Reply-To: References: Message-ID: <57E030AC.8070806@oracle.com> Hi Shafi, The fix is correct. However, the crash is not inside strncmp. I reproduced this bug inside gdb. When the crash happened, the parsed_name Symbol is "j", which has one character. By luck, it occupies the same space as a freed copy of the Symbol "java/lang/String". For space saving, the string content in the Symbol is not nul terminated. Thus: strncmp((const char*)parsed_name->bytes(), pkg, strlen(pkg)) would return true. However, when we later call char* name = parsed_name->as_C_string(); char* index = strrchr(name, '/'); *index = '\0'; // <------- crash name will be the 0-terminated string "j", and index would be NULL (because '/' is not contained inside name.) Storing into index would cause the SEGV. Thanks - Ioi On 9/18/16 10:14 PM, Shafi Ahmad wrote: > Hi, > > Please review the small code change for bug: "JDK-8157548: JVM crashes sometimes while starting" on jdk8u-dev > > Summary: > int strncmp(const char *s1, const char *s2, size_t n); > > s1 = "abcdefgh" // Assume this is not null terminated string. > s2 = "abcdefghijk" > n = 10 > > In case if s1 is not null terminated then for above input strncmp may crash. > > In expression marked as (B) parsed_name->bytes() returns base address of non-null terminated string buffer. > > + size_t pkglen = strlen(pkg); > if (!HAS_PENDING_EXCEPTION && > !class_loader.is_null() && > parsed_name != NULL && > - !strncmp((const char*)parsed_name->bytes(), pkg, strlen(pkg))) { > + parsed_name->utf8_length() >= (int)pkglen && // ------------------------------ (A) > + !strncmp((const char*)parsed_name->bytes(), pkg, pkglen)) { //------------------------------ (B) > > Adding expression marked as (A) avoid the above similar input scenario. > > Webrev: http://cr.openjdk.java.net/~shshahma/8157548/webrev.00/ > Jdk8 bug: https://bugs.openjdk.java.net/browse/JDK-8157548 > > Test: Run jprt > > Note: Thanks to Ioi for providing the code change. > > Regards, > Shafi From ioi.lam at oracle.com Mon Sep 19 18:47:43 2016 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 19 Sep 2016 11:47:43 -0700 Subject: [9] RFR(S): Crash with assert: symbol conversion failure in java_lang_String::create_from_symbol() In-Reply-To: <57DFAE0A.2090104@oracle.com> References: <57D2AE2F.1080607@oracle.com> <1203086c-f59f-40ce-f963-e958e570caf8@oracle.com> <57DF9D3A.3000802@oracle.com> <57DFAE0A.2090104@oracle.com> Message-ID: <57E032CF.4050602@oracle.com> On 9/19/16 2:21 AM, Tobias Hartmann wrote: > Hi David, > > On 19.09.2016 10:30, David Holmes wrote: >> On 19/09/2016 6:09 PM, Tobias Hartmann wrote: >>> Coleen, David, thanks for the reviews and sorry for the delay! >>> >>> I agree that we should validate the Strings we get through JNI, but the code in create_from_str() and create_from_symbol() was originally added to check the result of String compression with the Compact Strings feature and not to check for valid UTF-8. >>> >>> I think UTF-8 validity should be checked earlier (before or during Symbol creation) and this affects other JNI methods that take C-Strings as well (jni_NewString, jni_NewStringUTF, jni_DefineClass, ...). This is a general problem with JNI not validating input Strings. >>> >>> Are you fine with pushing the proposed fix (webrev.00) and open a follow up bug to fix JNI (and potentially re-enable the Compact Strings asserts because their existence is still justified)? >> As I wrote in the bug report: >> >> "I don't think I agree with the removed checks in either case. JNI doesn't do argument checking and so doesn't detect the invalid UTF-8 string. But the VM expects to work with valid strings. It seems quite reasonable to me that the VM should validate the UTF-8 and report some kind of failure to make these invalid inputs visible. Otherwise the bad JNI codes goes undetected. Maybe an assertion is too strong, perhaps we should throw InternalError?" > Yes, I've seen that and, as I said, I agree that we should validate the Strings. > >> If the VM does not detect this and respond to it exactly what happens with the bad UTF-8 string? > In the FindClass case, we just return NULL. I don't know if there are potential problems with invalid UTF-8 strings at other places in the code. > >> JNI doesn't do input validation - it is a "feature" of JNI so not to penalize correct code. > Right, but this check only affects debug builds. > >> If the test that triggered this is sending in bad UTF-8 because it expects JNI to do validation then it is an invalid test. I think the asserts serve their purpose exactly - to show where invalid inputs came from. If this was bad user code then it would be up to them to fix it. If it is a bad test then the test should be fixed. > Sure, the test is broken and will be fixed. So you're suggesting to leave the check in create_from_symbol() as it is and re-enable the check in create_from_str()? I also think this is the right approach. Fix (or disable) the test, but leave the C code the way it is. java_lang_String::create_from_str(jchar*, ...) could (potentially??) be called by non JNI code, in which cases we want the assert. Thanks - Ioi > I'm fine with that but as I remember, re-enabling the check in create_from_str() causes other test failures. > > Best regards, > Tobias > >> Sorry. >> >> David >> >>> Thanks, >>> Tobias >>> >>> On 12.09.2016 02:47, David Holmes wrote: >>>> On 10/09/2016 6:55 AM, Coleen Phillimore wrote: >>>>> This change is fine because it matches the commented out assert in >>>>> create_from_str(). We should probably figure out what it would take to >>>>> check the characters coming in from JNI and decide whether we should do >>>>> this. If not, it doesn't make sense to have commented out asserts. >>>>> But this is okay for jdk9. >>>> Grumble, grumble ... both are bad. If the VM doesn't validate this bad UTF-8 then where does it go? And how does the generator of the bad UTF-8 get informed? An assert may be too drastic but can we throw an exception (InternalError?) ? >>>> >>>> David >>>> >>>>> Thanks, >>>>> Coleen >>>>> >>>>> >>>>> On 9/9/16 8:42 AM, Tobias Hartmann wrote: >>>>>> Hi, >>>>>> >>>>>> please review the following patch: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8164561 >>>>>> http://cr.openjdk.java.net/~thartmann/8164561/webrev.00/ >>>>>> >>>>>> The verification code in java_lang_String::create_from_symbol() that >>>>>> was added by Compact Strings fails because the input symbol does not >>>>>> contain valid UTF8. The problem is that a JCK JNI test passes an >>>>>> invalid UTF8 string as class name to the JNI method "FindClass". In >>>>>> fact, the string contains garbage from reading past array boundaries >>>>>> because of a bug in the test [1]. The JNI spec [2] states that 'name' >>>>>> should be "a fully-qualified class name (that is, a package name, >>>>>> delimited by ?/?, followed by the class name). If the name begins with >>>>>> ?[? (the array signature character), it returns an array class. The >>>>>> string is encoded in modified UTF-8". >>>>>> >>>>>> I nevertheless think that we should not crash in the case of an >>>>>> invalid UTF8 string and therefore disabled the verification code with >>>>>> a comment. We did the same for java_lang_String::create_from_str() [3]. >>>>>> >>>>>> Tested with failing JCK test and JPRT (running). >>>>>> >>>>>> Thanks, >>>>>> Tobias >>>>>> >>>>>> [1] https://bugs.openjdk.java.net/browse/JCK-7307244 >>>>>> [2] >>>>>> https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/functions.html#FindClass >>>>>> >>>>>> [3] >>>>>> http://hg.openjdk.java.net/jdk9/hs/hotspot/file/d060826d0911/src/share/vm/classfile/javaClasses.cpp#l274 >>>>>> From brent.christian at oracle.com Mon Sep 19 19:35:17 2016 From: brent.christian at oracle.com (Brent Christian) Date: Mon, 19 Sep 2016 12:35:17 -0700 Subject: RFR 8165372 : StackWalker performance regression following JDK-8147039 Message-ID: <57E03DF5.1050003@oracle.com> Hi, Please review my fix for 8165372 : "StackWalker performance regression following JDK-8147039" Bug: https://bugs.openjdk.java.net/browse/JDK-8165372 Webrev: http://cr.openjdk.java.net/~bchristi/8165372/webrev.00/ 8147039 reimplemented stack walking using javaVFrames in place of vframeStream, in order to give correct results for the experimental LiveStackFrame feature. However, this also resulted in a significant StackWalker performance regression (25-60%, depending on specific operation and stack depth). This fix includes stack walking implemented both ways, encapsulated within C++ classes. JavaFrameStream provides the speed we had before for most use cases, while LiveFrameStream provides correct results when using LiveStackFrames. Performance is much improved, back within 5% or so of pre-8147039 levels, based on my measurements. Thanks, -Brent 1. http://hg.openjdk.java.net/jdk9/hs/hotspot/rev/6174ad93770c From coleen.phillimore at oracle.com Mon Sep 19 19:36:04 2016 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Mon, 19 Sep 2016 15:36:04 -0400 Subject: [9] RFR(S): Crash with assert: symbol conversion failure in java_lang_String::create_from_symbol() In-Reply-To: <57E032CF.4050602@oracle.com> References: <57D2AE2F.1080607@oracle.com> <1203086c-f59f-40ce-f963-e958e570caf8@oracle.com> <57DF9D3A.3000802@oracle.com> <57DFAE0A.2090104@oracle.com> <57E032CF.4050602@oracle.com> Message-ID: <2c907145-258c-2362-b525-438dbca34ec6@oracle.com> On 9/19/16 2:47 PM, Ioi Lam wrote: > > > On 9/19/16 2:21 AM, Tobias Hartmann wrote: >> Hi David, >> >> On 19.09.2016 10:30, David Holmes wrote: >>> On 19/09/2016 6:09 PM, Tobias Hartmann wrote: >>>> Coleen, David, thanks for the reviews and sorry for the delay! >>>> >>>> I agree that we should validate the Strings we get through JNI, but >>>> the code in create_from_str() and create_from_symbol() was >>>> originally added to check the result of String compression with the >>>> Compact Strings feature and not to check for valid UTF-8. >>>> >>>> I think UTF-8 validity should be checked earlier (before or during >>>> Symbol creation) and this affects other JNI methods that take >>>> C-Strings as well (jni_NewString, jni_NewStringUTF, >>>> jni_DefineClass, ...). This is a general problem with JNI not >>>> validating input Strings. >>>> >>>> Are you fine with pushing the proposed fix (webrev.00) and open a >>>> follow up bug to fix JNI (and potentially re-enable the Compact >>>> Strings asserts because their existence is still justified)? >>> As I wrote in the bug report: >>> >>> "I don't think I agree with the removed checks in either case. JNI >>> doesn't do argument checking and so doesn't detect the invalid UTF-8 >>> string. But the VM expects to work with valid strings. It seems >>> quite reasonable to me that the VM should validate the UTF-8 and >>> report some kind of failure to make these invalid inputs visible. >>> Otherwise the bad JNI codes goes undetected. Maybe an assertion is >>> too strong, perhaps we should throw InternalError?" >> Yes, I've seen that and, as I said, I agree that we should validate >> the Strings. >> >>> If the VM does not detect this and respond to it exactly what >>> happens with the bad UTF-8 string? >> In the FindClass case, we just return NULL. I don't know if there are >> potential problems with invalid UTF-8 strings at other places in the >> code. >> >>> JNI doesn't do input validation - it is a "feature" of JNI so not to >>> penalize correct code. >> Right, but this check only affects debug builds. >> >>> If the test that triggered this is sending in bad UTF-8 because it >>> expects JNI to do validation then it is an invalid test. I think the >>> asserts serve their purpose exactly - to show where invalid inputs >>> came from. If this was bad user code then it would be up to them to >>> fix it. If it is a bad test then the test should be fixed. >> Sure, the test is broken and will be fixed. So you're suggesting to >> leave the check in create_from_symbol() as it is and re-enable the >> check in create_from_str()? > I also think this is the right approach. Fix (or disable) the test, > but leave the C code the way it is. > > java_lang_String::create_from_str(jchar*, ...) could (potentially??) > be called by non JNI code, in which cases we want the assert. No, I think you need to leave the commented out code in create_from_str(). This could break testing and it's too late in jdk9 release for non-jigsaw to destabilize the testing. I also think the similar change needs to be added to create_from_symbol(), as in the proposed fix. And file a bug to fix the test and another to evaluate whether we should validate strings in JNI and remove the commented out code. I think the proposed fix is what we should do for now. This assert is a regression, and our non-checking of JNI strings and symbols is not. Thanks, Coleen > > Thanks > - Ioi > >> I'm fine with that but as I remember, re-enabling the check in >> create_from_str() causes other test failures. >> >> Best regards, >> Tobias >> >>> Sorry. >>> >>> David >>> >>>> Thanks, >>>> Tobias >>>> >>>> On 12.09.2016 02:47, David Holmes wrote: >>>>> On 10/09/2016 6:55 AM, Coleen Phillimore wrote: >>>>>> This change is fine because it matches the commented out assert in >>>>>> create_from_str(). We should probably figure out what it would >>>>>> take to >>>>>> check the characters coming in from JNI and decide whether we >>>>>> should do >>>>>> this. If not, it doesn't make sense to have commented out asserts. >>>>>> But this is okay for jdk9. >>>>> Grumble, grumble ... both are bad. If the VM doesn't validate this >>>>> bad UTF-8 then where does it go? And how does the generator of the >>>>> bad UTF-8 get informed? An assert may be too drastic but can we >>>>> throw an exception (InternalError?) ? >>>>> >>>>> David >>>>> >>>>>> Thanks, >>>>>> Coleen >>>>>> >>>>>> >>>>>> On 9/9/16 8:42 AM, Tobias Hartmann wrote: >>>>>>> Hi, >>>>>>> >>>>>>> please review the following patch: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8164561 >>>>>>> http://cr.openjdk.java.net/~thartmann/8164561/webrev.00/ >>>>>>> >>>>>>> The verification code in java_lang_String::create_from_symbol() >>>>>>> that >>>>>>> was added by Compact Strings fails because the input symbol does >>>>>>> not >>>>>>> contain valid UTF8. The problem is that a JCK JNI test passes an >>>>>>> invalid UTF8 string as class name to the JNI method "FindClass". In >>>>>>> fact, the string contains garbage from reading past array >>>>>>> boundaries >>>>>>> because of a bug in the test [1]. The JNI spec [2] states that >>>>>>> 'name' >>>>>>> should be "a fully-qualified class name (that is, a package name, >>>>>>> delimited by ?/?, followed by the class name). If the name >>>>>>> begins with >>>>>>> ?[? (the array signature character), it returns an array class. The >>>>>>> string is encoded in modified UTF-8". >>>>>>> >>>>>>> I nevertheless think that we should not crash in the case of an >>>>>>> invalid UTF8 string and therefore disabled the verification code >>>>>>> with >>>>>>> a comment. We did the same for >>>>>>> java_lang_String::create_from_str() [3]. >>>>>>> >>>>>>> Tested with failing JCK test and JPRT (running). >>>>>>> >>>>>>> Thanks, >>>>>>> Tobias >>>>>>> >>>>>>> [1] https://bugs.openjdk.java.net/browse/JCK-7307244 >>>>>>> [2] >>>>>>> https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/functions.html#FindClass >>>>>>> >>>>>>> >>>>>>> [3] >>>>>>> http://hg.openjdk.java.net/jdk9/hs/hotspot/file/d060826d0911/src/share/vm/classfile/javaClasses.cpp#l274 >>>>>>> >>>>>>> > From coleen.phillimore at oracle.com Mon Sep 19 19:46:01 2016 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Mon, 19 Sep 2016 15:46:01 -0400 Subject: RFR 8165372 : StackWalker performance regression following JDK-8147039 In-Reply-To: <57E03DF5.1050003@oracle.com> References: <57E03DF5.1050003@oracle.com> Message-ID: <3792574d-0461-bc6e-5b0a-646f223e60ed@oracle.com> http://cr.openjdk.java.net/~bchristi/8165372/webrev.00/src/share/vm/prims/stackwalk.cpp.udiff.html + if (need_method_info(mode) == false && get_caller_class(mode) && Change to + if (!need_method_info(mode) && get_caller_class(mode) && And the two instances of this: + Handle stackFrame(frames_array->obj_at(index)); Change to + Handle stackFrame(THREAD, frames_array->obj_at(index)); Other than these minor comments, I think this looks really good. thanks! Coleen On 9/19/16 3:35 PM, Brent Christian wrote: > Hi, > > Please review my fix for 8165372 : "StackWalker performance regression > following JDK-8147039" > > Bug: https://bugs.openjdk.java.net/browse/JDK-8165372 > Webrev: http://cr.openjdk.java.net/~bchristi/8165372/webrev.00/ > > 8147039 reimplemented stack walking using javaVFrames in place of > vframeStream, in order to give correct results for the experimental > LiveStackFrame feature. However, this also resulted in a significant > StackWalker performance regression (25-60%, depending on specific > operation and stack depth). > > This fix includes stack walking implemented both ways, encapsulated > within C++ classes. JavaFrameStream provides the speed we had before > for most use cases, while LiveFrameStream provides correct results > when using LiveStackFrames. > > Performance is much improved, back within 5% or so of pre-8147039 > levels, based on my measurements. > > Thanks, > -Brent > > 1. http://hg.openjdk.java.net/jdk9/hs/hotspot/rev/6174ad93770c > From mandy.chung at oracle.com Mon Sep 19 21:06:00 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 19 Sep 2016 14:06:00 -0700 Subject: RFR 8165372 : StackWalker performance regression following JDK-8147039 In-Reply-To: <57E03DF5.1050003@oracle.com> References: <57E03DF5.1050003@oracle.com> Message-ID: > On Sep 19, 2016, at 12:35 PM, Brent Christian wrote: > > Hi, > > Please review my fix for 8165372 : "StackWalker performance regression following JDK-8147039" > > Bug: https://bugs.openjdk.java.net/browse/JDK-8165372 > Webrev: http://cr.openjdk.java.net/~bchristi/8165372/webrev.00/ > This looks good and clean. I have the same comment what Coleen pointed out: + if (need_method_info(mode) == false && get_caller_class(mode) && and Handle stackFrame. Mandy From brent.christian at oracle.com Mon Sep 19 22:57:05 2016 From: brent.christian at oracle.com (Brent Christian) Date: Mon, 19 Sep 2016 15:57:05 -0700 Subject: RFR 8165372 : StackWalker performance regression following JDK-8147039 In-Reply-To: References: <57E03DF5.1050003@oracle.com> Message-ID: <57E06D41.8020507@oracle.com> Thanks for the good suggestions. Webrev updated in place: http://cr.openjdk.java.net/~bchristi/8165372/webrev.00/ -Brent From coleen.phillimore at oracle.com Mon Sep 19 23:43:15 2016 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Mon, 19 Sep 2016 19:43:15 -0400 Subject: RFR 8165372 : StackWalker performance regression following JDK-8147039 In-Reply-To: <57E06D41.8020507@oracle.com> References: <57E03DF5.1050003@oracle.com> <57E06D41.8020507@oracle.com> Message-ID: <233d5815-23e0-c9f5-8852-bda27916cadd@oracle.com> Looks good! Coleen On 9/19/16 6:57 PM, Brent Christian wrote: > Thanks for the good suggestions. Webrev updated in place: > http://cr.openjdk.java.net/~bchristi/8165372/webrev.00/ > > -Brent > From david.holmes at oracle.com Tue Sep 20 00:02:44 2016 From: david.holmes at oracle.com (David Holmes) Date: Tue, 20 Sep 2016 10:02:44 +1000 Subject: [9] RFR(S): Crash with assert: symbol conversion failure in java_lang_String::create_from_symbol() In-Reply-To: <57DFAE0A.2090104@oracle.com> References: <57D2AE2F.1080607@oracle.com> <1203086c-f59f-40ce-f963-e958e570caf8@oracle.com> <57DF9D3A.3000802@oracle.com> <57DFAE0A.2090104@oracle.com> Message-ID: <85ed982f-2e13-df25-e979-655009bf9bfe@oracle.com> Let me get this straight. The compact strings push occurred on Nov 3, 2015, some ten months ago! One of the assertions was commented out in that push because otherwise some tests failed. Someone made the decision that the tests were right and the assert was wrong - in which case it should just have been deleted from the code. Now 10 months later the other assert is suddenly failing and the claim is that it is the assert that is wrong! Why is it failing now - is this is a new test? If so we have to establish whether it is the test code or the VM code that is making the wrong assumptions. If the test is deemed valid then it is now passing invalid UTF-8 through a path never before tested with invalid UTF-8, so we need to know that the code will in fact handle that invalid input safely. Only then should the assert be removed, and that is only if it is determined that the test is valid. In this particular case a bad string is being passed to jni_FindClass. JNI doesn't do validation of the string (bar minimal sanity checks) but should the VM be being more vigilant? It may be that an invalidly formed UTF-8 string can travel safely through the VM, and in this case (as Tobias indicated) just result in the class not being found. Or it may be there are places that make assumptions about the validity of such a string. David ----- On 19/09/2016 7:21 PM, Tobias Hartmann wrote: > Hi David, > > On 19.09.2016 10:30, David Holmes wrote: >> On 19/09/2016 6:09 PM, Tobias Hartmann wrote: >>> Coleen, David, thanks for the reviews and sorry for the delay! >>> >>> I agree that we should validate the Strings we get through JNI, but the code in create_from_str() and create_from_symbol() was originally added to check the result of String compression with the Compact Strings feature and not to check for valid UTF-8. >>> >>> I think UTF-8 validity should be checked earlier (before or during Symbol creation) and this affects other JNI methods that take C-Strings as well (jni_NewString, jni_NewStringUTF, jni_DefineClass, ...). This is a general problem with JNI not validating input Strings. >>> >>> Are you fine with pushing the proposed fix (webrev.00) and open a follow up bug to fix JNI (and potentially re-enable the Compact Strings asserts because their existence is still justified)? >> >> As I wrote in the bug report: >> >> "I don't think I agree with the removed checks in either case. JNI doesn't do argument checking and so doesn't detect the invalid UTF-8 string. But the VM expects to work with valid strings. It seems quite reasonable to me that the VM should validate the UTF-8 and report some kind of failure to make these invalid inputs visible. Otherwise the bad JNI codes goes undetected. Maybe an assertion is too strong, perhaps we should throw InternalError? " > > Yes, I've seen that and, as I said, I agree that we should validate the Strings. > >> If the VM does not detect this and respond to it exactly what happens with the bad UTF-8 string? > > In the FindClass case, we just return NULL. I don't know if there are potential problems with invalid UTF-8 strings at other places in the code. > >> JNI doesn't do input validation - it is a "feature" of JNI so not to penalize correct code. > > Right, but this check only affects debug builds. > >> If the test that triggered this is sending in bad UTF-8 because it expects JNI to do validation then it is an invalid test. I think the asserts serve their purpose exactly - to show where invalid inputs came from. If this was bad user code then it would be up to them to fix it. If it is a bad test then the test should be fixed. > > Sure, the test is broken and will be fixed. So you're suggesting to leave the check in create_from_symbol() as it is and re-enable the check in create_from_str()? > > I'm fine with that but as I remember, re-enabling the check in create_from_str() causes other test failures. > > Best regards, > Tobias > >> Sorry. >> >> David >> >>> Thanks, >>> Tobias >>> >>> On 12.09.2016 02:47, David Holmes wrote: >>>> On 10/09/2016 6:55 AM, Coleen Phillimore wrote: >>>>> This change is fine because it matches the commented out assert in >>>>> create_from_str(). We should probably figure out what it would take to >>>>> check the characters coming in from JNI and decide whether we should do >>>>> this. If not, it doesn't make sense to have commented out asserts. >>>>> But this is okay for jdk9. >>>> >>>> Grumble, grumble ... both are bad. If the VM doesn't validate this bad UTF-8 then where does it go? And how does the generator of the bad UTF-8 get informed? An assert may be too drastic but can we throw an exception (InternalError?) ? >>>> >>>> David >>>> >>>>> Thanks, >>>>> Coleen >>>>> >>>>> >>>>> On 9/9/16 8:42 AM, Tobias Hartmann wrote: >>>>>> Hi, >>>>>> >>>>>> please review the following patch: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8164561 >>>>>> http://cr.openjdk.java.net/~thartmann/8164561/webrev.00/ >>>>>> >>>>>> The verification code in java_lang_String::create_from_symbol() that >>>>>> was added by Compact Strings fails because the input symbol does not >>>>>> contain valid UTF8. The problem is that a JCK JNI test passes an >>>>>> invalid UTF8 string as class name to the JNI method "FindClass". In >>>>>> fact, the string contains garbage from reading past array boundaries >>>>>> because of a bug in the test [1]. The JNI spec [2] states that 'name' >>>>>> should be "a fully-qualified class name (that is, a package name, >>>>>> delimited by ?/?, followed by the class name). If the name begins with >>>>>> ?[? (the array signature character), it returns an array class. The >>>>>> string is encoded in modified UTF-8". >>>>>> >>>>>> I nevertheless think that we should not crash in the case of an >>>>>> invalid UTF8 string and therefore disabled the verification code with >>>>>> a comment. We did the same for java_lang_String::create_from_str() [3]. >>>>>> >>>>>> Tested with failing JCK test and JPRT (running). >>>>>> >>>>>> Thanks, >>>>>> Tobias >>>>>> >>>>>> [1] https://bugs.openjdk.java.net/browse/JCK-7307244 >>>>>> [2] >>>>>> https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/functions.html#FindClass >>>>>> >>>>>> [3] >>>>>> http://hg.openjdk.java.net/jdk9/hs/hotspot/file/d060826d0911/src/share/vm/classfile/javaClasses.cpp#l274 >>>>>> >>>>> From mandy.chung at oracle.com Tue Sep 20 00:19:50 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 19 Sep 2016 17:19:50 -0700 Subject: RFR 8165372 : StackWalker performance regression following JDK-8147039 In-Reply-To: <57E06D41.8020507@oracle.com> References: <57E03DF5.1050003@oracle.com> <57E06D41.8020507@oracle.com> Message-ID: +1 Mandy > On Sep 19, 2016, at 3:57 PM, Brent Christian wrote: > > Thanks for the good suggestions. Webrev updated in place: > http://cr.openjdk.java.net/~bchristi/8165372/webrev.00/ > > -Brent > From david.holmes at oracle.com Tue Sep 20 01:28:37 2016 From: david.holmes at oracle.com (David Holmes) Date: Tue, 20 Sep 2016 11:28:37 +1000 Subject: RFR for JDK-8165482 java in ldoms, with cpu-arch=generic has problems In-Reply-To: <8046b263-8826-aa34-083a-9a83e105770f@oracle.com> References: <8046b263-8826-aa34-083a-9a83e105770f@oracle.com> Message-ID: <0a2a66db-f939-196f-bf23-a6a3df46b2d3@oracle.com> Hi Martin, Build changes must be reviewed by the build team - now cc'd On 20/09/2016 12:16 AM, Martin Walsh wrote: > Could I get a code review for the following bug: > > JDK-8165482 java in ldoms, with cpu-arch=generic has problems > > Webrev of the changes is available here: > > http://cr.openjdk.java.net/~mwalsh/JDK-8165482/ What is the devinfo library? Is it part of the normal Solaris installation, or does it need to be installed specifically? Is it available in our official build toolkits? Will checking the prom prior to using kstat change any of the values we currently see? (other than the generic case being fixed of course). Also note you will need to generate and push the closed generated-configure.sh file. Thanks, David > > Thanks, > > Martin From serguei.spitsyn at oracle.com Tue Sep 20 03:38:18 2016 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 19 Sep 2016 20:38:18 -0700 Subject: RFR: 8150758: [TESTBUG] need jvmti tests for module aware agents In-Reply-To: <93940e3d-48ae-9f04-045f-04618dc80d16@oracle.com> References: <93940e3d-48ae-9f04-045f-04618dc80d16@oracle.com> Message-ID: Hi Dmitry, Thanks a lot for this additional test coverage and discovering new bug: https://bugs.openjdk.java.net/browse/JDK-8165681 The tests look pretty good to me. A couple of minor comments. Dots are missed in the .c files comments. http://cr.openjdk.java.net/~ddmitriev/8150758/webrev.00/test/serviceability/jvmti 265 printf("Expecting to find '%s' class in ClassLoad events duringVM start phase.\n", EXPECTED_SIGNATURE); 266 if (class_in_class_load_events_vm_start == JNI_FALSE) { 267 throw_exc(env, "Unable to find expected class in ClassLoad events duringstart phase!\n"); 268 return FAILED; 269 } 270 271 if (class_prepare_events_vm_start_count == 0) { 272 throw_exc(env, "Didn't get ClassPrepare events in start phase!\n"); 273 return FAILED; 274 } 275 276 printf("Expecting to find '%s' class in ClassPrepare events duringVM start phase.\n", EXPECTED_SIGNATURE); 277 if (class_in_class_prepare_events_vm_start == JNI_FALSE) { 278 throw_exc(env, "Unable to find expected class in ClassPrepare events duringstart phase!\n"); 279 return FAILED; 280 } Could you, please, replace "start phase" with "early start phase"? It will be more clear that the start phase mode is "early". 288 if (class_in_class_prepare_events_vm_start == JNI_TRUE) { 289 throw_exc(env, "Class is found in ClassLoad events duringVM Start phase!\n"); 290 return FAILED; 291 } 292 293 printf("Expecting that '%s' class is absent in ClassPrepare events.\n", EXPECTED_SIGNATURE); 294 if (class_in_class_prepare_events_vm_start == JNI_TRUE) { 295 throw_exc(env, "Class is found in ClassPrepare events duringVM Start phase!\n"); 296 return FAILED; 297 } Could you, please, replace "VM Start phase" with "normal VM start phase"? It will be more clear that the start phase mode is "normal". Thanks, Serguei On 9/19/16 05:47, Dmitry Dmitriev wrote: > Hello, Please review new tests for module aware agents. There are 3 > tests: 1) MAAClassFileLoadHook.java - verifies ClassFileLoadHook event > with different combinations of can_generate_early_vmstart and > can_generate_early_class_hook_events JVMTI capabilities. Expects to > find(or not) class from java.base module in the right VM phase. 2) > MAAClassLoadPrepare.java - verifies ClassLoad and ClassPrepare events > with and without can_generate_early_vmstart JVMTI capability. Expects > to find(or not) class from java.base module in the VM start phase. 3) > MAAThreadStart.java - verifies ThreadStart event with > can_generate_early_vmstart JVMTI capability. Expect to find events in > the VM start phase. JBS: > https://bugs.openjdk.java.net/browse/JDK-8150758 webrev.00: > http://cr.openjdk.java.net/~ddmitriev/8150758/webrev.00/ > Testing: > RBT on all platforms Thanks, Dmitry From shafi.s.ahmad at oracle.com Tue Sep 20 04:36:52 2016 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Mon, 19 Sep 2016 21:36:52 -0700 (PDT) Subject: [8u] RFR for JDK-8157548: JVM crashes sometimes while starting In-Reply-To: <57E030AC.8070806@oracle.com> References: <57E030AC.8070806@oracle.com> Message-ID: <2f6aba53-0b1f-47cf-b3df-39ba5a7eaa8b@default> Hi Ioi, Thanks for the review. Yes, you are right. I mentioned my earlier mail 'it may crash' I think I should mentioned 'it may behave unexpectedly'. It totally dependent on the content of the buffer. s1 = ['j','\0','v','a','/','l','a',n',g',/ ',S',t',r',i',n',g']. // Crash scenario , other possiblilties are ['j','a','\0','a','/','l','a',n',g',/ ',S',t',r',i',n',g'], ['j','a','v,'\0','/','l','a',n',g',/ ',S',t',r',i',n',g'] and ['j','a','v','a','\0','l','a',n',g',/ ',S',t',r',i',n',g']. s2 = "java/" n = 5 As in the older code we are not checking length of buffer s1 so it returns true because s[1] = '\0' or s[2] = '\0' or s[3] = '\0' or s[4] = '\0' and it crashes later. Assume below scenario: s1 = ['j','a''v','a','/','l','a',n',g',/ ',S',t',r',i',n',g']. // Assume s1 is pointing to same space as a freed copy of the Symbol "java/lang/String" but with size 4 and let 's1 + 4' is not accessible. Note there is no null character. s2 = "java/" n = 5 In this scenario strncmp may crash when it tries to access memory location s1 + 4 as n is 5 and there is nothing to stop strncmp not to access 's1 + 4' With the current code change we are avoiding accessing memory location which is not owned by s1. Regards, Shafi > -----Original Message----- > From: Ioi Lam > Sent: Tuesday, September 20, 2016 12:09 AM > To: Shafi Ahmad; hotspot-dev at openjdk.java.net > Subject: Re: [8u] RFR for JDK-8157548: JVM crashes sometimes while starting > > Hi Shafi, > > The fix is correct. However, the crash is not inside strncmp. > > I reproduced this bug inside gdb. When the crash happened, the > parsed_name Symbol is "j", which has one character. By luck, it occupies the > same space as a freed copy of the Symbol "java/lang/String". For space > saving, the string content in the Symbol is not nul terminated. Thus: > > strncmp((const char*)parsed_name->bytes(), pkg, strlen(pkg)) > > would return true. However, when we later call > > char* name = parsed_name->as_C_string(); > char* index = strrchr(name, '/'); > *index = '\0'; // <------- crash > > name will be the 0-terminated string "j", and index would be NULL (because > '/' is not contained inside name.) Storing into index would cause the SEGV. > > Thanks > - Ioi > > > On 9/18/16 10:14 PM, Shafi Ahmad wrote: > > Hi, > > > > Please review the small code change for bug: "JDK-8157548: JVM crashes > > sometimes while starting" on jdk8u-dev > > > > Summary: > > int strncmp(const char *s1, const char *s2, size_t n); > > > > s1 = "abcdefgh" // Assume this is not null terminated string. > > s2 = "abcdefghijk" > > n = 10 > > > > In case if s1 is not null terminated then for above input strncmp may crash. > > > > In expression marked as (B) parsed_name->bytes() returns base address > of non-null terminated string buffer. > > > > + size_t pkglen = strlen(pkg); > > if (!HAS_PENDING_EXCEPTION && > > !class_loader.is_null() && > > parsed_name != NULL && > > - !strncmp((const char*)parsed_name->bytes(), pkg, strlen(pkg))) { > > + parsed_name->utf8_length() >= (int)pkglen && // ------- > ----------------------- (A) > > + !strncmp((const char*)parsed_name->bytes(), pkg, pkglen)) { > > + //------------------------------ (B) > > > > Adding expression marked as (A) avoid the above similar input scenario. > > > > Webrev: http://cr.openjdk.java.net/~shshahma/8157548/webrev.00/ > > Jdk8 bug: https://bugs.openjdk.java.net/browse/JDK-8157548 > > > > Test: Run jprt > > > > Note: Thanks to Ioi for providing the code change. > > > > Regards, > > Shafi > From shafi.s.ahmad at oracle.com Tue Sep 20 06:05:33 2016 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Mon, 19 Sep 2016 23:05:33 -0700 (PDT) Subject: [8u] RFR for JDK-6515172: Runtime.availableProcessors() ignores Linux taskset command In-Reply-To: References: <4a227590-41cd-4a4e-9b7d-9e8f8ad8f892@default> <11cf69a4-140b-90c4-e7af-ad0fb983f997@oracle.com> Message-ID: <570b7536-1007-4abb-a500-a519e0305a09@default> Adding jdk9 code change reviewer 'Daniel Daugherty' and 'Gerald Thornbrugh'. Regards, Shafi > -----Original Message----- > From: Shafi Ahmad > Sent: Monday, September 19, 2016 10:25 AM > To: David Holmes; Chris Plummer; hotspot-dev at openjdk.java.net > Cc: Roger Calnan > Subject: RE: [8u] RFR for JDK-6515172: Runtime.availableProcessors() ignores > Linux taskset command > > May I get the second review for this change. > > Regards, > Shafi > > > -----Original Message----- > > From: David Holmes > > Sent: Friday, September 16, 2016 7:16 AM > > To: Chris Plummer; Shafi Ahmad; hotspot-dev at openjdk.java.net > > Subject: Re: [8u] RFR for JDK-6515172: Runtime.availableProcessors() > > ignores Linux taskset command > > > > On 16/09/2016 4:17 AM, Chris Plummer wrote: > > > You should probably include the backport of JDK-8165153 since it was > > > introduced by JDK-6515172. > > > > This is a simplified backport of 6515172 suitable for Java 8. It does > > end up being a combination of that fix and 8165153, but you can't be a > > backport of two bugs, and there are no distinct parts that might map > > to each bug. That was why originally this was flagged under a separate > > bug id. But I agreed that this could be considered a simpified backport of > 6515172. > > > > Thanks, > > David > > > > > cheers, > > > > > > Chris > > > > > > On 9/15/16 2:51 AM, David Holmes wrote: > > >> Looks good. Reviewed. > > >> > > >> Thanks, > > >> David > > >> > > >> On 15/09/2016 4:40 PM, Shafi Ahmad wrote: > > >>> Hi, > > >>> > > >>> Please review the backport [modified] of bug: "JDK-6515172 > > >>> Runtime.availableProcessors() ignores Linux taskset command" to > jdk8u. > > >>> > > >>> Please note that the backport is not clean and we can't do as it is. > > >>> Please note > > >>> 1. The changes are made by 'Andreas Eriksson' who left Oracle. > > >>> 2. There is difference in logging mechanism in jdk9 and jdk8 is > > >>> different and file logTag.hpp doesn't exists in jdk8. > > >>> 3. Newly added test pass after this change. It fails without > > >>> this change. > > >>> > > >>> Webrev link: > > http://cr.openjdk.java.net/~shshahma/8154324/webrev.02/ > > >>> Jdk9 bug: https://bugs.openjdk.java.net/browse/JDK-6515172 > > >>> Original patch pushed to jdk9: > > >>> http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/c5480d4abfe4 > > >>> > > >>> Testing: jprt and running jtreg. > > >>> > > >>> Regards, > > >>> Shafi > > >>> > > > From tobias.hartmann at oracle.com Tue Sep 20 06:09:08 2016 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Tue, 20 Sep 2016 08:09:08 +0200 Subject: [9] RFR(S): Crash with assert: symbol conversion failure in java_lang_String::create_from_symbol() In-Reply-To: <2c907145-258c-2362-b525-438dbca34ec6@oracle.com> References: <57D2AE2F.1080607@oracle.com> <1203086c-f59f-40ce-f963-e958e570caf8@oracle.com> <57DF9D3A.3000802@oracle.com> <57DFAE0A.2090104@oracle.com> <57E032CF.4050602@oracle.com> <2c907145-258c-2362-b525-438dbca34ec6@oracle.com> Message-ID: <57E0D284.1030001@oracle.com> Ioi, Coleen, thanks again for the review! On 19.09.2016 21:36, Coleen Phillimore wrote: >>>> If the test that triggered this is sending in bad UTF-8 because it expects JNI to do validation then it is an invalid test. I think the asserts serve their purpose exactly - to show where invalid inputs came from. If this was bad user code then it would be up to them to fix it. If it is a bad test then the test should be fixed. >>> Sure, the test is broken and will be fixed. So you're suggesting to leave the check in create_from_symbol() as it is and re-enable the check in create_from_str()? >> I also think this is the right approach. Fix (or disable) the test, but leave the C code the way it is. >> >> java_lang_String::create_from_str(jchar*, ...) could (potentially??) be called by non JNI code, in which cases we want the assert. > > No, I think you need to leave the commented out code in create_from_str(). This could break testing and it's too late in jdk9 release for non-jigsaw to destabilize the testing. I agree, re-enabling the create_from_str() assert will cause other problems (see my latest comment in the bug). That's why I proposed to file a new bug for this. > I also think the similar change needs to be added to create_from_symbol(), as in the proposed fix. And file a bug to fix the test and another to evaluate whether we should validate strings in JNI and remove the commented out code. The test is quarantined and a JCK testbug was filed (see JIRA). > I think the proposed fix is what we should do for now. This assert is a regression, and our non-checking of JNI strings and symbols is not. Yes but as David mentioned in another email, this assert is only triggered by a broken test long after the code was pushed. I think it's safe to leave it in and open a new bug to investigate if we can re-enable the create_from_str() assert. Thanks, Tobias > > Thanks, > Coleen > > >> >> Thanks >> - Ioi >> >>> I'm fine with that but as I remember, re-enabling the check in create_from_str() causes other test failures. >>> >>> Best regards, >>> Tobias >>> >>>> Sorry. >>>> >>>> David >>>> >>>>> Thanks, >>>>> Tobias >>>>> >>>>> On 12.09.2016 02:47, David Holmes wrote: >>>>>> On 10/09/2016 6:55 AM, Coleen Phillimore wrote: >>>>>>> This change is fine because it matches the commented out assert in >>>>>>> create_from_str(). We should probably figure out what it would take to >>>>>>> check the characters coming in from JNI and decide whether we should do >>>>>>> this. If not, it doesn't make sense to have commented out asserts. >>>>>>> But this is okay for jdk9. >>>>>> Grumble, grumble ... both are bad. If the VM doesn't validate this bad UTF-8 then where does it go? And how does the generator of the bad UTF-8 get informed? An assert may be too drastic but can we throw an exception (InternalError?) ? >>>>>> >>>>>> David >>>>>> >>>>>>> Thanks, >>>>>>> Coleen >>>>>>> >>>>>>> >>>>>>> On 9/9/16 8:42 AM, Tobias Hartmann wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> please review the following patch: >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8164561 >>>>>>>> http://cr.openjdk.java.net/~thartmann/8164561/webrev.00/ >>>>>>>> >>>>>>>> The verification code in java_lang_String::create_from_symbol() that >>>>>>>> was added by Compact Strings fails because the input symbol does not >>>>>>>> contain valid UTF8. The problem is that a JCK JNI test passes an >>>>>>>> invalid UTF8 string as class name to the JNI method "FindClass". In >>>>>>>> fact, the string contains garbage from reading past array boundaries >>>>>>>> because of a bug in the test [1]. The JNI spec [2] states that 'name' >>>>>>>> should be "a fully-qualified class name (that is, a package name, >>>>>>>> delimited by ?/?, followed by the class name). If the name begins with >>>>>>>> ?[? (the array signature character), it returns an array class. The >>>>>>>> string is encoded in modified UTF-8". >>>>>>>> >>>>>>>> I nevertheless think that we should not crash in the case of an >>>>>>>> invalid UTF8 string and therefore disabled the verification code with >>>>>>>> a comment. We did the same for java_lang_String::create_from_str() [3]. >>>>>>>> >>>>>>>> Tested with failing JCK test and JPRT (running). >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Tobias >>>>>>>> >>>>>>>> [1] https://bugs.openjdk.java.net/browse/JCK-7307244 >>>>>>>> [2] >>>>>>>> https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/functions.html#FindClass >>>>>>>> >>>>>>>> [3] >>>>>>>> http://hg.openjdk.java.net/jdk9/hs/hotspot/file/d060826d0911/src/share/vm/classfile/javaClasses.cpp#l274 >>>>>>>> >> > From tobias.hartmann at oracle.com Tue Sep 20 06:20:59 2016 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Tue, 20 Sep 2016 08:20:59 +0200 Subject: [9] RFR(S): Crash with assert: symbol conversion failure in java_lang_String::create_from_symbol() In-Reply-To: <85ed982f-2e13-df25-e979-655009bf9bfe@oracle.com> References: <57D2AE2F.1080607@oracle.com> <1203086c-f59f-40ce-f963-e958e570caf8@oracle.com> <57DF9D3A.3000802@oracle.com> <57DFAE0A.2090104@oracle.com> <85ed982f-2e13-df25-e979-655009bf9bfe@oracle.com> Message-ID: <57E0D54B.4090504@oracle.com> Hi David, On 20.09.2016 02:02, David Holmes wrote: > Let me get this straight. > > The compact strings push occurred on Nov 3, 2015, some ten months ago! One of the assertions was commented out in that push because otherwise some tests failed. Someone made the decision that the tests were right and the assert was wrong - in which case it should just have been deleted from the code. Right, I referenced the failing (closed) test in my latest comment in the bug. It's not clear to me whether the test or the assert is "correct" and unfortunately I was not able to reproduce this old failure. As suggested, I would like to file a new bug to investigate re-enabling this assert. > Now 10 months later the other assert is suddenly failing and the claim is that it is the assert that is wrong! Why is it failing now - is this is a new test? If so we have to establish whether it is the test code or the VM code that is making the wrong assumptions. If the test is deemed valid then it is now passing invalid UTF-8 through a path never before tested with invalid UTF-8, so we need to know that the code will in fact handle that invalid input safely. Only then should the assert be removed, and that is only if it is determined that the test is valid. As described in the bug, the JNI test is broken and reads the UTF-8 strings from after the end of a char array. This garbage may or may not be invalid UTF-8. We were just unlucky to hit this now. So in this case it's the test that is wrong and not the assert. > In this particular case a bad string is being passed to jni_FindClass. JNI doesn't do validation of the string (bar minimal sanity checks) but should the VM be being more vigilant? It may be that an invalidly formed UTF-8 string can travel safely through the VM, and in this case (as Tobias indicated) just result in the class not being found. Or it may be there are places that make assumptions about the validity of such a string. Yes and to be on the safe side and to move forward, I would like to close this bug as "not an issue" and file a new bug to investigate re-enabling the create_from_str() assert. Are you okay with that? Thanks, Tobias > > David > ----- > > On 19/09/2016 7:21 PM, Tobias Hartmann wrote: >> Hi David, >> >> On 19.09.2016 10:30, David Holmes wrote: >>> On 19/09/2016 6:09 PM, Tobias Hartmann wrote: >>>> Coleen, David, thanks for the reviews and sorry for the delay! >>>> >>>> I agree that we should validate the Strings we get through JNI, but the code in create_from_str() and create_from_symbol() was originally added to check the result of String compression with the Compact Strings feature and not to check for valid UTF-8. >>>> >>>> I think UTF-8 validity should be checked earlier (before or during Symbol creation) and this affects other JNI methods that take C-Strings as well (jni_NewString, jni_NewStringUTF, jni_DefineClass, ...). This is a general problem with JNI not validating input Strings. >>>> >>>> Are you fine with pushing the proposed fix (webrev.00) and open a follow up bug to fix JNI (and potentially re-enable the Compact Strings asserts because their existence is still justified)? >>> >>> As I wrote in the bug report: >>> >>> "I don't think I agree with the removed checks in either case. JNI doesn't do argument checking and so doesn't detect the invalid UTF-8 string. But the VM expects to work with valid strings. It seems quite reasonable to me that the VM should validate the UTF-8 and report some kind of failure to make these invalid inputs visible. Otherwise the bad JNI codes goes undetected. Maybe an assertion is too strong, perhaps we should throw InternalError? " >> >> Yes, I've seen that and, as I said, I agree that we should validate the Strings. >> >>> If the VM does not detect this and respond to it exactly what happens with the bad UTF-8 string? >> >> In the FindClass case, we just return NULL. I don't know if there are potential problems with invalid UTF-8 strings at other places in the code. >> >>> JNI doesn't do input validation - it is a "feature" of JNI so not to penalize correct code. >> >> Right, but this check only affects debug builds. >> >>> If the test that triggered this is sending in bad UTF-8 because it expects JNI to do validation then it is an invalid test. I think the asserts serve their purpose exactly - to show where invalid inputs came from. If this was bad user code then it would be up to them to fix it. If it is a bad test then the test should be fixed. >> >> Sure, the test is broken and will be fixed. So you're suggesting to leave the check in create_from_symbol() as it is and re-enable the check in create_from_str()? >> >> I'm fine with that but as I remember, re-enabling the check in create_from_str() causes other test failures. >> >> Best regards, >> Tobias >> >>> Sorry. >>> >>> David >>> >>>> Thanks, >>>> Tobias >>>> >>>> On 12.09.2016 02:47, David Holmes wrote: >>>>> On 10/09/2016 6:55 AM, Coleen Phillimore wrote: >>>>>> This change is fine because it matches the commented out assert in >>>>>> create_from_str(). We should probably figure out what it would take to >>>>>> check the characters coming in from JNI and decide whether we should do >>>>>> this. If not, it doesn't make sense to have commented out asserts. >>>>>> But this is okay for jdk9. >>>>> >>>>> Grumble, grumble ... both are bad. If the VM doesn't validate this bad UTF-8 then where does it go? And how does the generator of the bad UTF-8 get informed? An assert may be too drastic but can we throw an exception (InternalError?) ? >>>>> >>>>> David >>>>> >>>>>> Thanks, >>>>>> Coleen >>>>>> >>>>>> >>>>>> On 9/9/16 8:42 AM, Tobias Hartmann wrote: >>>>>>> Hi, >>>>>>> >>>>>>> please review the following patch: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8164561 >>>>>>> http://cr.openjdk.java.net/~thartmann/8164561/webrev.00/ >>>>>>> >>>>>>> The verification code in java_lang_String::create_from_symbol() that >>>>>>> was added by Compact Strings fails because the input symbol does not >>>>>>> contain valid UTF8. The problem is that a JCK JNI test passes an >>>>>>> invalid UTF8 string as class name to the JNI method "FindClass". In >>>>>>> fact, the string contains garbage from reading past array boundaries >>>>>>> because of a bug in the test [1]. The JNI spec [2] states that 'name' >>>>>>> should be "a fully-qualified class name (that is, a package name, >>>>>>> delimited by ?/?, followed by the class name). If the name begins with >>>>>>> ?[? (the array signature character), it returns an array class. The >>>>>>> string is encoded in modified UTF-8". >>>>>>> >>>>>>> I nevertheless think that we should not crash in the case of an >>>>>>> invalid UTF8 string and therefore disabled the verification code with >>>>>>> a comment. We did the same for java_lang_String::create_from_str() [3]. >>>>>>> >>>>>>> Tested with failing JCK test and JPRT (running). >>>>>>> >>>>>>> Thanks, >>>>>>> Tobias >>>>>>> >>>>>>> [1] https://bugs.openjdk.java.net/browse/JCK-7307244 >>>>>>> [2] >>>>>>> https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/functions.html#FindClass >>>>>>> >>>>>>> [3] >>>>>>> http://hg.openjdk.java.net/jdk9/hs/hotspot/file/d060826d0911/src/share/vm/classfile/javaClasses.cpp#l274 >>>>>>> >>>>>> From david.holmes at oracle.com Tue Sep 20 07:00:10 2016 From: david.holmes at oracle.com (David Holmes) Date: Tue, 20 Sep 2016 17:00:10 +1000 Subject: [9] RFR(S): Crash with assert: symbol conversion failure in java_lang_String::create_from_symbol() In-Reply-To: <57E0D54B.4090504@oracle.com> References: <57D2AE2F.1080607@oracle.com> <1203086c-f59f-40ce-f963-e958e570caf8@oracle.com> <57DF9D3A.3000802@oracle.com> <57DFAE0A.2090104@oracle.com> <85ed982f-2e13-df25-e979-655009bf9bfe@oracle.com> <57E0D54B.4090504@oracle.com> Message-ID: <30ea6c33-eea5-5a30-d690-b4aba7b994c9@oracle.com> Hi Tobias, On 20/09/2016 4:20 PM, Tobias Hartmann wrote: > Hi David, > > On 20.09.2016 02:02, David Holmes wrote: >> Let me get this straight. >> >> The compact strings push occurred on Nov 3, 2015, some ten months ago! One of the assertions was commented out in that push because otherwise some tests failed. Someone made the decision that the tests were right and the assert was wrong - in which case it should just have been deleted from the code. > > Right, I referenced the failing (closed) test in my latest comment in the bug. It's not clear to me whether the test or the assert is "correct" and unfortunately I was not able to reproduce this old failure. As suggested, I would like to file a new bug to investigate re-enabling this assert. > >> Now 10 months later the other assert is suddenly failing and the claim is that it is the assert that is wrong! Why is it failing now - is this is a new test? If so we have to establish whether it is the test code or the VM code that is making the wrong assumptions. If the test is deemed valid then it is now passing invalid UTF-8 through a path never before tested with invalid UTF-8, so we need to know that the code will in fact handle that invalid input safely. Only then should the assert be removed, and that is only if it is determined that the test is valid. > > As described in the bug, the JNI test is broken and reads the UTF-8 strings from after the end of a char array. This garbage may or may not be invalid UTF-8. We were just unlucky to hit this now. So in this case it's the test that is wrong and not the assert. > >> In this particular case a bad string is being passed to jni_FindClass. JNI doesn't do validation of the string (bar minimal sanity checks) but should the VM be being more vigilant? It may be that an invalidly formed UTF-8 string can travel safely through the VM, and in this case (as Tobias indicated) just result in the class not being found. Or it may be there are places that make assumptions about the validity of such a string. > > Yes and to be on the safe side and to move forward, I would like to close this bug as "not an issue" and file a new bug to investigate re-enabling the create_from_str() assert. > > Are you okay with that? "not an issue" or a dup of the bug that has to fix the test? Either way I am okay with no change being made to the VM code for this case. Thanks, David > Thanks, > Tobias > >> >> David >> ----- >> >> On 19/09/2016 7:21 PM, Tobias Hartmann wrote: >>> Hi David, >>> >>> On 19.09.2016 10:30, David Holmes wrote: >>>> On 19/09/2016 6:09 PM, Tobias Hartmann wrote: >>>>> Coleen, David, thanks for the reviews and sorry for the delay! >>>>> >>>>> I agree that we should validate the Strings we get through JNI, but the code in create_from_str() and create_from_symbol() was originally added to check the result of String compression with the Compact Strings feature and not to check for valid UTF-8. >>>>> >>>>> I think UTF-8 validity should be checked earlier (before or during Symbol creation) and this affects other JNI methods that take C-Strings as well (jni_NewString, jni_NewStringUTF, jni_DefineClass, ...). This is a general problem with JNI not validating input Strings. >>>>> >>>>> Are you fine with pushing the proposed fix (webrev.00) and open a follow up bug to fix JNI (and potentially re-enable the Compact Strings asserts because their existence is still justified)? >>>> >>>> As I wrote in the bug report: >>>> >>>> "I don't think I agree with the removed checks in either case. JNI doesn't do argument checking and so doesn't detect the invalid UTF-8 string. But the VM expects to work with valid strings. It seems quite reasonable to me that the VM should validate the UTF-8 and report some kind of failure to make these invalid inputs visible. Otherwise the bad JNI codes goes undetected. Maybe an assertion is too strong, perhaps we should throw InternalError? " >>> >>> Yes, I've seen that and, as I said, I agree that we should validate the Strings. >>> >>>> If the VM does not detect this and respond to it exactly what happens with the bad UTF-8 string? >>> >>> In the FindClass case, we just return NULL. I don't know if there are potential problems with invalid UTF-8 strings at other places in the code. >>> >>>> JNI doesn't do input validation - it is a "feature" of JNI so not to penalize correct code. >>> >>> Right, but this check only affects debug builds. >>> >>>> If the test that triggered this is sending in bad UTF-8 because it expects JNI to do validation then it is an invalid test. I think the asserts serve their purpose exactly - to show where invalid inputs came from. If this was bad user code then it would be up to them to fix it. If it is a bad test then the test should be fixed. >>> >>> Sure, the test is broken and will be fixed. So you're suggesting to leave the check in create_from_symbol() as it is and re-enable the check in create_from_str()? >>> >>> I'm fine with that but as I remember, re-enabling the check in create_from_str() causes other test failures. >>> >>> Best regards, >>> Tobias >>> >>>> Sorry. >>>> >>>> David >>>> >>>>> Thanks, >>>>> Tobias >>>>> >>>>> On 12.09.2016 02:47, David Holmes wrote: >>>>>> On 10/09/2016 6:55 AM, Coleen Phillimore wrote: >>>>>>> This change is fine because it matches the commented out assert in >>>>>>> create_from_str(). We should probably figure out what it would take to >>>>>>> check the characters coming in from JNI and decide whether we should do >>>>>>> this. If not, it doesn't make sense to have commented out asserts. >>>>>>> But this is okay for jdk9. >>>>>> >>>>>> Grumble, grumble ... both are bad. If the VM doesn't validate this bad UTF-8 then where does it go? And how does the generator of the bad UTF-8 get informed? An assert may be too drastic but can we throw an exception (InternalError?) ? >>>>>> >>>>>> David >>>>>> >>>>>>> Thanks, >>>>>>> Coleen >>>>>>> >>>>>>> >>>>>>> On 9/9/16 8:42 AM, Tobias Hartmann wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> please review the following patch: >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8164561 >>>>>>>> http://cr.openjdk.java.net/~thartmann/8164561/webrev.00/ >>>>>>>> >>>>>>>> The verification code in java_lang_String::create_from_symbol() that >>>>>>>> was added by Compact Strings fails because the input symbol does not >>>>>>>> contain valid UTF8. The problem is that a JCK JNI test passes an >>>>>>>> invalid UTF8 string as class name to the JNI method "FindClass". In >>>>>>>> fact, the string contains garbage from reading past array boundaries >>>>>>>> because of a bug in the test [1]. The JNI spec [2] states that 'name' >>>>>>>> should be "a fully-qualified class name (that is, a package name, >>>>>>>> delimited by ?/?, followed by the class name). If the name begins with >>>>>>>> ?[? (the array signature character), it returns an array class. The >>>>>>>> string is encoded in modified UTF-8". >>>>>>>> >>>>>>>> I nevertheless think that we should not crash in the case of an >>>>>>>> invalid UTF8 string and therefore disabled the verification code with >>>>>>>> a comment. We did the same for java_lang_String::create_from_str() [3]. >>>>>>>> >>>>>>>> Tested with failing JCK test and JPRT (running). >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Tobias >>>>>>>> >>>>>>>> [1] https://bugs.openjdk.java.net/browse/JCK-7307244 >>>>>>>> [2] >>>>>>>> https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/functions.html#FindClass >>>>>>>> >>>>>>>> [3] >>>>>>>> http://hg.openjdk.java.net/jdk9/hs/hotspot/file/d060826d0911/src/share/vm/classfile/javaClasses.cpp#l274 >>>>>>>> >>>>>>> From tobias.hartmann at oracle.com Tue Sep 20 07:10:04 2016 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Tue, 20 Sep 2016 09:10:04 +0200 Subject: [9] RFR(S): Crash with assert: symbol conversion failure in java_lang_String::create_from_symbol() In-Reply-To: <30ea6c33-eea5-5a30-d690-b4aba7b994c9@oracle.com> References: <57D2AE2F.1080607@oracle.com> <1203086c-f59f-40ce-f963-e958e570caf8@oracle.com> <57DF9D3A.3000802@oracle.com> <57DFAE0A.2090104@oracle.com> <85ed982f-2e13-df25-e979-655009bf9bfe@oracle.com> <57E0D54B.4090504@oracle.com> <30ea6c33-eea5-5a30-d690-b4aba7b994c9@oracle.com> Message-ID: <57E0E0CC.6060403@oracle.com> Hi David, On 20.09.2016 09:00, David Holmes wrote: > Hi Tobias, > > On 20/09/2016 4:20 PM, Tobias Hartmann wrote: >> Hi David, >> >> On 20.09.2016 02:02, David Holmes wrote: >>> Let me get this straight. >>> >>> The compact strings push occurred on Nov 3, 2015, some ten months ago! One of the assertions was commented out in that push because otherwise some tests failed. Someone made the decision that the tests were right and the assert was wrong - in which case it should just have been deleted from the code. >> >> Right, I referenced the failing (closed) test in my latest comment in the bug. It's not clear to me whether the test or the assert is "correct" and unfortunately I was not able to reproduce this old failure. As suggested, I would like to file a new bug to investigate re-enabling this assert. >> >>> Now 10 months later the other assert is suddenly failing and the claim is that it is the assert that is wrong! Why is it failing now - is this is a new test? If so we have to establish whether it is the test code or the VM code that is making the wrong assumptions. If the test is deemed valid then it is now passing invalid UTF-8 through a path never before tested with invalid UTF-8, so we need to know that the code will in fact handle that invalid input safely. Only then should the assert be removed, and that is only if it is determined that the test is valid. >> >> As described in the bug, the JNI test is broken and reads the UTF-8 strings from after the end of a char array. This garbage may or may not be invalid UTF-8. We were just unlucky to hit this now. So in this case it's the test that is wrong and not the assert. >> >>> In this particular case a bad string is being passed to jni_FindClass. JNI doesn't do validation of the string (bar minimal sanity checks) but should the VM be being more vigilant? It may be that an invalidly formed UTF-8 string can travel safely through the VM, and in this case (as Tobias indicated) just result in the class not being found. Or it may be there are places that make assumptions about the validity of such a string. >> >> Yes and to be on the safe side and to move forward, I would like to close this bug as "not an issue" and file a new bug to investigate re-enabling the create_from_str() assert. >> >> Are you okay with that? > > "not an issue" or a dup of the bug that has to fix the test? Duplicate is probably better in this case as it's clearly a testbug. > Either way I am okay with no change being made to the VM code for this case. Okay, thank you. Best regards, Tobias > Thanks, > David > >> Thanks, >> Tobias >> >>> >>> David >>> ----- >>> >>> On 19/09/2016 7:21 PM, Tobias Hartmann wrote: >>>> Hi David, >>>> >>>> On 19.09.2016 10:30, David Holmes wrote: >>>>> On 19/09/2016 6:09 PM, Tobias Hartmann wrote: >>>>>> Coleen, David, thanks for the reviews and sorry for the delay! >>>>>> >>>>>> I agree that we should validate the Strings we get through JNI, but the code in create_from_str() and create_from_symbol() was originally added to check the result of String compression with the Compact Strings feature and not to check for valid UTF-8. >>>>>> >>>>>> I think UTF-8 validity should be checked earlier (before or during Symbol creation) and this affects other JNI methods that take C-Strings as well (jni_NewString, jni_NewStringUTF, jni_DefineClass, ...). This is a general problem with JNI not validating input Strings. >>>>>> >>>>>> Are you fine with pushing the proposed fix (webrev.00) and open a follow up bug to fix JNI (and potentially re-enable the Compact Strings asserts because their existence is still justified)? >>>>> >>>>> As I wrote in the bug report: >>>>> >>>>> "I don't think I agree with the removed checks in either case. JNI doesn't do argument checking and so doesn't detect the invalid UTF-8 string. But the VM expects to work with valid strings. It seems quite reasonable to me that the VM should validate the UTF-8 and report some kind of failure to make these invalid inputs visible. Otherwise the bad JNI codes goes undetected. Maybe an assertion is too strong, perhaps we should throw InternalError? " >>>> >>>> Yes, I've seen that and, as I said, I agree that we should validate the Strings. >>>> >>>>> If the VM does not detect this and respond to it exactly what happens with the bad UTF-8 string? >>>> >>>> In the FindClass case, we just return NULL. I don't know if there are potential problems with invalid UTF-8 strings at other places in the code. >>>> >>>>> JNI doesn't do input validation - it is a "feature" of JNI so not to penalize correct code. >>>> >>>> Right, but this check only affects debug builds. >>>> >>>>> If the test that triggered this is sending in bad UTF-8 because it expects JNI to do validation then it is an invalid test. I think the asserts serve their purpose exactly - to show where invalid inputs came from. If this was bad user code then it would be up to them to fix it. If it is a bad test then the test should be fixed. >>>> >>>> Sure, the test is broken and will be fixed. So you're suggesting to leave the check in create_from_symbol() as it is and re-enable the check in create_from_str()? >>>> >>>> I'm fine with that but as I remember, re-enabling the check in create_from_str() causes other test failures. >>>> >>>> Best regards, >>>> Tobias >>>> >>>>> Sorry. >>>>> >>>>> David >>>>> >>>>>> Thanks, >>>>>> Tobias >>>>>> >>>>>> On 12.09.2016 02:47, David Holmes wrote: >>>>>>> On 10/09/2016 6:55 AM, Coleen Phillimore wrote: >>>>>>>> This change is fine because it matches the commented out assert in >>>>>>>> create_from_str(). We should probably figure out what it would take to >>>>>>>> check the characters coming in from JNI and decide whether we should do >>>>>>>> this. If not, it doesn't make sense to have commented out asserts. >>>>>>>> But this is okay for jdk9. >>>>>>> >>>>>>> Grumble, grumble ... both are bad. If the VM doesn't validate this bad UTF-8 then where does it go? And how does the generator of the bad UTF-8 get informed? An assert may be too drastic but can we throw an exception (InternalError?) ? >>>>>>> >>>>>>> David >>>>>>> >>>>>>>> Thanks, >>>>>>>> Coleen >>>>>>>> >>>>>>>> >>>>>>>> On 9/9/16 8:42 AM, Tobias Hartmann wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> please review the following patch: >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8164561 >>>>>>>>> http://cr.openjdk.java.net/~thartmann/8164561/webrev.00/ >>>>>>>>> >>>>>>>>> The verification code in java_lang_String::create_from_symbol() that >>>>>>>>> was added by Compact Strings fails because the input symbol does not >>>>>>>>> contain valid UTF8. The problem is that a JCK JNI test passes an >>>>>>>>> invalid UTF8 string as class name to the JNI method "FindClass". In >>>>>>>>> fact, the string contains garbage from reading past array boundaries >>>>>>>>> because of a bug in the test [1]. The JNI spec [2] states that 'name' >>>>>>>>> should be "a fully-qualified class name (that is, a package name, >>>>>>>>> delimited by ?/?, followed by the class name). If the name begins with >>>>>>>>> ?[? (the array signature character), it returns an array class. The >>>>>>>>> string is encoded in modified UTF-8". >>>>>>>>> >>>>>>>>> I nevertheless think that we should not crash in the case of an >>>>>>>>> invalid UTF8 string and therefore disabled the verification code with >>>>>>>>> a comment. We did the same for java_lang_String::create_from_str() [3]. >>>>>>>>> >>>>>>>>> Tested with failing JCK test and JPRT (running). >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Tobias >>>>>>>>> >>>>>>>>> [1] https://bugs.openjdk.java.net/browse/JCK-7307244 >>>>>>>>> [2] >>>>>>>>> https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/functions.html#FindClass >>>>>>>>> >>>>>>>>> [3] >>>>>>>>> http://hg.openjdk.java.net/jdk9/hs/hotspot/file/d060826d0911/src/share/vm/classfile/javaClasses.cpp#l274 >>>>>>>>> >>>>>>>> From erik.joelsson at oracle.com Tue Sep 20 07:42:50 2016 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Tue, 20 Sep 2016 09:42:50 +0200 Subject: RFR for JDK-8165482 java in ldoms, with cpu-arch=generic has problems In-Reply-To: <0a2a66db-f939-196f-bf23-a6a3df46b2d3@oracle.com> References: <8046b263-8826-aa34-083a-9a83e105770f@oracle.com> <0a2a66db-f939-196f-bf23-a6a3df46b2d3@oracle.com> Message-ID: <4bfdead9-f0d9-f67e-e049-d48af5f19863@oracle.com> Hello, On 2016-09-20 03:28, David Holmes wrote: > Hi Martin, > > Build changes must be reviewed by the build team - now cc'd > Thanks for forwarding David, I can't keep up with all the lists to find these unless posted to build-dev. > On 20/09/2016 12:16 AM, Martin Walsh wrote: >> Could I get a code review for the following bug: >> >> JDK-8165482 java in ldoms, with cpu-arch=generic has problems >> >> Webrev of the changes is available here: >> >> http://cr.openjdk.java.net/~mwalsh/JDK-8165482/ > > What is the devinfo library? Is it part of the normal Solaris > installation, or does it need to be installed specifically? Is it > available in our official build toolkits? I did a bit of digging. It's part "system/libraries" so should be pretty standard. That package is in the devkit and I verified that libdevinfo.so is there too. Configure change looks fine. Just remember to also push the closed generated-configure.sh as David said. /Erik > > Will checking the prom prior to using kstat change any of the values > we currently see? (other than the generic case being fixed of course). > > Also note you will need to generate and push the closed > generated-configure.sh file. > > Thanks, > David > >> >> Thanks, >> >> Martin From tobias.hartmann at oracle.com Tue Sep 20 09:07:03 2016 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Tue, 20 Sep 2016 11:07:03 +0200 Subject: [9] RFR(S): Crash with assert: symbol conversion failure in java_lang_String::create_from_symbol() In-Reply-To: <57E0D284.1030001@oracle.com> References: <57D2AE2F.1080607@oracle.com> <1203086c-f59f-40ce-f963-e958e570caf8@oracle.com> <57DF9D3A.3000802@oracle.com> <57DFAE0A.2090104@oracle.com> <57E032CF.4050602@oracle.com> <2c907145-258c-2362-b525-438dbca34ec6@oracle.com> <57E0D284.1030001@oracle.com> Message-ID: <57E0FC37.4040003@oracle.com> Hi, On 20.09.2016 08:09, Tobias Hartmann wrote: > I think it's safe to leave it in and open a new bug to investigate if we can re-enable the create_from_str() assert. FYI, I filed JDK-8166358 to keep track of this. I think we should defer this to JDK 10. Thanks, Tobias From dmitry.samersoff at oracle.com Tue Sep 20 09:48:07 2016 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Tue, 20 Sep 2016 12:48:07 +0300 Subject: RFR: 8150758: [TESTBUG] need jvmti tests for module aware agents In-Reply-To: References: <93940e3d-48ae-9f04-045f-04618dc80d16@oracle.com> Message-ID: <9ee6b3cc-4032-213e-83ed-89a5d182a981@oracle.com> Dmitry, Please, also change !strcmp(...) to strcmp(...) == 0 because semantically strcmp result is not a boolean false. -Dmitry On 2016-09-20 06:38, serguei.spitsyn at oracle.com wrote: > Hi Dmitry, > > > Thanks a lot for this additional test coverage and discovering new bug: > https://bugs.openjdk.java.net/browse/JDK-8165681 > > The tests look pretty good to me. > > A couple of minor comments. > > Dots are missed in the .c files comments. > > > http://cr.openjdk.java.net/~ddmitriev/8150758/webrev.00/test/serviceability/jvmti > > > > 265 printf("Expecting to find '%s' class in ClassLoad events > duringVM start phase.\n", EXPECTED_SIGNATURE); > 266 if (class_in_class_load_events_vm_start == JNI_FALSE) { > 267 throw_exc(env, "Unable to find expected class in > ClassLoad events duringstart phase!\n"); > 268 return FAILED; > 269 } > 270 > 271 if (class_prepare_events_vm_start_count == 0) { > 272 throw_exc(env, "Didn't get ClassPrepare events in start > phase!\n"); > 273 return FAILED; > 274 } > 275 > 276 printf("Expecting to find '%s' class in ClassPrepare events > duringVM start phase.\n", EXPECTED_SIGNATURE); > 277 if (class_in_class_prepare_events_vm_start == JNI_FALSE) { > 278 throw_exc(env, "Unable to find expected class in > ClassPrepare events duringstart phase!\n"); > 279 return FAILED; > 280 } > > > Could you, please, replace "start phase" with "early start phase"? > It will be more clear that the start phase mode is "early". > > 288 if (class_in_class_prepare_events_vm_start == JNI_TRUE) { > 289 throw_exc(env, "Class is found in ClassLoad events > duringVM Start phase!\n"); > 290 return FAILED; > 291 } > 292 > 293 printf("Expecting that '%s' class is absent in ClassPrepare > events.\n", EXPECTED_SIGNATURE); > 294 if (class_in_class_prepare_events_vm_start == JNI_TRUE) { > 295 throw_exc(env, "Class is found in ClassPrepare events > duringVM Start phase!\n"); > 296 return FAILED; > 297 } > > Could you, please, replace "VM Start phase" with "normal VM start > phase"? It will be more clear that the start phase mode is "normal". > Thanks, Serguei On 9/19/16 05:47, Dmitry Dmitriev wrote: >> Hello, Please review new tests for module aware agents. There are 3 >> tests: 1) MAAClassFileLoadHook.java - verifies ClassFileLoadHook event >> with different combinations of can_generate_early_vmstart and >> can_generate_early_class_hook_events JVMTI capabilities. Expects to >> find(or not) class from java.base module in the right VM phase. 2) >> MAAClassLoadPrepare.java - verifies ClassLoad and ClassPrepare events >> with and without can_generate_early_vmstart JVMTI capability. Expects >> to find(or not) class from java.base module in the VM start phase. 3) >> MAAThreadStart.java - verifies ThreadStart event with >> can_generate_early_vmstart JVMTI capability. Expect to find events in >> the VM start phase. JBS: >> https://bugs.openjdk.java.net/browse/JDK-8150758 webrev.00: >> http://cr.openjdk.java.net/~ddmitriev/8150758/webrev.00/ >> Testing: >> RBT on all platforms Thanks, Dmitry -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From dmitry.dmitriev at oracle.com Tue Sep 20 10:17:26 2016 From: dmitry.dmitriev at oracle.com (Dmitry Dmitriev) Date: Tue, 20 Sep 2016 13:17:26 +0300 Subject: RFR: 8150758: [TESTBUG] need jvmti tests for module aware agents In-Reply-To: <9ee6b3cc-4032-213e-83ed-89a5d182a981@oracle.com> References: <93940e3d-48ae-9f04-045f-04618dc80d16@oracle.com> <9ee6b3cc-4032-213e-83ed-89a5d182a981@oracle.com> Message-ID: Serguei, Dmitry, Thank you for the feedback! Here is an updated webrev.01; http://cr.openjdk.java.net/~ddmitriev/8150758/webrev.01/ Dmitry On 20.09.2016 12:48, Dmitry Samersoff wrote: > Dmitry, > > Please, also change > > !strcmp(...) to strcmp(...) == 0 > > because semantically strcmp result is not a boolean false. > > -Dmitry > > On 2016-09-20 06:38, serguei.spitsyn at oracle.com wrote: >> Hi Dmitry, >> >> >> Thanks a lot for this additional test coverage and discovering new bug: >> https://bugs.openjdk.java.net/browse/JDK-8165681 >> >> The tests look pretty good to me. >> >> A couple of minor comments. >> >> Dots are missed in the .c files comments. >> >> >> http://cr.openjdk.java.net/~ddmitriev/8150758/webrev.00/test/serviceability/jvmti >> >> >> >> 265 printf("Expecting to find '%s' class in ClassLoad events >> duringVM start phase.\n", EXPECTED_SIGNATURE); >> 266 if (class_in_class_load_events_vm_start == JNI_FALSE) { >> 267 throw_exc(env, "Unable to find expected class in >> ClassLoad events duringstart phase!\n"); >> 268 return FAILED; >> 269 } >> 270 >> 271 if (class_prepare_events_vm_start_count == 0) { >> 272 throw_exc(env, "Didn't get ClassPrepare events in start >> phase!\n"); >> 273 return FAILED; >> 274 } >> 275 >> 276 printf("Expecting to find '%s' class in ClassPrepare events >> duringVM start phase.\n", EXPECTED_SIGNATURE); >> 277 if (class_in_class_prepare_events_vm_start == JNI_FALSE) { >> 278 throw_exc(env, "Unable to find expected class in >> ClassPrepare events duringstart phase!\n"); >> 279 return FAILED; >> 280 } >> >> >> Could you, please, replace "start phase" with "early start phase"? >> It will be more clear that the start phase mode is "early". >> >> 288 if (class_in_class_prepare_events_vm_start == JNI_TRUE) { >> 289 throw_exc(env, "Class is found in ClassLoad events >> duringVM Start phase!\n"); >> 290 return FAILED; >> 291 } >> 292 >> 293 printf("Expecting that '%s' class is absent in ClassPrepare >> events.\n", EXPECTED_SIGNATURE); >> 294 if (class_in_class_prepare_events_vm_start == JNI_TRUE) { >> 295 throw_exc(env, "Class is found in ClassPrepare events >> duringVM Start phase!\n"); >> 296 return FAILED; >> 297 } >> >> Could you, please, replace "VM Start phase" with "normal VM start >> phase"? It will be more clear that the start phase mode is "normal". >> Thanks, Serguei On 9/19/16 05:47, Dmitry Dmitriev wrote: >>> Hello, Please review new tests for module aware agents. There are 3 >>> tests: 1) MAAClassFileLoadHook.java - verifies ClassFileLoadHook event >>> with different combinations of can_generate_early_vmstart and >>> can_generate_early_class_hook_events JVMTI capabilities. Expects to >>> find(or not) class from java.base module in the right VM phase. 2) >>> MAAClassLoadPrepare.java - verifies ClassLoad and ClassPrepare events >>> with and without can_generate_early_vmstart JVMTI capability. Expects >>> to find(or not) class from java.base module in the VM start phase. 3) >>> MAAThreadStart.java - verifies ThreadStart event with >>> can_generate_early_vmstart JVMTI capability. Expect to find events in >>> the VM start phase. JBS: >>> https://bugs.openjdk.java.net/browse/JDK-8150758 webrev.00: >>> http://cr.openjdk.java.net/~ddmitriev/8150758/webrev.00/ >>> Testing: >>> RBT on all platforms Thanks, Dmitry > From dmitry.samersoff at oracle.com Tue Sep 20 10:39:28 2016 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Tue, 20 Sep 2016 13:39:28 +0300 Subject: RFR: 8150758: [TESTBUG] need jvmti tests for module aware agents In-Reply-To: References: <93940e3d-48ae-9f04-045f-04618dc80d16@oracle.com> <9ee6b3cc-4032-213e-83ed-89a5d182a981@oracle.com> Message-ID: <867e38fa-deea-939d-2a5b-a264b3e1cd7f@oracle.com> Dmitry, Looks good for me! Minor formatting nits: libMAAClassFileLoadHook.c 103: missed space after , libMAAClassFileLoadHook.c: 266: extra semicolon -Dmitry On 2016-09-20 13:17, Dmitry Dmitriev wrote: > Serguei, Dmitry, > > Thank you for the feedback! Here is an updated webrev.01; > http://cr.openjdk.java.net/~ddmitriev/8150758/webrev.01/ > > > Dmitry > > On 20.09.2016 12:48, Dmitry Samersoff wrote: >> Dmitry, >> >> Please, also change >> >> !strcmp(...) to strcmp(...) == 0 >> >> because semantically strcmp result is not a boolean false. >> >> -Dmitry >> >> On 2016-09-20 06:38, serguei.spitsyn at oracle.com wrote: >>> Hi Dmitry, >>> >>> >>> Thanks a lot for this additional test coverage and discovering new bug: >>> https://bugs.openjdk.java.net/browse/JDK-8165681 >>> >>> The tests look pretty good to me. >>> >>> A couple of minor comments. >>> >>> Dots are missed in the .c files comments. >>> >>> >>> http://cr.openjdk.java.net/~ddmitriev/8150758/webrev.00/test/serviceability/jvmti >>> >>> >>> >>> >>> 265 printf("Expecting to find '%s' class in ClassLoad events >>> duringVM start phase.\n", EXPECTED_SIGNATURE); >>> 266 if (class_in_class_load_events_vm_start == JNI_FALSE) { >>> 267 throw_exc(env, "Unable to find expected class in >>> ClassLoad events duringstart phase!\n"); >>> 268 return FAILED; >>> 269 } >>> 270 >>> 271 if (class_prepare_events_vm_start_count == 0) { >>> 272 throw_exc(env, "Didn't get ClassPrepare events in >>> start >>> phase!\n"); >>> 273 return FAILED; >>> 274 } >>> 275 >>> 276 printf("Expecting to find '%s' class in ClassPrepare >>> events >>> duringVM start phase.\n", EXPECTED_SIGNATURE); >>> 277 if (class_in_class_prepare_events_vm_start == JNI_FALSE) { >>> 278 throw_exc(env, "Unable to find expected class in >>> ClassPrepare events duringstart phase!\n"); >>> 279 return FAILED; >>> 280 } >>> >>> >>> Could you, please, replace "start phase" with "early start phase"? >>> It will be more clear that the start phase mode is "early". >>> >>> 288 if (class_in_class_prepare_events_vm_start == JNI_TRUE) { >>> 289 throw_exc(env, "Class is found in ClassLoad events >>> duringVM Start phase!\n"); >>> 290 return FAILED; >>> 291 } >>> 292 >>> 293 printf("Expecting that '%s' class is absent in >>> ClassPrepare >>> events.\n", EXPECTED_SIGNATURE); >>> 294 if (class_in_class_prepare_events_vm_start == JNI_TRUE) { >>> 295 throw_exc(env, "Class is found in ClassPrepare events >>> duringVM Start phase!\n"); >>> 296 return FAILED; >>> 297 } >>> >>> Could you, please, replace "VM Start phase" with "normal VM start >>> phase"? It will be more clear that the start phase mode is "normal". >>> Thanks, Serguei On 9/19/16 05:47, Dmitry Dmitriev wrote: >>>> Hello, Please review new tests for module aware agents. There are 3 >>>> tests: 1) MAAClassFileLoadHook.java - verifies ClassFileLoadHook event >>>> with different combinations of can_generate_early_vmstart and >>>> can_generate_early_class_hook_events JVMTI capabilities. Expects to >>>> find(or not) class from java.base module in the right VM phase. 2) >>>> MAAClassLoadPrepare.java - verifies ClassLoad and ClassPrepare events >>>> with and without can_generate_early_vmstart JVMTI capability. Expects >>>> to find(or not) class from java.base module in the VM start phase. 3) >>>> MAAThreadStart.java - verifies ThreadStart event with >>>> can_generate_early_vmstart JVMTI capability. Expect to find events in >>>> the VM start phase. JBS: >>>> https://bugs.openjdk.java.net/browse/JDK-8150758 webrev.00: >>>> http://cr.openjdk.java.net/~ddmitriev/8150758/webrev.00/ >>>> Testing: >>>> RBT on all platforms Thanks, Dmitry >> > -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From serguei.spitsyn at oracle.com Tue Sep 20 10:40:38 2016 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 20 Sep 2016 03:40:38 -0700 Subject: RFR: 8150758: [TESTBUG] need jvmti tests for module aware agents In-Reply-To: References: <93940e3d-48ae-9f04-045f-04618dc80d16@oracle.com> <9ee6b3cc-4032-213e-83ed-89a5d182a981@oracle.com> Message-ID: <7cc7978d-663b-1305-d3ea-01b8a8e11df2@oracle.com> Hi Dmitry, The fix looks good, thank you for the update. Thumbs up. Thanks, Serguei On 9/20/16 03:17, Dmitry Dmitriev wrote: > Serguei, Dmitry, > > Thank you for the feedback! Here is an updated webrev.01; > http://cr.openjdk.java.net/~ddmitriev/8150758/webrev.01/ > > > Dmitry > > On 20.09.2016 12:48, Dmitry Samersoff wrote: >> Dmitry, >> >> Please, also change >> >> !strcmp(...) to strcmp(...) == 0 >> >> because semantically strcmp result is not a boolean false. >> >> -Dmitry >> >> On 2016-09-20 06:38, serguei.spitsyn at oracle.com wrote: >>> Hi Dmitry, >>> >>> >>> Thanks a lot for this additional test coverage and discovering new bug: >>> https://bugs.openjdk.java.net/browse/JDK-8165681 >>> >>> The tests look pretty good to me. >>> >>> A couple of minor comments. >>> >>> Dots are missed in the .c files comments. >>> >>> >>> http://cr.openjdk.java.net/~ddmitriev/8150758/webrev.00/test/serviceability/jvmti >>> >>> >>> >>> >>> 265 printf("Expecting to find '%s' class in ClassLoad events >>> duringVM start phase.\n", EXPECTED_SIGNATURE); >>> 266 if (class_in_class_load_events_vm_start == JNI_FALSE) { >>> 267 throw_exc(env, "Unable to find expected class in >>> ClassLoad events duringstart phase!\n"); >>> 268 return FAILED; >>> 269 } >>> 270 >>> 271 if (class_prepare_events_vm_start_count == 0) { >>> 272 throw_exc(env, "Didn't get ClassPrepare events in >>> start >>> phase!\n"); >>> 273 return FAILED; >>> 274 } >>> 275 >>> 276 printf("Expecting to find '%s' class in ClassPrepare >>> events >>> duringVM start phase.\n", EXPECTED_SIGNATURE); >>> 277 if (class_in_class_prepare_events_vm_start == >>> JNI_FALSE) { >>> 278 throw_exc(env, "Unable to find expected class in >>> ClassPrepare events duringstart phase!\n"); >>> 279 return FAILED; >>> 280 } >>> >>> >>> Could you, please, replace "start phase" with "early start phase"? >>> It will be more clear that the start phase mode is "early". >>> >>> 288 if (class_in_class_prepare_events_vm_start == JNI_TRUE) { >>> 289 throw_exc(env, "Class is found in ClassLoad events >>> duringVM Start phase!\n"); >>> 290 return FAILED; >>> 291 } >>> 292 >>> 293 printf("Expecting that '%s' class is absent in >>> ClassPrepare >>> events.\n", EXPECTED_SIGNATURE); >>> 294 if (class_in_class_prepare_events_vm_start == JNI_TRUE) { >>> 295 throw_exc(env, "Class is found in ClassPrepare events >>> duringVM Start phase!\n"); >>> 296 return FAILED; >>> 297 } >>> >>> Could you, please, replace "VM Start phase" with "normal VM start >>> phase"? It will be more clear that the start phase mode is "normal". >>> Thanks, Serguei On 9/19/16 05:47, Dmitry Dmitriev wrote: >>>> Hello, Please review new tests for module aware agents. There are 3 >>>> tests: 1) MAAClassFileLoadHook.java - verifies ClassFileLoadHook event >>>> with different combinations of can_generate_early_vmstart and >>>> can_generate_early_class_hook_events JVMTI capabilities. Expects to >>>> find(or not) class from java.base module in the right VM phase. 2) >>>> MAAClassLoadPrepare.java - verifies ClassLoad and ClassPrepare events >>>> with and without can_generate_early_vmstart JVMTI capability. Expects >>>> to find(or not) class from java.base module in the VM start phase. 3) >>>> MAAThreadStart.java - verifies ThreadStart event with >>>> can_generate_early_vmstart JVMTI capability. Expect to find events in >>>> the VM start phase. JBS: >>>> https://bugs.openjdk.java.net/browse/JDK-8150758 webrev.00: >>>> http://cr.openjdk.java.net/~ddmitriev/8150758/webrev.00/ >>>> Testing: >>>> RBT on all platforms Thanks, Dmitry >> > From dmitry.dmitriev at oracle.com Tue Sep 20 10:45:46 2016 From: dmitry.dmitriev at oracle.com (Dmitry Dmitriev) Date: Tue, 20 Sep 2016 13:45:46 +0300 Subject: RFR: 8150758: [TESTBUG] need jvmti tests for module aware agents In-Reply-To: <867e38fa-deea-939d-2a5b-a264b3e1cd7f@oracle.com> References: <93940e3d-48ae-9f04-045f-04618dc80d16@oracle.com> <9ee6b3cc-4032-213e-83ed-89a5d182a981@oracle.com> <867e38fa-deea-939d-2a5b-a264b3e1cd7f@oracle.com> Message-ID: <2de3da26-eaf3-2346-079d-d96dd1d3b354@oracle.com> Dmitry, Sergeui, thank you for the review! For the record, webrev.02: http://cr.openjdk.java.net/~ddmitriev/8150758/webrev.02/ Dmitry On 20.09.2016 13:39, Dmitry Samersoff wrote: > Dmitry, > > Looks good for me! > > Minor formatting nits: > > libMAAClassFileLoadHook.c > > 103: missed space after , > > libMAAClassFileLoadHook.c: > 266: extra semicolon > > -Dmitry > > On 2016-09-20 13:17, Dmitry Dmitriev wrote: >> Serguei, Dmitry, >> >> Thank you for the feedback! Here is an updated webrev.01; >> http://cr.openjdk.java.net/~ddmitriev/8150758/webrev.01/ >> >> >> Dmitry >> >> On 20.09.2016 12:48, Dmitry Samersoff wrote: >>> Dmitry, >>> >>> Please, also change >>> >>> !strcmp(...) to strcmp(...) == 0 >>> >>> because semantically strcmp result is not a boolean false. >>> >>> -Dmitry >>> >>> On 2016-09-20 06:38, serguei.spitsyn at oracle.com wrote: >>>> Hi Dmitry, >>>> >>>> >>>> Thanks a lot for this additional test coverage and discovering new bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8165681 >>>> >>>> The tests look pretty good to me. >>>> >>>> A couple of minor comments. >>>> >>>> Dots are missed in the .c files comments. >>>> >>>> >>>> http://cr.openjdk.java.net/~ddmitriev/8150758/webrev.00/test/serviceability/jvmti >>>> >>>> >>>> >>>> >>>> 265 printf("Expecting to find '%s' class in ClassLoad events >>>> duringVM start phase.\n", EXPECTED_SIGNATURE); >>>> 266 if (class_in_class_load_events_vm_start == JNI_FALSE) { >>>> 267 throw_exc(env, "Unable to find expected class in >>>> ClassLoad events duringstart phase!\n"); >>>> 268 return FAILED; >>>> 269 } >>>> 270 >>>> 271 if (class_prepare_events_vm_start_count == 0) { >>>> 272 throw_exc(env, "Didn't get ClassPrepare events in >>>> start >>>> phase!\n"); >>>> 273 return FAILED; >>>> 274 } >>>> 275 >>>> 276 printf("Expecting to find '%s' class in ClassPrepare >>>> events >>>> duringVM start phase.\n", EXPECTED_SIGNATURE); >>>> 277 if (class_in_class_prepare_events_vm_start == JNI_FALSE) { >>>> 278 throw_exc(env, "Unable to find expected class in >>>> ClassPrepare events duringstart phase!\n"); >>>> 279 return FAILED; >>>> 280 } >>>> >>>> >>>> Could you, please, replace "start phase" with "early start phase"? >>>> It will be more clear that the start phase mode is "early". >>>> >>>> 288 if (class_in_class_prepare_events_vm_start == JNI_TRUE) { >>>> 289 throw_exc(env, "Class is found in ClassLoad events >>>> duringVM Start phase!\n"); >>>> 290 return FAILED; >>>> 291 } >>>> 292 >>>> 293 printf("Expecting that '%s' class is absent in >>>> ClassPrepare >>>> events.\n", EXPECTED_SIGNATURE); >>>> 294 if (class_in_class_prepare_events_vm_start == JNI_TRUE) { >>>> 295 throw_exc(env, "Class is found in ClassPrepare events >>>> duringVM Start phase!\n"); >>>> 296 return FAILED; >>>> 297 } >>>> >>>> Could you, please, replace "VM Start phase" with "normal VM start >>>> phase"? It will be more clear that the start phase mode is "normal". >>>> Thanks, Serguei On 9/19/16 05:47, Dmitry Dmitriev wrote: >>>>> Hello, Please review new tests for module aware agents. There are 3 >>>>> tests: 1) MAAClassFileLoadHook.java - verifies ClassFileLoadHook event >>>>> with different combinations of can_generate_early_vmstart and >>>>> can_generate_early_class_hook_events JVMTI capabilities. Expects to >>>>> find(or not) class from java.base module in the right VM phase. 2) >>>>> MAAClassLoadPrepare.java - verifies ClassLoad and ClassPrepare events >>>>> with and without can_generate_early_vmstart JVMTI capability. Expects >>>>> to find(or not) class from java.base module in the VM start phase. 3) >>>>> MAAThreadStart.java - verifies ThreadStart event with >>>>> can_generate_early_vmstart JVMTI capability. Expect to find events in >>>>> the VM start phase. JBS: >>>>> https://bugs.openjdk.java.net/browse/JDK-8150758 webrev.00: >>>>> http://cr.openjdk.java.net/~ddmitriev/8150758/webrev.00/ >>>>> Testing: >>>>> RBT on all platforms Thanks, Dmitry > From gerald.thornbrugh at oracle.com Tue Sep 20 13:20:58 2016 From: gerald.thornbrugh at oracle.com (Gerald Thornbrugh) Date: Tue, 20 Sep 2016 07:20:58 -0600 Subject: [8u] RFR for JDK-6515172: Runtime.availableProcessors() ignores Linux taskset command In-Reply-To: <570b7536-1007-4abb-a500-a519e0305a09@default> References: <4a227590-41cd-4a4e-9b7d-9e8f8ad8f892@default> <11cf69a4-140b-90c4-e7af-ad0fb983f997@oracle.com> <570b7536-1007-4abb-a500-a519e0305a09@default> Message-ID: <57E137BA.3070104@oracle.com> Hi Shafi, Your change looks good. Thanks for fixing this. Jerry > Adding jdk9 code change reviewer 'Daniel Daugherty' and 'Gerald Thornbrugh'. > > Regards, > Shafi > >> -----Original Message----- >> From: Shafi Ahmad >> Sent: Monday, September 19, 2016 10:25 AM >> To: David Holmes; Chris Plummer; hotspot-dev at openjdk.java.net >> Cc: Roger Calnan >> Subject: RE: [8u] RFR for JDK-6515172: Runtime.availableProcessors() ignores >> Linux taskset command >> >> May I get the second review for this change. >> >> Regards, >> Shafi >> >>> -----Original Message----- >>> From: David Holmes >>> Sent: Friday, September 16, 2016 7:16 AM >>> To: Chris Plummer; Shafi Ahmad; hotspot-dev at openjdk.java.net >>> Subject: Re: [8u] RFR for JDK-6515172: Runtime.availableProcessors() >>> ignores Linux taskset command >>> >>> On 16/09/2016 4:17 AM, Chris Plummer wrote: >>>> You should probably include the backport of JDK-8165153 since it was >>>> introduced by JDK-6515172. >>> This is a simplified backport of 6515172 suitable for Java 8. It does >>> end up being a combination of that fix and 8165153, but you can't be a >>> backport of two bugs, and there are no distinct parts that might map >>> to each bug. That was why originally this was flagged under a separate >>> bug id. But I agreed that this could be considered a simpified backport of >> 6515172. >>> Thanks, >>> David >>> >>>> cheers, >>>> >>>> Chris >>>> >>>> On 9/15/16 2:51 AM, David Holmes wrote: >>>>> Looks good. Reviewed. >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>> On 15/09/2016 4:40 PM, Shafi Ahmad wrote: >>>>>> Hi, >>>>>> >>>>>> Please review the backport [modified] of bug: "JDK-6515172 >>>>>> Runtime.availableProcessors() ignores Linux taskset command" to >> jdk8u. >>>>>> Please note that the backport is not clean and we can't do as it is. >>>>>> Please note >>>>>> 1. The changes are made by 'Andreas Eriksson' who left Oracle. >>>>>> 2. There is difference in logging mechanism in jdk9 and jdk8 is >>>>>> different and file logTag.hpp doesn't exists in jdk8. >>>>>> 3. Newly added test pass after this change. It fails without >>>>>> this change. >>>>>> >>>>>> Webrev link: >>> http://cr.openjdk.java.net/~shshahma/8154324/webrev.02/ >>>>>> Jdk9 bug: https://bugs.openjdk.java.net/browse/JDK-6515172 >>>>>> Original patch pushed to jdk9: >>>>>> http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/c5480d4abfe4 >>>>>> >>>>>> Testing: jprt and running jtreg. >>>>>> >>>>>> Regards, >>>>>> Shafi >>>>>> From shafi.s.ahmad at oracle.com Tue Sep 20 16:42:32 2016 From: shafi.s.ahmad at oracle.com (Shafi Ahmad) Date: Tue, 20 Sep 2016 09:42:32 -0700 (PDT) Subject: [8u] RFR for JDK-6515172: Runtime.availableProcessors() ignores Linux taskset command In-Reply-To: <57E137BA.3070104@oracle.com> References: <4a227590-41cd-4a4e-9b7d-9e8f8ad8f892@default> <11cf69a4-140b-90c4-e7af-ad0fb983f997@oracle.com> <570b7536-1007-4abb-a500-a519e0305a09@default> <57E137BA.3070104@oracle.com> Message-ID: Hi Gerald, Thank you for reviewing it. Regards, Shafi > -----Original Message----- > From: Gerald Thornbrugh > Sent: Tuesday, September 20, 2016 6:51 PM > To: Shafi Ahmad > Cc: David Holmes; Chris Plummer; hotspot-dev at openjdk.java.net; Daniel > Daugherty; Roger Calnan > Subject: Re: [8u] RFR for JDK-6515172: Runtime.availableProcessors() ignores > Linux taskset command > > Hi Shafi, > > Your change looks good. > > Thanks for fixing this. > > Jerry > > Adding jdk9 code change reviewer 'Daniel Daugherty' and 'Gerald > Thornbrugh'. > > > > Regards, > > Shafi > > > >> -----Original Message----- > >> From: Shafi Ahmad > >> Sent: Monday, September 19, 2016 10:25 AM > >> To: David Holmes; Chris Plummer; hotspot-dev at openjdk.java.net > >> Cc: Roger Calnan > >> Subject: RE: [8u] RFR for JDK-6515172: Runtime.availableProcessors() > >> ignores Linux taskset command > >> > >> May I get the second review for this change. > >> > >> Regards, > >> Shafi > >> > >>> -----Original Message----- > >>> From: David Holmes > >>> Sent: Friday, September 16, 2016 7:16 AM > >>> To: Chris Plummer; Shafi Ahmad; hotspot-dev at openjdk.java.net > >>> Subject: Re: [8u] RFR for JDK-6515172: Runtime.availableProcessors() > >>> ignores Linux taskset command > >>> > >>> On 16/09/2016 4:17 AM, Chris Plummer wrote: > >>>> You should probably include the backport of JDK-8165153 since it > >>>> was introduced by JDK-6515172. > >>> This is a simplified backport of 6515172 suitable for Java 8. It > >>> does end up being a combination of that fix and 8165153, but you > >>> can't be a backport of two bugs, and there are no distinct parts > >>> that might map to each bug. That was why originally this was flagged > >>> under a separate bug id. But I agreed that this could be considered > >>> a simpified backport of > >> 6515172. > >>> Thanks, > >>> David > >>> > >>>> cheers, > >>>> > >>>> Chris > >>>> > >>>> On 9/15/16 2:51 AM, David Holmes wrote: > >>>>> Looks good. Reviewed. > >>>>> > >>>>> Thanks, > >>>>> David > >>>>> > >>>>> On 15/09/2016 4:40 PM, Shafi Ahmad wrote: > >>>>>> Hi, > >>>>>> > >>>>>> Please review the backport [modified] of bug: "JDK-6515172 > >>>>>> Runtime.availableProcessors() ignores Linux taskset command" to > >> jdk8u. > >>>>>> Please note that the backport is not clean and we can't do as it is. > >>>>>> Please note > >>>>>> 1. The changes are made by 'Andreas Eriksson' who left Oracle. > >>>>>> 2. There is difference in logging mechanism in jdk9 and jdk8 > >>>>>> is different and file logTag.hpp doesn't exists in jdk8. > >>>>>> 3. Newly added test pass after this change. It fails without > >>>>>> this change. > >>>>>> > >>>>>> Webrev link: > >>> http://cr.openjdk.java.net/~shshahma/8154324/webrev.02/ > >>>>>> Jdk9 bug: https://bugs.openjdk.java.net/browse/JDK-6515172 > >>>>>> Original patch pushed to jdk9: > >>>>>> http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/c5480d4abfe4 > >>>>>> > >>>>>> Testing: jprt and running jtreg. > >>>>>> > >>>>>> Regards, > >>>>>> Shafi > >>>>>> > From serguei.spitsyn at oracle.com Tue Sep 20 18:33:09 2016 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 20 Sep 2016 11:33:09 -0700 Subject: RFR (S): 8147943 jvmti.h generated with GPL header Message-ID: Please, review the fix for: https://bugs.openjdk.java.net/browse/JDK-8147943 Webrev: http://cr.openjdk.java.net/~sspitsyn/webrevs/2016/hotspot/8147943-jvmti-header.hs1/ Summary: The problem is that the build/*/hotspot/variant-server/gensrc/jvmvtifiles/jvmti.h is currently generated with the GPL copyright comment. The JDK version of the header that we ship must havethe GPL + "Classpath" exception. So that the file is taken from the version that is checked into the jdk repository: jdk/src/java.base/share/native/include/jvmti.h. Now, the checked-in version of the jvmti.h in the jdk repository is manually updated from the jvmvtifiles/jvmti.h with the copyright comment replacement (very inconvenient). Replacement of the copyright comment in the jvmvtifiles/jvmti.h was discussed Alan and Iris, and they are Ok with the change. The jvmti.h is generated from the hotspot/src/share/vm/prims/jvmti.xml with the XSL scripts, and its copyright comment is inherited from the jvmti.xml. The fix is to update the XSL scripts to generate the GPL+CP copyright comment. The only part that is still taken from the jvmti.xml is the copyright year line. There is a separate bug that targets automatic installation of the generated jvmti.h and potential removal of the checked-in version from the jdk repository: https://bugs.openjdk.java.net/browse/JDK-8063154 Checked in jvmti.h not in sync with generated jvmti.h Testing: Checked the copyright comment in the generated gensrc/jvmvtifiles.jvmti.h. Thanks, Serguei From daniel.daugherty at oracle.com Tue Sep 20 22:21:32 2016 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 20 Sep 2016 16:21:32 -0600 Subject: RFR (S): 8147943 jvmti.h generated with GPL header In-Reply-To: References: Message-ID: On 9/20/16 12:33 PM, serguei.spitsyn at oracle.com wrote: > Please, review the fix for: > https://bugs.openjdk.java.net/browse/JDK-8147943 > > > Webrev: > http://cr.openjdk.java.net/~sspitsyn/webrevs/2016/hotspot/8147943-jvmti-header.hs1/ > src/share/vm/prims/jvmti.xml No comments. src/share/vm/prims/jvmtiH.xsl No comments. src/share/vm/prims/jvmtiLib.xsl No comments. Still can't say that I'm really any good at reviewing XML or XSL. As long as you are happy with the new generated file... :-) Thumbs up... Dan > > > Summary: > The problem is that the > build/*/hotspot/variant-server/gensrc/jvmvtifiles/jvmti.h > is currently generated with the GPL copyright comment. > The JDK version of the header that we ship must havethe GPL + > "Classpath" exception. > So that the file is taken from the version that is checked into the > jdk repository: > jdk/src/java.base/share/native/include/jvmti.h. > > Now, the checked-in version of the jvmti.h in the jdk repository is > manually updated > from the jvmvtifiles/jvmti.h with the copyright comment replacement > (very inconvenient). > > Replacement of the copyright comment in the jvmvtifiles/jvmti.h was > discussed > Alan and Iris, and they are Ok with the change. > > The jvmti.h is generated from the > hotspot/src/share/vm/prims/jvmti.xml with the XSL scripts, > and its copyright comment is inherited from the jvmti.xml. > The fix is to update the XSL scripts to generate the GPL+CP > copyright comment. > The only part that is still taken from the jvmti.xml is the > copyright year line. > > There is a separate bug that targets automatic installation of the > generated jvmti.h > and potential removal of the checked-in version from the jdk > repository: > https://bugs.openjdk.java.net/browse/JDK-8063154 > Checked in jvmti.h not in sync with generated jvmti.h > > Testing: > Checked the copyright comment in the generated > gensrc/jvmvtifiles.jvmti.h. > > > Thanks, > Serguei From serguei.spitsyn at oracle.com Tue Sep 20 23:44:59 2016 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 20 Sep 2016 16:44:59 -0700 Subject: RFR (S): 8147943 jvmti.h generated with GPL header In-Reply-To: References: Message-ID: <382156c0-d7ea-9e5f-3f0e-8d24ab9caf6e@oracle.com> On 9/20/16 15:21, Daniel D. Daugherty wrote: > On 9/20/16 12:33 PM, serguei.spitsyn at oracle.com wrote: >> Please, review the fix for: >> https://bugs.openjdk.java.net/browse/JDK-8147943 >> >> >> Webrev: >> http://cr.openjdk.java.net/~sspitsyn/webrevs/2016/hotspot/8147943-jvmti-header.hs1/ >> > > src/share/vm/prims/jvmti.xml > No comments. > > src/share/vm/prims/jvmtiH.xsl > No comments. > > src/share/vm/prims/jvmtiLib.xsl > No comments. > > Still can't say that I'm really any good at reviewing XML or XSL. I had a good practice but still can't say that. :-) > As long as you are happy with the new generated file... :-) I'm happy as the generated comment is correct and this version (I wrote two of them) is simpler than previous one. > > Thumbs up... Thank you so much, Dan! Serguei > > Dan > > >> >> >> Summary: >> The problem is that the >> build/*/hotspot/variant-server/gensrc/jvmvtifiles/jvmti.h >> is currently generated with the GPL copyright comment. >> The JDK version of the header that we ship must havethe GPL + >> "Classpath" exception. >> So that the file is taken from the version that is checked into the >> jdk repository: >> jdk/src/java.base/share/native/include/jvmti.h. >> >> Now, the checked-in version of the jvmti.h in the jdk repository is >> manually updated >> from the jvmvtifiles/jvmti.h with the copyright comment replacement >> (very inconvenient). >> >> Replacement of the copyright comment in the jvmvtifiles/jvmti.h was >> discussed >> Alan and Iris, and they are Ok with the change. >> >> The jvmti.h is generated from the >> hotspot/src/share/vm/prims/jvmti.xml with the XSL scripts, >> and its copyright comment is inherited from the jvmti.xml. >> The fix is to update the XSL scripts to generate the GPL+CP >> copyright comment. >> The only part that is still taken from the jvmti.xml is the >> copyright year line. >> >> There is a separate bug that targets automatic installation of the >> generated jvmti.h >> and potential removal of the checked-in version from the jdk >> repository: >> https://bugs.openjdk.java.net/browse/JDK-8063154 >> Checked in jvmti.h not in sync with generated jvmti.h >> >> Testing: >> Checked the copyright comment in the generated >> gensrc/jvmvtifiles.jvmti.h. >> >> >> Thanks, >> Serguei > From Alan.Bateman at oracle.com Wed Sep 21 02:43:40 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 20 Sep 2016 19:43:40 -0700 Subject: RFR (S): 8147943 jvmti.h generated with GPL header In-Reply-To: References: Message-ID: <5b258094-5317-0bce-2f8e-9e5f53b99e7e@oracle.com> On 20/09/2016 11:33, serguei.spitsyn at oracle.com wrote: > Please, review the fix for: > https://bugs.openjdk.java.net/browse/JDK-8147943 > > > Webrev: > http://cr.openjdk.java.net/~sspitsyn/webrevs/2016/hotspot/8147943-jvmti-header.hs1/ > > > Summary: > The problem is that the > build/*/hotspot/variant-server/gensrc/jvmvtifiles/jvmti.h > is currently generated with the GPL copyright comment. > The JDK version of the header that we ship must havethe GPL + > "Classpath" exception. > So that the file is taken from the version that is checked into the > jdk repository: > jdk/src/java.base/share/native/include/jvmti.h. The patch looks okay to me but I just wonder if it would be worth trying to consume license template from /make/templates rather than having it in jvmtiLib.xsl - you may have looked at this ready and discounted it. -Alan From serguei.spitsyn at oracle.com Wed Sep 21 08:40:24 2016 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 21 Sep 2016 01:40:24 -0700 Subject: RFR (S): 8147943 jvmti.h generated with GPL header In-Reply-To: <5b258094-5317-0bce-2f8e-9e5f53b99e7e@oracle.com> References: <5b258094-5317-0bce-2f8e-9e5f53b99e7e@oracle.com> Message-ID: Hi Alan, It is a nice suggestion to use the /make/templates/gpl-cp-header. But it does not seem easy to reach it from the XSL processing. So, I left the fix as it is. Thanks, Serguei On 9/20/16 19:43, Alan Bateman wrote: > On 20/09/2016 11:33, serguei.spitsyn at oracle.com wrote: >> Please, review the fix for: >> https://bugs.openjdk.java.net/browse/JDK-8147943 >> >> >> Webrev: >> http://cr.openjdk.java.net/~sspitsyn/webrevs/2016/hotspot/8147943-jvmti-header.hs1/ >> >> >> Summary: >> The problem is that the >> build/*/hotspot/variant-server/gensrc/jvmvtifiles/jvmti.h >> is currently generated with the GPL copyright comment. >> The JDK version of the header that we ship must have the GPL + >> "Classpath" exception. >> So that the file is taken from the version that is checked into the >> jdk repository: >> jdk/src/java.base/share/native/include/jvmti.h. > The patch looks okay to me but I just wonder if it would be worth > trying to consume license template from /make/templates rather > than having it in jvmtiLib.xsl - you may have looked at this ready and > discounted it. > > -Alan > From adinn at redhat.com Wed Sep 21 09:35:22 2016 From: adinn at redhat.com (Andrew Dinn) Date: Wed, 21 Sep 2016 10:35:22 +0100 Subject: [aarch64-port-dev ] RFR: AArch64: SEGV in stub code cipherBlockChaining_decryptAESCrypt In-Reply-To: References: Message-ID: <92c29040-7a82-3b73-7079-6b23b781dcd2@redhat.com> On 20/09/16 11:09, Ningsheng Jian wrote: > Jtreg test jdk/test/com/sun/crypto/provider/Cipher/CTS/CTSMode.java > failed with SIGSEGV on option "-Xcomp -XX:-TieredCompilation". > > The crash happens at stub code cipherBlockChaining_decryptAESCrypt. > > Checking the jdk source code CipherTextStealing.decryptFinal and its > callee CipherBlockChaining.decrypt/implDecrypt, we can see that > cipherLen which is passed to the stub code can be 0. The unexpected > len_reg value makes the load from invalid address. > > The following patch could fix this issue: > > http://people.linaro.org/~ningsheng.jian/webrev/cbc-stub-fix/webrev.00/ > > It aligns with the Java code implementation and passed JTreg tests. To > make code consistent, I also update the encrypt part. > > Could someone please help to take a look at it? I think this patch looks like it will suffice to fix the AArch64 code. However, I don't understand why this is only an AArch64 issue. For example, it looks to me as if the x86_64 code is also susceptible to the same problem should input value 0 be passed in len_reg (c_rarg4). So, this might need fixing for all archs. Alternatively, it might be better fixed in the jdk (Java) code. Could someone from the hotspot compiler/runtime team confirm: i) whether an input len_reg value of zero will cause a problem on x86 (or, indeed, other archs)? ii) whether that case should be caught in Java code or stub code? regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From david.holmes at oracle.com Wed Sep 21 10:16:17 2016 From: david.holmes at oracle.com (David Holmes) Date: Wed, 21 Sep 2016 20:16:17 +1000 Subject: [8u] RFR for JDK-6515172: Runtime.availableProcessors() ignores Linux taskset command In-Reply-To: References: Message-ID: <2f72e2f5-4a4c-8074-b392-7c139af7dbb0@oracle.com> For the record I confused the bug Chris was referring to with 8148987. A backport of 8165153 is needed. David ----- David Holmes david.holmes at oracle.com Fri Sep 16 01:45:32 UTC 2016 On 16/09/2016 4:17 AM, Chris Plummer wrote: > You should probably include the backport of JDK-8165153 since it was > introduced by JDK-6515172. This is a simplified backport of 6515172 suitable for Java 8. It does end up being a combination of that fix and 8165153, but you can't be a backport of two bugs, and there are no distinct parts that might map to each bug. That was why originally this was flagged under a separate bug id. But I agreed that this could be considered a simpified backport of 6515172. Thanks, David > cheers, > > Chris > > On 9/15/16 2:51 AM, David Holmes wrote: >> Looks good. Reviewed. >> >> Thanks, >> David >> >> On 15/09/2016 4:40 PM, Shafi Ahmad wrote: >>> Hi, >>> >>> Please review the backport [modified] of bug: "JDK-6515172 >>> Runtime.availableProcessors() ignores Linux taskset command" to jdk8u. >>> >>> Please note that the backport is not clean and we can't do as it is. >>> Please note >>> 1. The changes are made by 'Andreas Eriksson' who left Oracle. >>> 2. There is difference in logging mechanism in jdk9 and jdk8 is >>> different and file logTag.hpp doesn't exists in jdk8. >>> 3. Newly added test pass after this change. It fails without this >>> change. >>> >>> Webrev link: http://cr.openjdk.java.net/~shshahma/8154324/webrev.02/ >>> Jdk9 bug: https://bugs.openjdk.java.net/browse/JDK-6515172 >>> Original patch pushed to jdk9: >>> http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/c5480d4abfe4 >>> >>> Testing: jprt and running jtreg. >>> >>> Regards, >>> Shafi >>> > From Alan.Burlison at oracle.com Wed Sep 21 11:20:08 2016 From: Alan.Burlison at oracle.com (Alan Burlison) Date: Wed, 21 Sep 2016 12:20:08 +0100 Subject: RFR for JDK-8165482 java in ldoms, with cpu-arch=generic has problems In-Reply-To: <4bfdead9-f0d9-f67e-e049-d48af5f19863@oracle.com> References: <8046b263-8826-aa34-083a-9a83e105770f@oracle.com> <0a2a66db-f939-196f-bf23-a6a3df46b2d3@oracle.com> <4bfdead9-f0d9-f67e-e049-d48af5f19863@oracle.com> Message-ID: On 20/09/2016 08:42, Erik Joelsson wrote: >> What is the devinfo library? Is it part of the normal Solaris >> installation, or does it need to be installed specifically? Is it >> available in our official build toolkits? > I did a bit of digging. It's part "system/libraries" so should be pretty > standard. That package is in the devkit and I verified that > libdevinfo.so is there too. It's a standard system library, the manpage is here: http://docs.oracle.com/cd/E53394_01/html/E54772/libdevinfo-3lib.html -- Alan Burlison -- From igor.ignatyev at oracle.com Wed Sep 21 21:43:24 2016 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 22 Sep 2016 00:43:24 +0300 Subject: RFR(XXS) : 8166483 : gtest fmw should be updated to support null detection on SS >= 12u4 Message-ID: <24940507-7492-4FF1-86A3-F0399D126DA5@oracle.com> http://cr.openjdk.java.net/~iignatyev/8166483/webrev.00/ > 3 lines changed: 1 ins; 0 del; 2 mod; Hi all, could you please review this tiny fix which enables null detection functionality in gtest on solaris if it?s been build by Solaris Studio >= 12.4? Due to disabled null detection, we have to explicitly cast NULL to a type of an actual value, i.e. instead of nice and clear 'ASSERT_EQ(NULL, myPointer)? we have to write ?ASSERT_EQ((MyClass*) NULL, myPointer)?. Reviewing 8165433[1], we came to the conclusion that ASSERT_EQ(NULL, myPointer) is what we really want. W/O this patch, the latest version of 8165433[2] fails during build on solaris. JBS: https://bugs.openjdk.java.net/browse/JDK-8166483 webrev: http://cr.openjdk.java.net/~iignatyev/8166483/webrev.00/ testing: I?ve run jprt w/ 8165433 integrated [1] http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2016-September/020982.html [2] http://cr.openjdk.java.net/~kzhaldyb/webrevs/JDK-8165433/webrev.03/ Thanks, ? Igor From kirill.zhaldybin at oracle.com Wed Sep 21 22:21:47 2016 From: kirill.zhaldybin at oracle.com (Kirill Zhaldybin) Date: Thu, 22 Sep 2016 01:21:47 +0300 Subject: RFR(XXS) : 8166483 : gtest fmw should be updated to support null detection on SS >= 12u4 In-Reply-To: <24940507-7492-4FF1-86A3-F0399D126DA5@oracle.com> References: <24940507-7492-4FF1-86A3-F0399D126DA5@oracle.com> Message-ID: <57E307FB.203@oracle.com> Igor, Looks good to me, thank you for fixing this. Regards, Kirill On 22.09.2016 00:43, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev/8166483/webrev.00/ >> 3 lines changed: 1 ins; 0 del; 2 mod; > > Hi all, > > could you please review this tiny fix which enables null detection functionality in gtest on solaris if it?s been build by Solaris Studio >= 12.4? Due to disabled null detection, we have to explicitly cast NULL to a type of an actual value, i.e. instead of nice and clear 'ASSERT_EQ(NULL, myPointer)? we have to write ?ASSERT_EQ((MyClass*) NULL, myPointer)?. Reviewing 8165433[1], we came to the conclusion that ASSERT_EQ(NULL, myPointer) is what we really want. W/O this patch, the latest version of 8165433[2] fails during build on solaris. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8166483 > webrev: http://cr.openjdk.java.net/~iignatyev/8166483/webrev.00/ > testing: I?ve run jprt w/ 8165433 integrated > > [1] http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2016-September/020982.html > [2] http://cr.openjdk.java.net/~kzhaldyb/webrevs/JDK-8165433/webrev.03/ > > Thanks, > ? Igor > From igor.ignatyev at oracle.com Wed Sep 21 22:29:09 2016 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 22 Sep 2016 01:29:09 +0300 Subject: RFR(XXS) : 8166483 : gtest fmw should be updated to support null detection on SS >= 12u4 In-Reply-To: <57E307FB.203@oracle.com> References: <24940507-7492-4FF1-86A3-F0399D126DA5@oracle.com> <57E307FB.203@oracle.com> Message-ID: <42692CAB-9CF4-4F8A-B5EA-F0DCCC921080@oracle.com> Kirill, thank you for reviewing the patch. Cheers, ? Igor > On Sep 22, 2016, at 1:21 AM, Kirill Zhaldybin wrote: > > Igor, > > Looks good to me, thank you for fixing this. > > Regards, Kirill > > On 22.09.2016 00:43, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev/8166483/webrev.00/ >>> 3 lines changed: 1 ins; 0 del; 2 mod; >> >> Hi all, >> >> could you please review this tiny fix which enables null detection functionality in gtest on solaris if it?s been build by Solaris Studio >= 12.4? Due to disabled null detection, we have to explicitly cast NULL to a type of an actual value, i.e. instead of nice and clear 'ASSERT_EQ(NULL, myPointer)? we have to write ?ASSERT_EQ((MyClass*) NULL, myPointer)?. Reviewing 8165433[1], we came to the conclusion that ASSERT_EQ(NULL, myPointer) is what we really want. W/O this patch, the latest version of 8165433[2] fails during build on solaris. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8166483 >> webrev: http://cr.openjdk.java.net/~iignatyev/8166483/webrev.00/ >> testing: I?ve run jprt w/ 8165433 integrated >> >> [1] http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2016-September/020982.html >> [2] http://cr.openjdk.java.net/~kzhaldyb/webrevs/JDK-8165433/webrev.03/ >> >> Thanks, >> ? Igor >> > From david.holmes at oracle.com Thu Sep 22 00:59:39 2016 From: david.holmes at oracle.com (David Holmes) Date: Thu, 22 Sep 2016 10:59:39 +1000 Subject: RFR(XXS) : 8166483 : gtest fmw should be updated to support null detection on SS >= 12u4 In-Reply-To: <24940507-7492-4FF1-86A3-F0399D126DA5@oracle.com> References: <24940507-7492-4FF1-86A3-F0399D126DA5@oracle.com> Message-ID: <00098a17-0e90-4d88-c8c0-6830c1ea2123@oracle.com> Hi Igor, On 22/09/2016 7:43 AM, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev/8166483/webrev.00/ >> 3 lines changed: 1 ins; 0 del; 2 mod; > > Hi all, > > could you please review this tiny fix which enables null detection functionality in gtest on solaris if it?s been build by Solaris Studio >= 12.4? Due to disabled null detection, we have to explicitly cast NULL to a type of an actual value, i.e. instead of nice and clear 'ASSERT_EQ(NULL, myPointer)? we have to write ?ASSERT_EQ((MyClass*) NULL, myPointer)?. Reviewing 8165433[1], we came to the conclusion that ASSERT_EQ(NULL, myPointer) is what we really want. W/O this patch, the latest version of 8165433[2] fails during build on solaris. To be clear we are expecting/requiring to always build with SS12u4 or later to allow this fix to enable changes to the test logic - right? Thanks, David > JBS: https://bugs.openjdk.java.net/browse/JDK-8166483 > webrev: http://cr.openjdk.java.net/~iignatyev/8166483/webrev.00/ > testing: I?ve run jprt w/ 8165433 integrated > > [1] http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2016-September/020982.html > [2] http://cr.openjdk.java.net/~kzhaldyb/webrevs/JDK-8165433/webrev.03/ > > Thanks, > ? Igor > From igor.ignatyev at oracle.com Thu Sep 22 04:57:24 2016 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 22 Sep 2016 07:57:24 +0300 Subject: RFR(XXS) : 8166483 : gtest fmw should be updated to support null detection on SS >= 12u4 In-Reply-To: <00098a17-0e90-4d88-c8c0-6830c1ea2123@oracle.com> References: <24940507-7492-4FF1-86A3-F0399D126DA5@oracle.com> <00098a17-0e90-4d88-c8c0-6830c1ea2123@oracle.com> Message-ID: Hi David, > To be clear we are expecting/requiring to always build with SS12u4 or later to allow this fix to enable changes to the test logic - right? right. null detection is still disabled for SS12u3 or before, so we will get a compile-time error like 'The operation "const long == MyClass*" is illegal.? trying to compile 'ASSERT_EQ(NULL, myPointer)?. ? Igor > On Sep 22, 2016, at 3:59 AM, David Holmes wrote: > > Hi Igor, > > On 22/09/2016 7:43 AM, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev/8166483/webrev.00/ >>> 3 lines changed: 1 ins; 0 del; 2 mod; >> >> Hi all, >> >> could you please review this tiny fix which enables null detection functionality in gtest on solaris if it?s been build by Solaris Studio >= 12.4? Due to disabled null detection, we have to explicitly cast NULL to a type of an actual value, i.e. instead of nice and clear 'ASSERT_EQ(NULL, myPointer)? we have to write ?ASSERT_EQ((MyClass*) NULL, myPointer)?. Reviewing 8165433[1], we came to the conclusion that ASSERT_EQ(NULL, myPointer) is what we really want. W/O this patch, the latest version of 8165433[2] fails during build on solaris. > > To be clear we are expecting/requiring to always build with SS12u4 or later to allow this fix to enable changes to the test logic - right? > > Thanks, > David > > >> JBS: https://bugs.openjdk.java.net/browse/JDK-8166483 >> webrev: http://cr.openjdk.java.net/~iignatyev/8166483/webrev.00/ >> testing: I?ve run jprt w/ 8165433 integrated >> >> [1] http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2016-September/020982.html >> [2] http://cr.openjdk.java.net/~kzhaldyb/webrevs/JDK-8165433/webrev.03/ >> >> Thanks, >> ? Igor >> From david.holmes at oracle.com Thu Sep 22 05:53:29 2016 From: david.holmes at oracle.com (David Holmes) Date: Thu, 22 Sep 2016 15:53:29 +1000 Subject: RFR(XXS) : 8166483 : gtest fmw should be updated to support null detection on SS >= 12u4 In-Reply-To: References: <24940507-7492-4FF1-86A3-F0399D126DA5@oracle.com> <00098a17-0e90-4d88-c8c0-6830c1ea2123@oracle.com> Message-ID: On 22/09/2016 2:57 PM, Igor Ignatyev wrote: > Hi David, > >> To be clear we are expecting/requiring to always build with SS12u4 or later to allow this fix to enable changes to the test logic - right? > > right. null detection is still disabled for SS12u3 or before, so we will get a compile-time error like 'The operation "const long == MyClass*" is illegal.? trying to compile 'ASSERT_EQ(NULL, myPointer)?. Ok. Given we do compile with SS12u4 this seems okay then. Thanks, David > ? Igor > >> On Sep 22, 2016, at 3:59 AM, David Holmes wrote: >> >> Hi Igor, >> >> On 22/09/2016 7:43 AM, Igor Ignatyev wrote: >>> http://cr.openjdk.java.net/~iignatyev/8166483/webrev.00/ >>>> 3 lines changed: 1 ins; 0 del; 2 mod; >>> >>> Hi all, >>> >>> could you please review this tiny fix which enables null detection functionality in gtest on solaris if it?s been build by Solaris Studio >= 12.4? Due to disabled null detection, we have to explicitly cast NULL to a type of an actual value, i.e. instead of nice and clear 'ASSERT_EQ(NULL, myPointer)? we have to write ?ASSERT_EQ((MyClass*) NULL, myPointer)?. Reviewing 8165433[1], we came to the conclusion that ASSERT_EQ(NULL, myPointer) is what we really want. W/O this patch, the latest version of 8165433[2] fails during build on solaris. >> >> To be clear we are expecting/requiring to always build with SS12u4 or later to allow this fix to enable changes to the test logic - right? >> >> Thanks, >> David >> >> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8166483 >>> webrev: http://cr.openjdk.java.net/~iignatyev/8166483/webrev.00/ >>> testing: I?ve run jprt w/ 8165433 integrated >>> >>> [1] http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2016-September/020982.html >>> [2] http://cr.openjdk.java.net/~kzhaldyb/webrevs/JDK-8165433/webrev.03/ >>> >>> Thanks, >>> ? Igor >>> > From igor.ignatyev at oracle.com Thu Sep 22 07:24:47 2016 From: igor.ignatyev at oracle.com (=?utf-8?B?aWdvci5pZ25hdHlldkBvcmFjbGUuY29t?=) Date: Thu, 22 Sep 2016 10:24:47 +0300 Subject: =?utf-8?B?UmU6IFJGUihYWFMpIDogODE2NjQ4MyA6IGd0ZXN0IGZtdyBzaG91bGQgYmUgdXBk?= =?utf-8?B?YXRlZCB0byBzdXBwb3J0IG51bGwgZGV0ZWN0aW9uIG9uIFNTID49IDEydTQ=?= Message-ID: <201609220724.u8M7OlHo021836@aserv0121.oracle.com> David, As you mentioned, we use SS12u4 now, and I doubt we will use earlier releases of SS in JDK 9 and beyond. Hence I don't think it will be an issue for us. I can consider it as Reviewed by you, right? -- II ----- Reply message ----- From: "David Holmes" To: "Igor Ignatyev" Cc: "hotspot-dev at openjdk.java.net Developers" Subject: RFR(XXS) : 8166483 : gtest fmw should be updated to support null detection on SS >= 12u4 Date: Thu, Sep 22, 2016 08:53 On 22/09/2016 2:57 PM, Igor Ignatyev wrote: > Hi David, > >> To be clear we are expecting/requiring to always build with SS12u4 or later to allow this fix to enable changes to the test logic - right? > > right. null detection is still disabled for SS12u3 or before, so we will get a compile-time error like 'The operation "const long == MyClass*" is illegal.? trying to compile 'ASSERT_EQ(NULL, myPointer)?. Ok. Given we do compile with SS12u4 this seems okay then. Thanks, David > ? Igor > >> On Sep 22, 2016, at 3:59 AM, David Holmes wrote: >> >> Hi Igor, >> >> On 22/09/2016 7:43 AM, Igor Ignatyev wrote: >>> http://cr.openjdk.java.net/~iignatyev/8166483/webrev.00/ >>>> 3 lines changed: 1 ins; 0 del; 2 mod; >>> >>> Hi all, >>> >>> could you please review this tiny fix which enables null detection functionality in gtest on solaris if it?s been build by Solaris Studio >= 12.4? Due to disabled null detection, we have to explicitly cast NULL to a type of an actual value, i.e. instead of nice and clear 'ASSERT_EQ(NULL, myPointer)? we have to write ?ASSERT_EQ((MyClass*) NULL, myPointer)?. Reviewing 8165433[1], we came to the conclusion that ASSERT_EQ(NULL, myPointer) is what we really want. W/O this patch, the latest version of 8165433[2] fails during build on solaris. >> >> To be clear we are expecting/requiring to always build with SS12u4 or later to allow this fix to enable changes to the test logic - right? >> >> Thanks, >> David >> >> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8166483 >>> webrev: http://cr.openjdk.java.net/~iignatyev/8166483/webrev.00/ >>> testing: I?ve run jprt w/ 8165433 integrated >>> >>> [1] http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2016-September/020982.html >>> [2] http://cr.openjdk.java.net/~kzhaldyb/webrevs/JDK-8165433/webrev.03/ >>> >>> Thanks, >>> ? Igor >>> > From martin.walsh at oracle.com Thu Sep 22 09:34:51 2016 From: martin.walsh at oracle.com (Martin Walsh) Date: Thu, 22 Sep 2016 10:34:51 +0100 Subject: RFR for JDK-8165482 java in ldoms, with cpu-arch=generic has problems In-Reply-To: <8046b263-8826-aa34-083a-9a83e105770f@oracle.com> References: <8046b263-8826-aa34-083a-9a83e105770f@oracle.com> Message-ID: <4f2a4187-a492-1c0b-71ed-a79afe7d56d6@oracle.com> Polite reminder. Would anybody be able to review this? Webrev: http://cr.openjdk.java.net/~mwalsh/JDK-8165482/ Bug: https://bugs.openjdk.java.net/browse/JDK-8165482 Thanks, Martin On 19/09/2016 15:16, Martin Walsh wrote: > Could I get a code review for the following bug: > > JDK-8165482 java in ldoms, with cpu-arch=generic has problems > > > Webrev of the changes is available here: > > http://cr.openjdk.java.net/~mwalsh/JDK-8165482/ > > > Thanks, > > Martin From david.holmes at oracle.com Thu Sep 22 12:39:40 2016 From: david.holmes at oracle.com (David Holmes) Date: Thu, 22 Sep 2016 22:39:40 +1000 Subject: RFR(XXS) : 8166483 : gtest fmw should be updated to support null detection on SS >= 12u4 Message-ID: <64e3ec69-9f14-5665-6a7d-c1703acb0da1@oracle.com> On 22/09/2016 5:24 PM, igor.ignatyev at oracle.com wrote: > David, > > As you mentioned, we use SS12u4 now, and I doubt we will use earlier > releases of SS in JDK 9 and beyond. Hence I don't think it will be an > issue for us. > > I can consider it as Reviewed by you, right? Yes. Thanks, David > -- II > > ----- Reply message ----- > From: "David Holmes" > To: "Igor Ignatyev" > Cc: "hotspot-dev at openjdk.java.net Developers" > Subject: RFR(XXS) : 8166483 : gtest fmw should be updated to support > null detection on SS >= 12u4 > Date: Thu, Sep 22, 2016 08:53 > > On 22/09/2016 2:57 PM, Igor Ignatyev wrote: >> Hi David, >> >>> To be clear we are expecting/requiring to always build with SS12u4 or later to allow this fix to enable changes to the test logic - right? >> >> right. null detection is still disabled for SS12u3 or before, so we will get a compile-time error like 'The operation "const long == MyClass*" is illegal.? trying to compile 'ASSERT_EQ(NULL, myPointer)?. > > Ok. Given we do compile with SS12u4 this seems okay then. > > Thanks, > David > >> ? Igor >> >>> On Sep 22, 2016, at 3:59 AM, David Holmes wrote: >>> >>> Hi Igor, >>> >>> On 22/09/2016 7:43 AM, Igor Ignatyev wrote: >>>> http://cr.openjdk.java.net/~iignatyev/8166483/webrev.00/ >>>>> 3 lines changed: 1 ins; 0 del; 2 mod; >>>> >>>> Hi all, >>>> >>>> could you please review this tiny fix which enables null detection functionality in gtest on solaris if it?s been build by Solaris Studio >= 12.4? Due to disabled null detection, we have to explicitly cast NULL to a type of an actual value, i.e. instead of nice and clear 'ASSERT_EQ(NULL, myPointer)? we have to write ?ASSERT_EQ((MyClass*) NULL, myPointer)?. Reviewing 8165433[1], we came to the conclusion that ASSERT_EQ(NULL, myPointer) is what we really want. W/O this patch, the latest version of 8165433[2] fails during build on solaris. >>> >>> To be clear we are expecting/requiring to always build with SS12u4 or later to allow this fix to enable changes to the test logic - right? >>> >>> Thanks, >>> David >>> >>> >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8166483 >>>> webrev: http://cr.openjdk.java.net/~iignatyev/8166483/webrev.00/ >>>> testing: I?ve run jprt w/ 8165433 integrated >>>> >>>> [1] http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2016-September/020982.html >>>> [2] http://cr.openjdk.java.net/~kzhaldyb/webrevs/JDK-8165433/webrev.03/ >>>> >>>> Thanks, >>>> ? Igor >>>> >> > From brent.christian at oracle.com Thu Sep 22 16:04:56 2016 From: brent.christian at oracle.com (Brent Christian) Date: Thu, 22 Sep 2016 09:04:56 -0700 Subject: RFR 8166501 : compilation error in stackwalk.cpp on some gccs Message-ID: <57E40128.8010409@oracle.com> Hi, Looks like my 8165372 change broke the slowdebug build. Please review my fix (which also breaks up a pretty long line): --- a/src/share/vm/prims/stackwalk.cpp +++ b/src/share/vm/prims/stackwalk.cpp @@ -331,10 +331,12 @@ assert (use_frames_array(mode), "Bad mode for get live frame"); RegisterMap regMap(jt, true); LiveFrameStream stream(jt, ®Map); - return fetchFirstBatch(stream, stackStream, mode, skip_frames, frame_count, start_index, frames_array, CHECK_NULL); + return fetchFirstBatch(stream, stackStream, mode, skip_frames, frame_count, + start_index, frames_array, THREAD); } else { JavaFrameStream stream(jt, mode); - return fetchFirstBatch(stream, stackStream, mode, skip_frames, frame_count, start_index, frames_array, CHECK_NULL); + return fetchFirstBatch(stream, stackStream, mode, skip_frames, frame_count, + start_index, frames_array, THREAD); } } Thanks! -Brent From coleen.phillimore at oracle.com Thu Sep 22 17:07:07 2016 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Thu, 22 Sep 2016 13:07:07 -0400 Subject: RFR 8166501 : compilation error in stackwalk.cpp on some gccs In-Reply-To: <57E40128.8010409@oracle.com> References: <57E40128.8010409@oracle.com> Message-ID: Looks good! Thanks! Coleen On 9/22/16 12:04 PM, Brent Christian wrote: > Hi, > > Looks like my 8165372 change broke the slowdebug build. Please review > my fix (which also breaks up a pretty long line): > > --- a/src/share/vm/prims/stackwalk.cpp > +++ b/src/share/vm/prims/stackwalk.cpp > @@ -331,10 +331,12 @@ > assert (use_frames_array(mode), "Bad mode for get live frame"); > RegisterMap regMap(jt, true); > LiveFrameStream stream(jt, ®Map); > - return fetchFirstBatch(stream, stackStream, mode, skip_frames, > frame_count, start_index, frames_array, CHECK_NULL); > + return fetchFirstBatch(stream, stackStream, mode, skip_frames, > frame_count, > + start_index, frames_array, THREAD); > } else { > JavaFrameStream stream(jt, mode); > - return fetchFirstBatch(stream, stackStream, mode, skip_frames, > frame_count, start_index, frames_array, CHECK_NULL); > + return fetchFirstBatch(stream, stackStream, mode, skip_frames, > frame_count, > + start_index, frames_array, THREAD); > } > } > > Thanks! > -Brent > From coleen.phillimore at oracle.com Thu Sep 22 17:08:04 2016 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Thu, 22 Sep 2016 13:08:04 -0400 Subject: RFR 8166501 : compilation error in stackwalk.cpp on some gccs In-Reply-To: <57E40128.8010409@oracle.com> References: <57E40128.8010409@oracle.com> Message-ID: <5ee3e0fc-9aa7-f247-1b86-2760dad932eb@oracle.com> I should point out under the "trivial rules" this only needs one Reviewer - me. Coleen On 9/22/16 12:04 PM, Brent Christian wrote: > Hi, > > Looks like my 8165372 change broke the slowdebug build. Please review > my fix (which also breaks up a pretty long line): > > --- a/src/share/vm/prims/stackwalk.cpp > +++ b/src/share/vm/prims/stackwalk.cpp > @@ -331,10 +331,12 @@ > assert (use_frames_array(mode), "Bad mode for get live frame"); > RegisterMap regMap(jt, true); > LiveFrameStream stream(jt, ®Map); > - return fetchFirstBatch(stream, stackStream, mode, skip_frames, > frame_count, start_index, frames_array, CHECK_NULL); > + return fetchFirstBatch(stream, stackStream, mode, skip_frames, > frame_count, > + start_index, frames_array, THREAD); > } else { > JavaFrameStream stream(jt, mode); > - return fetchFirstBatch(stream, stackStream, mode, skip_frames, > frame_count, start_index, frames_array, CHECK_NULL); > + return fetchFirstBatch(stream, stackStream, mode, skip_frames, > frame_count, > + start_index, frames_array, THREAD); > } > } > > Thanks! > -Brent > From igor.ignatyev at oracle.com Thu Sep 22 17:43:02 2016 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 22 Sep 2016 20:43:02 +0300 Subject: RFR(S): 8165602: Convert TestChunkedList_test to GTest In-Reply-To: References: Message-ID: Kirill, looks good to me Thanks, ? Igor > On Sep 15, 2016, at 7:09 PM, Coleen Phillimore wrote: > > The GC group added this test, so copying them. > Coleen > > On 9/15/16 11:25 AM, Kirill Zhaldybin wrote: >> Dear all, >> >> Could you please review this fix for 8165602? >> >> WebRev: http://cr.openjdk.java.net/~kzhaldyb/webrevs/JDK-8165602/webrev.01/ >> CR: https://bugs.openjdk.java.net/browse/JDK-8165602 >> >> Thank you. >> >> Regards, Kirill > From kirill.zhaldybin at oracle.com Thu Sep 22 17:52:42 2016 From: kirill.zhaldybin at oracle.com (Kirill Zhaldybin) Date: Thu, 22 Sep 2016 20:52:42 +0300 Subject: RFR(S): 8165602: Convert TestChunkedList_test to GTest In-Reply-To: References: Message-ID: <806411b2-3381-362b-aa89-b819bfa24eb9@oracle.com> Igor, Thank you for review! Regards, Kirill On 22.09.2016 20:43, Igor Ignatyev wrote: > Kirill, > > looks good to me > > Thanks, > ? Igor > >> On Sep 15, 2016, at 7:09 PM, Coleen Phillimore wrote: >> >> The GC group added this test, so copying them. >> Coleen >> >> On 9/15/16 11:25 AM, Kirill Zhaldybin wrote: >>> Dear all, >>> >>> Could you please review this fix for 8165602? >>> >>> WebRev: http://cr.openjdk.java.net/~kzhaldyb/webrevs/JDK-8165602/webrev.01/ >>> CR: https://bugs.openjdk.java.net/browse/JDK-8165602 >>> >>> Thank you. >>> >>> Regards, Kirill From ningsheng.jian at linaro.org Thu Sep 22 05:28:00 2016 From: ningsheng.jian at linaro.org (Ningsheng Jian) Date: Thu, 22 Sep 2016 13:28:00 +0800 Subject: [aarch64-port-dev ] RFR: AArch64: SEGV in stub code cipherBlockChaining_decryptAESCrypt In-Reply-To: <92c29040-7a82-3b73-7079-6b23b781dcd2@redhat.com> References: <92c29040-7a82-3b73-7079-6b23b781dcd2@redhat.com> Message-ID: On 21 September 2016 at 17:35, Andrew Dinn wrote: > On 20/09/16 11:09, Ningsheng Jian wrote: >> Jtreg test jdk/test/com/sun/crypto/provider/Cipher/CTS/CTSMode.java >> failed with SIGSEGV on option "-Xcomp -XX:-TieredCompilation". >> >> The crash happens at stub code cipherBlockChaining_decryptAESCrypt. >> >> Checking the jdk source code CipherTextStealing.decryptFinal and its >> callee CipherBlockChaining.decrypt/implDecrypt, we can see that >> cipherLen which is passed to the stub code can be 0. The unexpected >> len_reg value makes the load from invalid address. >> >> The following patch could fix this issue: >> >> http://people.linaro.org/~ningsheng.jian/webrev/cbc-stub-fix/webrev.00/ >> >> It aligns with the Java code implementation and passed JTreg tests. To >> make code consistent, I also update the encrypt part. >> >> Could someone please help to take a look at it? > > I think this patch looks like it will suffice to fix the AArch64 code. > However, I don't understand why this is only an AArch64 issue. For > example, it looks to me as if the x86_64 code is also susceptible to the > same problem should input value 0 be passed in len_reg (c_rarg4). So, > this might need fixing for all archs. Alternatively, it might be better > fixed in the jdk (Java) code. > > Could someone from the hotspot compiler/runtime team confirm: > > i) whether an input len_reg value of zero will cause a problem on x86 > (or, indeed, other archs)? I cannot see this issue on x86. Not sure for other archs. Thanks, Ningsheng From dmitry.samersoff at oracle.com Thu Sep 22 18:43:42 2016 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Thu, 22 Sep 2016 21:43:42 +0300 Subject: RFR 8166501 : compilation error in stackwalk.cpp on some gccs In-Reply-To: <57E40128.8010409@oracle.com> References: <57E40128.8010409@oracle.com> Message-ID: <4724b2d4-a2ab-8251-86e3-37ffc0ea87c7@oracle.com> Brent, Looks good for me. -Dmitry On 2016-09-22 19:04, Brent Christian wrote: > Hi, > > Looks like my 8165372 change broke the slowdebug build. Please review > my fix (which also breaks up a pretty long line): > > --- a/src/share/vm/prims/stackwalk.cpp > +++ b/src/share/vm/prims/stackwalk.cpp > @@ -331,10 +331,12 @@ > assert (use_frames_array(mode), "Bad mode for get live frame"); > RegisterMap regMap(jt, true); > LiveFrameStream stream(jt, ®Map); > - return fetchFirstBatch(stream, stackStream, mode, skip_frames, > frame_count, start_index, frames_array, CHECK_NULL); > + return fetchFirstBatch(stream, stackStream, mode, skip_frames, > frame_count, > + start_index, frames_array, THREAD); > } else { > JavaFrameStream stream(jt, mode); > - return fetchFirstBatch(stream, stackStream, mode, skip_frames, > frame_count, start_index, frames_array, CHECK_NULL); > + return fetchFirstBatch(stream, stackStream, mode, skip_frames, > frame_count, > + start_index, frames_array, THREAD); > } > } > > Thanks! > -Brent > -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From kim.barrett at oracle.com Thu Sep 22 20:51:22 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 22 Sep 2016 16:51:22 -0400 Subject: RFR: 8166583: Add oopDesc::klass_or_null_acquire() Message-ID: Please review this addition of a new function to oopDesc. This is a variant of an existing function; the new function provides acquire semantics to the access. This is a subtask of JDK-8160369, providing additional infrastructure needed to fix that bug. CR: https://bugs.openjdk.java.net/browse/JDK-8166583 Webrev: http://cr.openjdk.java.net/~kbarrett/8166583/webrev.00/ Testing: Ran nightly tests with this being used by an in-development fix for part of JDK-8160369. From david.holmes at oracle.com Thu Sep 22 22:27:09 2016 From: david.holmes at oracle.com (David Holmes) Date: Fri, 23 Sep 2016 08:27:09 +1000 Subject: RFR: 8166583: Add oopDesc::klass_or_null_acquire() In-Reply-To: References: Message-ID: Looks good. Thanks, David On 23/09/2016 6:51 AM, Kim Barrett wrote: > Please review this addition of a new function to oopDesc. This is a > variant of an existing function; the new function provides acquire > semantics to the access. > > This is a subtask of JDK-8160369, providing additional infrastructure > needed to fix that bug. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8166583 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8166583/webrev.00/ > > Testing: > Ran nightly tests with this being used by an in-development fix for > part of JDK-8160369. > From kim.barrett at oracle.com Fri Sep 23 00:31:16 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 22 Sep 2016 20:31:16 -0400 Subject: RFR: 8166583: Add oopDesc::klass_or_null_acquire() In-Reply-To: References: Message-ID: <3DA9DF30-8D19-434D-86A5-F88CA3A44C32@oracle.com> > On Sep 22, 2016, at 6:27 PM, David Holmes wrote: > > Looks good. Thanks. > Thanks, > David > > On 23/09/2016 6:51 AM, Kim Barrett wrote: >> Please review this addition of a new function to oopDesc. This is a >> variant of an existing function; the new function provides acquire >> semantics to the access. >> >> This is a subtask of JDK-8160369, providing additional infrastructure >> needed to fix that bug. >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8166583 >> >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8166583/webrev.00/ >> >> Testing: >> Ran nightly tests with this being used by an in-development fix for >> part of JDK-8160369. From david.holmes at oracle.com Fri Sep 23 00:44:36 2016 From: david.holmes at oracle.com (David Holmes) Date: Fri, 23 Sep 2016 10:44:36 +1000 Subject: RFR 8166501 : compilation error in stackwalk.cpp on some gccs In-Reply-To: <57E40128.8010409@oracle.com> References: <57E40128.8010409@oracle.com> Message-ID: This is the second example I've seen in two days concerning misuse of CHECK_ macros. They expand into an if statement after the call, so can not appear on calls that are part of a return statement, or a conditional statement, or likely a number of other places, as the if code either becomes unreachable or else does not do what may be expected. David ----- On 23/09/2016 2:04 AM, Brent Christian wrote: > Hi, > > Looks like my 8165372 change broke the slowdebug build. Please review > my fix (which also breaks up a pretty long line): > > --- a/src/share/vm/prims/stackwalk.cpp > +++ b/src/share/vm/prims/stackwalk.cpp > @@ -331,10 +331,12 @@ > assert (use_frames_array(mode), "Bad mode for get live frame"); > RegisterMap regMap(jt, true); > LiveFrameStream stream(jt, ®Map); > - return fetchFirstBatch(stream, stackStream, mode, skip_frames, > frame_count, start_index, frames_array, CHECK_NULL); > + return fetchFirstBatch(stream, stackStream, mode, skip_frames, > frame_count, > + start_index, frames_array, THREAD); > } else { > JavaFrameStream stream(jt, mode); > - return fetchFirstBatch(stream, stackStream, mode, skip_frames, > frame_count, start_index, frames_array, CHECK_NULL); > + return fetchFirstBatch(stream, stackStream, mode, skip_frames, > frame_count, > + start_index, frames_array, THREAD); > } > } > > Thanks! > -Brent > From david.holmes at oracle.com Fri Sep 23 00:46:54 2016 From: david.holmes at oracle.com (David Holmes) Date: Fri, 23 Sep 2016 10:46:54 +1000 Subject: RFR for JDK-8165482 java in ldoms, with cpu-arch=generic has problems In-Reply-To: <4f2a4187-a492-1c0b-71ed-a79afe7d56d6@oracle.com> References: <8046b263-8826-aa34-083a-9a83e105770f@oracle.com> <4f2a4187-a492-1c0b-71ed-a79afe7d56d6@oracle.com> Message-ID: <5a1e7bed-d6d5-3739-1049-92782d600e73@oracle.com> On 22/09/2016 7:34 PM, Martin Walsh wrote: > Polite reminder. Would anybody be able to review this? I did but still have an unanswered question in my email. David > Webrev: > > http://cr.openjdk.java.net/~mwalsh/JDK-8165482/ > > Bug: > > https://bugs.openjdk.java.net/browse/JDK-8165482 > > > Thanks, > > Martin > > On 19/09/2016 15:16, Martin Walsh wrote: >> Could I get a code review for the following bug: >> >> JDK-8165482 java in ldoms, with cpu-arch=generic has problems >> >> >> Webrev of the changes is available here: >> >> http://cr.openjdk.java.net/~mwalsh/JDK-8165482/ >> >> >> Thanks, >> >> Martin > From goetz.lindenmaier at sap.com Fri Sep 23 05:52:31 2016 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 23 Sep 2016 05:52:31 +0000 Subject: RFR(M): 8166560: [s390] Basic enablement of s390 port. Message-ID: Hi, This change is part of the s390 port. It contains some basic adaptions needed for a full hotspot port for linux s390x. It defines the required macros, platform names and includes. The s390 port calles CodeCache::contains() in current_frame(), which is used in NMT. As NMT already collects stack traces before the CodeCache is initialized, contains() needs a check for this. Wherever a row of platforms are listed, I sorted them alphabetically. The jdk requires the file jvm.cfg. Please review. I please need a sponsor. http://cr.openjdk.java.net/~goetz/wr16/8166560-basic_s390/hotspot.wr01/ http://cr.openjdk.java.net/~goetz/wr16/8166560-basic_s390/jdk.wr01/ Best regards, Goetz. From david.holmes at oracle.com Fri Sep 23 06:11:20 2016 From: david.holmes at oracle.com (David Holmes) Date: Fri, 23 Sep 2016 16:11:20 +1000 Subject: RFR(M): 8166560: [s390] Basic enablement of s390 port. In-Reply-To: References: Message-ID: Hi Goetz, I see a change not related directly to S390 ie change from ARM to ARM32 in src/os/linux/vm/os_linux.cpp It will be a while before I can go through this in any detail. David On 23/09/2016 3:52 PM, Lindenmaier, Goetz wrote: > Hi, > > This change is part of the s390 port. It contains some basic adaptions needed for a full hotspot port for linux s390x. > > It defines the required macros, platform names and includes. > > The s390 port calles CodeCache::contains() in current_frame(), which is used in NMT. As NMT already collects stack traces before the CodeCache is initialized, contains() needs a check for this. > > Wherever a row of platforms are listed, I sorted them alphabetically. > > The jdk requires the file jvm.cfg. > > Please review. I please need a sponsor. > http://cr.openjdk.java.net/~goetz/wr16/8166560-basic_s390/hotspot.wr01/ > http://cr.openjdk.java.net/~goetz/wr16/8166560-basic_s390/jdk.wr01/ > > Best regards, > Goetz. > From thomas.schatzl at oracle.com Fri Sep 23 07:10:47 2016 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 23 Sep 2016 09:10:47 +0200 Subject: RFR: 8166583: Add oopDesc::klass_or_null_acquire() In-Reply-To: References: Message-ID: <1474614647.4481.4.camel@oracle.com> Hi Kim, On Thu, 2016-09-22 at 16:51 -0400, Kim Barrett wrote: > Please review this addition of a new function to oopDesc.??This is a > variant of an existing function; the new function provides acquire > semantics to the access. > > This is a subtask of JDK-8160369, providing additional infrastructure > needed to fix that bug. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8166583 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8166583/webrev.00/ > > Testing: > Ran nightly tests with this being used by an in-development fix for > part of JDK-8160369. > ? looks good. Is there a bug filed for fixing the "// Workaround for non-const load_acquire parameter." comment? I.e. it seems that actually more infrastructure changes are required that have been deferred for now. Thanks, ? Thomas From kim.barrett at oracle.com Fri Sep 23 07:40:53 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 23 Sep 2016 03:40:53 -0400 Subject: RFR: 8166583: Add oopDesc::klass_or_null_acquire() In-Reply-To: <1474614647.4481.4.camel@oracle.com> References: <1474614647.4481.4.camel@oracle.com> Message-ID: <1AFF5E0A-FD8B-40BA-B6C6-B3BD8BE46569@oracle.com> > On Sep 23, 2016, at 3:10 AM, Thomas Schatzl wrote: > > Hi Kim, > > On Thu, 2016-09-22 at 16:51 -0400, Kim Barrett wrote: >> Please review this addition of a new function to oopDesc. This is a >> variant of an existing function; the new function provides acquire >> semantics to the access. >> >> This is a subtask of JDK-8160369, providing additional infrastructure >> needed to fix that bug. >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8166583 >> >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8166583/webrev.00/ >> >> Testing: >> Ran nightly tests with this being used by an in-development fix for >> part of JDK-8160369. >> > > looks good. Thanks. > Is there a bug filed for fixing the "// Workaround for non-const > load_acquire parameter." comment? I.e. it seems that actually more > infrastructure changes are required that have been deferred for now. Not yet; I?ve drafted it but not actually entered it into the system. Planning to do that tomorrow. I spent a little time working on it, and have a changeset that I think should cover it, but I haven?t tested it except locally yet, and can?t test on some affected platforms. It touches quite a few files, and is unfortunately less mechanical than one might wish. It also doesn?t seem like something that needs to be done for JDK 9. I?ll probably add a pointer to a webrev when I submit the RFE. From martin.walsh at oracle.com Fri Sep 23 08:41:03 2016 From: martin.walsh at oracle.com (Martin Walsh) Date: Fri, 23 Sep 2016 09:41:03 +0100 Subject: RFR for JDK-8165482 java in ldoms, with cpu-arch=generic has problems In-Reply-To: <5a1e7bed-d6d5-3739-1049-92782d600e73@oracle.com> References: <8046b263-8826-aa34-083a-9a83e105770f@oracle.com> <4f2a4187-a492-1c0b-71ed-a79afe7d56d6@oracle.com> <5a1e7bed-d6d5-3739-1049-92782d600e73@oracle.com> Message-ID: On 23/09/2016 01:46, David Holmes wrote: > On 22/09/2016 7:34 PM, Martin Walsh wrote: >> Polite reminder. Would anybody be able to review this? > > I did but still have an unanswered question in my email. Sorry David. Seems my mail filters were playing up. Anyway, I have found the full thread now, so will reply accordingly. Thanks, Martin > > David > >> Webrev: >> >> http://cr.openjdk.java.net/~mwalsh/JDK-8165482/ >> >> Bug: >> >> https://bugs.openjdk.java.net/browse/JDK-8165482 >> >> >> Thanks, >> >> Martin >> >> On 19/09/2016 15:16, Martin Walsh wrote: >>> Could I get a code review for the following bug: >>> >>> JDK-8165482 java in ldoms, with cpu-arch=generic has problems >>> >>> >>> Webrev of the changes is available here: >>> >>> http://cr.openjdk.java.net/~mwalsh/JDK-8165482/ >>> >>> >>> Thanks, >>> >>> Martin >> From dmitry.fazunenko at oracle.com Fri Sep 23 13:26:57 2016 From: dmitry.fazunenko at oracle.com (Dmitry Fazunenko) Date: Fri, 23 Sep 2016 16:26:57 +0300 Subject: RFR(S): 8165602: Convert TestChunkedList_test to GTest In-Reply-To: References: Message-ID: Kirill, the fix looks good. Thanks, Dima On 15.09.2016 19:09, Coleen Phillimore wrote: > The GC group added this test, so copying them. > Coleen > > On 9/15/16 11:25 AM, Kirill Zhaldybin wrote: >> Dear all, >> >> Could you please review this fix for 8165602? >> >> WebRev: >> http://cr.openjdk.java.net/~kzhaldyb/webrevs/JDK-8165602/webrev.01/ >> CR: https://bugs.openjdk.java.net/browse/JDK-8165602 >> >> Thank you. >> >> Regards, Kirill > From martin.walsh at oracle.com Fri Sep 23 13:34:28 2016 From: martin.walsh at oracle.com (Martin Walsh) Date: Fri, 23 Sep 2016 14:34:28 +0100 Subject: RFR for JDK-8165482 java in ldoms, with cpu-arch=generic has problems In-Reply-To: <4bfdead9-f0d9-f67e-e049-d48af5f19863@oracle.com> References: <8046b263-8826-aa34-083a-9a83e105770f@oracle.com> <0a2a66db-f939-196f-bf23-a6a3df46b2d3@oracle.com> <4bfdead9-f0d9-f67e-e049-d48af5f19863@oracle.com> Message-ID: <994a0b2e-5874-cbdd-1544-d3b00e5165a2@oracle.com> On 20/09/2016 08:42, Erik Joelsson wrote: > Hello, > > > On 2016-09-20 03:28, David Holmes wrote: >> Hi Martin, >> >> Build changes must be reviewed by the build team - now cc'd >> > Thanks for forwarding David, I can't keep up with all the lists to find > these unless posted to build-dev. >> On 20/09/2016 12:16 AM, Martin Walsh wrote: >>> Could I get a code review for the following bug: >>> >>> JDK-8165482 java in ldoms, with cpu-arch=generic has problems >>> >>> Webrev of the changes is available here: >>> >>> http://cr.openjdk.java.net/~mwalsh/JDK-8165482/ >> >> What is the devinfo library? Is it part of the normal Solaris >> installation, or does it need to be installed specifically? Is it >> available in our official build toolkits? > I did a bit of digging. It's part "system/libraries" so should be pretty > standard. That package is in the devkit and I verified that > libdevinfo.so is there too. > > Configure change looks fine. Just remember to also push the closed > generated-configure.sh as David said. > > /Erik >> >> Will checking the prom prior to using kstat change any of the values >> we currently see? (other than the generic case being fixed of course). It shouldn't, as SPARC64 seems to be reserved for Fujitsu SPARC machines. However, to air on the side of caution I will investigate further and follow-up shortly. Thanks, Martin >> >> Also note you will need to generate and push the closed >> generated-configure.sh file. >> >> Thanks, >> David >> >>> >>> Thanks, >>> >>> Martin > From jon.masamitsu at oracle.com Fri Sep 23 18:25:41 2016 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Fri, 23 Sep 2016 11:25:41 -0700 Subject: Request for review (s) - 8155948: Add message for CMS deprecation for some internal builds In-Reply-To: <50bac55e-b92c-f5d5-cc29-f0a85fdc6923@oracle.com> References: <74eccd9a-1525-a9ef-1608-294de99d4483@oracle.com> <52db37a5-efa4-79dd-771f-45e1fc1ccaf5@oracle.com> <02968df4-b53d-c1d1-65e4-a612ca393bae@oracle.com> <5CB1B554-D733-40D6-881F-6237F9CE9E11@oracle.com> <50bac55e-b92c-f5d5-cc29-f0a85fdc6923@oracle.com> Message-ID: <3577baee-6ed2-d636-ee7d-7af33e0548b5@oracle.com> After some discussion I've removed the handling of the UseAutoGCSelectPolicy flag from this patch. It will be handled separately under https://bugs.openjdk.java.net/browse/JDK-8166461 The new full webrev is http://cr.openjdk.java.net/~jmasa/8155948/webrev.04/ The delta is http://cr.openjdk.java.net/~jmasa/8155948/webrev_delta.03_04/index.html Thanks. Jon On 09/08/2016 12:38 PM, Jon Masamitsu wrote: > Latest delta > > http://cr.openjdk.java.net/~jmasa/8155948/webrev_delta.01_02/ > > and full > > http://cr.openjdk.java.net/~jmasa/8155948/webrev.02/ > > Thanks. > > Jon > > On 09/08/2016 10:44 AM, Kim Barrett wrote: >>> On Sep 8, 2016, at 12:35 PM, Jon Masamitsu >>> wrote: >>> >>> Kim, >>> >>> Thanks for looking at this. Comments below. >>> >>> On 09/07/2016 03:12 PM, Kim Barrett wrote: >>>> src/share/vm/runtime/arguments.cpp >>>> 4064 void Arguments::handle_concgc_flags() { >>>> 4065 SpecialFlag flag; >>>> 4066 const char *flag_name = "UseConcMarkSweepGC"; >>>> 4067 if (log_is_enabled(Warning, gc) && >>>> 4068 lookup_special_flag(flag_name, flag)) { >>>> 4069 handle_aliases_and_deprecation(flag_name, /* print warning >>>> */ true); >>>> 4070 log_warning(gc)("-Xconcgc/-Xnoconcgc uses >>>> UseConcMarkSweepGC"); >>>> 4071 } >>>> 4072 } >>>> >>>> This non-trivial little dance is done both here and for >>>> UseAutoGCSelectPolicy. I think handle_concgc_flags could deal with >>>> both if it took an argument of the "offending" flag. >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> >>> -Xconcgc does not have a SpecialFlag associated with it currently. >>> Do you mean >>> add one for "concgc" (even through it is not a -XX flags)? Or do >>> you mean that >>> "offending" flag is just a parameter that chooses between the two cases >>> -Xconcgc/-Xnoconcgc and -XX:+UseAutoGCSelectPolicy? >>> >>> Jon >> A parameter that is used to tailor the warning, e.g. a const char* >> that is %s?ed into the log_warning message. >> > From kim.barrett at oracle.com Fri Sep 23 22:33:41 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 23 Sep 2016 18:33:41 -0400 Subject: Request for review (s) - 8155948: Add message for CMS deprecation for some internal builds In-Reply-To: <3577baee-6ed2-d636-ee7d-7af33e0548b5@oracle.com> References: <74eccd9a-1525-a9ef-1608-294de99d4483@oracle.com> <52db37a5-efa4-79dd-771f-45e1fc1ccaf5@oracle.com> <02968df4-b53d-c1d1-65e4-a612ca393bae@oracle.com> <5CB1B554-D733-40D6-881F-6237F9CE9E11@oracle.com> <50bac55e-b92c-f5d5-cc29-f0a85fdc6923@oracle.com> <3577baee-6ed2-d636-ee7d-7af33e0548b5@oracle.com> Message-ID: <14E40EE8-1A93-419E-8E30-A517E5C4F9E9@oracle.com> > On Sep 23, 2016, at 2:25 PM, Jon Masamitsu wrote: > > After some discussion I've removed the handling of the > UseAutoGCSelectPolicy flag from this patch. It will be handled > separately under > > https://bugs.openjdk.java.net/browse/JDK-8166461 > > > The new full webrev is > > http://cr.openjdk.java.net/~jmasa/8155948/webrev.04/ > > The delta is > > http://cr.openjdk.java.net/~jmasa/8155948/webrev_delta.03_04/index.html > > Thanks. > > Jon ------------------------------------------------------------------------------ src/share/vm/runtime/arguments.cpp I think this line should not be removed: 1807 handle_extra_cms_flags("UseAutoGCSelectPolicy uses UseConcMarkSweepGC"); Removal of this line means that commercial builds that otherwise warn about CMS being deprecated when it is selected won't warn if it is selected via this mechanism. Similarly for the test. So I liked the previous version and not so much this one. ------------------------------------------------------------------------------ From kim.barrett at oracle.com Fri Sep 23 23:06:22 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 23 Sep 2016 19:06:22 -0400 Subject: RFR: 8166583: Add oopDesc::klass_or_null_acquire() In-Reply-To: <1AFF5E0A-FD8B-40BA-B6C6-B3BD8BE46569@oracle.com> References: <1474614647.4481.4.camel@oracle.com> <1AFF5E0A-FD8B-40BA-B6C6-B3BD8BE46569@oracle.com> Message-ID: > On Sep 23, 2016, at 3:40 AM, Kim Barrett wrote: > >> On Sep 23, 2016, at 3:10 AM, Thomas Schatzl wrote: >> Is there a bug filed for fixing the "// Workaround for non-const >> load_acquire parameter." comment? I.e. it seems that actually more >> infrastructure changes are required that have been deferred for now. > > Not yet; I?ve drafted it but not actually entered it into the system. Planning to do that tomorrow. > > I spent a little time working on it, and have a changeset that I think should cover > it, but I haven?t tested it except locally yet, and can?t test on some affected platforms. > It touches quite a few files, and is unfortunately less mechanical than one might wish. > It also doesn?t seem like something that needs to be done for JDK 9. I?ll probably > add a pointer to a webrev when I submit the RFE. https://bugs.openjdk.java.net/browse/JDK-8166651 OrderAccess::load_acquire &etc should have const parameters From Alan.Bateman at oracle.com Sat Sep 24 10:08:49 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 24 Sep 2016 11:08:49 +0100 Subject: RFR(M): 8166560: [s390] Basic enablement of s390 port. In-Reply-To: References: Message-ID: <903643bb-9a78-5254-fd0e-108da29802bd@oracle.com> On 23/09/2016 06:52, Lindenmaier, Goetz wrote: > Hi, > > This change is part of the s390 port. It contains some basic adaptions needed for a full hotspot port for linux s390x. > > Out of curiosity, is there is JEP for the Linux/S390 port? There were JEPs for the Linux/AArch64 and PowerPC/AIX ports and just wondering if there is one coming for the S390 port too? -Alan From kirill.zhaldybin at oracle.com Sun Sep 25 19:08:15 2016 From: kirill.zhaldybin at oracle.com (Kirill Zhaldybin) Date: Sun, 25 Sep 2016 22:08:15 +0300 Subject: RFR(S): 8165602: Convert TestChunkedList_test to GTest In-Reply-To: References: Message-ID: Dmitry, Thank you for review! Regards, Kirill On 09/23/2016 04:26 PM, Dmitry Fazunenko wrote: > Kirill, > > the fix looks good. > > Thanks, > Dima > > On 15.09.2016 19:09, Coleen Phillimore wrote: >> The GC group added this test, so copying them. >> Coleen >> >> On 9/15/16 11:25 AM, Kirill Zhaldybin wrote: >>> Dear all, >>> >>> Could you please review this fix for 8165602? >>> >>> WebRev: >>> http://cr.openjdk.java.net/~kzhaldyb/webrevs/JDK-8165602/webrev.01/ >>> CR: https://bugs.openjdk.java.net/browse/JDK-8165602 >>> >>> Thank you. >>> >>> Regards, Kirill >> > From aph at redhat.com Mon Sep 26 15:25:14 2016 From: aph at redhat.com (Andrew Haley) Date: Mon, 26 Sep 2016 16:25:14 +0100 Subject: RFR: 165673: AArch64: Fix JNI floating point argument handling Message-ID: <2ba05d11-1119-5cf6-e84d-2f0cdb03dd98@redhat.com> This is an AArch64 critical bug. http://people.linaro.org/~ningsheng.jian/jni-fix/webrev.00/ We need a sponsor because of the test case. Will you help, please? Thanks, Andrew. From vladimir.kozlov at oracle.com Mon Sep 26 15:33:22 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 26 Sep 2016 08:33:22 -0700 Subject: RFR: 165673: AArch64: Fix JNI floating point argument handling In-Reply-To: <2ba05d11-1119-5cf6-e84d-2f0cdb03dd98@redhat.com> References: <2ba05d11-1119-5cf6-e84d-2f0cdb03dd98@redhat.com> Message-ID: <57E93FC2.8050401@oracle.com> No problem but why rename the test? It is difficult to see changes in test when it is renamed. Thanks, Vladimir On 9/26/16 8:25 AM, Andrew Haley wrote: > This is an AArch64 critical bug. > > http://people.linaro.org/~ningsheng.jian/jni-fix/webrev.00/ > > We need a sponsor because of the test case. Will you help, please? > > Thanks, > > Andrew. > From aph at redhat.com Mon Sep 26 15:36:17 2016 From: aph at redhat.com (Andrew Haley) Date: Mon, 26 Sep 2016 16:36:17 +0100 Subject: RFR: 165673: AArch64: Fix JNI floating point argument handling In-Reply-To: <57E93FC2.8050401@oracle.com> References: <2ba05d11-1119-5cf6-e84d-2f0cdb03dd98@redhat.com> <57E93FC2.8050401@oracle.com> Message-ID: On 26/09/16 16:33, Vladimir Kozlov wrote: > No problem but why rename the test? It is difficult to see changes in test when it is renamed. I have absolutely not idea why Ningsheng Jian did that. It doesn't matter. Andrew. From vladimir.kozlov at oracle.com Mon Sep 26 15:44:56 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 26 Sep 2016 08:44:56 -0700 Subject: RFR: 165673: AArch64: Fix JNI floating point argument handling In-Reply-To: References: <2ba05d11-1119-5cf6-e84d-2f0cdb03dd98@redhat.com> <57E93FC2.8050401@oracle.com> Message-ID: <57E94278.6020008@oracle.com> Looks like there were significant changes in the test, not just renaming. Either way it is fine with me. Please, file JBS bug and do normal review process. Thanks, Vladimir On 9/26/16 8:36 AM, Andrew Haley wrote: > On 26/09/16 16:33, Vladimir Kozlov wrote: >> No problem but why rename the test? It is difficult to see changes in test when it is renamed. > > I have absolutely not idea why Ningsheng Jian did that. It doesn't matter. > > Andrew. > > From aph at redhat.com Mon Sep 26 15:53:07 2016 From: aph at redhat.com (Andrew Haley) Date: Mon, 26 Sep 2016 16:53:07 +0100 Subject: RFR: 8165673: AArch64: Fix JNI floating point argument handling In-Reply-To: <57E94278.6020008@oracle.com> References: <2ba05d11-1119-5cf6-e84d-2f0cdb03dd98@redhat.com> <57E93FC2.8050401@oracle.com> <57E94278.6020008@oracle.com> Message-ID: On 26/09/16 16:44, Vladimir Kozlov wrote: > Looks like there were significant changes in the test, not just renaming. Either way it is fine with me. > > Please, file JBS bug and do normal review process. Bug is at https://bugs.openjdk.java.net/browse/JDK-8165673 Review is all done at http://mail.openjdk.java.net/pipermail/aarch64-port-dev/2016-September/003754.html Andrew. From adinn at redhat.com Mon Sep 26 15:58:01 2016 From: adinn at redhat.com (Andrew Dinn) Date: Mon, 26 Sep 2016 16:58:01 +0100 Subject: RFR: 165673: AArch64: Fix JNI floating point argument handling In-Reply-To: <57E94278.6020008@oracle.com> References: <2ba05d11-1119-5cf6-e84d-2f0cdb03dd98@redhat.com> <57E93FC2.8050401@oracle.com> <57E94278.6020008@oracle.com> Message-ID: On 26/09/16 16:44, Vladimir Kozlov wrote: > Looks like there were significant changes in the test, not just > renaming. Either way it is fine with me. > > Please, file JBS bug and do normal review process. This has already been reviewed by Andrew Haley and me on the aarch64-port-dev list. http://mail.openjdk.java.net/pipermail/aarch64-port-dev/2016-September/003754.html The bug is https://bugs.openjdk.java.net/browse/JDK-8165673 regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From vladimir.kozlov at oracle.com Mon Sep 26 16:22:03 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 26 Sep 2016 09:22:03 -0700 Subject: RFR: 165673: AArch64: Fix JNI floating point argument handling In-Reply-To: References: <2ba05d11-1119-5cf6-e84d-2f0cdb03dd98@redhat.com> <57E93FC2.8050401@oracle.com> <57E94278.6020008@oracle.com> Message-ID: <57E94B2B.8070609@oracle.com> Good. Consider me reviewed test changes. Please, prepare webrev and changeset on cr.openjdk.java.net. Ningsheng Jian is not Author in OpenJDK, should be listed as Contributed-by: Thanks, Vladimir On 9/26/16 8:58 AM, Andrew Dinn wrote: > On 26/09/16 16:44, Vladimir Kozlov wrote: >> Looks like there were significant changes in the test, not just >> renaming. Either way it is fine with me. >> >> Please, file JBS bug and do normal review process. > > This has already been reviewed by Andrew Haley and me on the > aarch64-port-dev list. > > > http://mail.openjdk.java.net/pipermail/aarch64-port-dev/2016-September/003754.html > > The bug is > > https://bugs.openjdk.java.net/browse/JDK-8165673 > > regards, > > > Andrew Dinn > ----------- > Senior Principal Software Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander > From christian.tornqvist at oracle.com Mon Sep 26 21:07:11 2016 From: christian.tornqvist at oracle.com (Christian Tornqvist) Date: Mon, 26 Sep 2016 17:07:11 -0400 Subject: RFR(S): 8166738 - Enable concurrency in Hotspot jtreg testing Message-ID: <375c01d21839$f3774c50$da65e4f0$@oracle.com> Hi everyone, Please review this small change that enables concurrency for the Hotspot jtreg tests when running them through the make system. Testing done so far shows very promising speed ups, both when running locally and in JPRT. It'll automatically calculate a sane concurrency number based on the number of cores / 2 (with a maximum of 12) and then sets MaxRAMFraction=concurrency*4 to make sure we don't run out of memory. The concurrency value can be overridden by setting TEST_JOBS to the desired concurrency value. On Windows, this change is temporarily commented out since we run into the intermittent issue described in JDK-8159799. Webrev: http://cr.openjdk.java.net/~ctornqvi/webrev/8166738/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8166738 Thanks, Christian From jon.masamitsu at oracle.com Mon Sep 26 21:18:08 2016 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Mon, 26 Sep 2016 14:18:08 -0700 Subject: Request for review (s) - 8155948: Add message for CMS deprecation for some internal builds In-Reply-To: <14E40EE8-1A93-419E-8E30-A517E5C4F9E9@oracle.com> References: <74eccd9a-1525-a9ef-1608-294de99d4483@oracle.com> <52db37a5-efa4-79dd-771f-45e1fc1ccaf5@oracle.com> <02968df4-b53d-c1d1-65e4-a612ca393bae@oracle.com> <5CB1B554-D733-40D6-881F-6237F9CE9E11@oracle.com> <50bac55e-b92c-f5d5-cc29-f0a85fdc6923@oracle.com> <3577baee-6ed2-d636-ee7d-7af33e0548b5@oracle.com> <14E40EE8-1A93-419E-8E30-A517E5C4F9E9@oracle.com> Message-ID: <67d0ba6f-014f-6cb7-0132-ae950d4e4af9@oracle.com> Kim, I thought that upshot of the discussion earlier in the week was that the UseAutoGCSelectPolicy flag should be deprecated itself and so would circumvent the warning about CMS being deprecated if UseAutoGCSelectPolicy picked CMS. Dima's point was that GC ergonomics should not pick a collector that was deprecated. Not so? Jon On 9/23/2016 3:33 PM, Kim Barrett wrote: >> On Sep 23, 2016, at 2:25 PM, Jon Masamitsu wrote: >> >> After some discussion I've removed the handling of the >> UseAutoGCSelectPolicy flag from this patch. It will be handled >> separately under >> >> https://bugs.openjdk.java.net/browse/JDK-8166461 >> >> >> The new full webrev is >> >> http://cr.openjdk.java.net/~jmasa/8155948/webrev.04/ >> >> The delta is >> >> http://cr.openjdk.java.net/~jmasa/8155948/webrev_delta.03_04/index.html >> >> Thanks. >> >> Jon > ------------------------------------------------------------------------------ > src/share/vm/runtime/arguments.cpp > > I think this line should not be removed: > > 1807 handle_extra_cms_flags("UseAutoGCSelectPolicy uses UseConcMarkSweepGC"); > > Removal of this line means that commercial builds that otherwise warn > about CMS being deprecated when it is selected won't warn if it is > selected via this mechanism. > > Similarly for the test. > > So I liked the previous version and not so much this one. > > ------------------------------------------------------------------------------ > From mikael.vidstedt at oracle.com Mon Sep 26 21:53:20 2016 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Mon, 26 Sep 2016 14:53:20 -0700 Subject: RFR(XS): 8166045: jdk/internal/misc/Unsafe tests fail due to timeout Message-ID: Please review the following change: Bug: https://bugs.openjdk.java.net/browse/JDK-8166045 Webrev: http://cr.openjdk.java.net/~mikael/webrevs/8166045/webrev.01/jdk/webrev/ Summary: Among the tests I introduced for Unsafe.copyMemory and Unsafe.copySwapMemory a few months ago are a couple of tests which rely on being able to allocate and operate on a large (~2GB) block of native memory. However, on some smallish machines the actual allocation succeeds, but the machine then goes on to swap (as in, page to/from disk) like crazy, making the test time out. In an attempt to work around this I?m using @requires in combination with the value of os.maxMemory. I could make the whole test(s) conditional on the memory size, but the vast majority of the test(s) actually operate on a very small block of memory (tens of bytes), so disabling the whole test seems unfortunate. Instead I decided to split up the test(s) in two - one set which tests the default/small size, and one set which tests the large size iff there is enough memory on the machine. I have tested manually that the @requires check has the right effect - the tests are skipped if there?s not enough memory on the machine - and that the tests still pass as expected. I also took the liberty of removing a superfluous import. Cheers, Mikael From serguei.spitsyn at oracle.com Mon Sep 26 22:46:08 2016 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 26 Sep 2016 15:46:08 -0700 Subject: RFR(S): 8166738 - Enable concurrency in Hotspot jtreg testing In-Reply-To: <375c01d21839$f3774c50$da65e4f0$@oracle.com> References: <375c01d21839$f3774c50$da65e4f0$@oracle.com> Message-ID: <553add11-1215-2fc9-0ca2-09423f51bea7@oracle.com> Hi Christian, Looks good to me. Thanks, Serguei On 9/26/16 14:07, Christian Tornqvist wrote: > Hi everyone, > > > > Please review this small change that enables concurrency for the Hotspot > jtreg tests when running them through the make system. Testing done so far > shows very promising speed ups, both when running locally and in JPRT. > > > > It'll automatically calculate a sane concurrency number based on the number > of cores / 2 (with a maximum of 12) and then sets > MaxRAMFraction=concurrency*4 to make sure we don't run out of memory. The > concurrency value can be overridden by setting TEST_JOBS to the desired > concurrency value. > > > > On Windows, this change is temporarily commented out since we run into the > intermittent issue described in JDK-8159799. > > > > Webrev: > > http://cr.openjdk.java.net/~ctornqvi/webrev/8166738/webrev.00/ > > > > Bug: > > https://bugs.openjdk.java.net/browse/JDK-8166738 > > > > Thanks, > > Christian > From david.holmes at oracle.com Mon Sep 26 22:46:36 2016 From: david.holmes at oracle.com (David Holmes) Date: Tue, 27 Sep 2016 08:46:36 +1000 Subject: RFR(XS): 8166045: jdk/internal/misc/Unsafe tests fail due to timeout In-Reply-To: References: Message-ID: <055580f5-8ab5-75fc-43b2-b5e601cd805a@oracle.com> Hi Mikael, Changes seem okay to me. Should the tests have an appropriate @bug line? Thanks, David On 27/09/2016 7:53 AM, Mikael Vidstedt wrote: > > Please review the following change: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8166045 > Webrev: http://cr.openjdk.java.net/~mikael/webrevs/8166045/webrev.01/jdk/webrev/ > > Summary: > > Among the tests I introduced for Unsafe.copyMemory and Unsafe.copySwapMemory a few months ago are a couple of tests which rely on being able to allocate and operate on a large (~2GB) block of native memory. However, on some smallish machines the actual allocation succeeds, but the machine then goes on to swap (as in, page to/from disk) like crazy, making the test time out. > > In an attempt to work around this I?m using @requires in combination with the value of os.maxMemory. I could make the whole test(s) conditional on the memory size, but the vast majority of the test(s) actually operate on a very small block of memory (tens of bytes), so disabling the whole test seems unfortunate. Instead I decided to split up the test(s) in two - one set which tests the default/small size, and one set which tests the large size iff there is enough memory on the machine. > > I have tested manually that the @requires check has the right effect - the tests are skipped if there?s not enough memory on the machine - and that the tests still pass as expected. > > I also took the liberty of removing a superfluous import. > > Cheers, > Mikael > From christian.tornqvist at oracle.com Mon Sep 26 23:11:37 2016 From: christian.tornqvist at oracle.com (Christian Tornqvist) Date: Mon, 26 Sep 2016 19:11:37 -0400 Subject: RFR(XS): 8166045: jdk/internal/misc/Unsafe tests fail due to timeout In-Reply-To: References: Message-ID: <6771FA24-EB00-4203-94F5-F2375E9FFB08@oracle.com> Hi Mikael, This looks good, thanks for fixing this. Thanks, Christian > On Sep 26, 2016, at 5:53 PM, Mikael Vidstedt wrote: > > > Please review the following change: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8166045 > > Webrev: > http://cr.openjdk.java.net/~mikael/webrevs/8166045/webrev.01/jdk/webrev/ > > > Summary: > > Among the tests I introduced for Unsafe.copyMemory and Unsafe.copySwapMemory > a few months ago are a couple of tests which rely on being able to allocate > and operate on a large (~2GB) block of native memory. However, on some > smallish machines the actual allocation succeeds, but the machine then goes > on to swap (as in, page to/from disk) like crazy, making the test time out. > > In an attempt to work around this I?m using @requires in combination with > the value of os.maxMemory. I could make the whole test(s) conditional on the > memory size, but the vast majority of the test(s) actually operate on a very > small block of memory (tens of bytes), so disabling the whole test seems > unfortunate. Instead I decided to split up the test(s) in two - one set > which tests the default/small size, and one set which tests the large size > iff there is enough memory on the machine. > > I have tested manually that the @requires check has the right effect - the > tests are skipped if there?s not enough memory on the machine - and that the > tests still pass as expected. > > I also took the liberty of removing a superfluous import. > > Cheers, > Mikael > From george.triantafillou at oracle.com Mon Sep 26 23:42:22 2016 From: george.triantafillou at oracle.com (George Triantafillou) Date: Mon, 26 Sep 2016 19:42:22 -0400 Subject: RFR(S): 8166738 - Enable concurrency in Hotspot jtreg testing In-Reply-To: <375c01d21839$f3774c50$da65e4f0$@oracle.com> References: <375c01d21839$f3774c50$da65e4f0$@oracle.com> Message-ID: Christian, Your changes look good! -George On 9/26/2016 5:07 PM, Christian Tornqvist wrote: > Hi everyone, > > > > Please review this small change that enables concurrency for the Hotspot > jtreg tests when running them through the make system. Testing done so far > shows very promising speed ups, both when running locally and in JPRT. > > > > It'll automatically calculate a sane concurrency number based on the number > of cores / 2 (with a maximum of 12) and then sets > MaxRAMFraction=concurrency*4 to make sure we don't run out of memory. The > concurrency value can be overridden by setting TEST_JOBS to the desired > concurrency value. > > > > On Windows, this change is temporarily commented out since we run into the > intermittent issue described in JDK-8159799. > > > > Webrev: > > http://cr.openjdk.java.net/~ctornqvi/webrev/8166738/webrev.00/ > > > > Bug: > > https://bugs.openjdk.java.net/browse/JDK-8166738 > > > > Thanks, > > Christian > From erik.joelsson at oracle.com Tue Sep 27 08:07:43 2016 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Tue, 27 Sep 2016 10:07:43 +0200 Subject: RFR(S): 8166738 - Enable concurrency in Hotspot jtreg testing In-Reply-To: <375c01d21839$f3774c50$da65e4f0$@oracle.com> References: <375c01d21839$f3774c50$da65e4f0$@oracle.com> Message-ID: <1de9e691-f75a-a301-b413-9a510ea3c1bc@oracle.com> Looks good to me. /Erik On 2016-09-26 23:07, Christian Tornqvist wrote: > > Hi everyone, > > Please review this small change that enables concurrency for the > Hotspot jtreg tests when running them through the make system. Testing > done so far shows very promising speed ups, both when running locally > and in JPRT. > > It?ll automatically calculate a sane concurrency number based on the > number of cores / 2 (with a maximum of 12) and then sets > MaxRAMFraction=concurrency*4 to make sure we don?t run out of memory. > The concurrency value can be overridden by setting TEST_JOBS to the > desired concurrency value. > > On Windows, this change is temporarily commented out since we run into > the intermittent issue described in JDK-8159799. > > Webrev: > > http://cr.openjdk.java.net/~ctornqvi/webrev/8166738/webrev.00/ > > > Bug: > > https://bugs.openjdk.java.net/browse/JDK-8166738 > > Thanks, > > Christian > From adinn at redhat.com Tue Sep 27 08:36:28 2016 From: adinn at redhat.com (Andrew Dinn) Date: Tue, 27 Sep 2016 09:36:28 +0100 Subject: RFR: 165673: AArch64: Fix JNI floating point argument handling In-Reply-To: <57E94B2B.8070609@oracle.com> References: <2ba05d11-1119-5cf6-e84d-2f0cdb03dd98@redhat.com> <57E93FC2.8050401@oracle.com> <57E94278.6020008@oracle.com> <57E94B2B.8070609@oracle.com> Message-ID: <2fc1edfe-2594-751e-64f1-b34aa940165d@redhat.com> > Good. Consider me reviewed test changes. > > Please, prepare webrev and changeset on cr.openjdk.java.net. > > Ningsheng Jian is not Author in OpenJDK, should be listed as Contributed-by: > > Thanks, > Vladimir Oops, apologies -- rather than upload the webrev I actually pushed this change to hs-comp. I hope that is alright. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From thomas.stuefe at gmail.com Tue Sep 27 08:45:01 2016 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 27 Sep 2016 10:45:01 +0200 Subject: Enhancement Proposal: Reduce metaspace waste by dynamically merging and splitting metaspace chunks. Message-ID: Dear all, please take a look at this Enhancement Proposal for the metaspace allocator. I hope these are the right groups for this discussion. https://bugs.openjdk.java.net/browse/JDK-8166690 Background: We at SAP see at times at customer installations OOMs in Metaspace (usually, with compressed class pointers enabled, in Compressed Class Space). The VM attempts to allocate metaspace and fails, hitting the CompressedClassSpaceSize limit. Note that we usually set the limit lower than the default, typically at 256M. When analyzing, we observed that a large part of the metaspace is indeed free but "locked in" into metaspace chunks of the wrong size: often we would find a lot of free small chunks, but the allocation request was for medium chunks, and failed. The reason was that if at some point in time a lot of class loaders were alive, each with only a few small classes loaded. This would lead to the metaspace being swamped with lots of small chunks. This is because each SpaceManager first allocates small chunks, only after a certain amount of allocation requests switches to larger chunks. These small chunks are free and wait in the freelist, but cannot be reused for allocation requests which require larger chunks, even if they are physically adjacent in the virtual space. We (at SAP) added a patch which allows on-the-fly metaspace chunk merging - to merge multiple adjacent smaller chunk to form a larger chunk. This, in combination with the reverse direction - splitting a large chunk to get smaller chunks - partly negates the "chunks-are-locked-in-into-their-size" limitation and provides for better reuse of metaspace chunks. It also provides better defragmentation as well. I discussed this fix off-list with Coleen Phillimore and Jon Masamitsu, and instead of just offering this as a fix, both recommended to open a JEP for this, because its scope would be beyond that of a simple fix. So here is my first JEP :) I hope it follows the right form. Please, if you have time, take a look and tell us what you think. Thank you, and Kind Regards, Thomas St?fe From jon.masamitsu at oracle.com Tue Sep 27 15:16:17 2016 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Tue, 27 Sep 2016 08:16:17 -0700 Subject: Request for review (s) - 8155948: Add message for CMS deprecation for some internal builds In-Reply-To: <67d0ba6f-014f-6cb7-0132-ae950d4e4af9@oracle.com> References: <74eccd9a-1525-a9ef-1608-294de99d4483@oracle.com> <52db37a5-efa4-79dd-771f-45e1fc1ccaf5@oracle.com> <02968df4-b53d-c1d1-65e4-a612ca393bae@oracle.com> <5CB1B554-D733-40D6-881F-6237F9CE9E11@oracle.com> <50bac55e-b92c-f5d5-cc29-f0a85fdc6923@oracle.com> <3577baee-6ed2-d636-ee7d-7af33e0548b5@oracle.com> <14E40EE8-1A93-419E-8E30-A517E5C4F9E9@oracle.com> <67d0ba6f-014f-6cb7-0132-ae950d4e4af9@oracle.com> Message-ID: <030ef60d-c99a-4a40-3603-dcd08f9ead5c@oracle.com> Kim, I fixed https://bugs.openjdk.java.net/browse/JDK-8166461 so that the fix version is 9. Jon On 9/26/2016 2:18 PM, Jon Masamitsu wrote: > Kim, > > I thought that upshot of the discussion earlier in the week was that > the UseAutoGCSelectPolicy flag should be deprecated itself and > so would circumvent the warning about CMS being deprecated if > UseAutoGCSelectPolicy picked CMS. Dima's point was that GC > ergonomics should not pick a collector that was deprecated. > > Not so? > > Jon > > On 9/23/2016 3:33 PM, Kim Barrett wrote: >>> On Sep 23, 2016, at 2:25 PM, Jon Masamitsu >>> wrote: >>> >>> After some discussion I've removed the handling of the >>> UseAutoGCSelectPolicy flag from this patch. It will be handled >>> separately under >>> >>> https://bugs.openjdk.java.net/browse/JDK-8166461 >>> >>> >>> The new full webrev is >>> >>> http://cr.openjdk.java.net/~jmasa/8155948/webrev.04/ >>> >>> The delta is >>> >>> http://cr.openjdk.java.net/~jmasa/8155948/webrev_delta.03_04/index.html >>> >>> Thanks. >>> >>> Jon >> ------------------------------------------------------------------------------ >> >> src/share/vm/runtime/arguments.cpp >> >> I think this line should not be removed: >> >> 1807 handle_extra_cms_flags("UseAutoGCSelectPolicy uses >> UseConcMarkSweepGC"); >> >> Removal of this line means that commercial builds that otherwise warn >> about CMS being deprecated when it is selected won't warn if it is >> selected via this mechanism. >> >> Similarly for the test. >> >> So I liked the previous version and not so much this one. >> >> ------------------------------------------------------------------------------ >> >> > From vladimir.kozlov at oracle.com Tue Sep 27 17:02:58 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 27 Sep 2016 10:02:58 -0700 Subject: RFR: 165673: AArch64: Fix JNI floating point argument handling In-Reply-To: <2fc1edfe-2594-751e-64f1-b34aa940165d@redhat.com> References: <2ba05d11-1119-5cf6-e84d-2f0cdb03dd98@redhat.com> <57E93FC2.8050401@oracle.com> <57E94278.6020008@oracle.com> <57E94B2B.8070609@oracle.com> <2fc1edfe-2594-751e-64f1-b34aa940165d@redhat.com> Message-ID: <39663abb-b293-8349-c815-aa0cabd9f5b3@oracle.com> Please, still create webrev on cr. server and add the link to the bug report. It is required to have webrev on cr for openjdk changes. Thanks, Vladimir On 9/27/16 1:36 AM, Andrew Dinn wrote: >> Good. Consider me reviewed test changes. >> >> Please, prepare webrev and changeset on cr.openjdk.java.net. >> >> Ningsheng Jian is not Author in OpenJDK, should be listed as Contributed-by: >> >> Thanks, >> Vladimir > > Oops, apologies -- rather than upload the webrev I actually pushed this > change to hs-comp. I hope that is alright. > > regards, > > > Andrew Dinn > ----------- > Senior Principal Software Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander > From volker.simonis at gmail.com Tue Sep 27 17:49:18 2016 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 27 Sep 2016 19:49:18 +0200 Subject: RFR: JEP draft for Linux/s3990x port Message-ID: Hi, can you please review and endorse the following draft JEP for the integration of the Linux/s390x port into the jkd9 master repository: https://bugs.openjdk.java.net/browse/JDK-8166730 As detailed in the JEP, the Linux/s390x requires very few shared changes and we therefore don't foresee any impact on the existing platforms at all. Following you can find a short description of the planned changes: hotspot: ======= Out for review: 8166560: [s390] Basic enablement of s390 port. http://cr.openjdk.java.net/~goetz/wr16/8166560-basic_s390/hotspot.wr01/ Reviewed: 8166562: C2: Suppress relocations in scratch emit. http://cr.openjdk.java.net/~goetz/wr16/8166562-scratch_emit/webrev.01/ Will send RFR soon (depends on 8166560): 8166561: [s390] Adaptions needed for s390 port in C1 and C2. http://cr.openjdk.java.net/~goetz/wr16/8166562-scratch_emit/webrev.01 We are still investigating the need of these shared changes: http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000011-pass_PC_to_retAddrOffset.patch http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000016-constant_table_offset.patch And finally the patch with the s390x-only platform files. We are still editing these to get them into OpenJdk style and shape. Hotspot passes most jck, jtreg and spec tests with these. http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000101-zFiles.patch top-level repository: =============== The following is just adding some s390x specific compiler flags to flags.m4 8166800: [s390] Top-level build changes required for Linux/s390x https://bugs.openjdk.java.net/browse/JDK-8166800 jdk repository: ============ This one just adds a new jvm.cfg file for s390x 8166801: [s390] Add jvm.cfg file for Linux/s390x https://bugs.openjdk.java.net/browse/JDK-8166801 And finally we plan to do one more change which fixes the jtreg test on Linux/s390x. But this is mainly for the correct detection of the platform and for excluding the tests which are not appropriate for s390x. Thank you and best regards, Volker From volker.simonis at gmail.com Tue Sep 27 17:49:26 2016 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 27 Sep 2016 19:49:26 +0200 Subject: RFR(M): 8166560: [s390] Basic enablement of s390 port. In-Reply-To: <903643bb-9a78-5254-fd0e-108da29802bd@oracle.com> References: <903643bb-9a78-5254-fd0e-108da29802bd@oracle.com> Message-ID: Hi Alan, I've created a JEP. You can find it here: https://bugs.openjdk.java.net/browse/JDK-8166730 I've also just sent an RFR for the JEP to the various list with some more detailed information. It would be great if you could review it for the class library. The changes to the jdk-repo are really minimal - just adding a jvm.cfg file for s390x. Thank you and best regards, Volker On Sat, Sep 24, 2016 at 12:08 PM, Alan Bateman wrote: > On 23/09/2016 06:52, Lindenmaier, Goetz wrote: > >> Hi, >> >> This change is part of the s390 port. It contains some basic adaptions >> needed for a full hotspot port for linux s390x. >> >> > Out of curiosity, is there is JEP for the Linux/S390 port? There were JEPs > for the Linux/AArch64 and PowerPC/AIX ports and just wondering if there is > one coming for the S390 port too? > > -Alan From volker.simonis at gmail.com Tue Sep 27 17:57:53 2016 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 27 Sep 2016 19:57:53 +0200 Subject: RFR(M): 8166560: [s390] Basic enablement of s390 port. In-Reply-To: References: Message-ID: On Fri, Sep 23, 2016 at 8:11 AM, David Holmes wrote: > Hi Goetz, > > I see a change not related directly to S390 ie change from ARM to ARM32 in > src/os/linux/vm/os_linux.cpp > The change looks a little confusing because Goetz reordered the ifdef cascades alphabetically (which I think is good). Besides that, the only real change not related to s390 is indeed the change from ARM to ARM32 which happend two times in the file. @Goetz: have you done this intentionally? > It will be a while before I can go through this in any detail. > > David > > > On 23/09/2016 3:52 PM, Lindenmaier, Goetz wrote: >> >> Hi, >> >> This change is part of the s390 port. It contains some basic adaptions >> needed for a full hotspot port for linux s390x. >> >> It defines the required macros, platform names and includes. >> >> The s390 port calles CodeCache::contains() in current_frame(), which is >> used in NMT. As NMT already collects stack traces before the CodeCache is >> initialized, contains() needs a check for this. >> >> Wherever a row of platforms are listed, I sorted them alphabetically. >> >> The jdk requires the file jvm.cfg. >> >> Please review. I please need a sponsor. >> http://cr.openjdk.java.net/~goetz/wr16/8166560-basic_s390/hotspot.wr01/ >> http://cr.openjdk.java.net/~goetz/wr16/8166560-basic_s390/jdk.wr01/ >> >> Best regards, >> Goetz. >> > From vladimir.kozlov at oracle.com Tue Sep 27 18:15:11 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 27 Sep 2016 11:15:11 -0700 Subject: RFR: JEP draft for Linux/s3990x port In-Reply-To: References: Message-ID: <72ddbd25-c7df-b02a-3827-d431f286c795@oracle.com> On 9/27/16 10:49 AM, Volker Simonis wrote: > Hi, > > can you please review and endorse the following draft JEP for the > integration of the Linux/s390x port into the jkd9 master repository: > > https://bugs.openjdk.java.net/browse/JDK-8166730 Good. Add links to webrevs in a comment. It will help to get umbrella FC extension approval. > > As detailed in the JEP, the Linux/s390x requires very few shared > changes and we therefore don't foresee any impact on the existing > platforms at all. Following you can find a short description of the > planned changes: > > hotspot: > ======= > > Out for review: > 8166560: [s390] Basic enablement of s390 port. > http://cr.openjdk.java.net/~goetz/wr16/8166560-basic_s390/hotspot.wr01/ > > Reviewed: > 8166562: C2: Suppress relocations in scratch emit. > http://cr.openjdk.java.net/~goetz/wr16/8166562-scratch_emit/webrev.01/ > > Will send RFR soon (depends on 8166560): > 8166561: [s390] Adaptions needed for s390 port in C1 and C2. > http://cr.openjdk.java.net/~goetz/wr16/8166562-scratch_emit/webrev.01 Wrong link. Thanks, Vladimir > > We are still investigating the need of these shared changes: > http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000011-pass_PC_to_retAddrOffset.patch > http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000016-constant_table_offset.patch > > And finally the patch with the s390x-only platform files. We are still > editing these to get them into OpenJdk style and shape. > Hotspot passes most jck, jtreg and spec tests with these. > http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000101-zFiles.patch > > top-level repository: > =============== > > The following is just adding some s390x specific compiler flags to flags.m4 > 8166800: [s390] Top-level build changes required for Linux/s390x > https://bugs.openjdk.java.net/browse/JDK-8166800 > > jdk repository: > ============ > > This one just adds a new jvm.cfg file for s390x > 8166801: [s390] Add jvm.cfg file for Linux/s390x > https://bugs.openjdk.java.net/browse/JDK-8166801 > > > And finally we plan to do one more change which fixes the jtreg test > on Linux/s390x. But this is mainly for the correct detection of the > platform and for excluding the tests which are not appropriate for > s390x. > > Thank you and best regards, > Volker > From coleen.phillimore at oracle.com Tue Sep 27 18:29:50 2016 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 27 Sep 2016 14:29:50 -0400 Subject: RFR(M): 8166560: [s390] Basic enablement of s390 port. In-Reply-To: References: Message-ID: <7b9201ce-a417-e489-7b46-c0488e8fda8f@oracle.com> Hi, I'm surprised there aren't more changes, probably because of the #include work that was done by you previously. Which is great. http://cr.openjdk.java.net/~goetz/wr16/8166560-basic_s390/hotspot.wr01/src/share/vm/runtime/mutex.hpp.udiff.html Can you describe this change? This makes all of the mutex's take a lot of space, and some don't have very long names. Was this added to avoid false sharing as the comment implies or to support longer names in monitors? thanks, Coleen On 9/27/16 1:57 PM, Volker Simonis wrote: > On Fri, Sep 23, 2016 at 8:11 AM, David Holmes wrote: >> Hi Goetz, >> >> I see a change not related directly to S390 ie change from ARM to ARM32 in >> src/os/linux/vm/os_linux.cpp >> > The change looks a little confusing because Goetz reordered the ifdef > cascades alphabetically (which I think is good). > > Besides that, the only real change not related to s390 is indeed the > change from ARM to ARM32 which happend two times in the file. > > @Goetz: have you done this intentionally? > >> It will be a while before I can go through this in any detail. >> >> David >> >> >> On 23/09/2016 3:52 PM, Lindenmaier, Goetz wrote: >>> Hi, >>> >>> This change is part of the s390 port. It contains some basic adaptions >>> needed for a full hotspot port for linux s390x. >>> >>> It defines the required macros, platform names and includes. >>> >>> The s390 port calles CodeCache::contains() in current_frame(), which is >>> used in NMT. As NMT already collects stack traces before the CodeCache is >>> initialized, contains() needs a check for this. >>> >>> Wherever a row of platforms are listed, I sorted them alphabetically. >>> >>> The jdk requires the file jvm.cfg. >>> >>> Please review. I please need a sponsor. >>> http://cr.openjdk.java.net/~goetz/wr16/8166560-basic_s390/hotspot.wr01/ >>> http://cr.openjdk.java.net/~goetz/wr16/8166560-basic_s390/jdk.wr01/ >>> >>> Best regards, >>> Goetz. >>> From kim.barrett at oracle.com Tue Sep 27 20:45:22 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 27 Sep 2016 16:45:22 -0400 Subject: Request for review (s) - 8155948: Add message for CMS deprecation for some internal builds In-Reply-To: <030ef60d-c99a-4a40-3603-dcd08f9ead5c@oracle.com> References: <74eccd9a-1525-a9ef-1608-294de99d4483@oracle.com> <52db37a5-efa4-79dd-771f-45e1fc1ccaf5@oracle.com> <02968df4-b53d-c1d1-65e4-a612ca393bae@oracle.com> <5CB1B554-D733-40D6-881F-6237F9CE9E11@oracle.com> <50bac55e-b92c-f5d5-cc29-f0a85fdc6923@oracle.com> <3577baee-6ed2-d636-ee7d-7af33e0548b5@oracle.com> <14E40EE8-1A93-419E-8E30-A517E5C4F9E9@oracle.com> <67d0ba6f-014f-6cb7-0132-ae950d4e4af9@oracle.com> <030ef60d-c99a-4a40-3603-dcd08f9ead5c@oracle.com> Message-ID: <54FC4ED1-B348-464F-BB85-D0C81F519DF8@oracle.com> > On Sep 27, 2016, at 11:16 AM, Jon Masamitsu wrote: > > Kim, > > I fixed > > https://bugs.openjdk.java.net/browse/JDK-8166461 > > so that the fix version is 9. Thanks. That addresses my objection to the latest version (webrev.04) for 8155948. > Jon > > On 9/26/2016 2:18 PM, Jon Masamitsu wrote: >> Kim, >> >> I thought that upshot of the discussion earlier in the week was that >> the UseAutoGCSelectPolicy flag should be deprecated itself and >> so would circumvent the warning about CMS being deprecated if >> UseAutoGCSelectPolicy picked CMS. Dima's point was that GC >> ergonomics should not pick a collector that was deprecated. >> >> Not so? >> >> Jon >> >> On 9/23/2016 3:33 PM, Kim Barrett wrote: >>>> On Sep 23, 2016, at 2:25 PM, Jon Masamitsu wrote: >>>> >>>> After some discussion I've removed the handling of the >>>> UseAutoGCSelectPolicy flag from this patch. It will be handled >>>> separately under >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8166461 >>>> >>>> >>>> The new full webrev is >>>> >>>> http://cr.openjdk.java.net/~jmasa/8155948/webrev.04/ >>>> >>>> The delta is >>>> >>>> http://cr.openjdk.java.net/~jmasa/8155948/webrev_delta.03_04/index.html >>>> >>>> Thanks. >>>> >>>> Jon >>> ------------------------------------------------------------------------------ >>> src/share/vm/runtime/arguments.cpp >>> >>> I think this line should not be removed: >>> >>> 1807 handle_extra_cms_flags("UseAutoGCSelectPolicy uses UseConcMarkSweepGC"); >>> >>> Removal of this line means that commercial builds that otherwise warn >>> about CMS being deprecated when it is selected won't warn if it is >>> selected via this mechanism. >>> >>> Similarly for the test. >>> >>> So I liked the previous version and not so much this one. >>> >>> ------------------------------------------------------------------------------ From mikael.vidstedt at oracle.com Tue Sep 27 22:07:49 2016 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Tue, 27 Sep 2016 15:07:49 -0700 Subject: RFR(XS): 8166045: jdk/internal/misc/Unsafe tests fail due to timeout In-Reply-To: <6771FA24-EB00-4203-94F5-F2375E9FFB08@oracle.com> References: <6771FA24-EB00-4203-94F5-F2375E9FFB08@oracle.com> Message-ID: <2604B8A4-955A-4829-A3E1-96337D026C67@oracle.com> Thanks for the reviews! Cheers, Mikael > On Sep 26, 2016, at 4:11 PM, Christian Tornqvist wrote: > > Hi Mikael, > > This looks good, thanks for fixing this. > > Thanks, > Christian > > On Sep 26, 2016, at 5:53 PM, Mikael Vidstedt > wrote: > >> >> Please review the following change: >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8166045 >> > >> Webrev: >> http://cr.openjdk.java.net/~mikael/webrevs/8166045/webrev.01/jdk/webrev/ >> > >> >> Summary: >> >> Among the tests I introduced for Unsafe.copyMemory and Unsafe.copySwapMemory >> a few months ago are a couple of tests which rely on being able to allocate >> and operate on a large (~2GB) block of native memory. However, on some >> smallish machines the actual allocation succeeds, but the machine then goes >> on to swap (as in, page to/from disk) like crazy, making the test time out. >> >> In an attempt to work around this I?m using @requires in combination with >> the value of os.maxMemory. I could make the whole test(s) conditional on the >> memory size, but the vast majority of the test(s) actually operate on a very >> small block of memory (tens of bytes), so disabling the whole test seems >> unfortunate. Instead I decided to split up the test(s) in two - one set >> which tests the default/small size, and one set which tests the large size >> iff there is enough memory on the machine. >> >> I have tested manually that the @requires check has the right effect - the >> tests are skipped if there?s not enough memory on the machine - and that the >> tests still pass as expected. >> >> I also took the liberty of removing a superfluous import. >> >> Cheers, >> Mikael >> From adinn at redhat.com Wed Sep 28 07:53:42 2016 From: adinn at redhat.com (Andrew Dinn) Date: Wed, 28 Sep 2016 08:53:42 +0100 Subject: RFR: 165673: AArch64: Fix JNI floating point argument handling In-Reply-To: <39663abb-b293-8349-c815-aa0cabd9f5b3@oracle.com> References: <2ba05d11-1119-5cf6-e84d-2f0cdb03dd98@redhat.com> <57E93FC2.8050401@oracle.com> <57E94278.6020008@oracle.com> <57E94B2B.8070609@oracle.com> <2fc1edfe-2594-751e-64f1-b34aa940165d@redhat.com> <39663abb-b293-8349-c815-aa0cabd9f5b3@oracle.com> Message-ID: On 27/09/16 18:02, Vladimir Kozlov wrote: > Please, still create webrev on cr. server and add the link to the bug > report. > It is required to have webrev on cr for openjdk changes. The webrev is available here http://cr.openjdk.java.net/~adinn/8165673/webrev.00 regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From goetz.lindenmaier at sap.com Wed Sep 28 09:52:40 2016 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 28 Sep 2016 09:52:40 +0000 Subject: RFR(M): 8166560: [s390] Basic enablement of s390 port. In-Reply-To: <7b9201ce-a417-e489-7b46-c0488e8fda8f@oracle.com> References: <7b9201ce-a417-e489-7b46-c0488e8fda8f@oracle.com> Message-ID: Hi Coleen, yes, the #include change simplified this a lot. Also maintaining the patch queue we have ... We reordered the fields of mutex and aligned it to cache line boundaries. This is the S390 part of this alignment change. S390 has 256B cache lines. I'll remove it as the other parts of this change are missing anyways. I?ll send a new webrev, soon. Best regards, Goetz > -----Original Message----- > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On > Behalf Of Coleen Phillimore > Sent: Dienstag, 27. September 2016 20:30 > To: hotspot-dev at openjdk.java.net > Subject: Re: RFR(M): 8166560: [s390] Basic enablement of s390 port. > > > Hi, I'm surprised there aren't more changes, probably because of the > #include work that was done by you previously. Which is great. > > http://cr.openjdk.java.net/~goetz/wr16/8166560- > basic_s390/hotspot.wr01/src/share/vm/runtime/mutex.hpp.udiff.html > > Can you describe this change? This makes all of the mutex's take a lot > of space, and some don't have very long names. Was this added to avoid > false sharing as the comment implies or to support longer names in > monitors? > > thanks, > Coleen > > > > On 9/27/16 1:57 PM, Volker Simonis wrote: > > On Fri, Sep 23, 2016 at 8:11 AM, David Holmes > wrote: > >> Hi Goetz, > >> > >> I see a change not related directly to S390 ie change from ARM to ARM32 > in > >> src/os/linux/vm/os_linux.cpp > >> > > The change looks a little confusing because Goetz reordered the ifdef > > cascades alphabetically (which I think is good). > > > > Besides that, the only real change not related to s390 is indeed the > > change from ARM to ARM32 which happend two times in the file. > > > > @Goetz: have you done this intentionally? > > > >> It will be a while before I can go through this in any detail. > >> > >> David > >> > >> > >> On 23/09/2016 3:52 PM, Lindenmaier, Goetz wrote: > >>> Hi, > >>> > >>> This change is part of the s390 port. It contains some basic adaptions > >>> needed for a full hotspot port for linux s390x. > >>> > >>> It defines the required macros, platform names and includes. > >>> > >>> The s390 port calles CodeCache::contains() in current_frame(), which is > >>> used in NMT. As NMT already collects stack traces before the CodeCache > is > >>> initialized, contains() needs a check for this. > >>> > >>> Wherever a row of platforms are listed, I sorted them alphabetically. > >>> > >>> The jdk requires the file jvm.cfg. > >>> > >>> Please review. I please need a sponsor. > >>> http://cr.openjdk.java.net/~goetz/wr16/8166560- > basic_s390/hotspot.wr01/ > >>> http://cr.openjdk.java.net/~goetz/wr16/8166560-basic_s390/jdk.wr01/ > >>> > >>> Best regards, > >>> Goetz. > >>> From goetz.lindenmaier at sap.com Wed Sep 28 10:26:47 2016 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 28 Sep 2016 10:26:47 +0000 Subject: RFR(M): 8166560: [s390] Basic enablement of s390 port. In-Reply-To: References: Message-ID: <4c1b77efa5014efc94d31a20e51ae657@DEWDFE13DE50.global.corp.sap> Hi, here a new webrev for this change: http://cr.openjdk.java.net/~goetz/wr16/8166560-basic_s390/hotspot.wr02/ I reverted the major reorderings in macros.hpp and os_linux.cpp. David asked me to do so, and I guess it makes reviewing more simple. Also this fixes the issue spotted by David which actually was wrong. The other renaming of ARM to ARM32 was correct, though, as AARCH64 is defined in both ARM 64-bit ports, and is checked before. So the existing case checking ARM is only reached if !LP64. I documented this ... Also I removed the change in mutex.hpp. Maybe we contribute the full change this was part of, but independent of the s390 port. I withdraw the part of the change to jdk introducing jvm.cfg. Volker wants to do a separate change for this. Best regards, Goetz. > -----Original Message----- > From: Volker Simonis [mailto:volker.simonis at gmail.com] > Sent: Dienstag, 27. September 2016 19:58 > To: David Holmes > Cc: Lindenmaier, Goetz ; hotspot- > dev at openjdk.java.net; core-libs-dev > Subject: Re: RFR(M): 8166560: [s390] Basic enablement of s390 port. > > On Fri, Sep 23, 2016 at 8:11 AM, David Holmes > wrote: > > Hi Goetz, > > > > I see a change not related directly to S390 ie change from ARM to ARM32 in > > src/os/linux/vm/os_linux.cpp > > > > The change looks a little confusing because Goetz reordered the ifdef > cascades alphabetically (which I think is good). > > Besides that, the only real change not related to s390 is indeed the > change from ARM to ARM32 which happend two times in the file. > > @Goetz: have you done this intentionally? > > > It will be a while before I can go through this in any detail. > > > > David > > > > > > On 23/09/2016 3:52 PM, Lindenmaier, Goetz wrote: > >> > >> Hi, > >> > >> This change is part of the s390 port. It contains some basic adaptions > >> needed for a full hotspot port for linux s390x. > >> > >> It defines the required macros, platform names and includes. > >> > >> The s390 port calles CodeCache::contains() in current_frame(), which is > >> used in NMT. As NMT already collects stack traces before the CodeCache > is > >> initialized, contains() needs a check for this. > >> > >> Wherever a row of platforms are listed, I sorted them alphabetically. > >> > >> The jdk requires the file jvm.cfg. > >> > >> Please review. I please need a sponsor. > >> http://cr.openjdk.java.net/~goetz/wr16/8166560- > basic_s390/hotspot.wr01/ > >> http://cr.openjdk.java.net/~goetz/wr16/8166560-basic_s390/jdk.wr01/ > >> > >> Best regards, > >> Goetz. > >> > > From bob.vandette at oracle.com Wed Sep 28 14:07:59 2016 From: bob.vandette at oracle.com (Bob Vandette) Date: Wed, 28 Sep 2016 10:07:59 -0400 Subject: Webrev of Oracle ARM & AARCH64 Sources Message-ID: <5A08C4A6-2B69-4D29-80F5-1CE11E350283@oracle.com> I?m am please to announce that I have completed our internal reviews and can now open up the sources to our ARM 32 & 64 bit implementations of JDK9. Here is a webrev that includes a patch that can be applied on top of the (http://hg.openjdk.java.net/aarch32-port/jdk9-arm3264/ ) forest. http://cr.openjdk.java.net/~bobv/arm3264/webrev Notes: 1. Counter to my initial email announcing the open sourcing effort, the pregenerated interpreter is not included in the provided patch. AOT is our long term solution for increased performance on Mobile and it was decided that we do not want to support the static interpreter beyond JDK 8. 2. In order to allow building the two different 64-bit ARM sources, I overloaded a new configure option ?abi-profile to allow for the selection of the Oracle 64-bit ARM port. We need a better solution before this support can be integrated into the mainline. To use the Oracle sources for an aarch64 build, select ?abi-profile=arm64. 3. The scripts provided below allows for the creation of the minimal, client and server VMs for the Oracle aarch64 build, however, the jvm.cfg file is not automatically updated during the build. To run a non server VM, this file will need to be updated. 4. In order to make it easier to keep the new open sources up to date with their closed counterparts in jdk9, I have added a comment in hotspot/src/cpu/arm/vm/vm_version_ext_arm.hpp which contains the mercurial revision that I last merged with. This will make it easier to produce patches to bring the new open files up to the latest version. /* ARM sources Merged up to hotspot/src/closed changeset 2316:b247a2ea70e4 */ Build Scripts: Here are a set of scripts that demonstrate how to build the three ARM configurations. We typically cross compile our ARM builds so these scripts will have to be adjusted in order to build natively on an ARM system. http://cr.openjdk.java.net/~bobv/arm3264/scripts/ aarch64.csh Builds the current open version of aarch64 binaries arm64.csh Builds the Oracle aarch64 binaries arm.csh Builds 32-bit ARM binaries My next step is to push this changeset into the jdk9-arm3264 forest assuming I achieve committer status for the aarch32 project this week. Bob Vandette From erik.helin at oracle.com Wed Sep 28 15:01:58 2016 From: erik.helin at oracle.com (Erik Helin) Date: Wed, 28 Sep 2016 17:01:58 +0200 Subject: RFR: 8166790: Add stress test GCBasher Message-ID: <20160928150158.GE20056@ehelin.jrpg.bea.com> Hi all, this patch adds a new GC stress test called GCBasher. GCBasher builds up large (well, for some definiton of large) object graphs by figuring out the relations between classes in the JDK. The test usually stresses the GC quite a lot, especially when run with a smaller heap. The changes in the top-level repository are for JPRT. JPRT will now run the jtreg test GCBasher instead of the old version. Enhancement: https://bugs.openjdk.java.net/browse/JDK-8166790 Webrev: - hotspot: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/00/ - top: http://cr.openjdk.java.net/~ehelin/8166790/top/00/ Testing: - Running the new jtreg test locally on Linux x86-64: $ jtreg -jdk:build/fastdebug/jdk hotspot/test/gc/stress/TestGCBasher.java (can also be run via: $ make test TEST=hotspot_fast_gc_gcbasher) - JPRT Thanks, Erik From erik.joelsson at oracle.com Wed Sep 28 15:08:46 2016 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Wed, 28 Sep 2016 17:08:46 +0200 Subject: RFR: 8166790: Add stress test GCBasher In-Reply-To: <20160928150158.GE20056@ehelin.jrpg.bea.com> References: <20160928150158.GE20056@ehelin.jrpg.bea.com> Message-ID: JPRT changes look good to me. /Erik On 2016-09-28 17:01, Erik Helin wrote: > Hi all, > > this patch adds a new GC stress test called GCBasher. GCBasher builds up > large (well, for some definiton of large) object graphs by figuring out > the relations between classes in the JDK. The test usually stresses the > GC quite a lot, especially when run with a smaller heap. > > The changes in the top-level repository are for JPRT. JPRT will now run > the jtreg test GCBasher instead of the old version. > > Enhancement: > https://bugs.openjdk.java.net/browse/JDK-8166790 > > Webrev: > - hotspot: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/00/ > - top: http://cr.openjdk.java.net/~ehelin/8166790/top/00/ > > Testing: > - Running the new jtreg test locally on Linux x86-64: > $ jtreg -jdk:build/fastdebug/jdk hotspot/test/gc/stress/TestGCBasher.java > > (can also be run via: $ make test TEST=hotspot_fast_gc_gcbasher) > - JPRT > > Thanks, > Erik From vladimir.kozlov at oracle.com Wed Sep 28 16:50:08 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 28 Sep 2016 09:50:08 -0700 Subject: RFR: 165673: AArch64: Fix JNI floating point argument handling In-Reply-To: References: <2ba05d11-1119-5cf6-e84d-2f0cdb03dd98@redhat.com> <57E93FC2.8050401@oracle.com> <57E94278.6020008@oracle.com> <57E94B2B.8070609@oracle.com> <2fc1edfe-2594-751e-64f1-b34aa940165d@redhat.com> <39663abb-b293-8349-c815-aa0cabd9f5b3@oracle.com> Message-ID: Thanks! Vladimir On 9/28/16 12:53 AM, Andrew Dinn wrote: > On 27/09/16 18:02, Vladimir Kozlov wrote: >> Please, still create webrev on cr. server and add the link to the bug >> report. >> It is required to have webrev on cr for openjdk changes. > > The webrev is available here > > http://cr.openjdk.java.net/~adinn/8165673/webrev.00 > > regards, > > > Andrew Dinn > ----------- > Senior Principal Software Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander > From volker.simonis at gmail.com Wed Sep 28 17:55:18 2016 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 28 Sep 2016 19:55:18 +0200 Subject: Webrev of Oracle ARM & AARCH64 Sources In-Reply-To: <5A08C4A6-2B69-4D29-80F5-1CE11E350283@oracle.com> References: <5A08C4A6-2B69-4D29-80F5-1CE11E350283@oracle.com> Message-ID: Hi Bob, congratulations! I'm pretty sure this was a pretty huge amount of paper and programming work to reach this point :) My general question is how is this code contribution supposed to work together with the current ARM ports in the OpenJDK. Currently we have a full-blown, supported arm64 bit port in the jdk9 main-line (contributed and maintained by RedHat). The aarch32 project has a full-blown implementation for arm32 for jdk8u contributed by Azul. Is your new port intended to live and be maintained alongside these already available ports? Where does your port differ from the available ports in terms of functionality and performance? Is Oracle committed to support the new ports? And finally, do you expect to integrate these port into JDK 9 mainline? Thanks a lot and best regards, Volker On Wed, Sep 28, 2016 at 4:07 PM, Bob Vandette wrote: > I?m am please to announce that I have completed our internal reviews and can now > open up the sources to our ARM 32 & 64 bit implementations of JDK9. > > Here is a webrev that includes a patch that can be applied on top of the > (http://hg.openjdk.java.net/aarch32-port/jdk9-arm3264/ ) forest. > > http://cr.openjdk.java.net/~bobv/arm3264/webrev > > Notes: > > 1. Counter to my initial email announcing the open sourcing effort, the > pregenerated interpreter is not included in the provided patch. AOT is > our long term solution for increased performance on Mobile and it was decided > that we do not want to support the static interpreter beyond JDK 8. > > 2. In order to allow building the two different 64-bit ARM sources, I overloaded > a new configure option ?abi-profile to allow for the selection of the Oracle > 64-bit ARM port. We need a better solution before this support can be > integrated into the mainline. To use the Oracle sources for an aarch64 build, > select ?abi-profile=arm64. > > 3. The scripts provided below allows for the creation of the minimal, client and > server VMs for the Oracle aarch64 build, however, the jvm.cfg file is not > automatically updated during the build. To run a non server VM, this file > will need to be updated. > > 4. In order to make it easier to keep the new open sources up to date with > their closed counterparts in jdk9, I have added a comment in > hotspot/src/cpu/arm/vm/vm_version_ext_arm.hpp which contains the mercurial > revision that I last merged with. This will make it easier to produce patches > to bring the new open files up to the latest version. > > /* ARM sources Merged up to hotspot/src/closed changeset 2316:b247a2ea70e4 */ > > Build Scripts: > > Here are a set of scripts that demonstrate how to build the three ARM > configurations. We typically cross compile our ARM builds so these > scripts will have to be adjusted in order to build natively on an ARM system. > > http://cr.openjdk.java.net/~bobv/arm3264/scripts/ > > aarch64.csh Builds the current open version of aarch64 binaries > arm64.csh Builds the Oracle aarch64 binaries > arm.csh Builds 32-bit ARM binaries > > My next step is to push this changeset into the jdk9-arm3264 forest assuming I > achieve committer status for the aarch32 project this week. > > Bob Vandette > > > > From bob.vandette at oracle.com Wed Sep 28 18:46:21 2016 From: bob.vandette at oracle.com (Bob Vandette) Date: Wed, 28 Sep 2016 14:46:21 -0400 Subject: Webrev of Oracle ARM & AARCH64 Sources In-Reply-To: References: <5A08C4A6-2B69-4D29-80F5-1CE11E350283@oracle.com> Message-ID: > On Sep 28, 2016, at 1:55 PM, Volker Simonis wrote: > > Hi Bob, > > congratulations! I'm pretty sure this was a pretty huge amount of > paper and programming work to reach this point :) > > My general question is how is this code contribution supposed to work > together with the current ARM ports in the OpenJDK. Currently we have > a full-blown, supported arm64 bit port in the jdk9 main-line > (contributed and maintained by RedHat). The aarch32 project has a > full-blown implementation for arm32 for jdk8u contributed by Azul. I didn?t think the arm32 project had a C2 server VM yet? > > Is your new port intended to live and be maintained alongside these > already available ports? At this point we are simply offering to contribute our ARM implementations to the community and no decisions on consolidation of ports has been decided. > Where does your port differ from the available ports in terms of > functionality and performance? The primary reason for putting the webrev out in the open is to allow for the review of our implementation by the aarch32 project team. ARM32 Our 32-bit ARM port is very mature and has been around for over 8 years. These sources support several 32-bit ARM architecture versions including ARMv5, ARMv6 and ARMv7 with VFP hard-float, soft-fp and full soft-float floating point support. Both 32 and 64 bit ARM ports support the building of the minimal, client and server hotspot VMs. We also support the compilation of native binaries with thumb2. AARCH64 Our AARCH64 bit port is fully functional and supports the same three VM combinations. MAINTAINABILITY As you can see from the sources, we designed the ARM ports to share as much of the sources for 32 and 64 bit ARM support as possible making it easier to maintain and add CPU specific features to hotspot. This design philosophy is consistent with our sparc and x86 ports. PERFORMANCE I have not done a performance comparison of the existing open ports. Note: Some of the Oracle commercial features may not be fully supported on some of the ARM platforms. > Is Oracle committed to support the new ports? Oracle is committed to producing 32 & 64 bit ARM binaries for at least the JDK 9 product life cycle. We will be producing our ARM binaries from our closed sources for JDK 9.0. > And finally, do you expect to integrate these ports into JDK 9 mainline? I would like to work with the aarch32 project team towards that goal. Bob. > Thanks a lot and best regards, > Volker > > > On Wed, Sep 28, 2016 at 4:07 PM, Bob Vandette wrote: >> I?m am please to announce that I have completed our internal reviews and can now >> open up the sources to our ARM 32 & 64 bit implementations of JDK9. >> >> Here is a webrev that includes a patch that can be applied on top of the >> (http://hg.openjdk.java.net/aarch32-port/jdk9-arm3264/ ) forest. >> >> http://cr.openjdk.java.net/~bobv/arm3264/webrev >> >> Notes: >> >> 1. Counter to my initial email announcing the open sourcing effort, the >> pregenerated interpreter is not included in the provided patch. AOT is >> our long term solution for increased performance on Mobile and it was decided >> that we do not want to support the static interpreter beyond JDK 8. >> >> 2. In order to allow building the two different 64-bit ARM sources, I overloaded >> a new configure option ?abi-profile to allow for the selection of the Oracle >> 64-bit ARM port. We need a better solution before this support can be >> integrated into the mainline. To use the Oracle sources for an aarch64 build, >> select ?abi-profile=arm64. >> >> 3. The scripts provided below allows for the creation of the minimal, client and >> server VMs for the Oracle aarch64 build, however, the jvm.cfg file is not >> automatically updated during the build. To run a non server VM, this file >> will need to be updated. >> >> 4. In order to make it easier to keep the new open sources up to date with >> their closed counterparts in jdk9, I have added a comment in >> hotspot/src/cpu/arm/vm/vm_version_ext_arm.hpp which contains the mercurial >> revision that I last merged with. This will make it easier to produce patches >> to bring the new open files up to the latest version. >> >> /* ARM sources Merged up to hotspot/src/closed changeset 2316:b247a2ea70e4 */ >> >> Build Scripts: >> >> Here are a set of scripts that demonstrate how to build the three ARM >> configurations. We typically cross compile our ARM builds so these >> scripts will have to be adjusted in order to build natively on an ARM system. >> >> http://cr.openjdk.java.net/~bobv/arm3264/scripts/ >> >> aarch64.csh Builds the current open version of aarch64 binaries >> arm64.csh Builds the Oracle aarch64 binaries >> arm.csh Builds 32-bit ARM binaries >> >> My next step is to push this changeset into the jdk9-arm3264 forest assuming I >> achieve committer status for the aarch32 project this week. >> >> Bob Vandette >> >> >> >> From aph at redhat.com Wed Sep 28 19:15:06 2016 From: aph at redhat.com (Andrew Haley) Date: Wed, 28 Sep 2016 20:15:06 +0100 Subject: [aarch64-port-dev ] Webrev of Oracle ARM & AARCH64 Sources In-Reply-To: References: <5A08C4A6-2B69-4D29-80F5-1CE11E350283@oracle.com> Message-ID: <7ea0939d-4aa1-05fc-1eac-f1da849920f4@redhat.com> On 28/09/16 18:55, Volker Simonis wrote: > My general question is how is this code contribution supposed to work > together with the current ARM ports in the OpenJDK. Currently we have > a full-blown, supported arm64 bit port in the jdk9 main-line > (contributed and maintained by RedHat). The aarch32 project has a > full-blown implementation for arm32 for jdk8u contributed by Azul. > > Is your new port intended to live and be maintained alongside these > already available ports? > Where does your port differ from the available ports in terms of > functionality and performance? > Is Oracle committed to support the new ports? > And finally, do you expect to integrate these port into JDK 9 mainline? We have to decide this by consensus. Azul seem to be happy enough to drop their own port and work on this one from Oracle: their work is new and rather immature, so it makes sense. In the case of AArch64, the situation is much less clear. We have a fairly mature and solid community-written (not just Red Hat!) implementation, and there's no obvious reason to push that implementation aside. It is the official AArch64 port of OpenJDK. I'll be posting more in the coming weeks. Andrew. From kostya at azul.com Wed Sep 28 19:22:20 2016 From: kostya at azul.com (Kostya Zolotnikov) Date: Wed, 28 Sep 2016 19:22:20 +0000 Subject: [aarch64-port-dev ] Webrev of Oracle ARM & AARCH64 Sources In-Reply-To: <7ea0939d-4aa1-05fc-1eac-f1da849920f4@redhat.com> References: <5A08C4A6-2B69-4D29-80F5-1CE11E350283@oracle.com> , <7ea0939d-4aa1-05fc-1eac-f1da849920f4@redhat.com> Message-ID: Hi, Andrew Not sure who told You about Azul's view. For us it is not that clear what make sense for You. Regards Kostya ?????????? ?? ????????? Sony Xperia? ----???????????? Andrew Haley ??????? ---- On 28/09/16 18:55, Volker Simonis wrote: > My general question is how is this code contribution supposed to work > together with the current ARM ports in the OpenJDK. Currently we have > a full-blown, supported arm64 bit port in the jdk9 main-line > (contributed and maintained by RedHat). The aarch32 project has a > full-blown implementation for arm32 for jdk8u contributed by Azul. > > Is your new port intended to live and be maintained alongside these > already available ports? > Where does your port differ from the available ports in terms of > functionality and performance? > Is Oracle committed to support the new ports? > And finally, do you expect to integrate these port into JDK 9 mainline? We have to decide this by consensus. Azul seem to be happy enough to drop their own port and work on this one from Oracle: their work is new and rather immature, so it makes sense. In the case of AArch64, the situation is much less clear. We have a fairly mature and solid community-written (not just Red Hat!) implementation, and there's no obvious reason to push that implementation aside. It is the official AArch64 port of OpenJDK. I'll be posting more in the coming weeks. Andrew. From david.holmes at oracle.com Thu Sep 29 04:14:18 2016 From: david.holmes at oracle.com (David Holmes) Date: Thu, 29 Sep 2016 14:14:18 +1000 Subject: RFR: 8166790: Add stress test GCBasher In-Reply-To: <20160928150158.GE20056@ehelin.jrpg.bea.com> References: <20160928150158.GE20056@ehelin.jrpg.bea.com> Message-ID: Hi Erik, On 29/09/2016 1:01 AM, Erik Helin wrote: > Hi all, > > this patch adds a new GC stress test called GCBasher. GCBasher builds up > large (well, for some definiton of large) object graphs by figuring out > the relations between classes in the JDK. The test usually stresses the > GC quite a lot, especially when run with a smaller heap. > > The changes in the top-level repository are for JPRT. JPRT will now run > the jtreg test GCBasher instead of the old version. > > Enhancement: > https://bugs.openjdk.java.net/browse/JDK-8166790 > > Webrev: > - hotspot: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/00/ Can't comment on actual test but placement etc looks fine. One query: test/gc/stress/gcbasher/TestGCBasher.java You are running with -server explicitly. Should you not also have @requires vm == server (or whatever the syntax is) as well? > - top: http://cr.openjdk.java.net/~ehelin/8166790/top/00/ Nit: The new end lines like: 337 solaris_sparcv9_5.11-product-c2-runThese8, 345 solaris_x64_5.11-product-c2-runThese8_Xcomp_vm, 370 windows_i586_6.3-product-c2-runThese8_Xcomp_vm, 378 windows_x64_6.3-product-c2-runThese8_Xcomp_vm, should not have the trailing comma. This seems wrong: 443 ${my.make.rule.test.targets.hotspot.reg.group:GROUP=hotspot_fast_gc_gcbasher}, \ 444 ${my.make.rule.test.targets.hotspot.reg.group:GROUP=hotspot_fast_runtime}, \ 445 ${my.make.rule.test.targets.hotspot.reg.group:GROUP=hotspot_fast_serviceability}, \ 446 ${my.make.rule.test.targets.hotspot.reg.group:GROUP=jdk_svc_sanity}, \ 447 solaris_sparcv9_5.11-product-c2-hotspot_fast_gc_gcbasher, \ 448 solaris_x64_5.11-product-c2-hotspot_fast_gc_gcbasher, \ 449 linux_i586_3.8-product-c2-hotspot_fast_gc_gcbasher, \ 450 linux_x64_3.8-product-c2-hotspot_fast_gc_gcbasher, \ 451 macosx_x64_10.9-product-c2-hotspot_fast_gc_gcbasher, \ 452 windows_i586_6.3-product-c2-hotspot_fast_gc_gcbasher, \ 453 windows_x64_6.3-product-c2-hotspot_fast_gc_gcbasher, \ You've added the new tests through a group substitution at line 443, but then have added each platform individually. I don't think 447 - 453 should be there else the test will run twice per platform. Thanks, David ----- > Testing: > - Running the new jtreg test locally on Linux x86-64: > $ jtreg -jdk:build/fastdebug/jdk hotspot/test/gc/stress/TestGCBasher.java > > (can also be run via: $ make test TEST=hotspot_fast_gc_gcbasher) > - JPRT > > Thanks, > Erik > From erik.joelsson at oracle.com Thu Sep 29 07:46:52 2016 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Thu, 29 Sep 2016 09:46:52 +0200 Subject: RFR: 8166790: Add stress test GCBasher In-Reply-To: References: <20160928150158.GE20056@ehelin.jrpg.bea.com> Message-ID: <3dc86351-c557-b785-3073-96b587ca08e9@oracle.com> On 2016-09-29 06:14, David Holmes wrote: > Hi Erik, > > On 29/09/2016 1:01 AM, Erik Helin wrote: > > > This seems wrong: > > 443 > ${my.make.rule.test.targets.hotspot.reg.group:GROUP=hotspot_fast_gc_gcbasher}, > \ > 444 > ${my.make.rule.test.targets.hotspot.reg.group:GROUP=hotspot_fast_runtime}, > \ > 445 > ${my.make.rule.test.targets.hotspot.reg.group:GROUP=hotspot_fast_serviceability}, > \ > 446 > ${my.make.rule.test.targets.hotspot.reg.group:GROUP=jdk_svc_sanity}, > \ > 447 solaris_sparcv9_5.11-product-c2-hotspot_fast_gc_gcbasher, > \ > 448 solaris_x64_5.11-product-c2-hotspot_fast_gc_gcbasher, > \ > 449 linux_i586_3.8-product-c2-hotspot_fast_gc_gcbasher, > \ > 450 linux_x64_3.8-product-c2-hotspot_fast_gc_gcbasher, > \ > 451 macosx_x64_10.9-product-c2-hotspot_fast_gc_gcbasher, > \ > 452 windows_i586_6.3-product-c2-hotspot_fast_gc_gcbasher, > \ > 453 windows_x64_6.3-product-c2-hotspot_fast_gc_gcbasher, > \ > > You've added the new tests through a group substitution at line 443, > but then have added each platform individually. I don't think 447 - > 453 should be there else the test will run twice per platform. > The groups only add fastdebug. The extra lines are for product. The other jtreg tests only run on fastdebug while gcbasher has traditionally been run on both product and fastdebug. I believe Erik just wanted to keep it the same. /Erik From erik.helin at oracle.com Thu Sep 29 07:52:57 2016 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 29 Sep 2016 09:52:57 +0200 Subject: RFR: 8166790: Add stress test GCBasher In-Reply-To: References: <20160928150158.GE20056@ehelin.jrpg.bea.com> Message-ID: <20160929075257.GF20056@ehelin.jrpg.bea.com> On 2016-09-29, David Holmes wrote: > Hi Erik, Hi David, thanks for reviewing! > On 29/09/2016 1:01 AM, Erik Helin wrote: > >Hi all, > > > >this patch adds a new GC stress test called GCBasher. GCBasher builds up > >large (well, for some definiton of large) object graphs by figuring out > >the relations between classes in the JDK. The test usually stresses the > >GC quite a lot, especially when run with a smaller heap. > > > >The changes in the top-level repository are for JPRT. JPRT will now run > >the jtreg test GCBasher instead of the old version. > > > >Enhancement: > >https://bugs.openjdk.java.net/browse/JDK-8166790 > > > >Webrev: > >- hotspot: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/00/ > > Can't comment on actual test but placement etc looks fine. Thanks! > One query: > > test/gc/stress/gcbasher/TestGCBasher.java > > You are running with -server explicitly. Should you not also have @requires > vm == server (or whatever the syntax is) as well? Yes, thanks, I will add that before I push. > >- top: http://cr.openjdk.java.net/~ehelin/8166790/top/00/ > > Nit: The new end lines like: > > 337 solaris_sparcv9_5.11-product-c2-runThese8, > 345 solaris_x64_5.11-product-c2-runThese8_Xcomp_vm, > 370 windows_i586_6.3-product-c2-runThese8_Xcomp_vm, > 378 windows_x64_6.3-product-c2-runThese8_Xcomp_vm, > > should not have the trailing comma. > > This seems wrong: > > 443 ${my.make.rule.test.targets.hotspot.reg.group:GROUP=hotspot_fast_gc_gcbasher}, > \ > 444 > ${my.make.rule.test.targets.hotspot.reg.group:GROUP=hotspot_fast_runtime}, > \ > 445 ${my.make.rule.test.targets.hotspot.reg.group:GROUP=hotspot_fast_serviceability}, > \ > 446 ${my.make.rule.test.targets.hotspot.reg.group:GROUP=jdk_svc_sanity}, > \ > 447 solaris_sparcv9_5.11-product-c2-hotspot_fast_gc_gcbasher, > \ > 448 solaris_x64_5.11-product-c2-hotspot_fast_gc_gcbasher, > \ > 449 linux_i586_3.8-product-c2-hotspot_fast_gc_gcbasher, > \ > 450 linux_x64_3.8-product-c2-hotspot_fast_gc_gcbasher, > \ > 451 macosx_x64_10.9-product-c2-hotspot_fast_gc_gcbasher, > \ > 452 windows_i586_6.3-product-c2-hotspot_fast_gc_gcbasher, > \ > 453 windows_x64_6.3-product-c2-hotspot_fast_gc_gcbasher, > \ > > You've added the new tests through a group substitution at line 443, but > then have added each platform individually. I don't think 447 - 453 should > be there else the test will run twice per platform. If you look at the current jprt.properties file you will see that GCBasher is being run for both product and fastdebug builds for 32 and 64 bit Linux, Windows, Mac, Solaris as well as Solaris SPARC. This is the behavior we want to match. Looking at my changes, you can see that the first change to my.make.rule.test.targets.hotspot.reg will cause the jtreg version of GCBasher to be run for all fastdebug builds for the above platforms. This is because my.make.rule.targets.hotspot.reg will expand a jtreg group to be to be a target for all fastdebug builds (by making use my.make.rule.targets.hotspot.reg.group with a substituion). Now, the previous paragraph takes care of fastdebug but doesn't run the test on product builds. We only want the jtreg version of GCBasher to run on the product builds, not all jtreg groups. Hence I added the product targets explicitly in my.make.rule.test.targets.hotspot.reg. Thanks, Erik > Thanks, > David > ----- > > >Testing: > >- Running the new jtreg test locally on Linux x86-64: > > $ jtreg -jdk:build/fastdebug/jdk hotspot/test/gc/stress/TestGCBasher.java > > > > (can also be run via: $ make test TEST=hotspot_fast_gc_gcbasher) > >- JPRT > > > >Thanks, > >Erik > > From david.holmes at oracle.com Thu Sep 29 08:27:42 2016 From: david.holmes at oracle.com (David Holmes) Date: Thu, 29 Sep 2016 18:27:42 +1000 Subject: RFR: 8166790: Add stress test GCBasher In-Reply-To: <20160929075257.GF20056@ehelin.jrpg.bea.com> References: <20160928150158.GE20056@ehelin.jrpg.bea.com> <20160929075257.GF20056@ehelin.jrpg.bea.com> Message-ID: <10920cb3-eb49-dd78-1074-203a5262284a@oracle.com> On 29/09/2016 5:52 PM, Erik Helin wrote: > On 2016-09-29, David Holmes wrote: >> Hi Erik, > > Hi David, thanks for reviewing! > >> On 29/09/2016 1:01 AM, Erik Helin wrote: >>> Hi all, >>> >>> this patch adds a new GC stress test called GCBasher. GCBasher builds up >>> large (well, for some definiton of large) object graphs by figuring out >>> the relations between classes in the JDK. The test usually stresses the >>> GC quite a lot, especially when run with a smaller heap. >>> >>> The changes in the top-level repository are for JPRT. JPRT will now run >>> the jtreg test GCBasher instead of the old version. >>> >>> Enhancement: >>> https://bugs.openjdk.java.net/browse/JDK-8166790 >>> >>> Webrev: >>> - hotspot: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/00/ >> >> Can't comment on actual test but placement etc looks fine. > > Thanks! > >> One query: >> >> test/gc/stress/gcbasher/TestGCBasher.java >> >> You are running with -server explicitly. Should you not also have @requires >> vm == server (or whatever the syntax is) as well? > > Yes, thanks, I will add that before I push. > >>> - top: http://cr.openjdk.java.net/~ehelin/8166790/top/00/ >> >> Nit: The new end lines like: >> >> 337 solaris_sparcv9_5.11-product-c2-runThese8, >> 345 solaris_x64_5.11-product-c2-runThese8_Xcomp_vm, >> 370 windows_i586_6.3-product-c2-runThese8_Xcomp_vm, >> 378 windows_x64_6.3-product-c2-runThese8_Xcomp_vm, >> >> should not have the trailing comma. >> >> This seems wrong: >> >> 443 ${my.make.rule.test.targets.hotspot.reg.group:GROUP=hotspot_fast_gc_gcbasher}, >> \ >> 444 >> ${my.make.rule.test.targets.hotspot.reg.group:GROUP=hotspot_fast_runtime}, >> \ >> 445 ${my.make.rule.test.targets.hotspot.reg.group:GROUP=hotspot_fast_serviceability}, >> \ >> 446 ${my.make.rule.test.targets.hotspot.reg.group:GROUP=jdk_svc_sanity}, >> \ >> 447 solaris_sparcv9_5.11-product-c2-hotspot_fast_gc_gcbasher, >> \ >> 448 solaris_x64_5.11-product-c2-hotspot_fast_gc_gcbasher, >> \ >> 449 linux_i586_3.8-product-c2-hotspot_fast_gc_gcbasher, >> \ >> 450 linux_x64_3.8-product-c2-hotspot_fast_gc_gcbasher, >> \ >> 451 macosx_x64_10.9-product-c2-hotspot_fast_gc_gcbasher, >> \ >> 452 windows_i586_6.3-product-c2-hotspot_fast_gc_gcbasher, >> \ >> 453 windows_x64_6.3-product-c2-hotspot_fast_gc_gcbasher, >> \ >> >> You've added the new tests through a group substitution at line 443, but >> then have added each platform individually. I don't think 447 - 453 should >> be there else the test will run twice per platform. > > If you look at the current jprt.properties file you will see that > GCBasher is being run for both product and fastdebug builds for 32 and > 64 bit Linux, Windows, Mac, Solaris as well as Solaris SPARC. This is > the behavior we want to match. > > Looking at my changes, you can see that the first change to > my.make.rule.test.targets.hotspot.reg will cause the jtreg version > of GCBasher to be run for all fastdebug builds for the above platforms. > This is because my.make.rule.targets.hotspot.reg will expand a jtreg > group to be to be a target for all fastdebug builds (by making use > my.make.rule.targets.hotspot.reg.group with a substituion). > > Now, the previous paragraph takes care of fastdebug but doesn't run the > test on product builds. We only want the jtreg version of GCBasher to > run on the product builds, not all jtreg groups. Hence I added the > product targets explicitly in my.make.rule.test.targets.hotspot.reg. Sorry I misread that. I'd be tempted to define a new group mechanism for my.make.rule.test.targets.hotspot.reg.product.group= \ solaris_sparcv9_5.11-product-c2-GROUP, \ ... and then use that. But I don't insist :) Thanks, David > Thanks, > Erik > >> Thanks, >> David >> ----- >> >>> Testing: >>> - Running the new jtreg test locally on Linux x86-64: >>> $ jtreg -jdk:build/fastdebug/jdk hotspot/test/gc/stress/TestGCBasher.java >>> >>> (can also be run via: $ make test TEST=hotspot_fast_gc_gcbasher) >>> - JPRT >>> >>> Thanks, >>> Erik >>> From aph at redhat.com Thu Sep 29 08:50:54 2016 From: aph at redhat.com (Andrew Haley) Date: Thu, 29 Sep 2016 09:50:54 +0100 Subject: [aarch64-port-dev ] Webrev of Oracle ARM & AARCH64 Sources In-Reply-To: References: <5A08C4A6-2B69-4D29-80F5-1CE11E350283@oracle.com> <7ea0939d-4aa1-05fc-1eac-f1da849920f4@redhat.com> Message-ID: <36626b7a-cb53-b13b-b529-d9c43ed28d0f@redhat.com> On 28/09/16 20:22, Kostya Zolotnikov wrote: > > Not sure who told You about Azul's view. > For us it is not that clear what make sense for You. I'm not quite sure what you mean by this, but I can only go by what Azul representatives have posted here. If I've got anything wrong, I am happy to be corrected. Andrew. From thomas.schatzl at oracle.com Thu Sep 29 09:47:57 2016 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 29 Sep 2016 11:47:57 +0200 Subject: RFR: 8166790: Add stress test GCBasher In-Reply-To: <20160928150158.GE20056@ehelin.jrpg.bea.com> References: <20160928150158.GE20056@ehelin.jrpg.bea.com> Message-ID: <1475142477.4444.1.camel@oracle.com> Hi, On Wed, 2016-09-28 at 17:01 +0200, Erik Helin wrote: > Hi all, > > this patch adds a new GC stress test called GCBasher. GCBasher builds > up > large (well, for some definiton of large) object graphs by figuring > out > the relations between classes in the JDK. The test usually stresses > the > GC quite a lot, especially when run with a smaller heap. > > The changes in the top-level repository are for JPRT. JPRT will now > run > the jtreg test GCBasher instead of the old version. > > Enhancement: > https://bugs.openjdk.java.net/browse/JDK-8166790 > > Webrev: > - hotspot: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/00/ > - top: http://cr.openjdk.java.net/~ehelin/8166790/top/00/ > ? I only looked at hotspot changes, mostly the actual test (settings etc). Looks good. Thanks, ? Thomas From erik.helin at oracle.com Thu Sep 29 09:52:28 2016 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 29 Sep 2016 11:52:28 +0200 Subject: RFR: 8166790: Add stress test GCBasher In-Reply-To: <10920cb3-eb49-dd78-1074-203a5262284a@oracle.com> References: <20160928150158.GE20056@ehelin.jrpg.bea.com> <20160929075257.GF20056@ehelin.jrpg.bea.com> <10920cb3-eb49-dd78-1074-203a5262284a@oracle.com> Message-ID: <20160929095228.GG20056@ehelin.jrpg.bea.com> On 2016-09-29, David Holmes wrote: > > > On 29/09/2016 5:52 PM, Erik Helin wrote: > >On 2016-09-29, David Holmes wrote: > >>Hi Erik, > > > >Hi David, thanks for reviewing! > > > >>On 29/09/2016 1:01 AM, Erik Helin wrote: > >>>Hi all, > >>> > >>>this patch adds a new GC stress test called GCBasher. GCBasher builds up > >>>large (well, for some definiton of large) object graphs by figuring out > >>>the relations between classes in the JDK. The test usually stresses the > >>>GC quite a lot, especially when run with a smaller heap. > >>> > >>>The changes in the top-level repository are for JPRT. JPRT will now run > >>>the jtreg test GCBasher instead of the old version. > >>> > >>>Enhancement: > >>>https://bugs.openjdk.java.net/browse/JDK-8166790 > >>> > >>>Webrev: > >>>- hotspot: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/00/ > >> > >>Can't comment on actual test but placement etc looks fine. > > > >Thanks! > > > >>One query: > >> > >>test/gc/stress/gcbasher/TestGCBasher.java > >> > >>You are running with -server explicitly. Should you not also have @requires > >>vm == server (or whatever the syntax is) as well? > > > >Yes, thanks, I will add that before I push. > > > >>>- top: http://cr.openjdk.java.net/~ehelin/8166790/top/00/ > >> > >>Nit: The new end lines like: > >> > >> 337 solaris_sparcv9_5.11-product-c2-runThese8, > >> 345 solaris_x64_5.11-product-c2-runThese8_Xcomp_vm, > >> 370 windows_i586_6.3-product-c2-runThese8_Xcomp_vm, > >> 378 windows_x64_6.3-product-c2-runThese8_Xcomp_vm, > >> > >>should not have the trailing comma. > >> > >>This seems wrong: > >> > >> 443 ${my.make.rule.test.targets.hotspot.reg.group:GROUP=hotspot_fast_gc_gcbasher}, > >>\ > >> 444 > >>${my.make.rule.test.targets.hotspot.reg.group:GROUP=hotspot_fast_runtime}, > >>\ > >> 445 ${my.make.rule.test.targets.hotspot.reg.group:GROUP=hotspot_fast_serviceability}, > >>\ > >> 446 ${my.make.rule.test.targets.hotspot.reg.group:GROUP=jdk_svc_sanity}, > >>\ > >> 447 solaris_sparcv9_5.11-product-c2-hotspot_fast_gc_gcbasher, > >>\ > >> 448 solaris_x64_5.11-product-c2-hotspot_fast_gc_gcbasher, > >>\ > >> 449 linux_i586_3.8-product-c2-hotspot_fast_gc_gcbasher, > >>\ > >> 450 linux_x64_3.8-product-c2-hotspot_fast_gc_gcbasher, > >>\ > >> 451 macosx_x64_10.9-product-c2-hotspot_fast_gc_gcbasher, > >>\ > >> 452 windows_i586_6.3-product-c2-hotspot_fast_gc_gcbasher, > >>\ > >> 453 windows_x64_6.3-product-c2-hotspot_fast_gc_gcbasher, > >>\ > >> > >>You've added the new tests through a group substitution at line 443, but > >>then have added each platform individually. I don't think 447 - 453 should > >>be there else the test will run twice per platform. > > > >If you look at the current jprt.properties file you will see that > >GCBasher is being run for both product and fastdebug builds for 32 and > >64 bit Linux, Windows, Mac, Solaris as well as Solaris SPARC. This is > >the behavior we want to match. > > > >Looking at my changes, you can see that the first change to > >my.make.rule.test.targets.hotspot.reg will cause the jtreg version > >of GCBasher to be run for all fastdebug builds for the above platforms. > >This is because my.make.rule.targets.hotspot.reg will expand a jtreg > >group to be to be a target for all fastdebug builds (by making use > >my.make.rule.targets.hotspot.reg.group with a substituion). > > > >Now, the previous paragraph takes care of fastdebug but doesn't run the > >test on product builds. We only want the jtreg version of GCBasher to > >run on the product builds, not all jtreg groups. Hence I added the > >product targets explicitly in my.make.rule.test.targets.hotspot.reg. > > Sorry I misread that. I'd be tempted to define a new group mechanism for > > my.make.rule.test.targets.hotspot.reg.product.group= \ > solaris_sparcv9_5.11-product-c2-GROUP, \ > ... > > and then use that. But I don't insist :) Me and Erik J discussed that and concluded that if the need arises for more jtreg tests to be run on both product and fastdebug, then we can do the abstraction then. Thanks for the review! Erik > Thanks, > David > > >Thanks, > >Erik > > > >>Thanks, > >>David > >>----- > >> > >>>Testing: > >>>- Running the new jtreg test locally on Linux x86-64: > >>> $ jtreg -jdk:build/fastdebug/jdk hotspot/test/gc/stress/TestGCBasher.java > >>> > >>> (can also be run via: $ make test TEST=hotspot_fast_gc_gcbasher) > >>>- JPRT > >>> > >>>Thanks, > >>>Erik > >>> From aph at redhat.com Thu Sep 29 10:43:36 2016 From: aph at redhat.com (Andrew Haley) Date: Thu, 29 Sep 2016 11:43:36 +0100 Subject: [aarch64-port-dev ] Webrev of Oracle ARM & AARCH64 Sources In-Reply-To: <9ECF7394-3D91-4A11-840C-37318CBA0CBF@gmail.com> References: <5A08C4A6-2B69-4D29-80F5-1CE11E350283@oracle.com> <7ea0939d-4aa1-05fc-1eac-f1da849920f4@redhat.com> <36626b7a-cb53-b13b-b529-d9c43ed28d0f@redhat.com> <9ECF7394-3D91-4A11-840C-37318CBA0CBF@gmail.com> Message-ID: <8fa2ec4d-3f6a-7d47-72f4-cafa1964d61a@redhat.com> On 29/09/16 11:13, Andrey Petushkov wrote: > Pardon, I?m too puzzled by your statement. To my best knowledge no > one from Azul considers this port as immature and wishes to drop > it. Nor AFAIK anyone from Azul has expressed that on the > maillist. My mistake, then. I should have waited for you to speak. I apologize. > In addition, aarch32 port shares much in common with RH?s aarch64 > implementation so well, if you keep aarch64 in main openjdk repos > it?s much easier to merge aarch32 into it, rather than merge with > Sun/Oracle?s arm implementation. There is no good technical reason for AArch64 to merge with AArch32, and IMO it would create a mess. AArch64 is a clean-sheet design which shares some of its DNA with AArch32, but that is all. It's not like x86-64, which is a 64-bit extension of x86. > And yes, aarch32 is missing c2 port, but this should not make any > difference long-term, right? I guess not, if a C2 port is forthcoming. Andrew. From aph at redhat.com Thu Sep 29 11:04:26 2016 From: aph at redhat.com (Andrew Haley) Date: Thu, 29 Sep 2016 12:04:26 +0100 Subject: [aarch64-port-dev ] Webrev of Oracle ARM & AARCH64 Sources In-Reply-To: <15B7B9FF-A282-4687-B847-ED17B0F7AC0E@gmail.com> References: <5A08C4A6-2B69-4D29-80F5-1CE11E350283@oracle.com> <7ea0939d-4aa1-05fc-1eac-f1da849920f4@redhat.com> <36626b7a-cb53-b13b-b529-d9c43ed28d0f@redhat.com> <9ECF7394-3D91-4A11-840C-37318CBA0CBF@gmail.com> <8fa2ec4d-3f6a-7d47-72f4-cafa1964d61a@redhat.com> <15B7B9FF-A282-4687-B847-ED17B0F7AC0E@gmail.com> Message-ID: <91d5a8c4-26e8-13a5-d45f-1f24be595361@redhat.com> On 29/09/16 11:54, Andrey Petushkov wrote: > >> On 29 Sep 2016, at 13:43, Andrew Haley wrote: >> >> On 29/09/16 11:13, Andrey Petushkov wrote: >> >>> In addition, aarch32 port shares much in common with RH?s aarch64 >>> implementation so well, if you keep aarch64 in main openjdk repos >>> it?s much easier to merge aarch32 into it, rather than merge with >>> Sun/Oracle?s arm implementation. >> >> There is no good technical reason for AArch64 to merge with AArch32, >> and IMO it would create a mess. AArch64 is a clean-sheet design which >> shares some of its DNA with AArch32, but that is all. It's not like >> x86-64, which is a 64-bit extension of x86. > > Well, it?s AArch32 which is not clean-sheet design but rather > borrows from AArch64 :) I meant to say: the AArch64 hardware architecture is a clean-sheet design which shares some of its DNA with AArch32, but that is all. > I admit that there are much more difference between architectures > than for x86 so you might be right that the difference in the code > could be too big. I just have a feeling it?s not. I can mistake of > course, I did not diff specifically Possibly. I don't really want to make the port a mess of #ifdefs and suchlike, so I'd fight pretty hard against it. It is possible to do some macro trickery to write common runtime routines, but with sufficient such trickery it'd be possible to do that with any two unrelated arches. While it might save some time, it adds another layer of complexity and makes it harder to write efficient and idiomatic code. In particular things such as predicated instructions, which are an important part of the AArch32 ISA, are absent on AArch64. Andrew. From bob.vandette at oracle.com Thu Sep 29 12:33:32 2016 From: bob.vandette at oracle.com (Bob Vandette) Date: Thu, 29 Sep 2016 08:33:32 -0400 Subject: [aarch64-port-dev ] Webrev of Oracle ARM & AARCH64 Sources In-Reply-To: <8441C732-88C4-4D7C-A6CA-3FB06C20FB55@gmail.com> References: <5A08C4A6-2B69-4D29-80F5-1CE11E350283@oracle.com> <7ea0939d-4aa1-05fc-1eac-f1da849920f4@redhat.com> <36626b7a-cb53-b13b-b529-d9c43ed28d0f@redhat.com> <9ECF7394-3D91-4A11-840C-37318CBA0CBF@gmail.com> <8fa2ec4d-3f6a-7d47-72f4-cafa1964d61a@redhat.com> <15B7B9FF-A282-4687-B847-ED17B0F7AC0E@gmail.com> <91d5a8c4-26e8-13a5-d45f-1f24be595361@redhat.com> <8441C732-88C4-4D7C-A6CA-3FB06C20FB55@gmail.com> Message-ID: <7F31BDA8-F324-4CAB-826D-D3054EE8CE43@oracle.com> > On Sep 29, 2016, at 7:10 AM, Andrey Petushkov wrote: > > >> On 29 Sep 2016, at 14:04, Andrew Haley wrote: >> >> On 29/09/16 11:54, Andrey Petushkov wrote: >>> >>>> On 29 Sep 2016, at 13:43, Andrew Haley wrote: >>>> >>>> On 29/09/16 11:13, Andrey Petushkov wrote: >>>> >>>>> In addition, aarch32 port shares much in common with RH?s aarch64 >>>>> implementation so well, if you keep aarch64 in main openjdk repos >>>>> it?s much easier to merge aarch32 into it, rather than merge with >>>>> Sun/Oracle?s arm implementation. >>>> >>>> There is no good technical reason for AArch64 to merge with AArch32, >>>> and IMO it would create a mess. AArch64 is a clean-sheet design which >>>> shares some of its DNA with AArch32, but that is all. It's not like >>>> x86-64, which is a 64-bit extension of x86. >>> >>> Well, it?s AArch32 which is not clean-sheet design but rather >>> borrows from AArch64 :) >> >> I meant to say: the AArch64 hardware architecture is a clean-sheet >> design which shares some of its DNA with AArch32, but that is all. > Ok, git it. Sorry for misunderstanding >> >>> I admit that there are much more difference between architectures >>> than for x86 so you might be right that the difference in the code >>> could be too big. I just have a feeling it?s not. I can mistake of >>> course, I did not diff specifically >> >> Possibly. I don't really want to make the port a mess of #ifdefs and >> suchlike, so I'd fight pretty hard against it. >> >> It is possible to do some macro trickery to write common runtime >> routines, but with sufficient such trickery it'd be possible to do >> that with any two unrelated arches. While it might save some time, it >> adds another layer of complexity and makes it harder to write >> efficient and idiomatic code. In particular things such as predicated >> instructions, which are an important part of the AArch32 ISA, are >> absent on AArch64. > Very much possible. I don?t want to propose it now, was just mentioning the probability. Ok it?s low, let?s dismiss it It is more that just possible. I have provided you with an implementation that does just that. I don?t think you should be so fast to dismiss it. Bob. >> >> Andrew. > From volker.simonis at gmail.com Thu Sep 29 16:55:21 2016 From: volker.simonis at gmail.com (Volker Simonis) Date: Thu, 29 Sep 2016 18:55:21 +0200 Subject: RFR: JEP draft for Linux/s3990x port In-Reply-To: <72ddbd25-c7df-b02a-3827-d431f286c795@oracle.com> References: <72ddbd25-c7df-b02a-3827-d431f286c795@oracle.com> Message-ID: Hi Vladimir, thanks a lot for reviewing and endorsing the JEP. I've linked all the relevant issues to the JEP (they all have a link to a webrev) and change the state to "Submitted". There's just one more small shared change we need for the port for which we haven't opened a bug now because we are still working on simplifying it. The current version looks as follows: http://cr.openjdk.java.net/~simonis/webrevs/2016/s390x/9000016-constant_table_offset.patch What are the next steps? Should I add a "jdk9-fc-request" label to t he JEP and add a corresponding "FC Extension Request" comment to it? Or will this be done automatically once I move it to "Candidate"? Is there anything left to do before I can move it to "Candidate" state? Thanks a lot and best regards, Volker On Tue, Sep 27, 2016 at 8:15 PM, Vladimir Kozlov wrote: > On 9/27/16 10:49 AM, Volker Simonis wrote: >> >> Hi, >> >> can you please review and endorse the following draft JEP for the >> integration of the Linux/s390x port into the jkd9 master repository: >> >> https://bugs.openjdk.java.net/browse/JDK-8166730 > > > Good. > Add links to webrevs in a comment. It will help to get umbrella FC extension > approval. > >> >> As detailed in the JEP, the Linux/s390x requires very few shared >> changes and we therefore don't foresee any impact on the existing >> platforms at all. Following you can find a short description of the >> planned changes: >> >> hotspot: >> ======= >> >> Out for review: >> 8166560: [s390] Basic enablement of s390 port. >> http://cr.openjdk.java.net/~goetz/wr16/8166560-basic_s390/hotspot.wr01/ >> >> Reviewed: >> 8166562: C2: Suppress relocations in scratch emit. >> http://cr.openjdk.java.net/~goetz/wr16/8166562-scratch_emit/webrev.01/ >> >> Will send RFR soon (depends on 8166560): >> 8166561: [s390] Adaptions needed for s390 port in C1 and C2. >> http://cr.openjdk.java.net/~goetz/wr16/8166562-scratch_emit/webrev.01 > > > Wrong link. > > Thanks, > Vladimir > > >> >> We are still investigating the need of these shared changes: >> >> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000011-pass_PC_to_retAddrOffset.patch >> >> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000016-constant_table_offset.patch >> >> And finally the patch with the s390x-only platform files. We are still >> editing these to get them into OpenJdk style and shape. >> Hotspot passes most jck, jtreg and spec tests with these. >> >> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000101-zFiles.patch >> >> top-level repository: >> =============== >> >> The following is just adding some s390x specific compiler flags to >> flags.m4 >> 8166800: [s390] Top-level build changes required for Linux/s390x >> https://bugs.openjdk.java.net/browse/JDK-8166800 >> >> jdk repository: >> ============ >> >> This one just adds a new jvm.cfg file for s390x >> 8166801: [s390] Add jvm.cfg file for Linux/s390x >> https://bugs.openjdk.java.net/browse/JDK-8166801 >> >> >> And finally we plan to do one more change which fixes the jtreg test >> on Linux/s390x. But this is mainly for the correct detection of the >> platform and for excluding the tests which are not appropriate for >> s390x. >> >> Thank you and best regards, >> Volker >> > From mikael.vidstedt at oracle.com Thu Sep 29 18:23:39 2016 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Thu, 29 Sep 2016 11:23:39 -0700 Subject: Merging jdk9/hs-comp with jdk9/hs Message-ID: All, Over the last two years we have worked towards simplifying the mercurial forest structure for Hotspot development by consolidating and removing forests. Last year we moved[1] the GC development work from jdk9/hs-gc[2] to jdk9/hs-rt[3], and earlier this year we further consolidated forests by moving[4] the jdk9/hs-rt work directly to the jdk9/hs[5] - aka. "hs main" - forest. The reduction in forests has led to less work spent on integration, faster quality feedback, more efficient hardware utilization, etc. All in all, the experience so far has been a good one. However, the Hotspot (JIT) compiler changes are still integrated through the jdk9/hs-comp[6] forest, and are tested separately before they are (bulk) integrated up to jdk9/hs. The separate forest of course comes with the expected integration/gatekeeper overhead and lead times. Since JDK 9 development is ramping down, we expect the number of changes to ramp down as well. This means that the benefit of having a separate jdk9/hs-comp forest is going to reduce over time. Now seems like a good time to look at doing a final hotspot forest consolidation. In line with this, we propose closing down jdk9/hs-comp and doing all remaining JDK 9 Hotspot development to jdk9/hs. We propose that this final forest consolidation to take place on Friday, October 7th. Much like the earlier forest consolidations the jdk9/hs-comp forest will be kept around, but will be locked down so that no accidental integrations are made to it. Any work in flight based on jdk9/hs-comp would have to be rebased on jdk9/hs. The earlier forest consolidations have gone very smoothly, and we expect this one to go smoothly as well, but if for some reason things don't work out we will of course have an option to go back to what we have today. That said, before using the escape hatch and reverting back, we will of course want to try to address any issues - thank you in advance for your patience while we work through any such issues. Please let us know if you have any feedback or questions! If no serious objections have been raised by 15:00 UTC Wednesday, 5 October 2016, we will go ahead with the forest consolidation. Cheers, Mikael [1]http://mail.openjdk.java.net/pipermail/hotspot-dev/2015-May/thread.html [2]http://hg.openjdk.java.net/jdk9/hs-gc [3]http://hg.openjdk.java.net/jdk9/hs-rt [4]http://mail.openjdk.java.net/pipermail/hotspot-dev/2016-March/022218.html [5]http://hg.openjdk.java.net/jdk9/hs [6]http://hg.openjdk.java.net/jdk9/hs-comp From vladimir.kozlov at oracle.com Thu Sep 29 18:25:29 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 29 Sep 2016 11:25:29 -0700 Subject: RFR: JEP draft for Linux/s3990x port In-Reply-To: References: <72ddbd25-c7df-b02a-3827-d431f286c795@oracle.com> Message-ID: <6261447a-fc39-4ea6-2ccf-3f0dcc396a36@oracle.com> You need to wait when Mark (OpenJDK Lead) move it to Candidate (or other) state: http://cr.openjdk.java.net/~mr/jep/jep-2.0-02.html Vladimir On 9/29/16 9:55 AM, Volker Simonis wrote: > Hi Vladimir, > > thanks a lot for reviewing and endorsing the JEP. > > I've linked all the relevant issues to the JEP (they all have a link > to a webrev) and change the state to "Submitted". > > There's just one more small shared change we need for the port for > which we haven't opened a bug now because we are still working on > simplifying it. The current version looks as follows: > > http://cr.openjdk.java.net/~simonis/webrevs/2016/s390x/9000016-constant_table_offset.patch > > What are the next steps? Should I add a "jdk9-fc-request" label to t > he JEP and add a corresponding "FC Extension Request" comment to it? > Or will this be done automatically once I move it to "Candidate"? > > Is there anything left to do before I can move it to "Candidate" state? > > Thanks a lot and best regards, > Volker > > > > > On Tue, Sep 27, 2016 at 8:15 PM, Vladimir Kozlov > wrote: >> On 9/27/16 10:49 AM, Volker Simonis wrote: >>> >>> Hi, >>> >>> can you please review and endorse the following draft JEP for the >>> integration of the Linux/s390x port into the jkd9 master repository: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8166730 >> >> >> Good. >> Add links to webrevs in a comment. It will help to get umbrella FC extension >> approval. >> >>> >>> As detailed in the JEP, the Linux/s390x requires very few shared >>> changes and we therefore don't foresee any impact on the existing >>> platforms at all. Following you can find a short description of the >>> planned changes: >>> >>> hotspot: >>> ======= >>> >>> Out for review: >>> 8166560: [s390] Basic enablement of s390 port. >>> http://cr.openjdk.java.net/~goetz/wr16/8166560-basic_s390/hotspot.wr01/ >>> >>> Reviewed: >>> 8166562: C2: Suppress relocations in scratch emit. >>> http://cr.openjdk.java.net/~goetz/wr16/8166562-scratch_emit/webrev.01/ >>> >>> Will send RFR soon (depends on 8166560): >>> 8166561: [s390] Adaptions needed for s390 port in C1 and C2. >>> http://cr.openjdk.java.net/~goetz/wr16/8166562-scratch_emit/webrev.01 >> >> >> Wrong link. >> >> Thanks, >> Vladimir >> >> >>> >>> We are still investigating the need of these shared changes: >>> >>> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000011-pass_PC_to_retAddrOffset.patch >>> >>> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000016-constant_table_offset.patch >>> >>> And finally the patch with the s390x-only platform files. We are still >>> editing these to get them into OpenJdk style and shape. >>> Hotspot passes most jck, jtreg and spec tests with these. >>> >>> http://cr.openjdk.java.net/~goetz/wr16/s390x_patch_queue/hotspot/9000101-zFiles.patch >>> >>> top-level repository: >>> =============== >>> >>> The following is just adding some s390x specific compiler flags to >>> flags.m4 >>> 8166800: [s390] Top-level build changes required for Linux/s390x >>> https://bugs.openjdk.java.net/browse/JDK-8166800 >>> >>> jdk repository: >>> ============ >>> >>> This one just adds a new jvm.cfg file for s390x >>> 8166801: [s390] Add jvm.cfg file for Linux/s390x >>> https://bugs.openjdk.java.net/browse/JDK-8166801 >>> >>> >>> And finally we plan to do one more change which fixes the jtreg test >>> on Linux/s390x. But this is mainly for the correct detection of the >>> platform and for excluding the tests which are not appropriate for >>> s390x. >>> >>> Thank you and best regards, >>> Volker >>> >> From daniel.daugherty at oracle.com Thu Sep 29 21:38:23 2016 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 29 Sep 2016 15:38:23 -0600 Subject: Does transition_and_fence really assume TSO? In-Reply-To: References: <57A13506.9070108@redhat.com> Message-ID: <75966da0-2808-7f5f-4ecd-069149d31bef@oracle.com> On 8/2/16 6:25 PM, David Holmes wrote: > Hi Andrew, > > On 3/08/2016 10:04 AM, Andrew Haley wrote: >> Here's an interesting comment: >> >> static inline void transition_and_fence(JavaThread *thread, >> JavaThreadState from, JavaThreadState to) { >> assert(thread->thread_state() == from, "coming from wrong thread >> state"); >> assert((from & 1) == 0 && (to & 1) == 0, "odd numbers are >> transitions states"); >> // Change to transition state (assumes total store ordering! -Urs) >> thread->set_thread_state((JavaThreadState)(from + 1)); >> >> The "assumes total store ordering!" comment is rather alarming. My >> processor is not TSO. But as far as I can see, all this really >> requires is single-variable coherency. Is that right? > > That comment is pre-historic. The code does not assume TSO. There is a > direct, or implicit, full fence between the two stores: > > // Make sure new state is seen by VM thread > if (os::is_MP()) { > if (UseMembar) { > // Force a fence between the write above and read below > OrderAccess::fence(); > } else { > // Must use this rather than serialization page in particular > on Windows > InterfaceSupport::serialize_memory(thread); > } > } > > Cheers, > David > >> Andrew. >> Filling in the history here. There are three instances of "assumes total store ordering" in src/share/vm/runtime/interfaceSupport.hpp: $ sgv src/share/vm/runtime/interfaceSupport.hpp | grep 'assumes total store ordering' 1.178 1.86 // Change to transition state (assumes total store ordering! -Urs) 1.168 // Change to transition state (assumes total store ordering! -Urs) 1.139.2.1 // Change to transition state (assumes total store ordering! -Urs) 568 lines No id keywords (cm7) $ sp -r1.86 src/share/vm/runtime/interfaceSupport.hpp src/share/vm/runtime/SCCS/s.interfaceSupport.hpp: D 1.86 98/06/10 11:10:44 urs 151 150 00005/00004/00290 MRs: COMMENTS: optimizations to entry & exit code of runtime routines $ sp -r1.139.2.1 src/share/vm/runtime/interfaceSupport.hpp src/share/vm/runtime/SCCS/s.interfaceSupport.hpp: D 1.139.2.1 02/10/22 00:51:52 huanghui 259 249 00017/00003/00455 MRs: COMMENTS: 4683006 add transition_from_native(), which also checks for pending suspend request $ sp -r1.168 src/share/vm/runtime/interfaceSupport.hpp src/share/vm/runtime/SCCS/s.interfaceSupport.hpp: D 1.168 05/08/29 15:14:41 kbr 290 289 00045/00020/00515 MRs: COMMENTS: 6304225 hotspot/runtime_syst IE crashes with b44 libjvm.dll The oldest delta has no bug ID: D 1.86 98/06/10 11:10:44 urs 151 150 00005/00004/00290 MRs: COMMENTS: optimizations to entry & exit code of runtime routines and here's the relevant part of the diff: 67,68c68,69 < assert(thread->thread_state() == from, "comming from wrong thread state"); < // Change to transition state --- > assert(thread->thread_state() == from, "coming from wrong thread state"); > // Change to transition state (assumes total store ordering! -Urs) The change from Hui has a nice comment: D 1.139.2.1 02/10/22 00:51:52 huanghui 259 249 00017/00003/00455 MRs: COMMENTS: 4683006 add transition_from_native(), which also checks for pending suspend request so we know that when Hui added transition_from_native() the comment was copied. Here's the relevant diffs: 102a104,115 > > static inline void transition_from_native(JavaThread *thread, JavaThreadState to) { > assert((to & 1) == 0, "odd numbers are transitions states"); > assert(thread->thread_state() == _thread_in_native, "coming from wrong thread state"); > // Change to transition state (assumes total store ordering! -Urs) > thread->set_thread_state(_thread_in_native_trans); > if (os::is_MP()) atomic::membar(); > if (SafepointSynchronize::do_call_back() || thread->is_external_suspend()) { > JavaThread::check_safepoint_and_suspend(thread); > } > thread->set_thread_state(to); > } The change from Ken has a bug ID: D 1.168 05/08/29 15:14:41 kbr 290 289 00045/00020/00515 MRs: COMMENTS: 6304225 hotspot/runtime_syst IE crashes with b44 libjvm.dll Here's the relevant diffs: > // transition_and_fence must be used on any thread state transition > // where there might not be a Java call stub on the stack, in > // particular on Windows where the Structured Exception Handler is > // set up in the call stub. os::write_memory_serialize_page() can > // fault and we can't recover from it on Windows without a SEH in > // place. > static inline void transition_and_fence(JavaThread *thread, JavaThreadState from, JavaThreadState to) { > assert(thread->thread_state() == from, "coming from wrong thread state"); > assert((from & 1) == 0 && (to & 1) == 0, "odd numbers are transitions states"); > // Change to transition state (assumes total store ordering! -Urs) > thread->set_thread_state((JavaThreadState)(from + 1)); > > // Make sure new state is seen by VM thread > if (os::is_MP()) { > // Must use this rather than serialization page in particular on Windows > InterfaceSupport::serialize_memory(thread); > } > > if (SafepointSynchronize::do_call_back()) { > SafepointSynchronize::block(thread); > } > thread->set_thread_state(to); > > CHECK_UNHANDLED_OOPS_ONLY(thread->clear_unhandled_oops();) > } Filed the following bug: JDK-8166927 interfaceSupport.hpp has ancient comments about TSO https://bugs.openjdk.java.net/browse/JDK-8166927 Dan From kim.barrett at oracle.com Thu Sep 29 22:49:09 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 29 Sep 2016 18:49:09 -0400 Subject: RFR: 8166607: G1 needs klass_or_null_acquire Message-ID: <98788D01-D3B4-45C4-B133-204B36A6799D@oracle.com> Please review this change to G1 to use klass_or_null_acquire where needed. [Reviewing on hotspot-dev; only GC code is involved, but some non-GC folks have expressed interest.] G1 BOT uses of klass_or_null have all been changed to instead use klass_or_null_acquire. This isn't necessary for the call in the revised version of oops_on_card_seq_iterate_careful (discussed below), but there can be (and are) other callers. The performance impact of unnecessary barriers in oops_on_card_xxx will hopefully not be too bad on non-TSO systems; the number reduces to one as the BOT gets filled in. Rather than splitting or otherwise conditionalizing, for now we just make it safe. [The CollectedHeap block_start &etc API needs some TLC in the face of collectors that use klass_or_null therein, since the answer to a call to block_start could change instantaneously. And some (all?) of the existing not-GC-specific callers probably should be using block_start_const rather than block_start.] Changed oops_on_card_seq_iterate_careful to use different code paths when the region is humongous or not. The humongous case is simplified because there can only be the one (potential) object in the region. Use klass_or_null_acquire to determine whether the object has been "published" or allocation is still in-progress. The acquire barrier ensures processing the humongous object is safe. When the klass is NULL the card marking must be stale and can be ignored. In the non-humongous case, we know we're dealing with an old-gen region. Because of that, we know we don't need to deal with in-progress allocations here. That allows many simplifications. As a result of the changes in oops_on_card_seq_iterate_careful, we now almost never fail to process the card. The only place where that can occur is a stale card in a humongous region with an in-progress allocation, where we can just ignore it. So the only caller, refine_card, no longer needs to examine the result of the call and enqueue the card for later reconsideration. Note that some of the complexity in refine_card / oops_on_card_xxx arises from the possibility of "stale" marked cards, especially during concurrent refinement. These are cards in regions that have been freed and then reallocated. At present the only source of stale cards in the concurrent case seems to be HCC eviction. (Eager reclaim of humongous objects flushes relevant remembered sets (possibly containing stale entries) for reconsideration during that GC, but in-GC card clipping to scan_top deals with that.) Doing HCC cleanup when freeing regions might remove the need for klass_or_null checking in the humongous case for concurrent refinement, so might be worth looking into later. CR: https://bugs.openjdk.java.net/browse/JDK-8166607 Webrev: http://cr.openjdk.java.net/~kbarrett/8166607/webrev.01/ Testing: Nightly test equiv. via rbt. (in progress) Local specjbb2015 (non-perf). Local jtreg hotspot_all. From aph at redhat.com Fri Sep 30 07:51:17 2016 From: aph at redhat.com (Andrew Haley) Date: Fri, 30 Sep 2016 08:51:17 +0100 Subject: Does transition_and_fence really assume TSO? In-Reply-To: <75966da0-2808-7f5f-4ecd-069149d31bef@oracle.com> References: <57A13506.9070108@redhat.com> <75966da0-2808-7f5f-4ecd-069149d31bef@oracle.com> Message-ID: On 29/09/16 22:38, Daniel D. Daugherty wrote: > Filed the following bug: > > JDK-8166927 interfaceSupport.hpp has ancient comments about TSO > https://bugs.openjdk.java.net/browse/JDK-8166927 :) Thanks, Andrew. From martin.walsh at oracle.com Fri Sep 30 10:03:06 2016 From: martin.walsh at oracle.com (Martin Walsh) Date: Fri, 30 Sep 2016 11:03:06 +0100 Subject: RFR for JDK-8165482 java in ldoms, with cpu-arch=generic has problems In-Reply-To: <994a0b2e-5874-cbdd-1544-d3b00e5165a2@oracle.com> References: <8046b263-8826-aa34-083a-9a83e105770f@oracle.com> <0a2a66db-f939-196f-bf23-a6a3df46b2d3@oracle.com> <4bfdead9-f0d9-f67e-e049-d48af5f19863@oracle.com> <994a0b2e-5874-cbdd-1544-d3b00e5165a2@oracle.com> Message-ID: On 23/09/2016 14:34, Martin Walsh wrote: > On 20/09/2016 08:42, Erik Joelsson wrote: >> Hello, >> >> >> On 2016-09-20 03:28, David Holmes wrote: >>> Hi Martin, >>> >>> Build changes must be reviewed by the build team - now cc'd >>> >> Thanks for forwarding David, I can't keep up with all the lists to find >> these unless posted to build-dev. >>> On 20/09/2016 12:16 AM, Martin Walsh wrote: >>>> Could I get a code review for the following bug: >>>> >>>> JDK-8165482 java in ldoms, with cpu-arch=generic has problems >>>> >>>> Webrev of the changes is available here: >>>> >>>> http://cr.openjdk.java.net/~mwalsh/JDK-8165482/ >>> >>> What is the devinfo library? Is it part of the normal Solaris >>> installation, or does it need to be installed specifically? Is it >>> available in our official build toolkits? >> I did a bit of digging. It's part "system/libraries" so should be pretty >> standard. That package is in the devkit and I verified that >> libdevinfo.so is there too. >> >> Configure change looks fine. Just remember to also push the closed >> generated-configure.sh as David said. >> >> /Erik >>> >>> Will checking the prom prior to using kstat change any of the values >>> we currently see? (other than the generic case being fixed of course). > > It shouldn't, as SPARC64 seems to be reserved for Fujitsu SPARC > machines. However, to air on the side of caution I will investigate > further and follow-up shortly. So, after further deliberation I have decided that although utilising the PROM may work on some occasions, it is not the correct solution. All Oracle SPARC LDOMS use the "sun4-cpu" string as the CPU implementation, therefore I think the best short term fix is to add an additional match that checks for this string. Long term, this could do with a re-write, but that is a JDK10 project. This updated fix also means there are no build changes. Updated the webrev accordingly. http://cr.openjdk.java.net/~mwalsh/JDK-8165482/ Thanks, Martin From leonid.mesnik at oracle.com Fri Sep 30 10:18:48 2016 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Fri, 30 Sep 2016 13:18:48 +0300 Subject: RFR: 8166790: Add stress test GCBasher In-Reply-To: <20160928150158.GE20056@ehelin.jrpg.bea.com> References: <20160928150158.GE20056@ehelin.jrpg.bea.com> Message-ID: <28d094d1-9355-9d69-4764-2fca30e910b5@oracle.com> Hi I think that it would be better to split this test into 4 tests. Currently this test is going to be skipped if any specific GCare set. Or it could even fail if any G1 specific options is set without setting G1 explicitly. So there is no way to run this with additional G1/CMS/ParGC specific options now. Leonid On 28.09.2016 18:01, Erik Helin wrote: > Hi all, > > this patch adds a new GC stress test called GCBasher. GCBasher builds up > large (well, for some definiton of large) object graphs by figuring out > the relations between classes in the JDK. The test usually stresses the > GC quite a lot, especially when run with a smaller heap. > > The changes in the top-level repository are for JPRT. JPRT will now run > the jtreg test GCBasher instead of the old version. > > Enhancement: > https://bugs.openjdk.java.net/browse/JDK-8166790 > > Webrev: > - hotspot: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/00/ > - top: http://cr.openjdk.java.net/~ehelin/8166790/top/00/ > > Testing: > - Running the new jtreg test locally on Linux x86-64: > $ jtreg -jdk:build/fastdebug/jdk hotspot/test/gc/stress/TestGCBasher.java > > (can also be run via: $ make test TEST=hotspot_fast_gc_gcbasher) > - JPRT > > Thanks, > Erik From dmitry.fazunenko at oracle.com Fri Sep 30 10:19:12 2016 From: dmitry.fazunenko at oracle.com (Dmitry Fazunenko) Date: Fri, 30 Sep 2016 13:19:12 +0300 Subject: RFR: 8166790: Add stress test GCBasher In-Reply-To: <20160928150158.GE20056@ehelin.jrpg.bea.com> References: <20160928150158.GE20056@ehelin.jrpg.bea.com> Message-ID: <9c76ac32-f336-44fd-7d29-ccfd2c9ed521@oracle.com> Hi Erik, The change looks good, thank you for doing that fix. -- Dima On 28.09.2016 18:01, Erik Helin wrote: > Hi all, > > this patch adds a new GC stress test called GCBasher. GCBasher builds up > large (well, for some definiton of large) object graphs by figuring out > the relations between classes in the JDK. The test usually stresses the > GC quite a lot, especially when run with a smaller heap. > > The changes in the top-level repository are for JPRT. JPRT will now run > the jtreg test GCBasher instead of the old version. > > Enhancement: > https://bugs.openjdk.java.net/browse/JDK-8166790 > > Webrev: > - hotspot: http://cr.openjdk.java.net/~ehelin/8166790/hotspot/00/ > - top: http://cr.openjdk.java.net/~ehelin/8166790/top/00/ > > Testing: > - Running the new jtreg test locally on Linux x86-64: > $ jtreg -jdk:build/fastdebug/jdk hotspot/test/gc/stress/TestGCBasher.java > > (can also be run via: $ make test TEST=hotspot_fast_gc_gcbasher) > - JPRT > > Thanks, > Erik From edward.nevill at gmail.com Fri Sep 30 10:32:01 2016 From: edward.nevill at gmail.com (Edward Nevill) Date: Fri, 30 Sep 2016 11:32:01 +0100 Subject: Webrev of Oracle ARM & AARCH64 Sources In-Reply-To: <5A08C4A6-2B69-4D29-80F5-1CE11E350283@oracle.com> References: <5A08C4A6-2B69-4D29-80F5-1CE11E350283@oracle.com> Message-ID: <1475231521.14313.29.camel@gmail.com> Hi Bob, On Wed, 2016-09-28 at 10:07 -0400, Bob Vandette wrote: > I?m am please to announce that I have completed our internal reviews and can now > open up the sources to our ARM 32 & 64 bit implementations of JDK9. Great news. > > Here is a webrev that includes a patch that can be applied on top of the > (http://hg.openjdk.java.net/aarch32-port/jdk9-arm3264/ ) forest.?? > > ?????http://cr.openjdk.java.net/~bobv/arm3264/webrev I have built this natively on armv7 and aarch64 without problems. In both cases I built the minimal,client,server combination. I have also tested the server build with JTreg hotspot & langtools. On aarch64 I got the following results:- hotspot: Test results: passed: 1,280; failed: 8; error: 43 langtools: Test results: passed: 3,700; failed: 1; error: 29 This is fairly typical, for example, using the Linaro 1609 build I get hotspot: Test results: passed: 1,271; failed: 17; error: 43 langtools: Test results: passed: 3,715; failed: 3; error: 12 On armv7 I first of all had to pin the jvm.cfg to use -server as it kept on using the client as I was not running on a 'server class machine' (since my armv7 device is a Samsung Chromebook this is fair enough). However, I then had problems with the test harness crashing with the error #??Internal Error (synchronizer.cpp:1576), pid=6533, tid=6542 #??guarantee(mid->header()->is_neutral()) failed: invariant (hs_err here?http://cr.openjdk.java.net/~enevill/aarch32/hs_err_pid6533.log) On a subsequent run the test harness locked up after 85 tests. It looks like there might be a problem with locks breaking with G1GC. I will keep investigating. I ran it again using the template interpreter for the test harness and the server for the jdk under test and it ran successfully. On armv7 I got hotspot: Test results: passed: 1,202; failed: 11; error: 36 langtools: Test results: passed: 3,718; failed: 5; error: 8 I also did some limited benchmarking which I am not going to share on a public forum. Lets just say the performance of the server JIT on armv7 was pleasing. Thanks for all your work getting to this stage. It looks good to push and you should shortly have committeer rights, if you don't get them within a few days ping me and I will give ops a nudge. All the best, Ed. From david.holmes at oracle.com Fri Sep 30 11:23:30 2016 From: david.holmes at oracle.com (David Holmes) Date: Fri, 30 Sep 2016 21:23:30 +1000 Subject: Webrev of Oracle ARM & AARCH64 Sources In-Reply-To: <1475231521.14313.29.camel@gmail.com> References: <5A08C4A6-2B69-4D29-80F5-1CE11E350283@oracle.com> <1475231521.14313.29.camel@gmail.com> Message-ID: <187f08d2-2f6d-a4f6-8662-7e8ac85308f7@oracle.com> Hi Ed, On 30/09/2016 8:32 PM, Edward Nevill wrote: > Hi Bob, > > On Wed, 2016-09-28 at 10:07 -0400, Bob Vandette wrote: >> I?m am please to announce that I have completed our internal reviews and can now >> open up the sources to our ARM 32 & 64 bit implementations of JDK9. > > Great news. > >> >> Here is a webrev that includes a patch that can be applied on top of the >> (http://hg.openjdk.java.net/aarch32-port/jdk9-arm3264/ ) forest. >> >> http://cr.openjdk.java.net/~bobv/arm3264/webrev > > I have built this natively on armv7 and aarch64 without problems. In both cases I built the minimal,client,server combination. > > I have also tested the server build with JTreg hotspot & langtools. > > On aarch64 I got the following results:- > > hotspot: Test results: passed: 1,280; failed: 8; error: 43 > langtools: Test results: passed: 3,700; failed: 1; error: 29 > > This is fairly typical, for example, using the Linaro 1609 build I get > > hotspot: Test results: passed: 1,271; failed: 17; error: 43 > langtools: Test results: passed: 3,715; failed: 3; error: 12 So most of the "errors" are probably tests that are ignored. > On armv7 I first of all had to pin the jvm.cfg to use -server as it kept on using the client as I was not running on a 'server class machine' (since my armv7 device is a Samsung Chromebook this is fair enough). > > However, I then had problems with the test harness crashing with the error > > # Internal Error (synchronizer.cpp:1576), pid=6533, tid=6542 > # guarantee(mid->header()->is_neutral()) failed: invariant > > (hs_err here http://cr.openjdk.java.net/~enevill/aarch32/hs_err_pid6533.log) Strange - that looks like our closed bug: https://bugs.openjdk.java.net/browse/JDK-8071540 which should be fixed. It was a missing memory barrier issue. Aside: we're going to have to figure out how to deal with currently closed bug reports. David ----- > On a subsequent run the test harness locked up after 85 tests. > > It looks like there might be a problem with locks breaking with G1GC. I will keep investigating. > > I ran it again using the template interpreter for the test harness and the server for the jdk under test and it ran successfully. > > On armv7 I got > > hotspot: Test results: passed: 1,202; failed: 11; error: 36 > langtools: Test results: passed: 3,718; failed: 5; error: 8 > > I also did some limited benchmarking which I am not going to share on a public forum. Lets just say the performance of the server JIT on armv7 was pleasing. > > Thanks for all your work getting to this stage. It looks good to push and you should shortly have committeer rights, if you don't get them within a few days ping me and I will give ops a nudge. > > All the best, > Ed. > > From thomas.schatzl at oracle.com Fri Sep 30 12:00:54 2016 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 30 Sep 2016 14:00:54 +0200 Subject: RFR: 8166607: G1 needs klass_or_null_acquire In-Reply-To: <98788D01-D3B4-45C4-B133-204B36A6799D@oracle.com> References: <98788D01-D3B4-45C4-B133-204B36A6799D@oracle.com> Message-ID: <1475236854.6301.70.camel@oracle.com> Hi Kim, On Thu, 2016-09-29 at 18:49 -0400, Kim Barrett wrote: > Please review this change to G1 to use klass_or_null_acquire where > needed. > > [Reviewing on hotspot-dev; only GC code is involved, but some non-GC > folks have expressed interest.] > > G1 BOT uses of klass_or_null have all been changed to instead use > klass_or_null_acquire.??This isn't necessary for the call in the > revised version of oops_on_card_seq_iterate_careful (discussed > below), but there can be (and are) other callers.??The performance > impact of unnecessary barriers in oops_on_card_xxx will hopefully not > be too bad on non-TSO systems; the number reduces to one as the BOT > gets filled in.??Rather than splitting or otherwise conditionalizing, > for now we just make it safe. Could you add a comment to?JDK-8162928? > ??[The CollectedHeap block_start &etc API needs some TLC in the face > of collectors that use klass_or_null TLC? > therein, since the answer to a call to block_start could change > instantaneously.??And some (all?) of the existing not-GC-specific > callers probably should be using block_start_const rather than > block_start.] > > Changed oops_on_card_seq_iterate_careful to use different code paths > when the region is humongous or not. That's nice. > The humongous case is simplified because there can only be the one > (potential) object in the region.??Use klass_or_null_acquire to > determine whether the object has been "published" or allocation is > still in-progress.??The acquire barrier ensures processing the > humongous object is safe.??When the klass is NULL the card marking > must be stale and can be ignored. > > In the non-humongous case, we know we're dealing with an old-gen > region.??Because of that, we know we don't need to deal with > in-progress allocations here.??That allows many simplifications. > > As a result of the changes in oops_on_card_seq_iterate_careful, we > now almost never fail to process the card.??The only place where that > can occur is a stale card in a humongous region with an in-progress > allocation, where we can just ignore it.??So the only caller, > refine_card, no longer needs to examine the result of the call and > enqueue the card for later reconsideration. > > Note that some of the complexity in refine_card / oops_on_card_xxx > arises from the possibility of "stale" marked cards, especially > during concurrent refinement.??These are cards in regions that have > been freed and then reallocated.??At present the only source of stale > cards in the concurrent case seems to be HCC eviction.??(Eager > reclaim of humongous objects flushes relevant remembered sets > (possibly containing stale entries) for reconsideration during that > GC, but in-GC card clipping to scan_top deals with that.)??Doing HCC > cleanup when freeing regions might remove the need for klass_or_null > checking in the humongous case for concurrent refinement, so might be > worth looking into later. CR for that? :) > > CR: > https://bugs.openjdk.java.net/browse/JDK-8166607 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8166607/webrev.01/ > > Testing: > Nightly test equiv. via rbt. (in progress) > Local specjbb2015 (non-perf). Just mentioning: this change already implements some changes suggested in JDK-8162928. So it will improve performance a little. > Local jtreg hotspot_all. > Some comments: - g1BlockOffsetTable.cpp:230: please add braces around the return. - in the new static method do_oops_on_card_in_humongous(), do you think it would be useful to avoid the if-cascading a little by doing early returns? - I think the comment at heapRegion.cpp:411 seems out of place (now?). I would rather put it somewhere around line 444. - what does "Non-humongous objects only reach the old-gen during GC." mean in the comment? Probably this means that "G1 only allocates into old gen during GC" which seems much more clear to me. Looks good otherwise. Thanks, ? Thomas From bob.vandette at oracle.com Fri Sep 30 13:58:55 2016 From: bob.vandette at oracle.com (Bob Vandette) Date: Fri, 30 Sep 2016 09:58:55 -0400 Subject: Webrev of Oracle ARM & AARCH64 Sources In-Reply-To: <187f08d2-2f6d-a4f6-8662-7e8ac85308f7@oracle.com> References: <5A08C4A6-2B69-4D29-80F5-1CE11E350283@oracle.com> <1475231521.14313.29.camel@gmail.com> <187f08d2-2f6d-a4f6-8662-7e8ac85308f7@oracle.com> Message-ID: <7561C178-318F-47D1-B22D-CEB73F4E2F7A@oracle.com> > On Sep 30, 2016, at 7:23 AM, David Holmes wrote: > > Hi Ed, > > On 30/09/2016 8:32 PM, Edward Nevill wrote: >> Hi Bob, >> >> On Wed, 2016-09-28 at 10:07 -0400, Bob Vandette wrote: >>> I?m am please to announce that I have completed our internal reviews and can now >>> open up the sources to our ARM 32 & 64 bit implementations of JDK9. >> >> Great news. >> >>> >>> Here is a webrev that includes a patch that can be applied on top of the >>> (http://hg.openjdk.java.net/aarch32-port/jdk9-arm3264/ ) forest. >>> >>> http://cr.openjdk.java.net/~bobv/arm3264/webrev >> >> I have built this natively on armv7 and aarch64 without problems. In both cases I built the minimal,client,server combination. >> >> I have also tested the server build with JTreg hotspot & langtools. >> >> On aarch64 I got the following results:- >> >> hotspot: Test results: passed: 1,280; failed: 8; error: 43 >> langtools: Test results: passed: 3,700; failed: 1; error: 29 >> >> This is fairly typical, for example, using the Linaro 1609 build I get >> >> hotspot: Test results: passed: 1,271; failed: 17; error: 43 >> langtools: Test results: passed: 3,715; failed: 3; error: 12 > > So most of the "errors" are probably tests that are ignored. > >> On armv7 I first of all had to pin the jvm.cfg to use -server as it kept on using the client as I was not running on a 'server class machine' (since my armv7 device is a Samsung Chromebook this is fair enough). >> >> However, I then had problems with the test harness crashing with the error >> >> # Internal Error (synchronizer.cpp:1576), pid=6533, tid=6542 >> # guarantee(mid->header()->is_neutral()) failed: invariant >> >> (hs_err here http://cr.openjdk.java.net/~enevill/aarch32/hs_err_pid6533.log) > > Strange - that looks like our closed bug: > > https://bugs.openjdk.java.net/browse/JDK-8071540 > > which should be fixed. It was a missing memory barrier issue. I checked and this fix was applied but later a newer implementation was added. The memory barriers were moved to cas_for_lock_acquire with optimized versions for aarch64. Be aware that G1 support on 32-bit ARM is not fully supported by the GC team. It?s experimental. This means that there was an attempt to implement it but it has not undergone the same level of testing as other platforms. I?ll forward this issue to the GC team. Bob. > > Aside: we're going to have to figure out how to deal with currently closed bug reports. > > David > ----- > >> On a subsequent run the test harness locked up after 85 tests. >> >> It looks like there might be a problem with locks breaking with G1GC. I will keep investigating. >> >> I ran it again using the template interpreter for the test harness and the server for the jdk under test and it ran successfully. >> >> On armv7 I got >> >> hotspot: Test results: passed: 1,202; failed: 11; error: 36 >> langtools: Test results: passed: 3,718; failed: 5; error: 8 >> >> I also did some limited benchmarking which I am not going to share on a public forum. Lets just say the performance of the server JIT on armv7 was pleasing. >> >> Thanks for all your work getting to this stage. It looks good to push and you should shortly have committeer rights, if you don't get them within a few days ping me and I will give ops a nudge. >> >> All the best, >> Ed. >> >> From bob.vandette at oracle.com Fri Sep 30 14:03:10 2016 From: bob.vandette at oracle.com (Bob Vandette) Date: Fri, 30 Sep 2016 10:03:10 -0400 Subject: Webrev of Oracle ARM & AARCH64 Sources In-Reply-To: <1475231521.14313.29.camel@gmail.com> References: <5A08C4A6-2B69-4D29-80F5-1CE11E350283@oracle.com> <1475231521.14313.29.camel@gmail.com> Message-ID: <25E205BE-9EE9-488A-9D08-DED365288D7F@oracle.com> Ed, Could you try the 32-bit ARM binaries from the early access page to see if the G1GC problem reproduces? https://jdk9.java.net/download/ Bob. > On Sep 30, 2016, at 6:32 AM, Edward Nevill wrote: > > Hi Bob, > > On Wed, 2016-09-28 at 10:07 -0400, Bob Vandette wrote: >> I?m am please to announce that I have completed our internal reviews and can now >> open up the sources to our ARM 32 & 64 bit implementations of JDK9. > > Great news. > >> >> Here is a webrev that includes a patch that can be applied on top of the >> (http://hg.openjdk.java.net/aarch32-port/jdk9-arm3264/ ) forest. >> >> http://cr.openjdk.java.net/~bobv/arm3264/webrev > > I have built this natively on armv7 and aarch64 without problems. In both cases I built the minimal,client,server combination. > > I have also tested the server build with JTreg hotspot & langtools. > > On aarch64 I got the following results:- > > hotspot: Test results: passed: 1,280; failed: 8; error: 43 > langtools: Test results: passed: 3,700; failed: 1; error: 29 > > This is fairly typical, for example, using the Linaro 1609 build I get > > hotspot: Test results: passed: 1,271; failed: 17; error: 43 > langtools: Test results: passed: 3,715; failed: 3; error: 12 > > On armv7 I first of all had to pin the jvm.cfg to use -server as it kept on using the client as I was not running on a 'server class machine' (since my armv7 device is a Samsung Chromebook this is fair enough). > > However, I then had problems with the test harness crashing with the error > > # Internal Error (synchronizer.cpp:1576), pid=6533, tid=6542 > # guarantee(mid->header()->is_neutral()) failed: invariant > > (hs_err here http://cr.openjdk.java.net/~enevill/aarch32/hs_err_pid6533.log) > > On a subsequent run the test harness locked up after 85 tests. > > It looks like there might be a problem with locks breaking with G1GC. I will keep investigating. > > I ran it again using the template interpreter for the test harness and the server for the jdk under test and it ran successfully. > > On armv7 I got > > hotspot: Test results: passed: 1,202; failed: 11; error: 36 > langtools: Test results: passed: 3,718; failed: 5; error: 8 > > I also did some limited benchmarking which I am not going to share on a public forum. Lets just say the performance of the server JIT on armv7 was pleasing. > > Thanks for all your work getting to this stage. It looks good to push and you should shortly have committeer rights, if you don't get them within a few days ping me and I will give ops a nudge. > > All the best, > Ed. > > From igor.ignatyev at oracle.com Fri Sep 30 15:34:55 2016 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 30 Sep 2016 18:34:55 +0300 Subject: RFR(S) : 8166925 : several native TESTs should be changed to TEST_VM Message-ID: http://cr.openjdk.java.net/~iignatyev/8166925/webrev.00 > 109 lines changed: 0 ins; 0 del; 109 mod; Hi all, could you please review this patch which fixes type of tests? the fix is basically 's/TEST/TEST_VM/?. there are several tests which depend on initialized JVM, but do not declare it (by using TEST_VM macros). here is the list of those tests along w/ observations where/when they need JVM: 1. all arrayOopDesc tests use arrayOopDesc::header_size_in_bytes which depends on a jvm state, it reads HeapWordSize and UseCompressedClassPointers and furthermore has an assert which checks that their values haven't changed. 2. all utilities/json tests use tty variable (as JSON::_st field), tty initialization is a port of jvm initialization. 3. SymbolTable.temp_new_symbol calls JavaThread::current() 4. LogFileOutput tests use ResourceMark which calls JavaThread::current() 5. LogDecorations.level, uptime, tags and timestamps tests directly or indirectly call os::javaTimeNanos() which assumes that some structures, e.g. Bsd::_timebase_info, are inited 6. LogConfigurationTest tests require ResourceMark or os::javaTimeNanos JBS: https://bugs.openjdk.java.net/browse/JDK-8166925 webrev: http://cr.openjdk.java.net/~iignatyev/8166925/webrev.00 Thanks, ? Igor From martin.doerr at sap.com Fri Sep 30 15:37:46 2016 From: martin.doerr at sap.com (Doerr, Martin) Date: Fri, 30 Sep 2016 15:37:46 +0000 Subject: RFR(M): 8166560: [s390] Basic enablement of s390 port. In-Reply-To: References: <7b9201ce-a417-e489-7b46-c0488e8fda8f@oracle.com> Message-ID: Hi Coleen, I can contribute the change which G?tz has mentioned. Therefore, I have created a bug: 8166970: Adapt mutex padding according to DEFAULT_CACHE_LINE_SIZE I will send out a RFR. Best regards, Martin -----Original Message----- From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Lindenmaier, Goetz Sent: Mittwoch, 28. September 2016 11:53 To: Coleen Phillimore ; hotspot-dev at openjdk.java.net Subject: RE: RFR(M): 8166560: [s390] Basic enablement of s390 port. Hi Coleen, yes, the #include change simplified this a lot. Also maintaining the patch queue we have ... We reordered the fields of mutex and aligned it to cache line boundaries. This is the S390 part of this alignment change. S390 has 256B cache lines. I'll remove it as the other parts of this change are missing anyways. I?ll send a new webrev, soon. Best regards, Goetz > -----Original Message----- > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On > Behalf Of Coleen Phillimore > Sent: Dienstag, 27. September 2016 20:30 > To: hotspot-dev at openjdk.java.net > Subject: Re: RFR(M): 8166560: [s390] Basic enablement of s390 port. > > > Hi, I'm surprised there aren't more changes, probably because of the > #include work that was done by you previously. Which is great. > > http://cr.openjdk.java.net/~goetz/wr16/8166560- > basic_s390/hotspot.wr01/src/share/vm/runtime/mutex.hpp.udiff.html > > Can you describe this change? This makes all of the mutex's take a lot > of space, and some don't have very long names. Was this added to avoid > false sharing as the comment implies or to support longer names in > monitors? > > thanks, > Coleen > > > > On 9/27/16 1:57 PM, Volker Simonis wrote: > > On Fri, Sep 23, 2016 at 8:11 AM, David Holmes > wrote: > >> Hi Goetz, > >> > >> I see a change not related directly to S390 ie change from ARM to ARM32 > in > >> src/os/linux/vm/os_linux.cpp > >> > > The change looks a little confusing because Goetz reordered the ifdef > > cascades alphabetically (which I think is good). > > > > Besides that, the only real change not related to s390 is indeed the > > change from ARM to ARM32 which happend two times in the file. > > > > @Goetz: have you done this intentionally? > > > >> It will be a while before I can go through this in any detail. > >> > >> David > >> > >> > >> On 23/09/2016 3:52 PM, Lindenmaier, Goetz wrote: > >>> Hi, > >>> > >>> This change is part of the s390 port. It contains some basic adaptions > >>> needed for a full hotspot port for linux s390x. > >>> > >>> It defines the required macros, platform names and includes. > >>> > >>> The s390 port calles CodeCache::contains() in current_frame(), which is > >>> used in NMT. As NMT already collects stack traces before the CodeCache > is > >>> initialized, contains() needs a check for this. > >>> > >>> Wherever a row of platforms are listed, I sorted them alphabetically. > >>> > >>> The jdk requires the file jvm.cfg. > >>> > >>> Please review. I please need a sponsor. > >>> http://cr.openjdk.java.net/~goetz/wr16/8166560- > basic_s390/hotspot.wr01/ > >>> http://cr.openjdk.java.net/~goetz/wr16/8166560-basic_s390/jdk.wr01/ > >>> > >>> Best regards, > >>> Goetz. > >>> From igor.ignatyev at oracle.com Fri Sep 30 16:06:31 2016 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 30 Sep 2016 19:06:31 +0300 Subject: RFR(S) : 8166129 : hitting vmassert during gtest execution doesn't generate core and hs_err files Message-ID: <1ACC38B9-8FD4-4F9E-A783-B05AFF98CA61@oracle.com> http://cr.openjdk.java.net/~iignatyev/8166129/webrev.00/ > 54 lines changed: 31 ins; 14 del; 9 mod; Hi all, could you please review the patch which enables core dumping and hs_err file generation in case of vmassert, guarantee and the like are hit during gtest tests execution? before this fix, we always initialize a jvm w/ -XX:+SuppressFatalErrorMessage and -XX:-CreateCoredumpOnCrash, now we do so only if we run assert tests, tests which we expect to hit an vmassert. JBS: https://bugs.openjdk.java.net/browse/JDK-8166129 webrev: http://cr.openjdk.java.net/~iignatyev/8166129/webrev.00/ Thanks, ? Igor From andrey.petushkov at gmail.com Thu Sep 29 10:13:47 2016 From: andrey.petushkov at gmail.com (Andrey Petushkov) Date: Thu, 29 Sep 2016 13:13:47 +0300 Subject: [aarch64-port-dev ] Webrev of Oracle ARM & AARCH64 Sources In-Reply-To: <36626b7a-cb53-b13b-b529-d9c43ed28d0f@redhat.com> References: <5A08C4A6-2B69-4D29-80F5-1CE11E350283@oracle.com> <7ea0939d-4aa1-05fc-1eac-f1da849920f4@redhat.com> <36626b7a-cb53-b13b-b529-d9c43ed28d0f@redhat.com> Message-ID: <9ECF7394-3D91-4A11-840C-37318CBA0CBF@gmail.com> Dear Andrew, Pardon, I?m too puzzled by your statement. To my best knowledge no one from Azul considers this port as immature and wishes to drop it. Nor AFAIK anyone from Azul has expressed that on the maillist. In addition, aarch32 port shares much in common with RH?s aarch64 implementation so well, if you keep aarch64 in main openjdk repos it?s much easier to merge aarch32 into it, rather than merge with Sun/Oracle?s arm implementation. And yes, aarch32 is missing c2 port, but this should not make any difference long-term, right? Regards, Andrey > On 29 Sep 2016, at 11:50, Andrew Haley wrote: > > On 28/09/16 20:22, Kostya Zolotnikov wrote: >> >> Not sure who told You about Azul's view. >> For us it is not that clear what make sense for You. > > I'm not quite sure what you mean by this, but I can only go by what > Azul representatives have posted here. If I've got anything wrong, > I am happy to be corrected. > > Andrew. > From andrey.petushkov at gmail.com Thu Sep 29 10:54:07 2016 From: andrey.petushkov at gmail.com (Andrey Petushkov) Date: Thu, 29 Sep 2016 13:54:07 +0300 Subject: [aarch64-port-dev ] Webrev of Oracle ARM & AARCH64 Sources In-Reply-To: <8fa2ec4d-3f6a-7d47-72f4-cafa1964d61a@redhat.com> References: <5A08C4A6-2B69-4D29-80F5-1CE11E350283@oracle.com> <7ea0939d-4aa1-05fc-1eac-f1da849920f4@redhat.com> <36626b7a-cb53-b13b-b529-d9c43ed28d0f@redhat.com> <9ECF7394-3D91-4A11-840C-37318CBA0CBF@gmail.com> <8fa2ec4d-3f6a-7d47-72f4-cafa1964d61a@redhat.com> Message-ID: <15B7B9FF-A282-4687-B847-ED17B0F7AC0E@gmail.com> > On 29 Sep 2016, at 13:43, Andrew Haley wrote: > > On 29/09/16 11:13, Andrey Petushkov wrote: > >> Pardon, I?m too puzzled by your statement. To my best knowledge no >> one from Azul considers this port as immature and wishes to drop >> it. Nor AFAIK anyone from Azul has expressed that on the >> maillist. > > My mistake, then. I should have waited for you to speak. I > apologize. > >> In addition, aarch32 port shares much in common with RH?s aarch64 >> implementation so well, if you keep aarch64 in main openjdk repos >> it?s much easier to merge aarch32 into it, rather than merge with >> Sun/Oracle?s arm implementation. > > There is no good technical reason for AArch64 to merge with AArch32, > and IMO it would create a mess. AArch64 is a clean-sheet design which > shares some of its DNA with AArch32, but that is all. It's not like > x86-64, which is a 64-bit extension of x86. Well, it?s AArch32 which is not clean-sheet design but rather borrows from AArch64 :) I admit that there are much more difference between architectures than for x86 so you might be right that the difference in the code could be too big. I just have a feeling it?s not. I can mistake of course, I did not diff specifically > >> And yes, aarch32 is missing c2 port, but this should not make any >> difference long-term, right? > > I guess not, if a C2 port is forthcoming. My idea here that Oracle?s Graal can make c2 obsolete and it might not worth to port c2 only for Java 9 Andrey > > Andrew. From andrey.petushkov at gmail.com Thu Sep 29 11:10:29 2016 From: andrey.petushkov at gmail.com (Andrey Petushkov) Date: Thu, 29 Sep 2016 14:10:29 +0300 Subject: [aarch64-port-dev ] Webrev of Oracle ARM & AARCH64 Sources In-Reply-To: <91d5a8c4-26e8-13a5-d45f-1f24be595361@redhat.com> References: <5A08C4A6-2B69-4D29-80F5-1CE11E350283@oracle.com> <7ea0939d-4aa1-05fc-1eac-f1da849920f4@redhat.com> <36626b7a-cb53-b13b-b529-d9c43ed28d0f@redhat.com> <9ECF7394-3D91-4A11-840C-37318CBA0CBF@gmail.com> <8fa2ec4d-3f6a-7d47-72f4-cafa1964d61a@redhat.com> <15B7B9FF-A282-4687-B847-ED17B0F7AC0E@gmail.com> <91d5a8c4-26e8-13a5-d45f-1f24be595361@redhat.com> Message-ID: <8441C732-88C4-4D7C-A6CA-3FB06C20FB55@gmail.com> > On 29 Sep 2016, at 14:04, Andrew Haley wrote: > > On 29/09/16 11:54, Andrey Petushkov wrote: >> >>> On 29 Sep 2016, at 13:43, Andrew Haley wrote: >>> >>> On 29/09/16 11:13, Andrey Petushkov wrote: >>> >>>> In addition, aarch32 port shares much in common with RH?s aarch64 >>>> implementation so well, if you keep aarch64 in main openjdk repos >>>> it?s much easier to merge aarch32 into it, rather than merge with >>>> Sun/Oracle?s arm implementation. >>> >>> There is no good technical reason for AArch64 to merge with AArch32, >>> and IMO it would create a mess. AArch64 is a clean-sheet design which >>> shares some of its DNA with AArch32, but that is all. It's not like >>> x86-64, which is a 64-bit extension of x86. >> >> Well, it?s AArch32 which is not clean-sheet design but rather >> borrows from AArch64 :) > > I meant to say: the AArch64 hardware architecture is a clean-sheet > design which shares some of its DNA with AArch32, but that is all. Ok, git it. Sorry for misunderstanding > >> I admit that there are much more difference between architectures >> than for x86 so you might be right that the difference in the code >> could be too big. I just have a feeling it?s not. I can mistake of >> course, I did not diff specifically > > Possibly. I don't really want to make the port a mess of #ifdefs and > suchlike, so I'd fight pretty hard against it. > > It is possible to do some macro trickery to write common runtime > routines, but with sufficient such trickery it'd be possible to do > that with any two unrelated arches. While it might save some time, it > adds another layer of complexity and makes it harder to write > efficient and idiomatic code. In particular things such as predicated > instructions, which are an important part of the AArch32 ISA, are > absent on AArch64. Very much possible. I don?t want to propose it now, was just mentioning the probability. Ok it?s low, let?s dismiss it > > Andrew. From coleen.phillimore at oracle.com Fri Sep 30 19:02:57 2016 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 30 Sep 2016 15:02:57 -0400 Subject: RFR (M) 8164921: Memory leaked when instrumentation.retransformClasses() is called repeatedly Message-ID: <27332fc5-4e48-d931-7603-f722c9c9bdd4@oracle.com> Summary: Return Metablocks smaller than dictionary's dark matter. This change contributed by Jon Masamitsu and myself. To reclaim "dark matter" this change adds an array of small blocks by size, created lazily, to return Metablocks smaller than the BinaryTreeDictionary entry's minimum size. This change also fixed a bug in small object double free and adds debugging code to check for this case. With this change, the submitted test case runs indefinitely. Also passed rbt tier 1-5 testing. open webrev at http://cr.openjdk.java.net/~coleenp/8164921.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8164921 Thanks, Coleen and Jon From dmitry.samersoff at oracle.com Fri Sep 30 19:46:15 2016 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Fri, 30 Sep 2016 22:46:15 +0300 Subject: RFR (M) 8164921: Memory leaked when instrumentation.retransformClasses() is called repeatedly In-Reply-To: <27332fc5-4e48-d931-7603-f722c9c9bdd4@oracle.com> References: <27332fc5-4e48-d931-7603-f722c9c9bdd4@oracle.com> Message-ID: <62fcf1e8-de12-5f25-5de2-9b9db39a0ca1@oracle.com> Coleen, My $.02 1. metaspace.cpp:279 we have assert(word_size >= _small_block_min_size, ...) so at 868 and 895 if (word_size < SmallBlocks::small_block_min_size()) { probably should be asserts as well 2. It might be better to rewrite loops at metaspace.cpp 261,268 for (uint i = _small_block_min_size; i < _small_block_max_size; i++) { as for (uint i = 0; i < (_small_block_max_size - _small_block_min_size); ++i) { ... -Dmitry On 2016-09-30 22:02, Coleen Phillimore wrote: > Summary: Return Metablocks smaller than dictionary's dark matter. > > This change contributed by Jon Masamitsu and myself. To reclaim "dark > matter" this change adds an array of small blocks by size, created > lazily, to return Metablocks smaller than the BinaryTreeDictionary > entry's minimum size. This change also fixed a bug in small object > double free and adds debugging code to check for this case. > > With this change, the submitted test case runs indefinitely. Also > passed rbt tier 1-5 testing. > > open webrev at http://cr.openjdk.java.net/~coleenp/8164921.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8164921 > > Thanks, > Coleen and Jon -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From dmitry.samersoff at oracle.com Fri Sep 30 20:18:54 2016 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Fri, 30 Sep 2016 23:18:54 +0300 Subject: RFR (M) 8164921: Memory leaked when instrumentation.retransformClasses() is called repeatedly In-Reply-To: <62fcf1e8-de12-5f25-5de2-9b9db39a0ca1@oracle.com> References: <27332fc5-4e48-d931-7603-f722c9c9bdd4@oracle.com> <62fcf1e8-de12-5f25-5de2-9b9db39a0ca1@oracle.com> Message-ID: Coleen, 1. Never mind, should read bug report more carefully. Sorry! -Dmitry On 2016-09-30 22:46, Dmitry Samersoff wrote: > Coleen, > > My $.02 > > 1. > metaspace.cpp:279 we have assert(word_size >= _small_block_min_size, > ...) so at 868 and 895 > if (word_size < SmallBlocks::small_block_min_size()) { > probably should be asserts as well > > > 2. > It might be better to rewrite loops at metaspace.cpp 261,268 > for (uint i = _small_block_min_size; i < _small_block_max_size; i++) { > as > for (uint i = 0; > i < (_small_block_max_size - _small_block_min_size); ++i) { > ... > > -Dmitry > > On 2016-09-30 22:02, Coleen Phillimore wrote: >> Summary: Return Metablocks smaller than dictionary's dark matter. >> >> This change contributed by Jon Masamitsu and myself. To reclaim "dark >> matter" this change adds an array of small blocks by size, created >> lazily, to return Metablocks smaller than the BinaryTreeDictionary >> entry's minimum size. This change also fixed a bug in small object >> double free and adds debugging code to check for this case. >> >> With this change, the submitted test case runs indefinitely. Also >> passed rbt tier 1-5 testing. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8164921.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8164921 >> >> Thanks, >> Coleen and Jon > > -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From cthalinger at twitter.com Fri Sep 30 22:42:18 2016 From: cthalinger at twitter.com (Christian Thalinger) Date: Fri, 30 Sep 2016 12:42:18 -1000 Subject: [aarch64-port-dev ] Webrev of Oracle ARM & AARCH64 Sources In-Reply-To: <15B7B9FF-A282-4687-B847-ED17B0F7AC0E@gmail.com> References: <5A08C4A6-2B69-4D29-80F5-1CE11E350283@oracle.com> <7ea0939d-4aa1-05fc-1eac-f1da849920f4@redhat.com> <36626b7a-cb53-b13b-b529-d9c43ed28d0f@redhat.com> <9ECF7394-3D91-4A11-840C-37318CBA0CBF@gmail.com> <8fa2ec4d-3f6a-7d47-72f4-cafa1964d61a@redhat.com> <15B7B9FF-A282-4687-B847-ED17B0F7AC0E@gmail.com> Message-ID: <7FFCC804-286B-4CDF-94A7-A8B46DCFC812@twitter.com> > On Sep 29, 2016, at 12:54 AM, Andrey Petushkov wrote: > >> >> On 29 Sep 2016, at 13:43, Andrew Haley wrote: >> >> On 29/09/16 11:13, Andrey Petushkov wrote: >> >>> Pardon, I?m too puzzled by your statement. To my best knowledge no >>> one from Azul considers this port as immature and wishes to drop >>> it. Nor AFAIK anyone from Azul has expressed that on the >>> maillist. >> >> My mistake, then. I should have waited for you to speak. I >> apologize. >> >>> In addition, aarch32 port shares much in common with RH?s aarch64 >>> implementation so well, if you keep aarch64 in main openjdk repos >>> it?s much easier to merge aarch32 into it, rather than merge with >>> Sun/Oracle?s arm implementation. >> >> There is no good technical reason for AArch64 to merge with AArch32, >> and IMO it would create a mess. AArch64 is a clean-sheet design which >> shares some of its DNA with AArch32, but that is all. It's not like >> x86-64, which is a 64-bit extension of x86. > Well, it?s AArch32 which is not clean-sheet design but rather borrows from AArch64 :) > I admit that there are much more difference between architectures than for x86 so you might be right that the difference in the code could be too big. I just have a feeling it?s not. I can mistake of course, I did not diff specifically >> >>> And yes, aarch32 is missing c2 port, but this should not make any >>> difference long-term, right? >> >> I guess not, if a C2 port is forthcoming. > My idea here that Oracle?s Graal can make c2 obsolete and it might not worth to port c2 only for Java 9 Listen to this man :-) From coleen.phillimore at oracle.com Fri Sep 30 22:44:27 2016 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 30 Sep 2016 18:44:27 -0400 Subject: RFR (M) 8164921: Memory leaked when instrumentation.retransformClasses() is called repeatedly In-Reply-To: <62fcf1e8-de12-5f25-5de2-9b9db39a0ca1@oracle.com> References: <27332fc5-4e48-d931-7603-f722c9c9bdd4@oracle.com> <62fcf1e8-de12-5f25-5de2-9b9db39a0ca1@oracle.com> Message-ID: <4efb2443-9ade-fb56-2016-f46bb5b2eb57@oracle.com> On 9/30/16 3:46 PM, Dmitry Samersoff wrote: > Coleen, > > My $.02 > > 1. > metaspace.cpp:279 we have assert(word_size >= _small_block_min_size, > ...) so at 868 and 895 > if (word_size < SmallBlocks::small_block_min_size()) { > probably should be asserts as well > > > 2. > It might be better to rewrite loops at metaspace.cpp 261,268 > for (uint i = _small_block_min_size; i < _small_block_max_size; i++) { > as > for (uint i = 0; > i < (_small_block_max_size - _small_block_min_size); ++i) { > ... for (uint i = _small_block_min_size; i < _small_block_max_size; i++) { _small_lists[i - _small_block_min_size].set_size(i); } In this loop I want 'i' to set the size. The index of small_lists starts at 0 but the size is the size of the blocks. This might make more sense like: for (uint i = _small_block_min_size; i < _small_block_max_size; i++) { int k = i - _small_block_min_size; // zero based addressing _small_lists[k].set_size(i); } Or something but the loops seem simple enough to me. Coleen > > -Dmitry > > On 2016-09-30 22:02, Coleen Phillimore wrote: >> Summary: Return Metablocks smaller than dictionary's dark matter. >> >> This change contributed by Jon Masamitsu and myself. To reclaim "dark >> matter" this change adds an array of small blocks by size, created >> lazily, to return Metablocks smaller than the BinaryTreeDictionary >> entry's minimum size. This change also fixed a bug in small object >> double free and adds debugging code to check for this case. >> >> With this change, the submitted test case runs indefinitely. Also >> passed rbt tier 1-5 testing. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8164921.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8164921 >> >> Thanks, >> Coleen and Jon > From cthalinger at twitter.com Fri Sep 30 22:48:23 2016 From: cthalinger at twitter.com (Christian Thalinger) Date: Fri, 30 Sep 2016 12:48:23 -1000 Subject: Merging jdk9/hs-comp with jdk9/hs In-Reply-To: References: Message-ID: <8DAC3F7E-485D-483A-8F8B-9B280C712015@twitter.com> End of an era?? > On Sep 29, 2016, at 8:23 AM, Mikael Vidstedt wrote: > > > All, > > Over the last two years we have worked towards simplifying the mercurial forest structure for Hotspot development by consolidating and removing forests. Last year we moved[1] the GC development work from jdk9/hs-gc[2] to jdk9/hs-rt[3], and earlier this year we further consolidated forests by moving[4] the jdk9/hs-rt work directly to the jdk9/hs[5] - aka. "hs main" - forest. The reduction in forests has led to less work spent on integration, faster quality feedback, more efficient hardware utilization, etc. All in all, the experience so far has been a good one. > > However, the Hotspot (JIT) compiler changes are still integrated through the jdk9/hs-comp[6] forest, and are tested separately before they are (bulk) integrated up to jdk9/hs. The separate forest of course comes with the expected integration/gatekeeper overhead and lead times. Since JDK 9 development is ramping down, we expect the number of changes to ramp down as well. This means that the benefit of having a separate jdk9/hs-comp forest is going to reduce over time. Now seems like a good time to look at doing a final hotspot forest consolidation. > > In line with this, we propose closing down jdk9/hs-comp and doing all remaining JDK 9 Hotspot development to jdk9/hs. We propose that this final forest consolidation to take place on Friday, October 7th. Much like the earlier forest consolidations the jdk9/hs-comp forest will be kept around, but will be locked down so that no accidental integrations are made to it. Any work in flight based on jdk9/hs-comp would have to be rebased on jdk9/hs. > > The earlier forest consolidations have gone very smoothly, and we expect this one to go smoothly as well, but if for some reason things don't work out we will of course have an option to go back to what we have today. That said, before using the escape hatch and reverting back, we will of course want to try to address any issues - thank you in advance for your patience while we work through any such issues. > > Please let us know if you have any feedback or questions! If no serious objections have been raised by 15:00 UTC Wednesday, 5 October 2016, we will go ahead with the forest consolidation. > > Cheers, > Mikael > > [1]http://mail.openjdk.java.net/pipermail/hotspot-dev/2015-May/thread.html > [2]http://hg.openjdk.java.net/jdk9/hs-gc > [3]http://hg.openjdk.java.net/jdk9/hs-rt > [4]http://mail.openjdk.java.net/pipermail/hotspot-dev/2016-March/022218.html > [5]http://hg.openjdk.java.net/jdk9/hs > [6]http://hg.openjdk.java.net/jdk9/hs-comp > From kim.barrett at oracle.com Fri Sep 30 23:46:20 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 30 Sep 2016 19:46:20 -0400 Subject: RFR: 8166607: G1 needs klass_or_null_acquire In-Reply-To: <1475236854.6301.70.camel@oracle.com> References: <98788D01-D3B4-45C4-B133-204B36A6799D@oracle.com> <1475236854.6301.70.camel@oracle.com> Message-ID: > On Sep 30, 2016, at 8:00 AM, Thomas Schatzl wrote: > On Thu, 2016-09-29 at 18:49 -0400, Kim Barrett wrote: >> Please review this change to G1 to use klass_or_null_acquire where >> needed. >> >> [Reviewing on hotspot-dev; only GC code is involved, but some non-GC >> folks have expressed interest.] >> >> G1 BOT uses of klass_or_null have all been changed to instead use >> klass_or_null_acquire. This isn't necessary for the call in the >> revised version of oops_on_card_seq_iterate_careful (discussed >> below), but there can be (and are) other callers. The performance >> impact of unnecessary barriers in oops_on_card_xxx will hopefully not >> be too bad on non-TSO systems; the number reduces to one as the BOT >> gets filled in. Rather than splitting or otherwise conditionalizing, >> for now we just make it safe. > > Could you add a comment to JDK-8162928? Done. >> [The CollectedHeap block_start &etc API needs some TLC in the face >> of collectors that use klass_or_null > > TLC? TLC = Tender Loving Care The API around CollectedHeap::block_start and friends doesn't deal very well with the possibility of querying a location within an in-progress allocation. This is clearly an issue for CMS and G1. CMS even goes so far as having its own specialized variants of parts that API, and JDK-8162928 suggests doing similarly for G1. But even for a non-concurrent collector it could be an issue; consider os::print_location, which could be applied to any address at any time. Descriptions of these functions make no mention of that problem. Some callers know how to cope, generally when dealing with a specific collector. Not sure how to cope in generic code; the API might need rework. I suppose I should file that as an RFE. >> [?] >> Note that some of the complexity in refine_card / oops_on_card_xxx >> arises from the possibility of "stale" marked cards, especially >> during concurrent refinement. These are cards in regions that have >> been freed and then reallocated. At present the only source of stale >> cards in the concurrent case seems to be HCC eviction. (Eager >> reclaim of humongous objects flushes relevant remembered sets >> (possibly containing stale entries) for reconsideration during that >> GC, but in-GC card clipping to scan_top deals with that.) Doing HCC >> cleanup when freeing regions might remove the need for klass_or_null >> checking in the humongous case for concurrent refinement, so might be >> worth looking into later. > > CR for that? :) https://bugs.openjdk.java.net/browse/JDK-8166995 Consider removing stale cards from HCC during cleanup >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8166607 >> >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8166607/webrev.01/ >> >> Testing: >> Nightly test equiv. via rbt. (in progress) >> Local specjbb2015 (non-perf). > > Just mentioning: this change already implements some changes suggested > in JDK-8162928. So it will improve performance a little. > >> Local jtreg hotspot_all. >> > > Some comments: > > - g1BlockOffsetTable.cpp:230: please add braces around the return. Present code is similar to other nearby code. > - in the new static method do_oops_on_card_in_humongous(), do you think > it would be useful to avoid the if-cascading a little by doing early > returns? I wrote it somewhat like that earlier, but prefer how it reads now. > - I think the comment at heapRegion.cpp:411 seems out of place (now?). > I would rather put it somewhere around line 444. Good point. Done. > - what does "Non-humongous objects only reach the old-gen during GC." > mean in the comment? Probably this means that "G1 only allocates into > old gen during GC" which seems much more clear to me. Good point. Reworded a bit. Non-humongous objects are only allocated in the old-gen during GC. > Looks good otherwise. Thanks. From serguei.spitsyn at oracle.com Fri Sep 30 23:57:31 2016 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 30 Sep 2016 16:57:31 -0700 Subject: RFR (M) 8164921: Memory leaked when instrumentation.retransformClasses() is called repeatedly In-Reply-To: <27332fc5-4e48-d931-7603-f722c9c9bdd4@oracle.com> References: <27332fc5-4e48-d931-7603-f722c9c9bdd4@oracle.com> Message-ID: <101e4a9e-2e91-6415-9dd9-0d57522ce7f6@oracle.com> Hi Coleen, Just wanted to share a couple of comments I have so far. http://cr.openjdk.java.net/~coleenp/8164921.01/webrev/src/share/vm/memory/metaspace.cpp.frames.html 302 debug_only(GrowableArray* _all_blocks); 855 DEBUG_ONLY(_all_blocks = new (ResourceObj::C_HEAP, mtClass) GrowableArray(100, true)); 863 DEBUG_ONLY(delete _all_blocks;) 891 DEBUG_ONLY(_all_blocks->remove((Metablock*)new_block)); 908 DEBUG_ONLY(_all_blocks->remove((Metablock*)free_block)); 922 DEBUG_ONLY(_all_blocks->remove((Metablock*)new_block)); Not clear, why different macros debug_only() is used at L302. Could it be DEBUG_ONLY as well? 866 void BlockFreelist::return_block(MetaWord* p, size_t word_size) { 867 Metablock* free_chunk = ::new (p) Metablock(word_size); 868 if (word_size < SmallBlocks::small_block_min_size()) { 869 // Dark matter 870 return; 871 } else if (word_size < SmallBlocks::small_block_max_size()) { In case of dark matter the free_chunk is leaked. It is better to rearrange the fragment to something like this: void BlockFreelist::return_block(MetaWord* p, size_t word_size) { if (word_size < SmallBlocks::small_block_min_size()) { return; // Dark matter } Metablock* free_chunk = ::new (p) Metablock(word_size); if (word_size < SmallBlocks::small_block_max_size()) { 886 MetaWord* BlockFreelist::get_block(size_t word_size) { 887 if (word_size < SmallBlocks::small_block_max_size() && small_blocks()->list_at(word_size).count() > 0) { . . . 892 return new_block; 893 } 894 895 if (word_size < BlockFreelist::min_size()) { 896 // Dark matter. Too small for dictionary. 897 return NULL; 898 } It'd be better to reorder the fragments 887-893 and 895-898. Thanks, Serguei On 9/30/16 12:02, Coleen Phillimore wrote: > Summary: Return Metablocks smaller than dictionary's dark matter. This > change contributed by Jon Masamitsu and myself. To reclaim "dark > matter" this change adds an array of small blocks by size, created > lazily, to return Metablocks smaller than the BinaryTreeDictionary > entry's minimum size. This change also fixed a bug in small object > double free and adds debugging code to check for this case. With this > change, the submitted test case runs indefinitely. Also passed rbt > tier 1-5 testing. open webrev at > http://cr.openjdk.java.net/~coleenp/8164921.01/webrev bug link > https://bugs.openjdk.java.net/browse/JDK-8164921 Thanks, Coleen and Jon