From varming at gmail.com Sun Nov 1 19:48:10 2015 From: varming at gmail.com (Carsten Varming) Date: Sun, 1 Nov 2015 14:48:10 -0500 Subject: Let jvmtiGen exit with a non-zero exit code upon failure In-Reply-To: <5633EC3E.5020701@oracle.com> References: <5632251B.80606@oracle.com> <5632A09F.1060205@oracle.com> <5632A148.6070102@oracle.com> <56333474.3050406@oracle.com> <4C0FA96C-1122-4236-84B2-B4CBAB2F795E@oracle.com> <5633EC3E.5020701@oracle.com> Message-ID: Dear Serguei and Steffan, Sorry about the late reply. The office was a little busy Friday. Steffan's suggestion is to remove all the exception handling code sound reasonable. I am not a fan of the main method throwing checked exceptions, so I wrapped all the exception handling into a tiny handler that simply prints the exception and exits with a non-zero value. I never cared much for the stacktrace, so I put printing of the stacktrace under the verbose flag. Speaking of cleaning up, there is code in there to use Apache xalan instead of the standard xslt processor due to an ancient bug fixed in Java 1.5. I removed that as well. There was also a comment about the field "document" being package globally accessible due to: "ref'd by the tree-adapter". I have no idea what that means, but changing document to a local variable in main didn't break the build, so I guess that comment is out-of-date. Please let me know if I am missing something here. There was also a use of java.io.PrintStream. That class suppresses IOExceptions: "Unlike other output streams, a PrintStream never throws an IOException", so I removed the PrintStream as I want to get a failure when something goes wrong. I have put a new webrev at http://cr.openjdk.java.net/~cvarming/jvmtiGen.02/ Carsten ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Mon Nov 2 01:40:45 2015 From: david.holmes at oracle.com (David Holmes) Date: Mon, 2 Nov 2015 11:40:45 +1000 Subject: RFR(S): JDK-7041183 Improve error handling in Improve error handling in src/share/vm/services/management.cpp In-Reply-To: <56334E21.4020500@oracle.com> References: <56334E21.4020500@oracle.com> Message-ID: <5636BF1D.8030903@oracle.com> Hi, On 30/10/2015 9:01 PM, Dmitry Samersoff wrote: > Everybody, > > (* On behalf of Jini George *) > > Please review the fix > > http://cr.openjdk.java.net/~dsamersoff/sponsorship/jingeorg/JDK-7041183/webrev.01/ If you throw IllegalArgumentException it should be obvious at the Java level which argument is illegal and why. I can't tell at the Java level how that can come about - these seem to be methods invoked on a MemoryPoolMXBean - see for example: https://bugs.openjdk.java.net/browse/JDK-8025089 so the "illegal argument" would seem to be "this" ??? I don't see any changes to the code that would currently throw Internal Error ?? The asserts for pool!=NULL seem rather pointless as the method has been changed to never return null. If you really want the assert add it to the end of the method being called, rather than placing at all the call sites. Thanks, David > -Dmitry > From david.holmes at oracle.com Mon Nov 2 02:11:54 2015 From: david.holmes at oracle.com (David Holmes) Date: Mon, 2 Nov 2015 12:11:54 +1000 Subject: RFR: JDK-8140556: Add force rotation option to VM.log jcmd In-Reply-To: <5634D9FC.6080502@gmail.com> References: <562101D8.5080909@gmail.com> <562DF8D4.40501@oracle.com> <562EBF4B.4020607@gmail.com> <562F6B46.5050500@oracle.com> <563226AB.2030206@gmail.com> <5632341F.4060400@oracle.com> <56338125.9030802@gmail.com> <56338D58.9090709@oracle.com> <5634D9FC.6080502@gmail.com> Message-ID: <5636C66A.6020304@oracle.com> On 1/11/2015 1:10 AM, Yasumasa Suenaga wrote: >> This in turn means that we are logging before the thread local storage >> has been initialized, and the JVM will crash/hit the assert because we >> are trying to take the rotation lock. > > I guess that you are saying about class member (not TLS - e.g. > pthread_setspecific()). You can't safely lock a mutex until the current thread has attached to the VM and Thread::current() returns non-null (that is what Thread::initialize_thread_local_storage() currently does). You may get away with it, particularly in product builds, during VM initialization, as the fast-lock path doesn't require access to Thread::current(). David > Most of fields (include _rotation_lock) in LogFileOutput are initialized > at c'tor. > Other field (_stream) is initialized at LogFileOutput::initialize(). > > We can avoid problem if we can move initialize() to c'tor because > LogFileOutput > will be instantiated at LogConfiguration::new_output(). > > Currently, LogFileOutput will be added to LogConfiguration::_outputs and > to LogTagSet after calling LogFileOutput::initialize(). > I wonder why we cannot avoid crash/assert with current implementation... > > > I will keep current implementation about _rotation_lock if the above is > incorrect. > > > Thanks, > > Yasumasa > > > On 2015/10/31 0:31, Marcus Larsson wrote: >> Hey, >> >> On 2015-10-30 15:39, Yasumasa Suenaga wrote: >>> Hi Marcus, >>> >>> Thank you for your comment. >>> >>> >>>> Still missing the renaming of rotate_all_logfile to rotate_all_outputs. >>> >>> Sorry, I will fix. >>> >> >> Thanks! >> >>> >>>> This rotate function works I guess, but it will probably need some >>>> rework once there are other types of log outputs (rotate doesn't >>>> make sense on all types of log outputs). >>> >>> I will add is_rotetable() to LogOutput as virtual function. >>> This function return false (can not rotate) by default, return true >>> if _file_count is >>> greater than 0 in LogFileOutput. >>> >>> LogConfiguration::rotate_all_outputs() will rotate if this function >>> returns true. >>> >> >> Small typo, please make that is_rotatable(). >> >>> >>>> If we configure a rotated file output and then log something on that >>>> output before thread local storage initialization is complete, we >>>> will crash/hit an assert. The previous implementation avoided this >>>> as long as no rotation was needed before this initialization was >>>> complete. (This can be triggered using the following arguments >>>> "-Xlog:all=trace:file.txt::filesize=10,filecount=2 -Xlog:invalid", >>>> for example.) >>> >>> I do not think so. >>> Each -Xlog option creates unique instance of LogOutput (and subclass). >>> So they are isolated. >> >> My example includes a second (and intentionally incorrect) -Xlog >> option only to trigger logging on the newly configured rotating >> LogFileOutput. This will cause some logging about the invalid -Xlog >> argument, and this logging happens during the argument parsing. This >> in turn means that we are logging before the thread local storage has >> been initialized, and the JVM will crash/hit the assert because we are >> trying to take the rotation lock. This has nothing to do with >> concurrent fprintfs or freopens, even if that is a problem. >> >>> >>> However, we might have to rotate at safepoint. >> >> Doing stuff at a safepoint will not give us much in this case. There >> are threads that continue execution through a safepoint, which means >> we will see log calls being made even during this time. >> >>> Currently, LogOutput might be used by multiple threads concurrently. >>> If we rotate log when another thread is writing string to output, we >>> encounter unexpected behavior. >>> >>> LogFileOutput::rotate() uses freopen(). >>> LogFileStreamOutput::write() uses vfprintf() through jio_fprintf(). >>> freopen() and vfprintf() are not atomic. >> >> You're right. There seems to be an overlooked concurrency issue with >> vfprintfs during freopen. I don't think we should resolve this as part >> of this particular fix, so instead I'll create a separate issue for it. >> >>> >>> I think that cause of crash/assertion which you say is it. >>> (GC log will be rotated at safepoint.) >>> >> >> See my comment above for what crash/assert I'm talking about. >> >> Thanks, >> Marcus >> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> On 2015/10/29 23:58, Marcus Larsson wrote: >>>> Hi, >>>> >>>> On 2015-10-29 15:01, Yasumasa Suenaga wrote: >>>>> Hi Marcus, >>>>> >>>>> Thank you for your comment. >>>>> >>>>>> I'll sponsor it. >>>>> >>>>> Thank you so much! >>>>> I've uploaded new webrev: >>>>> >>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.01/ >>>>> >>>>> >>>> >>>> Still missing the renaming of rotate_all_logfile to rotate_all_outputs. >>>> >>>>>> logConfiguration.cpp/hpp: >>>>>> >>>>>> * I don't think we should rely on the archive_name or the output's >>>>>> name to decide whether or not an output should be rotated. It >>>>>> would be better to introduce an is_rotated() test function in >>>>>> LogOutput that could be used here. >>>>>> >>>>>> * rotate_all_logfile() should be renamed to rotate_all_outputs(). >>>>>> Currently there are only LogFileOutputs (other than >>>>>> stdout/stderr), but in the future there might be other types of >>>>>> outputs so I prefer having a more general name. >>>>>> Also, please use static_cast instead of the >>>>>> C-style cast. (Additional logic will be required here once more >>>>>> types of log outputs are added, but I don't think we need to worry >>>>>> about this right now.) >>>>>> >>>>>> * Don't print an error if an output is not rotatable, since it's >>>>>> not an error to have some log outputs rotated while others are >>>>>> not. If you really want some traceability here I suggest adding >>>>>> log messages on trace level, tagged with 'logging'. >>>>> >>>>> I added LogOutput::rotate(bool) as virtual function. >>>>> This function works nothing by default, but will rotate all logs at >>>>> LogFileOutput. >>>> >>>> This rotate function works I guess, but it will probably need some >>>> rework once there are other types of log outputs (rotate doesn't >>>> make sense on all types of log outputs). >>>> >>>>> >>>>> If true is passed to this function, all files will be rotated. >>>>> >>>>> >>>>>> logDiagnosticCommand.cpp/hpp: >>>>>> >>>>>> * I think the description could be improved to something like >>>>>> "Lists current log configuration, enables/disables/configures a >>>>>> log output, or disables/rotates all logs." >>>>>> >>>>>> * The rotate option description should clarify that all logs will >>>>>> be rotated ("current log" is too ambiguous). >>>>> >>>>> I've fixed. >>>> >>>> Great, thanks! >>>> >>>>> >>>>> >>>>>> logFileOutput.cpp/hpp: >>>>>> >>>>>> * Moving the MutexLocker like this introduces a race condition >>>>>> where a log might be rotated multiple times by different threads, >>>>>> instead of just once. >>>>>> Instead of making the rotate() function public and moving the >>>>>> mutexlocker, I suggest adding something like a public >>>>>> force_rotation() function that grabs the lock and calls the >>>>>> private rotate(). >>>>>> >>>>>> * Given the addition of is_rotated() in LogOutput, then >>>>>> get_archive_name() should be removed. >>>>> >>>>> I moved MutexLocker and should_rotate() to rotate(). >>>>> I think this change can avoid race condition. >>>>> >>>> >>>> There is a subtle problem with taking the lock to check if we should >>>> rotate. If we configure a rotated file output and then log something >>>> on that output before thread local storage initialization is >>>> complete, we will crash/hit an assert. The previous implementation >>>> avoided this as long as no rotation was needed before this >>>> initialization was complete. (This can be triggered using the >>>> following arguments >>>> "-Xlog:all=trace:file.txt::filesize=10,filecount=2 -Xlog:invalid", >>>> for example.) >>>> >>>> Thanks, >>>> Marcus >>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2015/10/27 21:17, Marcus Larsson wrote: >>>>>> Hi, >>>>>> >>>>>> On 2015-10-27 01:03, Yasumasa Suenaga wrote: >>>>>>> Hi Marcus, >>>>>>> >>>>>>> Thank you for replying. >>>>>>> >>>>>>> I filed this feature to JBS and change subject of this email. >>>>>>> Could you be a sponsor and review it? >>>>>> >>>>>> I'll sponsor it. >>>>>> >>>>>>> >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.00/ >>>>>> >>>>>> logConfiguration.cpp/hpp: >>>>>> >>>>>> * I don't think we should rely on the archive_name or the output's >>>>>> name to decide whether or not an output should be rotated. It >>>>>> would be better to introduce an is_rotated() test function in >>>>>> LogOutput that could be used here. >>>>>> >>>>>> * rotate_all_logfile() should be renamed to rotate_all_outputs(). >>>>>> Currently there are only LogFileOutputs (other than >>>>>> stdout/stderr), but in the future there might be other types of >>>>>> outputs so I prefer having a more general name. >>>>>> Also, please use static_cast instead of the >>>>>> C-style cast. (Additional logic will be required here once more >>>>>> types of log outputs are added, but I don't think we need to worry >>>>>> about this right now.) >>>>>> >>>>>> * Don't print an error if an output is not rotatable, since it's >>>>>> not an error to have some log outputs rotated while others are >>>>>> not. If you really want some traceability here I suggest adding >>>>>> log messages on trace level, tagged with 'logging'. >>>>>> >>>>>> >>>>>> logDiagnosticCommand.cpp/hpp: >>>>>> >>>>>> * I think the description could be improved to something like >>>>>> "Lists current log configuration, enables/disables/configures a >>>>>> log output, or disables/rotates all logs." >>>>>> >>>>>> * The rotate option description should clarify that all logs will >>>>>> be rotated ("current log" is too ambiguous). >>>>>> >>>>>> >>>>>> logFileOutput.cpp/hpp: >>>>>> >>>>>> * Moving the MutexLocker like this introduces a race condition >>>>>> where a log might be rotated multiple times by different threads, >>>>>> instead of just once. >>>>>> Instead of making the rotate() function public and moving the >>>>>> mutexlocker, I suggest adding something like a public >>>>>> force_rotation() function that grabs the lock and calls the >>>>>> private rotate(). >>>>>> >>>>>> * Given the addition of is_rotated() in LogOutput, then >>>>>> get_archive_name() should be removed. >>>>>> >>>>>> Thanks, >>>>>> Marcus >>>>>> >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2015/10/26 18:56, Marcus Larsson wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> Sorry for my late reply. >>>>>>>> >>>>>>>> I think being able to force rotation via jcmd seems like a good >>>>>>>> feature. Files are currently opened in append mode so it should >>>>>>>> already be possible to use external log rotation tools by >>>>>>>> copying and truncating the files. Still I think it would be nice >>>>>>>> to provide the jcmd for rotation as well. >>>>>>>> >>>>>>>> I can see some small issues with the implementation, but we can >>>>>>>> deal with that during the review. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Marcus >>>>>>>> >>>>>>>> >>>>>>>> On 2015-10-26 00:26, Yasumasa Suenaga wrote: >>>>>>>>> Hi all, >>>>>>>>> >>>>>>>>> Have you ever seen my change? >>>>>>>>> I (and my customers) need log rotation function via external tool. >>>>>>>>> >>>>>>>>> I want to merge it by Feature Complete. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> 2015/10/16 22:55 "Yasumasa Suenaga" : >>>>>>>>> >>>>>>>>>> Hi all, >>>>>>>>>> >>>>>>>>>> I contributed JDK-7090324: gclog rotation via external tool to be >>>>>>>>>> synchronized with >>>>>>>>>> logrotated tool on Linux. >>>>>>>>>> >>>>>>>>>> I think JEP 158 is in progress. >>>>>>>>>> However, this JEP does not contain log rotation via external >>>>>>>>>> tool in the >>>>>>>>>> spec. >>>>>>>>>> I want to rotate logs via jcmd in this JEP. >>>>>>>>>> >>>>>>>>>> I've updated a patch for it: >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/jvmlogging-logrotate/ >>>>>>>>>> >>>>>>>>>> This patch provides new option "rotate" in VM.log command. >>>>>>>>>> If this change can be accepted, I will file it to JBS and send >>>>>>>>>> RFR. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>>> >>>>>> >>>> >> From yasuenag at gmail.com Mon Nov 2 02:40:28 2015 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Mon, 2 Nov 2015 11:40:28 +0900 Subject: RFR: JDK-8140556: Add force rotation option to VM.log jcmd In-Reply-To: <5636C66A.6020304@oracle.com> References: <562101D8.5080909@gmail.com> <562DF8D4.40501@oracle.com> <562EBF4B.4020607@gmail.com> <562F6B46.5050500@oracle.com> <563226AB.2030206@gmail.com> <5632341F.4060400@oracle.com> <56338125.9030802@gmail.com> <56338D58.9090709@oracle.com> <5634D9FC.6080502@gmail.com> <5636C66A.6020304@oracle.com> Message-ID: Thank you David! I understood. I will try to create a patch for this enhancement with avoiding this issue. Yasumasa 2015/11/02 11:12 "David Holmes" : > On 1/11/2015 1:10 AM, Yasumasa Suenaga wrote: > >> This in turn means that we are logging before the thread local storage >>> has been initialized, and the JVM will crash/hit the assert because we >>> are trying to take the rotation lock. >>> >> >> I guess that you are saying about class member (not TLS - e.g. >> pthread_setspecific()). >> > > You can't safely lock a mutex until the current thread has attached to the > VM and Thread::current() returns non-null (that is what > Thread::initialize_thread_local_storage() currently does). You may get away > with it, particularly in product builds, during VM initialization, as the > fast-lock path doesn't require access to Thread::current(). > > David > > Most of fields (include _rotation_lock) in LogFileOutput are initialized >> at c'tor. >> Other field (_stream) is initialized at LogFileOutput::initialize(). >> >> We can avoid problem if we can move initialize() to c'tor because >> LogFileOutput >> will be instantiated at LogConfiguration::new_output(). >> >> Currently, LogFileOutput will be added to LogConfiguration::_outputs and >> to LogTagSet after calling LogFileOutput::initialize(). >> I wonder why we cannot avoid crash/assert with current implementation... >> >> >> I will keep current implementation about _rotation_lock if the above is >> incorrect. >> >> >> Thanks, >> >> Yasumasa >> >> >> On 2015/10/31 0:31, Marcus Larsson wrote: >> >>> Hey, >>> >>> On 2015-10-30 15:39, Yasumasa Suenaga wrote: >>> >>>> Hi Marcus, >>>> >>>> Thank you for your comment. >>>> >>>> >>>> Still missing the renaming of rotate_all_logfile to rotate_all_outputs. >>>>> >>>> >>>> Sorry, I will fix. >>>> >>>> >>> Thanks! >>> >>> >>>> This rotate function works I guess, but it will probably need some >>>>> rework once there are other types of log outputs (rotate doesn't >>>>> make sense on all types of log outputs). >>>>> >>>> >>>> I will add is_rotetable() to LogOutput as virtual function. >>>> This function return false (can not rotate) by default, return true >>>> if _file_count is >>>> greater than 0 in LogFileOutput. >>>> >>>> LogConfiguration::rotate_all_outputs() will rotate if this function >>>> returns true. >>>> >>>> >>> Small typo, please make that is_rotatable(). >>> >>> >>>> If we configure a rotated file output and then log something on that >>>>> output before thread local storage initialization is complete, we >>>>> will crash/hit an assert. The previous implementation avoided this >>>>> as long as no rotation was needed before this initialization was >>>>> complete. (This can be triggered using the following arguments >>>>> "-Xlog:all=trace:file.txt::filesize=10,filecount=2 -Xlog:invalid", >>>>> for example.) >>>>> >>>> >>>> I do not think so. >>>> Each -Xlog option creates unique instance of LogOutput (and subclass). >>>> So they are isolated. >>>> >>> >>> My example includes a second (and intentionally incorrect) -Xlog >>> option only to trigger logging on the newly configured rotating >>> LogFileOutput. This will cause some logging about the invalid -Xlog >>> argument, and this logging happens during the argument parsing. This >>> in turn means that we are logging before the thread local storage has >>> been initialized, and the JVM will crash/hit the assert because we are >>> trying to take the rotation lock. This has nothing to do with >>> concurrent fprintfs or freopens, even if that is a problem. >>> >>> >>>> However, we might have to rotate at safepoint. >>>> >>> >>> Doing stuff at a safepoint will not give us much in this case. There >>> are threads that continue execution through a safepoint, which means >>> we will see log calls being made even during this time. >>> >>> Currently, LogOutput might be used by multiple threads concurrently. >>>> If we rotate log when another thread is writing string to output, we >>>> encounter unexpected behavior. >>>> >>>> LogFileOutput::rotate() uses freopen(). >>>> LogFileStreamOutput::write() uses vfprintf() through jio_fprintf(). >>>> freopen() and vfprintf() are not atomic. >>>> >>> >>> You're right. There seems to be an overlooked concurrency issue with >>> vfprintfs during freopen. I don't think we should resolve this as part >>> of this particular fix, so instead I'll create a separate issue for it. >>> >>> >>>> I think that cause of crash/assertion which you say is it. >>>> (GC log will be rotated at safepoint.) >>>> >>>> >>> See my comment above for what crash/assert I'm talking about. >>> >>> Thanks, >>> Marcus >>> >>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> On 2015/10/29 23:58, Marcus Larsson wrote: >>>> >>>>> Hi, >>>>> >>>>> On 2015-10-29 15:01, Yasumasa Suenaga wrote: >>>>> >>>>>> Hi Marcus, >>>>>> >>>>>> Thank you for your comment. >>>>>> >>>>>> I'll sponsor it. >>>>>>> >>>>>> >>>>>> Thank you so much! >>>>>> I've uploaded new webrev: >>>>>> >>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.01/ >>>>>> >>>>>> >>>>>> >>>>> Still missing the renaming of rotate_all_logfile to rotate_all_outputs. >>>>> >>>>> logConfiguration.cpp/hpp: >>>>>>> >>>>>>> * I don't think we should rely on the archive_name or the output's >>>>>>> name to decide whether or not an output should be rotated. It >>>>>>> would be better to introduce an is_rotated() test function in >>>>>>> LogOutput that could be used here. >>>>>>> >>>>>>> * rotate_all_logfile() should be renamed to rotate_all_outputs(). >>>>>>> Currently there are only LogFileOutputs (other than >>>>>>> stdout/stderr), but in the future there might be other types of >>>>>>> outputs so I prefer having a more general name. >>>>>>> Also, please use static_cast instead of the >>>>>>> C-style cast. (Additional logic will be required here once more >>>>>>> types of log outputs are added, but I don't think we need to worry >>>>>>> about this right now.) >>>>>>> >>>>>>> * Don't print an error if an output is not rotatable, since it's >>>>>>> not an error to have some log outputs rotated while others are >>>>>>> not. If you really want some traceability here I suggest adding >>>>>>> log messages on trace level, tagged with 'logging'. >>>>>>> >>>>>> >>>>>> I added LogOutput::rotate(bool) as virtual function. >>>>>> This function works nothing by default, but will rotate all logs at >>>>>> LogFileOutput. >>>>>> >>>>> >>>>> This rotate function works I guess, but it will probably need some >>>>> rework once there are other types of log outputs (rotate doesn't >>>>> make sense on all types of log outputs). >>>>> >>>>> >>>>>> If true is passed to this function, all files will be rotated. >>>>>> >>>>>> >>>>>> logDiagnosticCommand.cpp/hpp: >>>>>>> >>>>>>> * I think the description could be improved to something like >>>>>>> "Lists current log configuration, enables/disables/configures a >>>>>>> log output, or disables/rotates all logs." >>>>>>> >>>>>>> * The rotate option description should clarify that all logs will >>>>>>> be rotated ("current log" is too ambiguous). >>>>>>> >>>>>> >>>>>> I've fixed. >>>>>> >>>>> >>>>> Great, thanks! >>>>> >>>>> >>>>>> >>>>>> logFileOutput.cpp/hpp: >>>>>>> >>>>>>> * Moving the MutexLocker like this introduces a race condition >>>>>>> where a log might be rotated multiple times by different threads, >>>>>>> instead of just once. >>>>>>> Instead of making the rotate() function public and moving the >>>>>>> mutexlocker, I suggest adding something like a public >>>>>>> force_rotation() function that grabs the lock and calls the >>>>>>> private rotate(). >>>>>>> >>>>>>> * Given the addition of is_rotated() in LogOutput, then >>>>>>> get_archive_name() should be removed. >>>>>>> >>>>>> >>>>>> I moved MutexLocker and should_rotate() to rotate(). >>>>>> I think this change can avoid race condition. >>>>>> >>>>>> >>>>> There is a subtle problem with taking the lock to check if we should >>>>> rotate. If we configure a rotated file output and then log something >>>>> on that output before thread local storage initialization is >>>>> complete, we will crash/hit an assert. The previous implementation >>>>> avoided this as long as no rotation was needed before this >>>>> initialization was complete. (This can be triggered using the >>>>> following arguments >>>>> "-Xlog:all=trace:file.txt::filesize=10,filecount=2 -Xlog:invalid", >>>>> for example.) >>>>> >>>>> Thanks, >>>>> Marcus >>>>> >>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2015/10/27 21:17, Marcus Larsson wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> On 2015-10-27 01:03, Yasumasa Suenaga wrote: >>>>>>> >>>>>>>> Hi Marcus, >>>>>>>> >>>>>>>> Thank you for replying. >>>>>>>> >>>>>>>> I filed this feature to JBS and change subject of this email. >>>>>>>> Could you be a sponsor and review it? >>>>>>>> >>>>>>> >>>>>>> I'll sponsor it. >>>>>>> >>>>>>> >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.00/ >>>>>>>> >>>>>>> >>>>>>> logConfiguration.cpp/hpp: >>>>>>> >>>>>>> * I don't think we should rely on the archive_name or the output's >>>>>>> name to decide whether or not an output should be rotated. It >>>>>>> would be better to introduce an is_rotated() test function in >>>>>>> LogOutput that could be used here. >>>>>>> >>>>>>> * rotate_all_logfile() should be renamed to rotate_all_outputs(). >>>>>>> Currently there are only LogFileOutputs (other than >>>>>>> stdout/stderr), but in the future there might be other types of >>>>>>> outputs so I prefer having a more general name. >>>>>>> Also, please use static_cast instead of the >>>>>>> C-style cast. (Additional logic will be required here once more >>>>>>> types of log outputs are added, but I don't think we need to worry >>>>>>> about this right now.) >>>>>>> >>>>>>> * Don't print an error if an output is not rotatable, since it's >>>>>>> not an error to have some log outputs rotated while others are >>>>>>> not. If you really want some traceability here I suggest adding >>>>>>> log messages on trace level, tagged with 'logging'. >>>>>>> >>>>>>> >>>>>>> logDiagnosticCommand.cpp/hpp: >>>>>>> >>>>>>> * I think the description could be improved to something like >>>>>>> "Lists current log configuration, enables/disables/configures a >>>>>>> log output, or disables/rotates all logs." >>>>>>> >>>>>>> * The rotate option description should clarify that all logs will >>>>>>> be rotated ("current log" is too ambiguous). >>>>>>> >>>>>>> >>>>>>> logFileOutput.cpp/hpp: >>>>>>> >>>>>>> * Moving the MutexLocker like this introduces a race condition >>>>>>> where a log might be rotated multiple times by different threads, >>>>>>> instead of just once. >>>>>>> Instead of making the rotate() function public and moving the >>>>>>> mutexlocker, I suggest adding something like a public >>>>>>> force_rotation() function that grabs the lock and calls the >>>>>>> private rotate(). >>>>>>> >>>>>>> * Given the addition of is_rotated() in LogOutput, then >>>>>>> get_archive_name() should be removed. >>>>>>> >>>>>>> Thanks, >>>>>>> Marcus >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> On 2015/10/26 18:56, Marcus Larsson wrote: >>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> Sorry for my late reply. >>>>>>>>> >>>>>>>>> I think being able to force rotation via jcmd seems like a good >>>>>>>>> feature. Files are currently opened in append mode so it should >>>>>>>>> already be possible to use external log rotation tools by >>>>>>>>> copying and truncating the files. Still I think it would be nice >>>>>>>>> to provide the jcmd for rotation as well. >>>>>>>>> >>>>>>>>> I can see some small issues with the implementation, but we can >>>>>>>>> deal with that during the review. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Marcus >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2015-10-26 00:26, Yasumasa Suenaga wrote: >>>>>>>>> >>>>>>>>>> Hi all, >>>>>>>>>> >>>>>>>>>> Have you ever seen my change? >>>>>>>>>> I (and my customers) need log rotation function via external tool. >>>>>>>>>> >>>>>>>>>> I want to merge it by Feature Complete. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> 2015/10/16 22:55 "Yasumasa Suenaga" : >>>>>>>>>> >>>>>>>>>> Hi all, >>>>>>>>>>> >>>>>>>>>>> I contributed JDK-7090324: gclog rotation via external tool to be >>>>>>>>>>> synchronized with >>>>>>>>>>> logrotated tool on Linux. >>>>>>>>>>> >>>>>>>>>>> I think JEP 158 is in progress. >>>>>>>>>>> However, this JEP does not contain log rotation via external >>>>>>>>>>> tool in the >>>>>>>>>>> spec. >>>>>>>>>>> I want to rotate logs via jcmd in this JEP. >>>>>>>>>>> >>>>>>>>>>> I've updated a patch for it: >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/jvmlogging-logrotate/ >>>>>>>>>>> >>>>>>>>>>> This patch provides new option "rotate" in VM.log command. >>>>>>>>>>> If this change can be accepted, I will file it to JBS and send >>>>>>>>>>> RFR. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>> >>>>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From staffan.larsen at oracle.com Mon Nov 2 07:37:52 2015 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Mon, 2 Nov 2015 08:37:52 +0100 Subject: Let jvmtiGen exit with a non-zero exit code upon failure In-Reply-To: References: <5632251B.80606@oracle.com> <5632A09F.1060205@oracle.com> <5632A148.6070102@oracle.com> <56333474.3050406@oracle.com> <4C0FA96C-1122-4236-84B2-B4CBAB2F795E@oracle.com> <5633EC3E.5020701@oracle.com> Message-ID: <0467332A-2C0C-477D-B669-587B764D3C13@oracle.com> Hi Carsten, Thanks for spending the time on this - this version looks a lot cleaner. I would prefer to always print the exception stack trace. The reason is that if something goes wrong, it will likely happen on a build server where it isn?t simple to find and insert the extra ?-verbose? flag. Better to log as much information as possible in case of failure so it?s easier to find the root cause. That leaves warnings from the parser as the only usage of -verbose. Are there any warnings presently? If not, I would remove the -verbose flag altogether and always print the warnings as well. Thanks, /Staffan > On 1 nov. 2015, at 20:48, Carsten Varming wrote: > > Dear Serguei and Steffan, > > Sorry about the late reply. The office was a little busy Friday. > > Steffan's suggestion is to remove all the exception handling code sound reasonable. I am not a fan of the main method throwing checked exceptions, so I wrapped all the exception handling into a tiny handler that simply prints the exception and exits with a non-zero value. I never cared much for the stacktrace, so I put printing of the stacktrace under the verbose flag. > > Speaking of cleaning up, there is code in there to use Apache xalan instead of the standard xslt processor due to an ancient bug fixed in Java 1.5. I removed that as well. > > There was also a comment about the field "document" being package globally accessible due to: "ref'd by the tree-adapter". I have no idea what that means, but changing document to a local variable in main didn't break the build, so I guess that comment is out-of-date. Please let me know if I am missing something here. > > There was also a use of java.io.PrintStream. That class suppresses IOExceptions: "Unlike other output streams, a PrintStream never throws an IOException", so I removed the PrintStream as I want to get a failure when something goes wrong. > > I have put a new webrev at http://cr.openjdk.java.net/~cvarming/jvmtiGen.02/ > > Carsten > ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jaroslav.bachorik at oracle.com Mon Nov 2 08:08:28 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Mon, 2 Nov 2015 09:08:28 +0100 Subject: RFR 8140481: NoClassDefFoundError thrown by ManagementFactory.getPlatformMBeanServer Message-ID: <563719FC.20502@oracle.com> Please, review the following build change Issue : https://bugs.openjdk.java.net/browse/JDK-8140481 Webrev: http://cr.openjdk.java.net/~jbachorik/8140481/webrev.00 Currently, the 'jdk.management' module is not a part of the JRE image even though it should be. This patch adds 'jdk.management' to PROVIDER_MODULES where it belongs. Thanks, -JB- From dmitry.samersoff at oracle.com Mon Nov 2 08:21:44 2015 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Mon, 2 Nov 2015 11:21:44 +0300 Subject: RFR 8140481: NoClassDefFoundError thrown by ManagementFactory.getPlatformMBeanServer In-Reply-To: <563719FC.20502@oracle.com> References: <563719FC.20502@oracle.com> Message-ID: <56371D18.2000605@oracle.com> Jaroslav, Looks good for me (not a reviewer). -Dmitry On 2015-11-02 11:08, Jaroslav Bachorik wrote: > Please, review the following build change > > Issue : https://bugs.openjdk.java.net/browse/JDK-8140481 > Webrev: http://cr.openjdk.java.net/~jbachorik/8140481/webrev.00 > > Currently, the 'jdk.management' module is not a part of the JRE image > even though it should be. This patch adds 'jdk.management' to > PROVIDER_MODULES where it belongs. > > Thanks, > > -JB- -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From Alan.Bateman at oracle.com Mon Nov 2 08:28:29 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 2 Nov 2015 08:28:29 +0000 Subject: RFR 8140481: NoClassDefFoundError thrown by ManagementFactory.getPlatformMBeanServer In-Reply-To: <563719FC.20502@oracle.com> References: <563719FC.20502@oracle.com> Message-ID: <56371EAD.7010204@oracle.com> On 02/11/2015 08:08, Jaroslav Bachorik wrote: > Please, review the following build change > > Issue : https://bugs.openjdk.java.net/browse/JDK-8140481 > Webrev: http://cr.openjdk.java.net/~jbachorik/8140481/webrev.00 > > Currently, the 'jdk.management' module is not a part of the JRE image > even though it should be. This patch adds 'jdk.management' to > PROVIDER_MODULES where it belongs. jdk.management exports an API so I assume it should be in MAIN_MODULES. -Alan From erik.joelsson at oracle.com Mon Nov 2 08:34:08 2015 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Mon, 2 Nov 2015 09:34:08 +0100 Subject: RFR 8140481: NoClassDefFoundError thrown by ManagementFactory.getPlatformMBeanServer In-Reply-To: <563719FC.20502@oracle.com> References: <563719FC.20502@oracle.com> Message-ID: <56372000.4020603@oracle.com> The change looks good to me, but I think someone else from the jigsaw team should review this as well. /Erik On 2015-11-02 09:08, Jaroslav Bachorik wrote: > Please, review the following build change > > Issue : https://bugs.openjdk.java.net/browse/JDK-8140481 > Webrev: http://cr.openjdk.java.net/~jbachorik/8140481/webrev.00 > > Currently, the 'jdk.management' module is not a part of the JRE image > even though it should be. This patch adds 'jdk.management' to > PROVIDER_MODULES where it belongs. > > Thanks, > > -JB- From jaroslav.bachorik at oracle.com Mon Nov 2 08:34:53 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Mon, 2 Nov 2015 09:34:53 +0100 Subject: RFR 8140481: NoClassDefFoundError thrown by ManagementFactory.getPlatformMBeanServer In-Reply-To: <56371EAD.7010204@oracle.com> References: <563719FC.20502@oracle.com> <56371EAD.7010204@oracle.com> Message-ID: <5637202D.701@oracle.com> On 2.11.2015 09:28, Alan Bateman wrote: > > > On 02/11/2015 08:08, Jaroslav Bachorik wrote: >> Please, review the following build change >> >> Issue : https://bugs.openjdk.java.net/browse/JDK-8140481 >> Webrev: http://cr.openjdk.java.net/~jbachorik/8140481/webrev.00 >> >> Currently, the 'jdk.management' module is not a part of the JRE image >> even though it should be. This patch adds 'jdk.management' to >> PROVIDER_MODULES where it belongs. > jdk.management exports an API so I assume it should be in MAIN_MODULES. Ok. So, PROVIDER_MODULES shall contain modules *only* providing services, right? -JB- > > -Alan From Alan.Bateman at oracle.com Mon Nov 2 08:46:52 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 2 Nov 2015 08:46:52 +0000 Subject: RFR 8140481: NoClassDefFoundError thrown by ManagementFactory.getPlatformMBeanServer In-Reply-To: <5637202D.701@oracle.com> References: <563719FC.20502@oracle.com> <56371EAD.7010204@oracle.com> <5637202D.701@oracle.com> Message-ID: <563722FC.50909@oracle.com> On 02/11/2015 08:34, Jaroslav Bachorik wrote: > On 2.11.2015 09:28, Alan Bateman wrote: >> >> >> On 02/11/2015 08:08, Jaroslav Bachorik wrote: >>> Please, review the following build change >>> >>> Issue : https://bugs.openjdk.java.net/browse/JDK-8140481 >>> Webrev: http://cr.openjdk.java.net/~jbachorik/8140481/webrev.00 >>> >>> Currently, the 'jdk.management' module is not a part of the JRE image >>> even though it should be. This patch adds 'jdk.management' to >>> PROVIDER_MODULES where it belongs. >> jdk.management exports an API so I assume it should be in MAIN_MODULES. > > Ok. So, PROVIDER_MODULES shall contain modules *only* providing > services, right? Yes, although there are a few inconsistencies. I expect we'll clean this up in the jake forest so that it is easier to maintain. -Alan From jaroslav.bachorik at oracle.com Mon Nov 2 10:28:12 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Mon, 2 Nov 2015 11:28:12 +0100 Subject: RFR 8140481: NoClassDefFoundError thrown by ManagementFactory.getPlatformMBeanServer In-Reply-To: <56371EAD.7010204@oracle.com> References: <563719FC.20502@oracle.com> <56371EAD.7010204@oracle.com> Message-ID: <56373ABC.2050109@oracle.com> On 2.11.2015 09:28, Alan Bateman wrote: > > > On 02/11/2015 08:08, Jaroslav Bachorik wrote: >> Please, review the following build change >> >> Issue : https://bugs.openjdk.java.net/browse/JDK-8140481 >> Webrev: http://cr.openjdk.java.net/~jbachorik/8140481/webrev.00 >> >> Currently, the 'jdk.management' module is not a part of the JRE image >> even though it should be. This patch adds 'jdk.management' to >> PROVIDER_MODULES where it belongs. > jdk.management exports an API so I assume it should be in MAIN_MODULES. Updated the patch to add the module to MAIN_MODULES - http://cr.openjdk.java.net/~jbachorik/8140481/webrev.01 -JB- > > -Alan From Alan.Bateman at oracle.com Mon Nov 2 10:37:45 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 2 Nov 2015 10:37:45 +0000 Subject: RFR 8140481: NoClassDefFoundError thrown by ManagementFactory.getPlatformMBeanServer In-Reply-To: <56373ABC.2050109@oracle.com> References: <563719FC.20502@oracle.com> <56371EAD.7010204@oracle.com> <56373ABC.2050109@oracle.com> Message-ID: <56373CF9.5070709@oracle.com> On 02/11/2015 10:28, Jaroslav Bachorik wrote: > On 2.11.2015 09:28, Alan Bateman wrote: >> >> >> On 02/11/2015 08:08, Jaroslav Bachorik wrote: >>> Please, review the following build change >>> >>> Issue : https://bugs.openjdk.java.net/browse/JDK-8140481 >>> Webrev: http://cr.openjdk.java.net/~jbachorik/8140481/webrev.00 >>> >>> Currently, the 'jdk.management' module is not a part of the JRE image >>> even though it should be. This patch adds 'jdk.management' to >>> PROVIDER_MODULES where it belongs. >> jdk.management exports an API so I assume it should be in MAIN_MODULES. > > Updated the patch to add the module to MAIN_MODULES - > http://cr.openjdk.java.net/~jbachorik/8140481/webrev.01 This looks okay to me. -Alan From erik.joelsson at oracle.com Mon Nov 2 12:43:25 2015 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Mon, 2 Nov 2015 13:43:25 +0100 Subject: RFR 8140481: NoClassDefFoundError thrown by ManagementFactory.getPlatformMBeanServer In-Reply-To: <56373ABC.2050109@oracle.com> References: <563719FC.20502@oracle.com> <56371EAD.7010204@oracle.com> <56373ABC.2050109@oracle.com> Message-ID: <56375A6D.2080401@oracle.com> Looks good to me. /Erik On 2015-11-02 11:28, Jaroslav Bachorik wrote: > On 2.11.2015 09:28, Alan Bateman wrote: >> >> >> On 02/11/2015 08:08, Jaroslav Bachorik wrote: >>> Please, review the following build change >>> >>> Issue : https://bugs.openjdk.java.net/browse/JDK-8140481 >>> Webrev: http://cr.openjdk.java.net/~jbachorik/8140481/webrev.00 >>> >>> Currently, the 'jdk.management' module is not a part of the JRE image >>> even though it should be. This patch adds 'jdk.management' to >>> PROVIDER_MODULES where it belongs. >> jdk.management exports an API so I assume it should be in MAIN_MODULES. > > Updated the patch to add the module to MAIN_MODULES - > http://cr.openjdk.java.net/~jbachorik/8140481/webrev.01 > > -JB- > >> >> -Alan > From yasuenag at gmail.com Mon Nov 2 13:32:44 2015 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Mon, 2 Nov 2015 22:32:44 +0900 Subject: RFR: JDK-8140556: Add force rotation option to VM.log jcmd In-Reply-To: <5636C66A.6020304@oracle.com> References: <562101D8.5080909@gmail.com> <562DF8D4.40501@oracle.com> <562EBF4B.4020607@gmail.com> <562F6B46.5050500@oracle.com> <563226AB.2030206@gmail.com> <5632341F.4060400@oracle.com> <56338125.9030802@gmail.com> <56338D58.9090709@oracle.com> <5634D9FC.6080502@gmail.com> <5636C66A.6020304@oracle.com> Message-ID: <563765FC.2090804@gmail.com> Marcus, David, I've uploaded new webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.02/ Could you review it? >>>>> Still missing the renaming of rotate_all_logfile to rotate_all_outputs. I've fixed. >>> Small typo, please make that is_rotatable(). I've added, and have used at LogConfiguration::rotate_all_outputs(). > You can't safely lock a mutex until the current thread has attached to the VM and Thread::current() returns non-null (that is what Thread::initialize_thread_local_storage() currently does). You may get away with it, particularly in product builds, during VM initialization, as the fast-lock path doesn't require access to Thread::current(). I will check whether Thread::current() is NULL or not at LogFileOutput::is_rotatable() is_rotatable() is called at LogFileOutput::rotate() before locking _rotation_lock. So we can avoid crash/assert which is related to TLS. Thanks, Yasumasa On 2015/11/02 11:11, David Holmes wrote: > On 1/11/2015 1:10 AM, Yasumasa Suenaga wrote: >>> This in turn means that we are logging before the thread local storage >>> has been initialized, and the JVM will crash/hit the assert because we >>> are trying to take the rotation lock. >> >> I guess that you are saying about class member (not TLS - e.g. >> pthread_setspecific()). > > You can't safely lock a mutex until the current thread has attached to the VM and Thread::current() returns non-null (that is what Thread::initialize_thread_local_storage() currently does). You may get away with it, particularly in product builds, during VM initialization, as the fast-lock path doesn't require access to Thread::current(). > > David > >> Most of fields (include _rotation_lock) in LogFileOutput are initialized >> at c'tor. >> Other field (_stream) is initialized at LogFileOutput::initialize(). >> >> We can avoid problem if we can move initialize() to c'tor because >> LogFileOutput >> will be instantiated at LogConfiguration::new_output(). >> >> Currently, LogFileOutput will be added to LogConfiguration::_outputs and >> to LogTagSet after calling LogFileOutput::initialize(). >> I wonder why we cannot avoid crash/assert with current implementation... >> >> >> I will keep current implementation about _rotation_lock if the above is >> incorrect. >> >> >> Thanks, >> >> Yasumasa >> >> >> On 2015/10/31 0:31, Marcus Larsson wrote: >>> Hey, >>> >>> On 2015-10-30 15:39, Yasumasa Suenaga wrote: >>>> Hi Marcus, >>>> >>>> Thank you for your comment. >>>> >>>> >>>>> Still missing the renaming of rotate_all_logfile to rotate_all_outputs. >>>> >>>> Sorry, I will fix. >>>> >>> >>> Thanks! >>> >>>> >>>>> This rotate function works I guess, but it will probably need some >>>>> rework once there are other types of log outputs (rotate doesn't >>>>> make sense on all types of log outputs). >>>> >>>> I will add is_rotetable() to LogOutput as virtual function. >>>> This function return false (can not rotate) by default, return true >>>> if _file_count is >>>> greater than 0 in LogFileOutput. >>>> >>>> LogConfiguration::rotate_all_outputs() will rotate if this function >>>> returns true. >>>> >>> >>> Small typo, please make that is_rotatable(). >>> >>>> >>>>> If we configure a rotated file output and then log something on that >>>>> output before thread local storage initialization is complete, we >>>>> will crash/hit an assert. The previous implementation avoided this >>>>> as long as no rotation was needed before this initialization was >>>>> complete. (This can be triggered using the following arguments >>>>> "-Xlog:all=trace:file.txt::filesize=10,filecount=2 -Xlog:invalid", >>>>> for example.) >>>> >>>> I do not think so. >>>> Each -Xlog option creates unique instance of LogOutput (and subclass). >>>> So they are isolated. >>> >>> My example includes a second (and intentionally incorrect) -Xlog >>> option only to trigger logging on the newly configured rotating >>> LogFileOutput. This will cause some logging about the invalid -Xlog >>> argument, and this logging happens during the argument parsing. This >>> in turn means that we are logging before the thread local storage has >>> been initialized, and the JVM will crash/hit the assert because we are >>> trying to take the rotation lock. This has nothing to do with >>> concurrent fprintfs or freopens, even if that is a problem. >>> >>>> >>>> However, we might have to rotate at safepoint. >>> >>> Doing stuff at a safepoint will not give us much in this case. There >>> are threads that continue execution through a safepoint, which means >>> we will see log calls being made even during this time. >>> >>>> Currently, LogOutput might be used by multiple threads concurrently. >>>> If we rotate log when another thread is writing string to output, we >>>> encounter unexpected behavior. >>>> >>>> LogFileOutput::rotate() uses freopen(). >>>> LogFileStreamOutput::write() uses vfprintf() through jio_fprintf(). >>>> freopen() and vfprintf() are not atomic. >>> >>> You're right. There seems to be an overlooked concurrency issue with >>> vfprintfs during freopen. I don't think we should resolve this as part >>> of this particular fix, so instead I'll create a separate issue for it. >>> >>>> >>>> I think that cause of crash/assertion which you say is it. >>>> (GC log will be rotated at safepoint.) >>>> >>> >>> See my comment above for what crash/assert I'm talking about. >>> >>> Thanks, >>> Marcus >>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> On 2015/10/29 23:58, Marcus Larsson wrote: >>>>> Hi, >>>>> >>>>> On 2015-10-29 15:01, Yasumasa Suenaga wrote: >>>>>> Hi Marcus, >>>>>> >>>>>> Thank you for your comment. >>>>>> >>>>>>> I'll sponsor it. >>>>>> >>>>>> Thank you so much! >>>>>> I've uploaded new webrev: >>>>>> >>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.01/ >>>>>> >>>>>> >>>>> >>>>> Still missing the renaming of rotate_all_logfile to rotate_all_outputs. >>>>> >>>>>>> logConfiguration.cpp/hpp: >>>>>>> >>>>>>> * I don't think we should rely on the archive_name or the output's >>>>>>> name to decide whether or not an output should be rotated. It >>>>>>> would be better to introduce an is_rotated() test function in >>>>>>> LogOutput that could be used here. >>>>>>> >>>>>>> * rotate_all_logfile() should be renamed to rotate_all_outputs(). >>>>>>> Currently there are only LogFileOutputs (other than >>>>>>> stdout/stderr), but in the future there might be other types of >>>>>>> outputs so I prefer having a more general name. >>>>>>> Also, please use static_cast instead of the >>>>>>> C-style cast. (Additional logic will be required here once more >>>>>>> types of log outputs are added, but I don't think we need to worry >>>>>>> about this right now.) >>>>>>> >>>>>>> * Don't print an error if an output is not rotatable, since it's >>>>>>> not an error to have some log outputs rotated while others are >>>>>>> not. If you really want some traceability here I suggest adding >>>>>>> log messages on trace level, tagged with 'logging'. >>>>>> >>>>>> I added LogOutput::rotate(bool) as virtual function. >>>>>> This function works nothing by default, but will rotate all logs at >>>>>> LogFileOutput. >>>>> >>>>> This rotate function works I guess, but it will probably need some >>>>> rework once there are other types of log outputs (rotate doesn't >>>>> make sense on all types of log outputs). >>>>> >>>>>> >>>>>> If true is passed to this function, all files will be rotated. >>>>>> >>>>>> >>>>>>> logDiagnosticCommand.cpp/hpp: >>>>>>> >>>>>>> * I think the description could be improved to something like >>>>>>> "Lists current log configuration, enables/disables/configures a >>>>>>> log output, or disables/rotates all logs." >>>>>>> >>>>>>> * The rotate option description should clarify that all logs will >>>>>>> be rotated ("current log" is too ambiguous). >>>>>> >>>>>> I've fixed. >>>>> >>>>> Great, thanks! >>>>> >>>>>> >>>>>> >>>>>>> logFileOutput.cpp/hpp: >>>>>>> >>>>>>> * Moving the MutexLocker like this introduces a race condition >>>>>>> where a log might be rotated multiple times by different threads, >>>>>>> instead of just once. >>>>>>> Instead of making the rotate() function public and moving the >>>>>>> mutexlocker, I suggest adding something like a public >>>>>>> force_rotation() function that grabs the lock and calls the >>>>>>> private rotate(). >>>>>>> >>>>>>> * Given the addition of is_rotated() in LogOutput, then >>>>>>> get_archive_name() should be removed. >>>>>> >>>>>> I moved MutexLocker and should_rotate() to rotate(). >>>>>> I think this change can avoid race condition. >>>>>> >>>>> >>>>> There is a subtle problem with taking the lock to check if we should >>>>> rotate. If we configure a rotated file output and then log something >>>>> on that output before thread local storage initialization is >>>>> complete, we will crash/hit an assert. The previous implementation >>>>> avoided this as long as no rotation was needed before this >>>>> initialization was complete. (This can be triggered using the >>>>> following arguments >>>>> "-Xlog:all=trace:file.txt::filesize=10,filecount=2 -Xlog:invalid", >>>>> for example.) >>>>> >>>>> Thanks, >>>>> Marcus >>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2015/10/27 21:17, Marcus Larsson wrote: >>>>>>> Hi, >>>>>>> >>>>>>> On 2015-10-27 01:03, Yasumasa Suenaga wrote: >>>>>>>> Hi Marcus, >>>>>>>> >>>>>>>> Thank you for replying. >>>>>>>> >>>>>>>> I filed this feature to JBS and change subject of this email. >>>>>>>> Could you be a sponsor and review it? >>>>>>> >>>>>>> I'll sponsor it. >>>>>>> >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.00/ >>>>>>> >>>>>>> logConfiguration.cpp/hpp: >>>>>>> >>>>>>> * I don't think we should rely on the archive_name or the output's >>>>>>> name to decide whether or not an output should be rotated. It >>>>>>> would be better to introduce an is_rotated() test function in >>>>>>> LogOutput that could be used here. >>>>>>> >>>>>>> * rotate_all_logfile() should be renamed to rotate_all_outputs(). >>>>>>> Currently there are only LogFileOutputs (other than >>>>>>> stdout/stderr), but in the future there might be other types of >>>>>>> outputs so I prefer having a more general name. >>>>>>> Also, please use static_cast instead of the >>>>>>> C-style cast. (Additional logic will be required here once more >>>>>>> types of log outputs are added, but I don't think we need to worry >>>>>>> about this right now.) >>>>>>> >>>>>>> * Don't print an error if an output is not rotatable, since it's >>>>>>> not an error to have some log outputs rotated while others are >>>>>>> not. If you really want some traceability here I suggest adding >>>>>>> log messages on trace level, tagged with 'logging'. >>>>>>> >>>>>>> >>>>>>> logDiagnosticCommand.cpp/hpp: >>>>>>> >>>>>>> * I think the description could be improved to something like >>>>>>> "Lists current log configuration, enables/disables/configures a >>>>>>> log output, or disables/rotates all logs." >>>>>>> >>>>>>> * The rotate option description should clarify that all logs will >>>>>>> be rotated ("current log" is too ambiguous). >>>>>>> >>>>>>> >>>>>>> logFileOutput.cpp/hpp: >>>>>>> >>>>>>> * Moving the MutexLocker like this introduces a race condition >>>>>>> where a log might be rotated multiple times by different threads, >>>>>>> instead of just once. >>>>>>> Instead of making the rotate() function public and moving the >>>>>>> mutexlocker, I suggest adding something like a public >>>>>>> force_rotation() function that grabs the lock and calls the >>>>>>> private rotate(). >>>>>>> >>>>>>> * Given the addition of is_rotated() in LogOutput, then >>>>>>> get_archive_name() should be removed. >>>>>>> >>>>>>> Thanks, >>>>>>> Marcus >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> On 2015/10/26 18:56, Marcus Larsson wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> Sorry for my late reply. >>>>>>>>> >>>>>>>>> I think being able to force rotation via jcmd seems like a good >>>>>>>>> feature. Files are currently opened in append mode so it should >>>>>>>>> already be possible to use external log rotation tools by >>>>>>>>> copying and truncating the files. Still I think it would be nice >>>>>>>>> to provide the jcmd for rotation as well. >>>>>>>>> >>>>>>>>> I can see some small issues with the implementation, but we can >>>>>>>>> deal with that during the review. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Marcus >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2015-10-26 00:26, Yasumasa Suenaga wrote: >>>>>>>>>> Hi all, >>>>>>>>>> >>>>>>>>>> Have you ever seen my change? >>>>>>>>>> I (and my customers) need log rotation function via external tool. >>>>>>>>>> >>>>>>>>>> I want to merge it by Feature Complete. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> 2015/10/16 22:55 "Yasumasa Suenaga" : >>>>>>>>>> >>>>>>>>>>> Hi all, >>>>>>>>>>> >>>>>>>>>>> I contributed JDK-7090324: gclog rotation via external tool to be >>>>>>>>>>> synchronized with >>>>>>>>>>> logrotated tool on Linux. >>>>>>>>>>> >>>>>>>>>>> I think JEP 158 is in progress. >>>>>>>>>>> However, this JEP does not contain log rotation via external >>>>>>>>>>> tool in the >>>>>>>>>>> spec. >>>>>>>>>>> I want to rotate logs via jcmd in this JEP. >>>>>>>>>>> >>>>>>>>>>> I've updated a patch for it: >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/jvmlogging-logrotate/ >>>>>>>>>>> >>>>>>>>>>> This patch provides new option "rotate" in VM.log command. >>>>>>>>>>> If this change can be accepted, I will file it to JBS and send >>>>>>>>>>> RFR. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>>> >>>>>>> >>>>> >>> From varming at gmail.com Mon Nov 2 13:34:58 2015 From: varming at gmail.com (Carsten Varming) Date: Mon, 2 Nov 2015 08:34:58 -0500 Subject: Let jvmtiGen exit with a non-zero exit code upon failure In-Reply-To: <0467332A-2C0C-477D-B669-587B764D3C13@oracle.com> References: <5632251B.80606@oracle.com> <5632A09F.1060205@oracle.com> <5632A148.6070102@oracle.com> <56333474.3050406@oracle.com> <4C0FA96C-1122-4236-84B2-B4CBAB2F795E@oracle.com> <5633EC3E.5020701@oracle.com> <0467332A-2C0C-477D-B669-587B764D3C13@oracle.com> Message-ID: Dear Staffan, Stacktraces: I can see your point. I'll add them back unconditionally. Warnings: The OpenJDK build gets a little noisy when you start printing warnings. The fallback on failed includes in hotspot/src/share/vm/trace/*.xml cause a lot of printing; see below. That is why I introduced the verbose flag in the first place. This is not a big deal as it pollutes only the build log, but I would still prefer the program to be silent on normal execution. Please let me know if you still prefer to discard the verbose flag and print all warnings. I'll update the webrev after your reply. Thank you, Carsten Noisy warnings: jvmtiGen warning: Include operation failed, reverting to fallback. Resource error reading file as XML (href='../../../closed/share/vm/trace/traceeventtypes.xml'). Reason: /Users/cvarming/workspace/jdk9/hotspot/src/closed/share/vm/trace/traceeventtypes.xml (No such file or directory) jvmtiGen warning: Include operation failed, reverting to fallback. Resource error reading file as XML (href='../../../closed/share/vm/trace/traceevents.xml'). Reason: /Users/cvarming/workspace/jdk9/hotspot/src/closed/share/vm/trace/traceevents.xml (No such file or directory) jvmtiGen warning: Include operation failed, reverting to fallback. Resource error reading file as XML (href='../../../closed/share/vm/trace/traceeventtypes.xml'). Reason: /Users/cvarming/workspace/jdk9/hotspot/src/closed/share/vm/trace/traceeventtypes.xml (No such file or directory) jvmtiGen warning: Include operation failed, reverting to fallback. Resource error reading file as XML (href='../../../closed/share/vm/trace/traceevents.xml'). Reason: /Users/cvarming/workspace/jdk9/hotspot/src/closed/share/vm/trace/traceevents.xml (No such file or directory) jvmtiGen warning: Include operation failed, reverting to fallback. Resource error reading file as XML (href='../../../closed/share/vm/trace/traceeventtypes.xml'). Reason: /Users/cvarming/workspace/jdk9/hotspot/src/closed/share/vm/trace/traceeventtypes.xml (No such file or directory) jvmtiGen warning: Include operation failed, reverting to fallback. Resource error reading file as XML (href='../../../closed/share/vm/trace/traceevents.xml'). Reason: /Users/cvarming/workspace/jdk9/hotspot/src/closed/share/vm/trace/traceevents.xml (No such file or directory) On Mon, Nov 2, 2015 at 2:37 AM, Staffan Larsen wrote: > Hi Carsten, > > Thanks for spending the time on this - this version looks a lot cleaner. > > I would prefer to always print the exception stack trace. The reason is > that if something goes wrong, it will likely happen on a build server where > it isn?t simple to find and insert the extra ?-verbose? flag. Better to log > as much information as possible in case of failure so it?s easier to find > the root cause. > > That leaves warnings from the parser as the only usage of -verbose. Are > there any warnings presently? If not, I would remove the -verbose flag > altogether and always print the warnings as well. > > Thanks, > /Staffan > > On 1 nov. 2015, at 20:48, Carsten Varming wrote: > > Dear Serguei and Steffan, > > Sorry about the late reply. The office was a little busy Friday. > > Steffan's suggestion is to remove all the exception handling code sound > reasonable. I am not a fan of the main method throwing checked exceptions, > so I wrapped all the exception handling into a tiny handler that simply > prints the exception and exits with a non-zero value. I never cared much > for the stacktrace, so I put printing of the stacktrace under the verbose > flag. > > Speaking of cleaning up, there is code in there to use Apache xalan > instead of the standard xslt processor due to an ancient bug fixed in Java > 1.5. I removed that as well. > > There was also a comment about the field "document" being package globally > accessible due to: "ref'd by the tree-adapter". I have no idea what that > means, but changing document to a local variable in main didn't break the > build, so I guess that comment is out-of-date. Please let me know if I am > missing something here. > > There was also a use of java.io.PrintStream. That class suppresses > IOExceptions: "Unlike other output streams, a PrintStream never throws an > IOException", so I removed the PrintStream as I want to get a failure when > something goes wrong. > > I have put a new webrev at > http://cr.openjdk.java.net/~cvarming/jvmtiGen.02/ > > Carsten > ? > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.fuchs at oracle.com Mon Nov 2 15:19:51 2015 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Mon, 2 Nov 2015 16:19:51 +0100 Subject: jmx-dev RFR 6425769: jmx remote bind address In-Reply-To: <1446460682.10865.20.camel@redhat.com> References: <1446460682.10865.20.camel@redhat.com> Message-ID: <56377F17.4050006@oracle.com> Hi Severin, Adding serviceability-dev at openjdk.java.net into the loop - that's a better alias than hotspot-dev for this kind of changes - maybe someone from serviceability-dev will offer to sponsor :-) I will let serviceability team members comment on the hotspot changes. ConnectorBootstrap.java I have one suggestion and one concern: Suggestion: I would suggest to reuse 'csf' (Client Socket Factory) and ssf (Server Socket Factory) variables rather than introduce the two new variables rmiServerSocketFactory and rmiClientSocketFactory. You might want to create a new boolean 'useSocketFactory' variable, if that makes the code more readable. Concern: If I understand correctly how RMI socket factories work, the client socket factory will be serialized and sent to the client side. This is problematic for interoperability, as the class may not present in the remote client - if the remote client is e.g. jdk 8. As far as I can see, your new DefaultClientSocketFactory doesn't do anything useful - so I would suggest to simply get rid of it, and only set the Server Socket Factory when SSL is not involved. Tests: Concerning the tests - we're trying to get rid of shell scripts rather than introducing new ones :-) Could the test be rewritten in pure java using the Process API? I believe there's even a test library that will let you do that easily jdk/test/lib/testlibrary/jdk/testlibrary/ (see ProcessTools.java) Other: Also - I believe the new option should be documented in: src/java.management/share/conf/management.properties best regards, -- daniel On 02/11/15 11:38, Severin Gehwolf wrote: > Hi, > > Here is a patch addressing JDK-6425769. The issue is that the JMX agent > binds to the wildcard address by default, preventing users to use > system properties for JMX agents on multi-homed hosts. Given a host > with local network interfaces, 192.168.0.1 and 192.168.0.2 say, it's > impossible to start two JMX agents binding to fixed ports but to > different network interfaces, 192.168.0.1:{9111,9112} and > 192.168.0.2:{9111,9112} say. > > The JDK would bind to all local interfaces by default. In the above > example to 192.168.0.1 *and* 192.168.0.2. The effect is that the second > Java process would get a "Connection refused" error because another > process has already been bound to the specified JMX/RMI port pairs. > > Bug: https://bugs.openjdk.java.net/browse/JDK-6425769 > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-6425769/00/ > > Testing done: > jdk_jmx and jdk_management tests all pass after this change (no > regressions). There is also a new JTREG test which fails prior this > change and passes after. Updates to the diagnostic command have been > tested manually. > > I understand that, if approved, the JDK and hotspot changes should get > pushed together? Hotspot changes are fairly trivial since it's only a > doc-update for the new JDK property in the relevant diagnostic command. > > Could someone please review and sponsor this change? Please let me know > if there are questions. > > Thanks, > Severin > From jaroslav.bachorik at oracle.com Mon Nov 2 15:54:03 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Mon, 2 Nov 2015 16:54:03 +0100 Subject: jmx-dev RFR 6425769: jmx remote bind address In-Reply-To: <56377F17.4050006@oracle.com> References: <1446460682.10865.20.camel@redhat.com> <56377F17.4050006@oracle.com> Message-ID: <5637871B.30705@oracle.com> Hi, On 2.11.2015 16:19, Daniel Fuchs wrote: > Hi Severin, > > Adding serviceability-dev at openjdk.java.net into the loop - that's > a better alias than hotspot-dev for this kind of changes - maybe > someone from serviceability-dev will offer to sponsor :-) > > I will let serviceability team members comment on the hotspot changes. > > ConnectorBootstrap.java > > I have one suggestion and one concern: > > Suggestion: I would suggest to reuse 'csf' (Client Socket Factory) and > ssf (Server Socket Factory) variables rather than introduce the two > new variables rmiServerSocketFactory and rmiClientSocketFactory. > You might want to create a new boolean 'useSocketFactory' variable, > if that makes the code more readable. > > Concern: If I understand correctly how RMI socket factories work, > the client socket factory will be serialized and sent to the client > side. This is problematic for interoperability, as the class may not > present in the remote client - if the remote client is e.g. jdk 8. > > As far as I can see, your new DefaultClientSocketFactory doesn't do > anything useful - so I would suggest to simply get rid of it, and only > set the Server Socket Factory when SSL is not involved. > > Tests: > > Concerning the tests - we're trying to get rid of shell scripts > rather than introducing new ones :-) > Could the test be rewritten in pure java using the Process API? > > I believe there's even a test library that will let you do that > easily jdk/test/lib/testlibrary/jdk/testlibrary/ > (see ProcessTools.java) > > Other: > > Also - I believe the new option should be documented in: > src/java.management/share/conf/management.properties I share Daniel's concerns. Also, the part of the changeset is related to javax.rmi.ssl - someone maintaining this library should also comment here. Also, the change is introducing new API (system property) and changing the existing one (adding SslRmiServerSocketFactory public constructors) so compatibility review process will have to be involved. -JB- > > best regards, > > -- daniel > > On 02/11/15 11:38, Severin Gehwolf wrote: > > Hi, > > > > Here is a patch addressing JDK-6425769. The issue is that the JMX agent > > binds to the wildcard address by default, preventing users to use > > system properties for JMX agents on multi-homed hosts. Given a host > > with local network interfaces, 192.168.0.1 and 192.168.0.2 say, it's > > impossible to start two JMX agents binding to fixed ports but to > > different network interfaces, 192.168.0.1:{9111,9112} and > > 192.168.0.2:{9111,9112} say. > > > > The JDK would bind to all local interfaces by default. In the above > > example to 192.168.0.1 *and* 192.168.0.2. The effect is that the second > > Java process would get a "Connection refused" error because another > > process has already been bound to the specified JMX/RMI port pairs. > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-6425769 > > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-6425769/00/ > > > > Testing done: > > jdk_jmx and jdk_management tests all pass after this change (no > > regressions). There is also a new JTREG test which fails prior this > > change and passes after. Updates to the diagnostic command have been > > tested manually. > > > > I understand that, if approved, the JDK and hotspot changes should get > > pushed together? Hotspot changes are fairly trivial since it's only a > > doc-update for the new JDK property in the relevant diagnostic command. > > > > Could someone please review and sponsor this change? Please let me know > > if there are questions. > > > > Thanks, > > Severin > > > From rachel.protacio at oracle.com Mon Nov 2 18:03:42 2015 From: rachel.protacio at oracle.com (Rachel Protacio) Date: Mon, 2 Nov 2015 13:03:42 -0500 Subject: RFR: 8138916: Logging write function does not allow for long enough messages In-Reply-To: <5631F66E.8090601@oracle.com> References: <562E9406.5040207@oracle.com> <562F6567.6040707@oracle.com> <5630E029.5040500@oracle.com> <5630EC54.4020402@oracle.com> <5630EF9E.6090401@oracle.com> <5630F328.8@oracle.com> <5630F77D.1050907@oracle.com> <56310AFC.10609@oracle.com> <56312A94.1060202@oracle.com> <56312B2B.5090305@oracle.com> <56313726.3080507@oracle.com> <5631F66E.8090601@oracle.com> Message-ID: <5637A57E.4050103@oracle.com> Thanks for the comments, Marcus. I've undone the "logtest" tag and instead created an InternalVMTest in log.cpp that writes the 518-character string to a temporary file and looks for it there. Using "logging" tag in the "develop" level. Testing succeeds. Webrev: http://cr.openjdk.java.net/~rprotacio/8139564.03/ Thank you, Rachel On 10/29/2015 6:35 AM, Marcus Larsson wrote: > Hi, > > On 2015-10-28 21:59, Rachel Protacio wrote: >> Including serviceability-dev. Recap: >> >> Summary: In logging/log.hpp, the logging vwrite function previously >> asserted that the buffer remains within 512 characters, which is >> too short for logging message of non-pre-determined length, e.g. for >> vtable and itable function names. Now, the function resizes the buffer >> to the appropriate length, asserting only that the resulting buffer >> is valid. Includes tag "logtest" to test that logging can print an >> output >> of 518 characters. >> >> Open webrev: http://cr.openjdk.java.net/~rprotacio/8138916.02 >> Bug: https://bugs.openjdk.java.net/browse/JDK-8138916 > > I don't like that the test log message is included in the product, or > that a completely new tag is introduced for it. > > I think a much better approach would be to do the test logging in an > internal VM test, reusing the logging tag. This way it won't be logged > unless we are actually testing stuff, and we don't introduce a new > tag. Also it would look better if it were logged on develop level. > > Thanks, > Marcus > >> >> Includes own acceptance test, and passes JPRT and remote hotspot/test >> tests. >> >> Thank you, >> Rachel >> >> On 10/28/2015 4:08 PM, Rachel Protacio wrote: >>> Thanks for the review and all the feedback! >>> Rachel >>> >>> On 10/28/2015 4:05 PM, Dmitry Samersoff wrote: >>>> Rachel, >>>> >>>> Looks good for me! >>>> >>>> -Dmitry >>>> >>>> On 2015-10-28 20:50, Rachel Protacio wrote: >>>>> Thank you both. I've removed the re-declaration and added >>>>> prefix_len to >>>>> newbuf_len. As for line 115, that line is not for the sake of >>>>> recalculating prefix_len, but for writing the prefix to the buffer. >>>>> >>>>> http://cr.openjdk.java.net/~rprotacio/8138916.02/ >>>>> >>>>> Thanks, >>>>> Rachel >>>>> >>>>> On 10/28/2015 12:27 PM, Coleen Phillimore wrote: >>>>>> Dmitry, Thank you for reviewing and providing comments. The second >>>>>> version is better because of these things that I didn't notice until >>>>>> you pointed it out. >>>>>> >>>>>> The malloc won't be frequent. I think it'll be okay and follows >>>>>> other >>>>>> coding patterns in the jvm. >>>>>> >>>>>> Thanks! >>>>>> Coleen >>>>>> >>>>>> On 10/28/15 12:09 PM, Dmitry Samersoff wrote: >>>>>>> Coleen, >>>>>>> >>>>>>> OK. I'm leaving it in good hands ;) >>>>>>> >>>>>>> -Dmitry >>>>>>> >>>>>>> On 2015-10-28 18:54, Coleen Phillimore wrote: >>>>>>>> Hi, I'm also reviewing this. I think this looks good. I don't >>>>>>>> think >>>>>>>> the solution to allocate a static buffer is very nice because >>>>>>>> it still >>>>>>>> has limits and we don't want to allocate something large on the >>>>>>>> stack. >>>>>>>> The malloc case is the infrequent case. If it becomes frequent >>>>>>>> we can >>>>>>>> increase LogBufferSize from 512. >>>>>>>> >>>>>>>> I think 111 the assert is good because if the windows code is >>>>>>>> wrong (or >>>>>>>> the vscprintf does something wrong an unexpected) we can catch the >>>>>>>> error >>>>>>>> with the assert before writing over memory. >>>>>>>> >>>>>>>> I agree 116 shouldn't redeclare 'ret'. >>>>>>>> >>>>>>>> Otherwise, I think it looks really good and I don't need to see >>>>>>>> another >>>>>>>> revision. >>>>>>>> >>>>>>>> Thanks! >>>>>>>> Coleen >>>>>>>> >>>>>>>> >>>>>>>> On 10/28/15 11:40 AM, Dmitry Samersoff wrote: >>>>>>>>> Rachel, >>>>>>>>> >>>>>>>>> On 2015-10-28 17:48, Rachel Protacio wrote: >>>>>>>>>> Thank you for the comments, both Dmitry and Mikael. Replies >>>>>>>>>> inline. >>>>>>>>>> >>>>>>>>>> Modified webrev: >>>>>>>>>> http://cr.openjdk.java.net/~rprotacio/8138916.01/ >>>>>>>>> 111 Assert is never happens on Windows, so it's better to move >>>>>>>>> it to >>>>>>>>> os_posix >>>>>>>>> >>>>>>>>> 113 newbuf_len doesn't account prefix_len >>>>>>>>> >>>>>>>>> 115 you may consider to use pre-calculated at ll. 108 prefix >>>>>>>>> rather >>>>>>>>> than >>>>>>>>> calculate it again. >>>>>>>>> >>>>>>>>> 116 int ret redefined it causes a warning. >>>>>>>>> >>>>>>>>> -Dmitry >>>>>>>>> >>>>>>>>> >>>>>>>>>> (Retested and works fine.) >>>>>>>>>> >>>>>>>>>> On 10/27/2015 7:52 AM, Dmitry Samersoff wrote: >>>>>>>>>>> Rachel, >>>>>>>>>>> >>>>>>>>>>> In worst case (windows, overflow) the code would parse >>>>>>>>>>> format string >>>>>>>>>>> four times. >>>>>>>>>> You're right. I've added a check. New amount of times >>>>>>>>>> vnsprintf() is >>>>>>>>>> called: >>>>>>>>>> posix, fits => 1 >>>>>>>>>> posix, overflow => 2 >>>>>>>>>> windows, fits => 1 >>>>>>>>>> windows, overflow => 3 (the best performance we can get) >>>>>>>>>> >>>>>>>>>>> Also NEW_C_HEAP_ARRAY takes a lock and might become a >>>>>>>>>>> performance bottleneck. >>>>>>>>>>> >>>>>>>>>>> POSIX equivalent of _vscprintf(fmt, args) is vsnprintf(NULL, >>>>>>>>>>> 0, fmt, >>>>>>>>>>> args) >>>>>>>>>>> >>>>>>>>>>> So it might be better to create os::log_vcprintf(fmt, args), >>>>>>>>>>> then >>>>>>>>>>> write >>>>>>>>>>> code at log.hpp as: >>>>>>>>>>> >>>>>>>>>>> int buf_len = os::log_vcprintf(fmt, args); >>>>>>>>>>> assert(buf_len < 2*page_size, "Stack overflow in logging"); >>>>>>>>>>> char buf[buf_len]; >>>>>>>>>>> vsprintf(buf, fmt, args); >>>>>>>>>> I don't think this would work anyway. A char[] needs a static >>>>>>>>>> size. >>>>>>>>>> Thanks, >>>>>>>>>> Rachel >>>>>>>>>>> -Dmitry >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 2015-10-26 23:58, Rachel Protacio wrote: >>>>>>>>>>>> Hello, all, >>>>>>>>>>>> >>>>>>>>>>>> Please review this fix to the faulty write function from the >>>>>>>>>>>> Unified >>>>>>>>>>>> Logging framework. >>>>>>>>>>>> >>>>>>>>>>>> Summary: In logging/log.hpp, the logging vwrite function >>>>>>>>>>>> previously >>>>>>>>>>>> asserted that the buffer remains within 512 characters, >>>>>>>>>>>> which is >>>>>>>>>>>> too >>>>>>>>>>>> short for logging message of non-pre-determined length, >>>>>>>>>>>> e.g. for >>>>>>>>>>>> vtable >>>>>>>>>>>> and itable function names. Now, the function resizes the >>>>>>>>>>>> buffer >>>>>>>>>>>> to the >>>>>>>>>>>> appropriate length, asserting only that the resulting >>>>>>>>>>>> buffer is >>>>>>>>>>>> valid. >>>>>>>>>>>> Includes tag "logtest" to test that logging can print an >>>>>>>>>>>> output >>>>>>>>>>>> of 518 >>>>>>>>>>>> characters. >>>>>>>>>>>> >>>>>>>>>>>> Open webrev:http://cr.openjdk.java.net/~rprotacio/8138916/ >>>>>>>>>>>> Bug:https://bugs.openjdk.java.net/browse/JDK-8138916 >>>>>>>>>>>> >>>>>>>>>>>> Includes own acceptance test, and passes JPRT and remote >>>>>>>>>>>> hotspot/test >>>>>>>>>>>> tests. >>>>>>>>>>>> >>>>>>>>>>>> Thank you, >>>>>>>>>>>> Rachel >>>> >>> >> > From sgehwolf at redhat.com Mon Nov 2 18:06:55 2015 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Mon, 02 Nov 2015 19:06:55 +0100 Subject: jmx-dev RFR 6425769: jmx remote bind address In-Reply-To: <5637871B.30705@oracle.com> References: <1446460682.10865.20.camel@redhat.com> <56377F17.4050006@oracle.com> <5637871B.30705@oracle.com> Message-ID: <1446487615.10865.57.camel@redhat.com> Hi, Thanks Jaroslav and Daniel for the reviews! Comments inline. On Mon, 2015-11-02 at 16:54 +0100, Jaroslav Bachorik wrote: > Hi, > > On 2.11.2015 16:19, Daniel Fuchs wrote: > > Hi Severin, > > > > Adding serviceability-dev at openjdk.java.net into the loop - that's > > a better alias than hotspot-dev for this kind of changes - maybe > > someone from serviceability-dev will offer to sponsor :-) > > > > I will let serviceability team members comment on the hotspot > > changes. > > > > ConnectorBootstrap.java > > > > I have one suggestion and one concern: > > > > Suggestion: I would suggest to reuse 'csf' (Client Socket Factory) > > and > > ssf??(Server Socket Factory) variables rather than introduce the > > two > > new variables rmiServerSocketFactory and rmiClientSocketFactory. > > You might want to create a new boolean 'useSocketFactory' variable, > > if that makes the code more readable. > > > > Concern: If I understand correctly how RMI socket factories work, > > the client socket factory will be serialized and sent to the client > > side. This is problematic for interoperability, as the class may > > not > > present in the remote client - if the remote client is e.g. jdk 8. > > > > As far as I can see, your new DefaultClientSocketFactory doesn't do > > anything useful - so I would suggest to simply get rid of it, and > > only > > set the Server Socket Factory when SSL is not involved. Thanks. Fixed in updated webrev. > > Tests: > > > > Concerning the tests - we're trying to get rid of shell scripts > > rather than introducing new ones :-) > > Could the test be rewritten in pure java using the Process API? > > > > I believe there's even a test library that will let you do that > > easily jdk/test/lib/testlibrary/jdk/testlibrary/ > > (see ProcessTools.java) It'll take me a bit to rewrite the test in pure Java, but should be fine. This is not yet fixed in the updated webrev. > > Other: > > > > Also - I believe the new option should be documented in: > > src/java.management/share/conf/management.properties Docs have been updated in?src/java.management/share/conf/management.properties. > I share Daniel's concerns. Also, the part of the changeset is related > to javax.rmi.ssl - someone maintaining this library should also > comment here. > > Also, the change is introducing new API (system property) and > changing the existing one (adding SslRmiServerSocketFactory public > constructors) so compatibility review process will have to be > involved. OK. What exactly is there for me to do? I'm not familiar with this process. Note that the intent would be to get this backported to JDK 8. New webrev at: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-6425769/01/ Thanks, Severin > -JB- > > > > best regards, > > > > -- daniel > > > > On 02/11/15 11:38, Severin Gehwolf wrote: > > > Hi, > > > > > > Here is a patch addressing JDK-6425769. The issue is that the JMX > > > agent > > > binds to the wildcard address by default, preventing users to use > > > system properties for JMX agents on multi-homed hosts. Given a > > > host > > > with local network interfaces, 192.168.0.1 and 192.168.0.2 say, > > > it's > > > impossible to start two JMX agents binding to fixed ports but to > > > different network interfaces, 192.168.0.1:{9111,9112} and > > > 192.168.0.2:{9111,9112} say. > > > > > > The JDK would bind to all local interfaces by default. In the > > > above > > > example to 192.168.0.1 *and* 192.168.0.2. The effect is that the > > > second > > > Java process would get a "Connection refused" error because > > > another > > > process has already been bound to the specified JMX/RMI port > > > pairs. > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-6425769 > > > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-6425769/ > > > 00/ > > > > > > Testing done: > > > jdk_jmx and jdk_management tests all pass after this change (no > > > regressions). There is also a new JTREG test which fails prior > > > this > > > change and passes after. Updates to the diagnostic command have > > > been > > > tested manually. > > > > > > I understand that, if approved, the JDK and hotspot changes > > > should get > > > pushed together? Hotspot changes are fairly trivial since it's > > > only a > > > doc-update for the new JDK property in the relevant diagnostic > > > command. > > > > > > Could someone please review and sponsor this change? Please let > > > me know > > > if there are questions. > > > > > > Thanks, > > > Severin > > > > > > From varming at gmail.com Mon Nov 2 18:54:16 2015 From: varming at gmail.com (Carsten Varming) Date: Mon, 2 Nov 2015 13:54:16 -0500 Subject: Let jvmtiGen exit with a non-zero exit code upon failure In-Reply-To: References: <5632251B.80606@oracle.com> <5632A09F.1060205@oracle.com> <5632A148.6070102@oracle.com> <56333474.3050406@oracle.com> <4C0FA96C-1122-4236-84B2-B4CBAB2F795E@oracle.com> <5633EC3E.5020701@oracle.com> <0467332A-2C0C-477D-B669-587B764D3C13@oracle.com> Message-ID: Dear Staffan, I updated the webrev at http://cr.openjdk.java.net/~cvarming/jvmtiGen.02/ I kept the verbose flag to avoid printing the fallback warnings. Carsten On Mon, Nov 2, 2015 at 8:34 AM, Carsten Varming wrote: > Dear Staffan, > > Stacktraces: I can see your point. I'll add them back unconditionally. > > Warnings: The OpenJDK build gets a little noisy when you start printing > warnings. The fallback on failed includes in > hotspot/src/share/vm/trace/*.xml cause a lot of printing; see below. That > is why I introduced the verbose flag in the first place. This is not a big > deal as it pollutes only the build log, but I would still prefer the > program to be silent on normal execution. Please let me know if you still > prefer to discard the verbose flag and print all warnings. > > I'll update the webrev after your reply. > > Thank you, > Carsten > > Noisy warnings: > > jvmtiGen warning: Include operation failed, reverting to fallback. > Resource error reading file as XML > (href='../../../closed/share/vm/trace/traceeventtypes.xml'). Reason: > /Users/cvarming/workspace/jdk9/hotspot/src/closed/share/vm/trace/traceeventtypes.xml > (No such file or directory) > jvmtiGen warning: Include operation failed, reverting to fallback. > Resource error reading file as XML > (href='../../../closed/share/vm/trace/traceevents.xml'). Reason: > /Users/cvarming/workspace/jdk9/hotspot/src/closed/share/vm/trace/traceevents.xml > (No such file or directory) > jvmtiGen warning: Include operation failed, reverting to fallback. > Resource error reading file as XML > (href='../../../closed/share/vm/trace/traceeventtypes.xml'). Reason: > /Users/cvarming/workspace/jdk9/hotspot/src/closed/share/vm/trace/traceeventtypes.xml > (No such file or directory) > jvmtiGen warning: Include operation failed, reverting to fallback. > Resource error reading file as XML > (href='../../../closed/share/vm/trace/traceevents.xml'). Reason: > /Users/cvarming/workspace/jdk9/hotspot/src/closed/share/vm/trace/traceevents.xml > (No such file or directory) > jvmtiGen warning: Include operation failed, reverting to fallback. > Resource error reading file as XML > (href='../../../closed/share/vm/trace/traceeventtypes.xml'). Reason: > /Users/cvarming/workspace/jdk9/hotspot/src/closed/share/vm/trace/traceeventtypes.xml > (No such file or directory) > jvmtiGen warning: Include operation failed, reverting to fallback. > Resource error reading file as XML > (href='../../../closed/share/vm/trace/traceevents.xml'). Reason: > /Users/cvarming/workspace/jdk9/hotspot/src/closed/share/vm/trace/traceevents.xml > (No such file or directory) > > > > On Mon, Nov 2, 2015 at 2:37 AM, Staffan Larsen > wrote: > >> Hi Carsten, >> >> Thanks for spending the time on this - this version looks a lot cleaner. >> >> I would prefer to always print the exception stack trace. The reason is >> that if something goes wrong, it will likely happen on a build server where >> it isn?t simple to find and insert the extra ?-verbose? flag. Better to log >> as much information as possible in case of failure so it?s easier to find >> the root cause. >> >> That leaves warnings from the parser as the only usage of -verbose. Are >> there any warnings presently? If not, I would remove the -verbose flag >> altogether and always print the warnings as well. >> >> Thanks, >> /Staffan >> >> On 1 nov. 2015, at 20:48, Carsten Varming wrote: >> >> Dear Serguei and Steffan, >> >> Sorry about the late reply. The office was a little busy Friday. >> >> Steffan's suggestion is to remove all the exception handling code sound >> reasonable. I am not a fan of the main method throwing checked exceptions, >> so I wrapped all the exception handling into a tiny handler that simply >> prints the exception and exits with a non-zero value. I never cared much >> for the stacktrace, so I put printing of the stacktrace under the verbose >> flag. >> >> Speaking of cleaning up, there is code in there to use Apache xalan >> instead of the standard xslt processor due to an ancient bug fixed in Java >> 1.5. I removed that as well. >> >> There was also a comment about the field "document" being package >> globally accessible due to: "ref'd by the tree-adapter". I have no idea >> what that means, but changing document to a local variable in main didn't >> break the build, so I guess that comment is out-of-date. Please let me know >> if I am missing something here. >> >> There was also a use of java.io.PrintStream. That class suppresses >> IOExceptions: "Unlike other output streams, a PrintStream never throws an >> IOException", so I removed the PrintStream as I want to get a failure when >> something goes wrong. >> >> I have put a new webrev at >> http://cr.openjdk.java.net/~cvarming/jvmtiGen.02/ >> >> Carsten >> ? >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mandy.chung at oracle.com Mon Nov 2 20:19:29 2015 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 2 Nov 2015 12:19:29 -0800 Subject: RFR 8140481: NoClassDefFoundError thrown by ManagementFactory.getPlatformMBeanServer In-Reply-To: <56373ABC.2050109@oracle.com> References: <563719FC.20502@oracle.com> <56371EAD.7010204@oracle.com> <56373ABC.2050109@oracle.com> Message-ID: <10C93905-19D7-4D65-9062-12D490A2C77C@oracle.com> > On Nov 2, 2015, at 2:28 AM, Jaroslav Bachorik wrote: > >>> Currently, the 'jdk.management' module is not a part of the JRE image >>> even though it should be. This patch adds 'jdk.management' to >>> PROVIDER_MODULES where it belongs. >> jdk.management exports an API so I assume it should be in MAIN_MODULES. > > Updated the patch to add the module to MAIN_MODULES - http://cr.openjdk.java.net/~jbachorik/8140481/webrev.01 I agree that jdk.management should be in MAIN_MODULES but not PROVIDER_MODULES list. Looks good. Mandy From serguei.spitsyn at oracle.com Mon Nov 2 20:28:59 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 2 Nov 2015 12:28:59 -0800 Subject: Let jvmtiGen exit with a non-zero exit code upon failure In-Reply-To: References: <5632251B.80606@oracle.com> <5632A09F.1060205@oracle.com> <5632A148.6070102@oracle.com> <56333474.3050406@oracle.com> <4C0FA96C-1122-4236-84B2-B4CBAB2F795E@oracle.com> <5633EC3E.5020701@oracle.com> <0467332A-2C0C-477D-B669-587B764D3C13@oracle.com> Message-ID: <5637C78B.2010501@oracle.com> Hi Carsten, The fix looks good to me. Thank you for the update! I'm waiting for a reply and thumbs up from Staffan. Thanks, Serguei On 11/2/15 10:54, Carsten Varming wrote: > Dear Staffan, > > I updated the webrev at > http://cr.openjdk.java.net/~cvarming/jvmtiGen.02/ > > > I kept the verbose flag to avoid printing the fallback warnings. > > Carsten > > On Mon, Nov 2, 2015 at 8:34 AM, Carsten Varming > wrote: > > Dear Staffan, > > Stacktraces: I can see your point. I'll add them back unconditionally. > > Warnings: The OpenJDK build gets a little noisy when you start > printing warnings. The fallback on failed includes in > hotspot/src/share/vm/trace/*.xml cause a lot of printing; see > below. That is why I introduced the verbose flag in the first > place. This is not a big deal as it pollutes only the build log, > but I would still prefer the program to be silent on normal > execution. Please let me know if you still prefer to discard the > verbose flag and print all warnings. > > I'll update the webrev after your reply. > > Thank you, > Carsten > > Noisy warnings: > > jvmtiGen warning: Include operation failed, reverting to fallback. > Resource error reading file as XML > (href='../../../closed/share/vm/trace/traceeventtypes.xml'). > Reason: > /Users/cvarming/workspace/jdk9/hotspot/src/closed/share/vm/trace/traceeventtypes.xml > (No such file or directory) > jvmtiGen warning: Include operation failed, reverting to fallback. > Resource error reading file as XML > (href='../../../closed/share/vm/trace/traceevents.xml'). Reason: > /Users/cvarming/workspace/jdk9/hotspot/src/closed/share/vm/trace/traceevents.xml > (No such file or directory) > jvmtiGen warning: Include operation failed, reverting to fallback. > Resource error reading file as XML > (href='../../../closed/share/vm/trace/traceeventtypes.xml'). > Reason: > /Users/cvarming/workspace/jdk9/hotspot/src/closed/share/vm/trace/traceeventtypes.xml > (No such file or directory) > jvmtiGen warning: Include operation failed, reverting to fallback. > Resource error reading file as XML > (href='../../../closed/share/vm/trace/traceevents.xml'). Reason: > /Users/cvarming/workspace/jdk9/hotspot/src/closed/share/vm/trace/traceevents.xml > (No such file or directory) > jvmtiGen warning: Include operation failed, reverting to fallback. > Resource error reading file as XML > (href='../../../closed/share/vm/trace/traceeventtypes.xml'). > Reason: > /Users/cvarming/workspace/jdk9/hotspot/src/closed/share/vm/trace/traceeventtypes.xml > (No such file or directory) > jvmtiGen warning: Include operation failed, reverting to fallback. > Resource error reading file as XML > (href='../../../closed/share/vm/trace/traceevents.xml'). Reason: > /Users/cvarming/workspace/jdk9/hotspot/src/closed/share/vm/trace/traceevents.xml > (No such file or directory) > > > > On Mon, Nov 2, 2015 at 2:37 AM, Staffan Larsen > > wrote: > > Hi Carsten, > > Thanks for spending the time on this - this version looks a > lot cleaner. > > I would prefer to always print the exception stack trace. The > reason is that if something goes wrong, it will likely happen > on a build server where it isn?t simple to find and insert the > extra ?-verbose? flag. Better to log as much information as > possible in case of failure so it?s easier to find the root cause. > > That leaves warnings from the parser as the only usage of > -verbose. Are there any warnings presently? If not, I would > remove the -verbose flag altogether and always print the > warnings as well. > > Thanks, > /Staffan > >> On 1 nov. 2015, at 20:48, Carsten Varming > > wrote: >> >> Dear Serguei and Steffan, >> >> Sorry about the late reply. The office was a little busy Friday. >> >> Steffan's suggestion is to remove all the exception handling >> code sound reasonable. I am not a fan of the main method >> throwing checked exceptions, so I wrapped all the exception >> handling into a tiny handler that simply prints the exception >> and exits with a non-zero value. I never cared much for the >> stacktrace, so I put printing of the stacktrace under the >> verbose flag. >> >> Speaking of cleaning up, there is code in there to use Apache >> xalan instead of the standard xslt processor due to an >> ancient bug fixed in Java 1.5. I removed that as well. >> >> There was also a comment about the field "document" being >> package globally accessible due to: "ref'd by the >> tree-adapter". I have no idea what that means, but changing >> document to a local variable in main didn't break the build, >> so I guess that comment is out-of-date. Please let me know if >> I am missing something here. >> >> There was also a use of java.io.PrintStream. That class >> suppresses IOExceptions: "Unlike other output streams, a >> PrintStream never throws an IOException", so I removed the >> PrintStream as I want to get a failure when something goes wrong. >> >> I have put a new webrev at >> http://cr.openjdk.java.net/~cvarming/jvmtiGen.02/ >> >> >> Carsten >> ? > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Tue Nov 3 05:05:52 2015 From: david.holmes at oracle.com (David Holmes) Date: Tue, 3 Nov 2015 15:05:52 +1000 Subject: RFR(M): 8140482: Various minor code improvements (runtime) In-Reply-To: <4295855A5C1DE049A61835A1887419CC41E9F0AA@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC41E9F0AA@DEWDFEMB12A.global.corp.sap> Message-ID: <563840B0.10401@oracle.com> (adding serviceability-dev as they own SA code and some other parts - eg SR_signal_num issue) Hi Goetz, Sorry, lots of folks very busy at the moment as we try to get features finalized before the deadline. On 27/10/2015 5:39 PM, Lindenmaier, Goetz wrote: > Hi, > > SAP requires us to fix a row of issues in hotspot. I'd like to share these > with openjdk: > http://cr.openjdk.java.net/~goetz/webrevs/8140482-covRt/webrev.00 > > Please review this change. I please need a sponsor. > > The fixes in detail: > > libproc_impl.c: > Do strncpy in case getenv returned a bad string. > Strcat could overflow the buffer. Use strncat instead. Okay. > ps_core.c: > Pread not necessarily terminates interp_name which is printed thereafter. > Increase buffer size by 1 and add '\0'. Given: #define BUF_SIZE (PATH_MAX + NAME_MAX + 1) isn't it impossible to encounter that problem? > stubRoutines_x86.cpp: > Cast to proper type. This way, left and right of '&' have the same type. I think you could just have changed the uint64_t to uint32_t as applied to the shift rather than casting the 1 to uint_32t. End result is the same though. > src/cpu/x86/vm/templateTable_x86.cpp Use of CONST64 seems okay. > attachListener_linux.cpp: > Read does not terminate buf. Size for '\0' is already considered. Looks a little odd being done on each iteration, but okay I guess. > os_linux.cpp: > Array sigflags[] has size MAXSIGNUM==32. _NSIG is bigger than > MAXSIGNUM (_NSIG == 65 on my machine). > sig is checked to be smaller than _NSIG. Later, in set_our_sigflags(), > sig is used to access sigflags[MAXSIGNUM] which can overflow the array. > Should we also increase MAXSIGNUM? Need to let the SR folk comment here as something definitely seems wrong, but I'm not 100% sure the what the correct answer is. If _JAVA_SR_SIGNUM is too big it should be validated somewhere and an error or warning reported. > os::get_core_path(): read does not terminate string, but strlen is > called on it. The size already foresees one char for the '\0' byte. Ok. > codeBuffer.cpp: > New_capacity is not initialized. Figure_expanded_capacities() handles this > correctly, but initializing this is cheap and safe. Hmmm ... I hate redundancy - this is pure wasted cycles. If we had to do it would memset not be better? Or would the code-checker not realize what memset was doing? > dict.cpp: > If j-- is executed for j==0, the loop aborts because j is unsigned (0-- >= b->_cnt). > Instead, only do j++ if necessary. Not at all obvious to me that it is possible to do j-- when j==0, but the change seems reasonable. Lots of spacing changes in that code make it hard to see the real changes. 145 // SAPJVM GL j-- can underflow, which will cause the loop to abort. Seems unnecessary with the code change as noone will understand what j-- you are referring to. 150 nb->_keyvals[nbcnt + nbcnt ] = key; 151 nb->_keyvals[nbcnt + nbcnt + 1] = b->_keyvals[j+j+1]; hotspot-style doesn't align array index expressions like that. Ditto 154/155. > generateOopMap.cpp: > Idx is read from String. This is only called with constant strings, so compare > should be folded away by optimizing compilers if inlined. Not a fan of adding conditions that should never be false (hence the assert) and relying on the compiler to elide them. > deoptimization.cpp: > If buflen == 0, buf[-1] is accessed. Okay - but an assert(buflen>0) would be better I think as we should never be calling with a zero-length buffer. > task.cpp: > Fatal can return if -XX:SuppressErrorAt is used. Just don't access the > array in this case. Okay. I would not be surprised if we have a lot of potential errors if a fatal/guarantee failure is suppressed. > attachListener.hpp: > Do strncpy to not overflow buffer. Don't write more chars than before. Again we have the assert to catch an error in the caller using an invalid name. > heapDumper.cpp: > strncpy does not null terminate. 1973 if (total_length >= sizeof(base_path)) { total_length already adds +1 for the nul character so the == case is fine AFAICS. strncpy wont nul-terminate if the string exceeds the buffer size. But we have already established that total_length <= sizeof(base_path), and total_path includes space for a bunch of stuff other than HeapDumpPath, so the strncpy of HeapDumpPath has to copy the nul character. > src/share/vm/services/memoryService.cpp Ok. > src/share/vm/utilities/xmlstream.cpp Ok - I'm more concerned about the "magic" 10 in that piece of code. > Some of these, as the issue in codeBuffer.cpp, are actually handled correctly. > Nevertheless this is not that obvious so that somebody changing the code > Could oversee he has to add the initialization. Not an argument I buy en-masse as it leads to a lot of redundancy through the code paths. Plus these tools that are being run should show if a code changes requires initialization that is not present. Thanks, David > Some of these fixes are part of SAP JVM for a long time. This change has > been tested with our nightly build of openJDK. > > Best regards, > Goetz,. > From david.holmes at oracle.com Tue Nov 3 05:37:40 2015 From: david.holmes at oracle.com (David Holmes) Date: Tue, 3 Nov 2015 15:37:40 +1000 Subject: RFR: JDK-8140556: Add force rotation option to VM.log jcmd In-Reply-To: <563765FC.2090804@gmail.com> References: <562101D8.5080909@gmail.com> <562DF8D4.40501@oracle.com> <562EBF4B.4020607@gmail.com> <562F6B46.5050500@oracle.com> <563226AB.2030206@gmail.com> <5632341F.4060400@oracle.com> <56338125.9030802@gmail.com> <56338D58.9090709@oracle.com> <5634D9FC.6080502@gmail.com> <5636C66A.6020304@oracle.com> <563765FC.2090804@gmail.com> Message-ID: <56384824.4080904@oracle.com> On 2/11/2015 11:32 PM, Yasumasa Suenaga wrote: > Marcus, David, > > I've uploaded new webrev: > http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.02/ > > Could you review it? > > >>>>>> Still missing the renaming of rotate_all_logfile to >>>>>> rotate_all_outputs. > > I've fixed. > > >>>> Small typo, please make that is_rotatable(). > > I've added, and have used at LogConfiguration::rotate_all_outputs(). > > >> You can't safely lock a mutex until the current thread has attached to >> the VM and Thread::current() returns non-null (that is what >> Thread::initialize_thread_local_storage() currently does). You may get >> away with it, particularly in product builds, during VM >> initialization, as the fast-lock path doesn't require access to >> Thread::current(). > > I will check whether Thread::current() is NULL or not at > LogFileOutput::is_rotatable() > is_rotatable() is called at LogFileOutput::rotate() before locking > _rotation_lock. > So we can avoid crash/assert which is related to TLS. I'm unclear of the context in which this could be called by an unattached thread (ie during early VM initialization). Could you clarify the code-path for me please? It seems an odd check to make on all rotate() requests rather then doing something only in the problematic case. Note I am not reviewing this in general as I am not familiar with the logging code, I am just commenting on the thread-safety issue. Thanks, David > > Thanks, > > Yasumasa > > > On 2015/11/02 11:11, David Holmes wrote: >> On 1/11/2015 1:10 AM, Yasumasa Suenaga wrote: >>>> This in turn means that we are logging before the thread local storage >>>> has been initialized, and the JVM will crash/hit the assert because we >>>> are trying to take the rotation lock. >>> >>> I guess that you are saying about class member (not TLS - e.g. >>> pthread_setspecific()). >> >> You can't safely lock a mutex until the current thread has attached to >> the VM and Thread::current() returns non-null (that is what >> Thread::initialize_thread_local_storage() currently does). You may get >> away with it, particularly in product builds, during VM >> initialization, as the fast-lock path doesn't require access to >> Thread::current(). >> >> David >> >>> Most of fields (include _rotation_lock) in LogFileOutput are initialized >>> at c'tor. >>> Other field (_stream) is initialized at LogFileOutput::initialize(). >>> >>> We can avoid problem if we can move initialize() to c'tor because >>> LogFileOutput >>> will be instantiated at LogConfiguration::new_output(). >>> >>> Currently, LogFileOutput will be added to LogConfiguration::_outputs and >>> to LogTagSet after calling LogFileOutput::initialize(). >>> I wonder why we cannot avoid crash/assert with current implementation... >>> >>> >>> I will keep current implementation about _rotation_lock if the above is >>> incorrect. >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> On 2015/10/31 0:31, Marcus Larsson wrote: >>>> Hey, >>>> >>>> On 2015-10-30 15:39, Yasumasa Suenaga wrote: >>>>> Hi Marcus, >>>>> >>>>> Thank you for your comment. >>>>> >>>>> >>>>>> Still missing the renaming of rotate_all_logfile to >>>>>> rotate_all_outputs. >>>>> >>>>> Sorry, I will fix. >>>>> >>>> >>>> Thanks! >>>> >>>>> >>>>>> This rotate function works I guess, but it will probably need some >>>>>> rework once there are other types of log outputs (rotate doesn't >>>>>> make sense on all types of log outputs). >>>>> >>>>> I will add is_rotetable() to LogOutput as virtual function. >>>>> This function return false (can not rotate) by default, return true >>>>> if _file_count is >>>>> greater than 0 in LogFileOutput. >>>>> >>>>> LogConfiguration::rotate_all_outputs() will rotate if this function >>>>> returns true. >>>>> >>>> >>>> Small typo, please make that is_rotatable(). >>>> >>>>> >>>>>> If we configure a rotated file output and then log something on that >>>>>> output before thread local storage initialization is complete, we >>>>>> will crash/hit an assert. The previous implementation avoided this >>>>>> as long as no rotation was needed before this initialization was >>>>>> complete. (This can be triggered using the following arguments >>>>>> "-Xlog:all=trace:file.txt::filesize=10,filecount=2 -Xlog:invalid", >>>>>> for example.) >>>>> >>>>> I do not think so. >>>>> Each -Xlog option creates unique instance of LogOutput (and subclass). >>>>> So they are isolated. >>>> >>>> My example includes a second (and intentionally incorrect) -Xlog >>>> option only to trigger logging on the newly configured rotating >>>> LogFileOutput. This will cause some logging about the invalid -Xlog >>>> argument, and this logging happens during the argument parsing. This >>>> in turn means that we are logging before the thread local storage has >>>> been initialized, and the JVM will crash/hit the assert because we are >>>> trying to take the rotation lock. This has nothing to do with >>>> concurrent fprintfs or freopens, even if that is a problem. >>>> >>>>> >>>>> However, we might have to rotate at safepoint. >>>> >>>> Doing stuff at a safepoint will not give us much in this case. There >>>> are threads that continue execution through a safepoint, which means >>>> we will see log calls being made even during this time. >>>> >>>>> Currently, LogOutput might be used by multiple threads concurrently. >>>>> If we rotate log when another thread is writing string to output, we >>>>> encounter unexpected behavior. >>>>> >>>>> LogFileOutput::rotate() uses freopen(). >>>>> LogFileStreamOutput::write() uses vfprintf() through jio_fprintf(). >>>>> freopen() and vfprintf() are not atomic. >>>> >>>> You're right. There seems to be an overlooked concurrency issue with >>>> vfprintfs during freopen. I don't think we should resolve this as part >>>> of this particular fix, so instead I'll create a separate issue for it. >>>> >>>>> >>>>> I think that cause of crash/assertion which you say is it. >>>>> (GC log will be rotated at safepoint.) >>>>> >>>> >>>> See my comment above for what crash/assert I'm talking about. >>>> >>>> Thanks, >>>> Marcus >>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2015/10/29 23:58, Marcus Larsson wrote: >>>>>> Hi, >>>>>> >>>>>> On 2015-10-29 15:01, Yasumasa Suenaga wrote: >>>>>>> Hi Marcus, >>>>>>> >>>>>>> Thank you for your comment. >>>>>>> >>>>>>>> I'll sponsor it. >>>>>>> >>>>>>> Thank you so much! >>>>>>> I've uploaded new webrev: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.01/ >>>>>>> >>>>>>> >>>>>> >>>>>> Still missing the renaming of rotate_all_logfile to >>>>>> rotate_all_outputs. >>>>>> >>>>>>>> logConfiguration.cpp/hpp: >>>>>>>> >>>>>>>> * I don't think we should rely on the archive_name or the output's >>>>>>>> name to decide whether or not an output should be rotated. It >>>>>>>> would be better to introduce an is_rotated() test function in >>>>>>>> LogOutput that could be used here. >>>>>>>> >>>>>>>> * rotate_all_logfile() should be renamed to rotate_all_outputs(). >>>>>>>> Currently there are only LogFileOutputs (other than >>>>>>>> stdout/stderr), but in the future there might be other types of >>>>>>>> outputs so I prefer having a more general name. >>>>>>>> Also, please use static_cast instead of the >>>>>>>> C-style cast. (Additional logic will be required here once more >>>>>>>> types of log outputs are added, but I don't think we need to worry >>>>>>>> about this right now.) >>>>>>>> >>>>>>>> * Don't print an error if an output is not rotatable, since it's >>>>>>>> not an error to have some log outputs rotated while others are >>>>>>>> not. If you really want some traceability here I suggest adding >>>>>>>> log messages on trace level, tagged with 'logging'. >>>>>>> >>>>>>> I added LogOutput::rotate(bool) as virtual function. >>>>>>> This function works nothing by default, but will rotate all logs at >>>>>>> LogFileOutput. >>>>>> >>>>>> This rotate function works I guess, but it will probably need some >>>>>> rework once there are other types of log outputs (rotate doesn't >>>>>> make sense on all types of log outputs). >>>>>> >>>>>>> >>>>>>> If true is passed to this function, all files will be rotated. >>>>>>> >>>>>>> >>>>>>>> logDiagnosticCommand.cpp/hpp: >>>>>>>> >>>>>>>> * I think the description could be improved to something like >>>>>>>> "Lists current log configuration, enables/disables/configures a >>>>>>>> log output, or disables/rotates all logs." >>>>>>>> >>>>>>>> * The rotate option description should clarify that all logs will >>>>>>>> be rotated ("current log" is too ambiguous). >>>>>>> >>>>>>> I've fixed. >>>>>> >>>>>> Great, thanks! >>>>>> >>>>>>> >>>>>>> >>>>>>>> logFileOutput.cpp/hpp: >>>>>>>> >>>>>>>> * Moving the MutexLocker like this introduces a race condition >>>>>>>> where a log might be rotated multiple times by different threads, >>>>>>>> instead of just once. >>>>>>>> Instead of making the rotate() function public and moving the >>>>>>>> mutexlocker, I suggest adding something like a public >>>>>>>> force_rotation() function that grabs the lock and calls the >>>>>>>> private rotate(). >>>>>>>> >>>>>>>> * Given the addition of is_rotated() in LogOutput, then >>>>>>>> get_archive_name() should be removed. >>>>>>> >>>>>>> I moved MutexLocker and should_rotate() to rotate(). >>>>>>> I think this change can avoid race condition. >>>>>>> >>>>>> >>>>>> There is a subtle problem with taking the lock to check if we should >>>>>> rotate. If we configure a rotated file output and then log something >>>>>> on that output before thread local storage initialization is >>>>>> complete, we will crash/hit an assert. The previous implementation >>>>>> avoided this as long as no rotation was needed before this >>>>>> initialization was complete. (This can be triggered using the >>>>>> following arguments >>>>>> "-Xlog:all=trace:file.txt::filesize=10,filecount=2 -Xlog:invalid", >>>>>> for example.) >>>>>> >>>>>> Thanks, >>>>>> Marcus >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2015/10/27 21:17, Marcus Larsson wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> On 2015-10-27 01:03, Yasumasa Suenaga wrote: >>>>>>>>> Hi Marcus, >>>>>>>>> >>>>>>>>> Thank you for replying. >>>>>>>>> >>>>>>>>> I filed this feature to JBS and change subject of this email. >>>>>>>>> Could you be a sponsor and review it? >>>>>>>> >>>>>>>> I'll sponsor it. >>>>>>>> >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.00/ >>>>>>>> >>>>>>>> logConfiguration.cpp/hpp: >>>>>>>> >>>>>>>> * I don't think we should rely on the archive_name or the output's >>>>>>>> name to decide whether or not an output should be rotated. It >>>>>>>> would be better to introduce an is_rotated() test function in >>>>>>>> LogOutput that could be used here. >>>>>>>> >>>>>>>> * rotate_all_logfile() should be renamed to rotate_all_outputs(). >>>>>>>> Currently there are only LogFileOutputs (other than >>>>>>>> stdout/stderr), but in the future there might be other types of >>>>>>>> outputs so I prefer having a more general name. >>>>>>>> Also, please use static_cast instead of the >>>>>>>> C-style cast. (Additional logic will be required here once more >>>>>>>> types of log outputs are added, but I don't think we need to worry >>>>>>>> about this right now.) >>>>>>>> >>>>>>>> * Don't print an error if an output is not rotatable, since it's >>>>>>>> not an error to have some log outputs rotated while others are >>>>>>>> not. If you really want some traceability here I suggest adding >>>>>>>> log messages on trace level, tagged with 'logging'. >>>>>>>> >>>>>>>> >>>>>>>> logDiagnosticCommand.cpp/hpp: >>>>>>>> >>>>>>>> * I think the description could be improved to something like >>>>>>>> "Lists current log configuration, enables/disables/configures a >>>>>>>> log output, or disables/rotates all logs." >>>>>>>> >>>>>>>> * The rotate option description should clarify that all logs will >>>>>>>> be rotated ("current log" is too ambiguous). >>>>>>>> >>>>>>>> >>>>>>>> logFileOutput.cpp/hpp: >>>>>>>> >>>>>>>> * Moving the MutexLocker like this introduces a race condition >>>>>>>> where a log might be rotated multiple times by different threads, >>>>>>>> instead of just once. >>>>>>>> Instead of making the rotate() function public and moving the >>>>>>>> mutexlocker, I suggest adding something like a public >>>>>>>> force_rotation() function that grabs the lock and calls the >>>>>>>> private rotate(). >>>>>>>> >>>>>>>> * Given the addition of is_rotated() in LogOutput, then >>>>>>>> get_archive_name() should be removed. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Marcus >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2015/10/26 18:56, Marcus Larsson wrote: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> Sorry for my late reply. >>>>>>>>>> >>>>>>>>>> I think being able to force rotation via jcmd seems like a good >>>>>>>>>> feature. Files are currently opened in append mode so it should >>>>>>>>>> already be possible to use external log rotation tools by >>>>>>>>>> copying and truncating the files. Still I think it would be nice >>>>>>>>>> to provide the jcmd for rotation as well. >>>>>>>>>> >>>>>>>>>> I can see some small issues with the implementation, but we can >>>>>>>>>> deal with that during the review. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Marcus >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 2015-10-26 00:26, Yasumasa Suenaga wrote: >>>>>>>>>>> Hi all, >>>>>>>>>>> >>>>>>>>>>> Have you ever seen my change? >>>>>>>>>>> I (and my customers) need log rotation function via external >>>>>>>>>>> tool. >>>>>>>>>>> >>>>>>>>>>> I want to merge it by Feature Complete. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> 2015/10/16 22:55 "Yasumasa Suenaga" : >>>>>>>>>>> >>>>>>>>>>>> Hi all, >>>>>>>>>>>> >>>>>>>>>>>> I contributed JDK-7090324: gclog rotation via external tool >>>>>>>>>>>> to be >>>>>>>>>>>> synchronized with >>>>>>>>>>>> logrotated tool on Linux. >>>>>>>>>>>> >>>>>>>>>>>> I think JEP 158 is in progress. >>>>>>>>>>>> However, this JEP does not contain log rotation via external >>>>>>>>>>>> tool in the >>>>>>>>>>>> spec. >>>>>>>>>>>> I want to rotate logs via jcmd in this JEP. >>>>>>>>>>>> >>>>>>>>>>>> I've updated a patch for it: >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/jvmlogging-logrotate/ >>>>>>>>>>>> >>>>>>>>>>>> This patch provides new option "rotate" in VM.log command. >>>>>>>>>>>> If this change can be accepted, I will file it to JBS and send >>>>>>>>>>>> RFR. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Yasumasa >>>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>> >>>> From david.holmes at oracle.com Tue Nov 3 06:27:53 2015 From: david.holmes at oracle.com (David Holmes) Date: Tue, 3 Nov 2015 16:27:53 +1000 Subject: RFR: JDK-8134030: test/serviceability/dcmd/gc/HeapDumpTest fails to verify the dump In-Reply-To: <563378B1.6080607@oracle.com> References: <563378B1.6080607@oracle.com> Message-ID: <563853E9.6090901@oracle.com> Hi Andreas, On 31/10/2015 12:03 AM, Andreas Eriksson wrote: > Hi, > > Please review this change to heap dump logic. > Including runtime list, since this is related to class loading. > > Bug: > JDK-8134030: test/serviceability/dcmd/gc/HeapDumpTest fails to verify > the dump > https://bugs.openjdk.java.net/browse/JDK-8134030 > > Webrev: http://cr.openjdk.java.net/~aeriksso/8134030/webrev.00/ > > A heap dump can occur at a point where an InstanceKlass doesn't have a > Java class mirror yet. > To avoid a crash because of this JDK-8131600 [1] excluded classes that > are not linked yet from the dump. > > The problem here is that since a class that has been loaded is already > exposed to the Java side, another class can have a reference to the > excluded class. > So, if it has been *loaded* but not *linked* it will be excluded, but > can still be referenced by other classes in the dump. I'm having trouble seeing exactly when the class is marked as "loaded" but it certainly seems to be after the mirror has been set. However linking happens even later than that - so if you still see a crash excluding linked classes then I fear you will also see problems excluding loaded classes! David ----- > This gives the following warning in HeapDumpTest: > WARNING: Failed to resolve object id 0x6c003d348 [...]. > > This fix changes heapDumper.cpp to only exclude classes that are not yet > loaded instead. > > I'd like confirmation on this from someone who knows more about loaded > vs linked, but I think that this is correct: > When a class has been loaded, we are guaranteed that a Java class mirror > is setup, so the crash seen in JDK-8131600 still cannot happen. > This seems to be confirmed by the manual reproducer from JDK-8131600, > which still succeeds with this fix. > > Also in this change is the re-addition of the actual heap dump > verification call in HeapDumpTest, which was accidentally removed when > the test was re-factored. > > Regards, > Andreas > > [1]: https://bugs.openjdk.java.net/browse/JDK-8131600 From goetz.lindenmaier at sap.com Tue Nov 3 09:33:37 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 3 Nov 2015 09:33:37 +0000 Subject: RFR(M): 8140482: Various minor code improvements (runtime) In-Reply-To: <563840B0.10401@oracle.com> References: <4295855A5C1DE049A61835A1887419CC41E9F0AA@DEWDFEMB12A.global.corp.sap> <563840B0.10401@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC41EB1746@DEWDFEMB12A.global.corp.sap> Hi David, > Sorry, lots of folks very busy at the moment as we try to get features > finalized before the deadline. thanks for looking at this! I know the Dec. 10 deadline, but I guess that also holds for us ... at least J1 is over now. (Unfortunately we could not attend this year.) > > ps_core.c: > > Pread not necessarily terminates interp_name which is printed thereafter. > > Increase buffer size by 1 and add '\0'. > > Given: > #define BUF_SIZE (PATH_MAX + NAME_MAX + 1) > isn't it impossible to encounter that problem? As I understand, pread does not null-terminate what it read. So the null must come from the file. This protects against a corrupted file. > > stubRoutines_x86.cpp: > > Cast to proper type. This way, left and right of '&' have the same type. > > I think you could just have changed the uint64_t to uint32_t as applied > to the shift rather than casting the 1 to uint_32t. End result is the > same though. What you propose did not work. It was my first fix, too. > > attachListener_linux.cpp: > > Read does not terminate buf. Size for '\0' is already considered. > > Looks a little odd being done on each iteration, but okay I guess. I'll try to move it out of the loop. Better: I'll check whether the scan groks it if I move it out of the loop :) > > os_linux.cpp: > > Array sigflags[] has size MAXSIGNUM==32. _NSIG is bigger than > > MAXSIGNUM (_NSIG == 65 on my machine). > > sig is checked to be smaller than _NSIG. Later, in set_our_sigflags(), > > sig is used to access sigflags[MAXSIGNUM] which can overflow the array. > > Should we also increase MAXSIGNUM? > > Need to let the SR folk comment here as something definitely seems > wrong, but I'm not 100% sure the what the correct answer is. If > _JAVA_SR_SIGNUM is too big it should be validated somewhere and an error > or warning reported. I'm also not sure how to best handle this. Might even require a fix exceeding this change. But I think this is the best finding. > > codeBuffer.cpp: > > New_capacity is not initialized. Figure_expanded_capacities() handles this > > correctly, but initializing this is cheap and safe. > > Hmmm ... I hate redundancy - this is pure wasted cycles. If we had to do > it would memset not be better? Or would the code-checker not realize > what memset was doing? I guess it would work with memset, too. But I thought the 3-deep loop will be unrolled completely so that only three stores remain. > > dict.cpp: > > If j-- is executed for j==0, the loop aborts because j is unsigned (0-- >= b- > >_cnt). > > Instead, only do j++ if necessary. > > Not at all obvious to me that it is possible to do j-- when j==0, but > the change seems reasonable. Yes, the scan does not understand there is j++ right after j-- because of the loop iteration. I saw it complaining about this pattern several times. > Lots of spacing changes in that code make it hard to see the real changes. Before, I was asked to fix indentation issues in a function I touch. Does that only hold for compiler files? > 145 // SAPJVM GL j-- can underflow, which will cause the loop to abort. > Seems unnecessary with the code change as noone will understand what j-- > you are referring to. Didn't mean to leave this in here. Removed. > 150 nb->_keyvals[nbcnt + nbcnt ] = key; > 151 nb->_keyvals[nbcnt + nbcnt + 1] = b->_keyvals[j+j+1]; > hotspot-style doesn't align array index expressions like that. Ditto > 154/155. Fixed. > > generateOopMap.cpp: > > Idx is read from String. This is only called with constant strings, so compare > > should be folded away by optimizing compilers if inlined. > > Not a fan of adding conditions that should never be false (hence the > assert) and relying on the compiler to elide them. OK, removed. > > deoptimization.cpp: > > If buflen == 0, buf[-1] is accessed. > > Okay - but an assert(buflen>0) would be better I think as we should > never be calling with a zero-length buffer. Ok, I added the assert. As this isn't critical code, I would like to leave the check in there, still. > > task.cpp: > > Fatal can return if -XX:SuppressErrorAt is used. Just don't access the > > array in this case. > > Okay. I would not be surprised if we have a lot of potential errors if a > fatal/guarantee failure is suppressed. > > > attachListener.hpp: > > Do strncpy to not overflow buffer. Don't write more chars than before. > > Again we have the assert to catch an error in the caller using an > invalid name. Hmm, the command comes from outside of the VM. It's not checked very thoroughly, see, e.g., attachListener_windows.cpp:194. Arg0 is checked twice, arg1 and arg2 are not checked at all. I add fixes for attachListener_windows.cpp to this change. > > heapDumper.cpp: > > strncpy does not null terminate. > > 1973 if (total_length >= sizeof(base_path)) { > > total_length already adds +1 for the nul character so the == case is > fine AFAICS. > > strncpy wont nul-terminate if the string exceeds the buffer size. But we > have already established that total_length <= sizeof(base_path), and > total_path includes space for a bunch of stuff other than HeapDumpPath, > so the strncpy of HeapDumpPath has to copy the nul character. Ok, removed. > > src/share/vm/services/memoryService.cpp > > Ok. > > > src/share/vm/utilities/xmlstream.cpp > > Ok - I'm more concerned about the "magic" 10 in that piece of code. I assume the 10 is the space needed for the "_done" plus some waste ... I'll do another run of the scan. That takes a day. I'll post a new webrev after that. Thank again for this thorough review, Goetz > > Some of these, as the issue in codeBuffer.cpp, are actually handled correctly. > > Nevertheless this is not that obvious so that somebody changing the code > > Could oversee he has to add the initialization. > > Not an argument I buy en-masse as it leads to a lot of redundancy > through the code paths. Plus these tools that are being run should show > if a code changes requires initialization that is not present. > > Thanks, > David > > > Some of these fixes are part of SAP JVM for a long time. This change has > > been tested with our nightly build of openJDK. > > > > Best regards, > > Goetz,. > > From staffan.larsen at oracle.com Tue Nov 3 09:59:35 2015 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Tue, 3 Nov 2015 10:59:35 +0100 Subject: Let jvmtiGen exit with a non-zero exit code upon failure In-Reply-To: References: <5632251B.80606@oracle.com> <5632A09F.1060205@oracle.com> <5632A148.6070102@oracle.com> <56333474.3050406@oracle.com> <4C0FA96C-1122-4236-84B2-B4CBAB2F795E@oracle.com> <5633EC3E.5020701@oracle.com> <0467332A-2C0C-477D-B669-587B764D3C13@oracle.com> Message-ID: Looks good! Thanks, /Staffan > On 2 nov. 2015, at 19:54, Carsten Varming wrote: > > Dear Staffan, > > I updated the webrev at http://cr.openjdk.java.net/~cvarming/jvmtiGen.02/ > > I kept the verbose flag to avoid printing the fallback warnings. > > Carsten > > On Mon, Nov 2, 2015 at 8:34 AM, Carsten Varming > wrote: > Dear Staffan, > > Stacktraces: I can see your point. I'll add them back unconditionally. > > Warnings: The OpenJDK build gets a little noisy when you start printing warnings. The fallback on failed includes in hotspot/src/share/vm/trace/*.xml cause a lot of printing; see below. That is why I introduced the verbose flag in the first place. This is not a big deal as it pollutes only the build log, but I would still prefer the program to be silent on normal execution. Please let me know if you still prefer to discard the verbose flag and print all warnings. > > I'll update the webrev after your reply. > > Thank you, > Carsten > > Noisy warnings: > > jvmtiGen warning: Include operation failed, reverting to fallback. Resource error reading file as XML (href='../../../closed/share/vm/trace/traceeventtypes.xml'). Reason: /Users/cvarming/workspace/jdk9/hotspot/src/closed/share/vm/trace/traceeventtypes.xml (No such file or directory) > jvmtiGen warning: Include operation failed, reverting to fallback. Resource error reading file as XML (href='../../../closed/share/vm/trace/traceevents.xml'). Reason: /Users/cvarming/workspace/jdk9/hotspot/src/closed/share/vm/trace/traceevents.xml (No such file or directory) > jvmtiGen warning: Include operation failed, reverting to fallback. Resource error reading file as XML (href='../../../closed/share/vm/trace/traceeventtypes.xml'). Reason: /Users/cvarming/workspace/jdk9/hotspot/src/closed/share/vm/trace/traceeventtypes.xml (No such file or directory) > jvmtiGen warning: Include operation failed, reverting to fallback. Resource error reading file as XML (href='../../../closed/share/vm/trace/traceevents.xml'). Reason: /Users/cvarming/workspace/jdk9/hotspot/src/closed/share/vm/trace/traceevents.xml (No such file or directory) > jvmtiGen warning: Include operation failed, reverting to fallback. Resource error reading file as XML (href='../../../closed/share/vm/trace/traceeventtypes.xml'). Reason: /Users/cvarming/workspace/jdk9/hotspot/src/closed/share/vm/trace/traceeventtypes.xml (No such file or directory) > jvmtiGen warning: Include operation failed, reverting to fallback. Resource error reading file as XML (href='../../../closed/share/vm/trace/traceevents.xml'). Reason: /Users/cvarming/workspace/jdk9/hotspot/src/closed/share/vm/trace/traceevents.xml (No such file or directory) > > > > On Mon, Nov 2, 2015 at 2:37 AM, Staffan Larsen > wrote: > Hi Carsten, > > Thanks for spending the time on this - this version looks a lot cleaner. > > I would prefer to always print the exception stack trace. The reason is that if something goes wrong, it will likely happen on a build server where it isn?t simple to find and insert the extra ?-verbose? flag. Better to log as much information as possible in case of failure so it?s easier to find the root cause. > > That leaves warnings from the parser as the only usage of -verbose. Are there any warnings presently? If not, I would remove the -verbose flag altogether and always print the warnings as well. > > Thanks, > /Staffan > >> On 1 nov. 2015, at 20:48, Carsten Varming > wrote: >> >> Dear Serguei and Steffan, >> >> Sorry about the late reply. The office was a little busy Friday. >> >> Steffan's suggestion is to remove all the exception handling code sound reasonable. I am not a fan of the main method throwing checked exceptions, so I wrapped all the exception handling into a tiny handler that simply prints the exception and exits with a non-zero value. I never cared much for the stacktrace, so I put printing of the stacktrace under the verbose flag. >> >> Speaking of cleaning up, there is code in there to use Apache xalan instead of the standard xslt processor due to an ancient bug fixed in Java 1.5. I removed that as well. >> >> There was also a comment about the field "document" being package globally accessible due to: "ref'd by the tree-adapter". I have no idea what that means, but changing document to a local variable in main didn't break the build, so I guess that comment is out-of-date. Please let me know if I am missing something here. >> >> There was also a use of java.io.PrintStream. That class suppresses IOExceptions: "Unlike other output streams, a PrintStream never throws an IOException", so I removed the PrintStream as I want to get a failure when something goes wrong. >> >> I have put a new webrev at http://cr.openjdk.java.net/~cvarming/jvmtiGen.02/ >> >> Carsten >> ? > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Tue Nov 3 10:07:28 2015 From: david.holmes at oracle.com (David Holmes) Date: Tue, 3 Nov 2015 20:07:28 +1000 Subject: RFR(M): 8140482: Various minor code improvements (runtime) In-Reply-To: <4295855A5C1DE049A61835A1887419CC41EB1746@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC41E9F0AA@DEWDFEMB12A.global.corp.sap> <563840B0.10401@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB1746@DEWDFEMB12A.global.corp.sap> Message-ID: <56388760.60800@oracle.com> Hi Goetz, Quick follow up on a couple of things ... On 3/11/2015 7:33 PM, Lindenmaier, Goetz wrote: > Hi David, > >> Sorry, lots of folks very busy at the moment as we try to get features >> finalized before the deadline. > thanks for looking at this! I know the Dec. 10 deadline, but I guess that also > holds for us ... at least J1 is over now. (Unfortunately we could not attend > this year.) Me neither :) >>> ps_core.c: >>> Pread not necessarily terminates interp_name which is printed thereafter. >>> Increase buffer size by 1 and add '\0'. >> >> Given: >> #define BUF_SIZE (PATH_MAX + NAME_MAX + 1) >> isn't it impossible to encounter that problem? > As I understand, pread does not null-terminate what it read. So the > null must come from the file. This protects against a corrupted file. So are you saying the nul is not present in the file? I'm not familiar with the ELF format. >>> stubRoutines_x86.cpp: >>> Cast to proper type. This way, left and right of '&' have the same type. >> >> I think you could just have changed the uint64_t to uint32_t as applied >> to the shift rather than casting the 1 to uint_32t. End result is the >> same though. > What you propose did not work. It was my first fix, too. Hmm okay. The result of the shift must be an unsigned type and the constant 1 is signed, so needs the cast (or use the unsigned constant form - 1ud? ) >>> attachListener_linux.cpp: >>> Read does not terminate buf. Size for '\0' is already considered. >> >> Looks a little odd being done on each iteration, but okay I guess. > I'll try to move it out of the loop. Better: I'll check whether the > scan groks it if I move it out of the loop :) > >>> os_linux.cpp: >>> Array sigflags[] has size MAXSIGNUM==32. _NSIG is bigger than >>> MAXSIGNUM (_NSIG == 65 on my machine). >>> sig is checked to be smaller than _NSIG. Later, in set_our_sigflags(), >>> sig is used to access sigflags[MAXSIGNUM] which can overflow the array. >>> Should we also increase MAXSIGNUM? >> >> Need to let the SR folk comment here as something definitely seems >> wrong, but I'm not 100% sure the what the correct answer is. If >> _JAVA_SR_SIGNUM is too big it should be validated somewhere and an error >> or warning reported. > I'm also not sure how to best handle this. Might even require a fix > exceeding this change. But I think this is the best finding. > >>> codeBuffer.cpp: >>> New_capacity is not initialized. Figure_expanded_capacities() handles this >>> correctly, but initializing this is cheap and safe. >> >> Hmmm ... I hate redundancy - this is pure wasted cycles. If we had to do >> it would memset not be better? Or would the code-checker not realize >> what memset was doing? > I guess it would work with memset, too. But I thought the 3-deep loop > will be unrolled completely so that only three stores remain. I tend not to try and imagine what the compiler may or may not do. Happy to take other opinions. Though again I'd prefer if the checker could be shown that there is no missing initialization. >>> dict.cpp: >>> If j-- is executed for j==0, the loop aborts because j is unsigned (0-- >= b- >>> _cnt). >>> Instead, only do j++ if necessary. >> >> Not at all obvious to me that it is possible to do j-- when j==0, but >> the change seems reasonable. > Yes, the scan does not understand there is j++ right after j-- because > of the loop iteration. I saw it complaining about this pattern several times. > >> Lots of spacing changes in that code make it hard to see the real changes. > Before, I was asked to fix indentation issues in a function I touch. > Does that only hold for compiler files? Yes/no/maybe :) Fixing up bad formatting when you are touching an area can be convenient, however it can also obscure the real changes, so it depends on the ratio of functional changes to format changes. >> 145 // SAPJVM GL j-- can underflow, which will cause the loop to abort. >> Seems unnecessary with the code change as noone will understand what j-- >> you are referring to. > Didn't mean to leave this in here. Removed. > >> 150 nb->_keyvals[nbcnt + nbcnt ] = key; >> 151 nb->_keyvals[nbcnt + nbcnt + 1] = b->_keyvals[j+j+1]; >> hotspot-style doesn't align array index expressions like that. Ditto >> 154/155. > Fixed. > >>> generateOopMap.cpp: >>> Idx is read from String. This is only called with constant strings, so compare >>> should be folded away by optimizing compilers if inlined. >> >> Not a fan of adding conditions that should never be false (hence the >> assert) and relying on the compiler to elide them. > OK, removed. > >>> deoptimization.cpp: >>> If buflen == 0, buf[-1] is accessed. >> >> Okay - but an assert(buflen>0) would be better I think as we should >> never be calling with a zero-length buffer. > Ok, I added the assert. As this isn't critical code, I would like to leave the > check in there, still. > >>> task.cpp: >>> Fatal can return if -XX:SuppressErrorAt is used. Just don't access the >>> array in this case. >> >> Okay. I would not be surprised if we have a lot of potential errors if a >> fatal/guarantee failure is suppressed. >> >>> attachListener.hpp: >>> Do strncpy to not overflow buffer. Don't write more chars than before. >> >> Again we have the assert to catch an error in the caller using an >> invalid name. > Hmm, the command comes from outside of the VM. It's not checked > very thoroughly, see, e.g., attachListener_windows.cpp:194. Arg0 is > checked twice, arg1 and arg2 are not checked at all. The libattach code is still part of our codebase so should be doing the right things. The linux and solaris code seems to be doing the expected name length check. On Windows the name is set using cmd, which is also subject to a length check: if (strlen(cmd) > AttachOperation::name_length_max) return ATTACH_ERROR_ILLEGALARG; > I add fixes for attachListener_windows.cpp to this change. > >>> heapDumper.cpp: >>> strncpy does not null terminate. >> >> 1973 if (total_length >= sizeof(base_path)) { >> >> total_length already adds +1 for the nul character so the == case is >> fine AFAICS. >> >> strncpy wont nul-terminate if the string exceeds the buffer size. But we >> have already established that total_length <= sizeof(base_path), and >> total_path includes space for a bunch of stuff other than HeapDumpPath, >> so the strncpy of HeapDumpPath has to copy the nul character. > Ok, removed. > >> > src/share/vm/services/memoryService.cpp >> >> Ok. >> >> > src/share/vm/utilities/xmlstream.cpp >> >> Ok - I'm more concerned about the "magic" 10 in that piece of code. > I assume the 10 is the space needed for the "_done" plus some waste ... > > I'll do another run of the scan. That takes a day. I'll post a new webrev after > that. Thanks, David > Thank again for this thorough review, > Goetz > > > > >>> Some of these, as the issue in codeBuffer.cpp, are actually handled correctly. >>> Nevertheless this is not that obvious so that somebody changing the code >>> Could oversee he has to add the initialization. >> >> Not an argument I buy en-masse as it leads to a lot of redundancy >> through the code paths. Plus these tools that are being run should show >> if a code changes requires initialization that is not present. >> >> Thanks, >> David >> >>> Some of these fixes are part of SAP JVM for a long time. This change has >>> been tested with our nightly build of openJDK. >>> >>> Best regards, >>> Goetz,. >>> From david.holmes at oracle.com Tue Nov 3 10:10:16 2015 From: david.holmes at oracle.com (David Holmes) Date: Tue, 3 Nov 2015 20:10:16 +1000 Subject: RFR(M): 8140482: Various minor code improvements (runtime) In-Reply-To: <56388760.60800@oracle.com> References: <4295855A5C1DE049A61835A1887419CC41E9F0AA@DEWDFEMB12A.global.corp.sap> <563840B0.10401@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB1746@DEWDFEMB12A.global.corp.sap> <56388760.60800@oracle.com> Message-ID: <56388808.7090109@oracle.com> Oops see what you mean about attachListener_windows.cpp now: if (strlen(arg0) > AttachOperation::arg_length_max) return ATTACH_ERROR_ILLEGALARG; if (strlen(arg0) > AttachOperation::arg_length_max) return ATTACH_ERROR_ILLEGALARG; Should be arg0, arg1 and arg2. David On 3/11/2015 8:07 PM, David Holmes wrote: > Hi Goetz, > > Quick follow up on a couple of things ... > > On 3/11/2015 7:33 PM, Lindenmaier, Goetz wrote: >> Hi David, >> >>> Sorry, lots of folks very busy at the moment as we try to get features >>> finalized before the deadline. >> thanks for looking at this! I know the Dec. 10 deadline, but I guess >> that also >> holds for us ... at least J1 is over now. (Unfortunately we could not >> attend >> this year.) > > Me neither :) > >>>> ps_core.c: >>>> Pread not necessarily terminates interp_name which is printed >>>> thereafter. >>>> Increase buffer size by 1 and add '\0'. >>> >>> Given: >>> #define BUF_SIZE (PATH_MAX + NAME_MAX + 1) >>> isn't it impossible to encounter that problem? >> As I understand, pread does not null-terminate what it read. So the >> null must come from the file. This protects against a corrupted file. > > So are you saying the nul is not present in the file? I'm not familiar > with the ELF format. > >>>> stubRoutines_x86.cpp: >>>> Cast to proper type. This way, left and right of '&' have the same >>>> type. >>> >>> I think you could just have changed the uint64_t to uint32_t as applied >>> to the shift rather than casting the 1 to uint_32t. End result is the >>> same though. >> What you propose did not work. It was my first fix, too. > > Hmm okay. The result of the shift must be an unsigned type and the > constant 1 is signed, so needs the cast (or use the unsigned constant > form - 1ud? ) > >>>> attachListener_linux.cpp: >>>> Read does not terminate buf. Size for '\0' is already considered. >>> >>> Looks a little odd being done on each iteration, but okay I guess. >> I'll try to move it out of the loop. Better: I'll check whether the >> scan groks it if I move it out of the loop :) >> >>>> os_linux.cpp: >>>> Array sigflags[] has size MAXSIGNUM==32. _NSIG is bigger than >>>> MAXSIGNUM (_NSIG == 65 on my machine). >>>> sig is checked to be smaller than _NSIG. Later, in set_our_sigflags(), >>>> sig is used to access sigflags[MAXSIGNUM] which can overflow the array. >>>> Should we also increase MAXSIGNUM? >>> >>> Need to let the SR folk comment here as something definitely seems >>> wrong, but I'm not 100% sure the what the correct answer is. If >>> _JAVA_SR_SIGNUM is too big it should be validated somewhere and an error >>> or warning reported. >> I'm also not sure how to best handle this. Might even require a fix >> exceeding this change. But I think this is the best finding. >> >>>> codeBuffer.cpp: >>>> New_capacity is not initialized. Figure_expanded_capacities() >>>> handles this >>>> correctly, but initializing this is cheap and safe. >>> >>> Hmmm ... I hate redundancy - this is pure wasted cycles. If we had to do >>> it would memset not be better? Or would the code-checker not realize >>> what memset was doing? >> I guess it would work with memset, too. But I thought the 3-deep loop >> will be unrolled completely so that only three stores remain. > > I tend not to try and imagine what the compiler may or may not do. Happy > to take other opinions. Though again I'd prefer if the checker could be > shown that there is no missing initialization. > >>>> dict.cpp: >>>> If j-- is executed for j==0, the loop aborts because j is unsigned >>>> (0-- >= b- >>>> _cnt). >>>> Instead, only do j++ if necessary. >>> >>> Not at all obvious to me that it is possible to do j-- when j==0, but >>> the change seems reasonable. >> Yes, the scan does not understand there is j++ right after j-- because >> of the loop iteration. I saw it complaining about this pattern >> several times. >> >>> Lots of spacing changes in that code make it hard to see the real >>> changes. >> Before, I was asked to fix indentation issues in a function I touch. >> Does that only hold for compiler files? > > Yes/no/maybe :) Fixing up bad formatting when you are touching an area > can be convenient, however it can also obscure the real changes, so it > depends on the ratio of functional changes to format changes. > >>> 145 // SAPJVM GL j-- can underflow, which will cause the loop to >>> abort. >>> Seems unnecessary with the code change as noone will understand what j-- >>> you are referring to. >> Didn't mean to leave this in here. Removed. >> >>> 150 nb->_keyvals[nbcnt + nbcnt ] = key; >>> 151 nb->_keyvals[nbcnt + nbcnt + 1] = b->_keyvals[j+j+1]; >>> hotspot-style doesn't align array index expressions like that. Ditto >>> 154/155. >> Fixed. >> >>>> generateOopMap.cpp: >>>> Idx is read from String. This is only called with constant strings, >>>> so compare >>>> should be folded away by optimizing compilers if inlined. >>> >>> Not a fan of adding conditions that should never be false (hence the >>> assert) and relying on the compiler to elide them. >> OK, removed. >> >>>> deoptimization.cpp: >>>> If buflen == 0, buf[-1] is accessed. >>> >>> Okay - but an assert(buflen>0) would be better I think as we should >>> never be calling with a zero-length buffer. >> Ok, I added the assert. As this isn't critical code, I would like to >> leave the >> check in there, still. >> >>>> task.cpp: >>>> Fatal can return if -XX:SuppressErrorAt is used. Just don't access the >>>> array in this case. >>> >>> Okay. I would not be surprised if we have a lot of potential errors if a >>> fatal/guarantee failure is suppressed. >>> >>>> attachListener.hpp: >>>> Do strncpy to not overflow buffer. Don't write more chars than before. >>> >>> Again we have the assert to catch an error in the caller using an >>> invalid name. >> Hmm, the command comes from outside of the VM. It's not checked >> very thoroughly, see, e.g., attachListener_windows.cpp:194. Arg0 is >> checked twice, arg1 and arg2 are not checked at all. > > The libattach code is still part of our codebase so should be doing the > right things. The linux and solaris code seems to be doing the expected > name length check. On Windows the name is set using cmd, which is also > subject to a length check: > > if (strlen(cmd) > AttachOperation::name_length_max) return > ATTACH_ERROR_ILLEGALARG; > >> I add fixes for attachListener_windows.cpp to this change. >> >>>> heapDumper.cpp: >>>> strncpy does not null terminate. >>> >>> 1973 if (total_length >= sizeof(base_path)) { >>> >>> total_length already adds +1 for the nul character so the == case is >>> fine AFAICS. >>> >>> strncpy wont nul-terminate if the string exceeds the buffer size. But we >>> have already established that total_length <= sizeof(base_path), and >>> total_path includes space for a bunch of stuff other than HeapDumpPath, >>> so the strncpy of HeapDumpPath has to copy the nul character. >> Ok, removed. >> >>> > src/share/vm/services/memoryService.cpp >>> >>> Ok. >>> >>> > src/share/vm/utilities/xmlstream.cpp >>> >>> Ok - I'm more concerned about the "magic" 10 in that piece of code. >> I assume the 10 is the space needed for the "_done" plus some waste ... >> >> I'll do another run of the scan. That takes a day. I'll post a new >> webrev after >> that. > > Thanks, > David > >> Thank again for this thorough review, >> Goetz >> >> >> >> >>>> Some of these, as the issue in codeBuffer.cpp, are actually handled >>>> correctly. >>>> Nevertheless this is not that obvious so that somebody changing the >>>> code >>>> Could oversee he has to add the initialization. >>> >>> Not an argument I buy en-masse as it leads to a lot of redundancy >>> through the code paths. Plus these tools that are being run should show >>> if a code changes requires initialization that is not present. >>> >>> Thanks, >>> David >>> >>>> Some of these fixes are part of SAP JVM for a long time. This >>>> change has >>>> been tested with our nightly build of openJDK. >>>> >>>> Best regards, >>>> Goetz,. >>>> From andreas.eriksson at oracle.com Tue Nov 3 11:10:42 2015 From: andreas.eriksson at oracle.com (Andreas Eriksson) Date: Tue, 3 Nov 2015 12:10:42 +0100 Subject: RFR: JDK-8134030: test/serviceability/dcmd/gc/HeapDumpTest fails to verify the dump In-Reply-To: <563853E9.6090901@oracle.com> References: <563378B1.6080607@oracle.com> <563853E9.6090901@oracle.com> Message-ID: <56389632.3080805@oracle.com> On 2015-11-03 07:27, David Holmes wrote: > Hi Andreas, > > On 31/10/2015 12:03 AM, Andreas Eriksson wrote: >> Hi, >> >> Please review this change to heap dump logic. >> Including runtime list, since this is related to class loading. >> >> Bug: >> JDK-8134030: test/serviceability/dcmd/gc/HeapDumpTest fails to verify >> the dump >> https://bugs.openjdk.java.net/browse/JDK-8134030 >> >> Webrev: http://cr.openjdk.java.net/~aeriksso/8134030/webrev.00/ >> >> A heap dump can occur at a point where an InstanceKlass doesn't have a >> Java class mirror yet. >> To avoid a crash because of this JDK-8131600 [1] excluded classes that >> are not linked yet from the dump. >> >> The problem here is that since a class that has been loaded is already >> exposed to the Java side, another class can have a reference to the >> excluded class. >> So, if it has been *loaded* but not *linked* it will be excluded, but >> can still be referenced by other classes in the dump. > > I'm having trouble seeing exactly when the class is marked as "loaded" > but it certainly seems to be after the mirror has been set. However > linking happens even later than that - so if you still see a crash > excluding linked classes then I fear you will also see problems > excluding loaded classes! > > David > ----- > Hi, The crash does not happen when excluding linked classes, sorry if that was unclear. Let me try to explain it again: The crash was fixed by excluding non-linked classes (in JDK-8131600). But that excludes some classes that are already referenced in the heap dump by other classes. By changing to excluding non-loaded classes, the crash can still not happen, and all classes that are referenced in the dump are also included. Not dumping these classes is only a problem when we try to look at the heap dump at a later point (and even then it is only a problem of missing information, not a crash). For example, HeapDumpTest gives the following warning when it tries to follow a reference to a class that was excluded from the dump: WARNING: Failed to resolve object id 0x6c003d348 [...]. VisualVM on the other hand will ignore the value, and tell the user the field was null (which is very misleading). I hope that made sense. Thanks, Andreas > >> This gives the following warning in HeapDumpTest: >> WARNING: Failed to resolve object id 0x6c003d348 [...]. >> >> This fix changes heapDumper.cpp to only exclude classes that are not yet >> loaded instead. >> >> I'd like confirmation on this from someone who knows more about loaded >> vs linked, but I think that this is correct: >> When a class has been loaded, we are guaranteed that a Java class mirror >> is setup, so the crash seen in JDK-8131600 still cannot happen. >> This seems to be confirmed by the manual reproducer from JDK-8131600, >> which still succeeds with this fix. >> >> Also in this change is the re-addition of the actual heap dump >> verification call in HeapDumpTest, which was accidentally removed when >> the test was re-factored. >> >> Regards, >> Andreas >> >> [1]: https://bugs.openjdk.java.net/browse/JDK-8131600 From goetz.lindenmaier at sap.com Tue Nov 3 14:10:13 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 3 Nov 2015 14:10:13 +0000 Subject: RFR(M): 8140482: Various minor code improvements (runtime) In-Reply-To: <56388760.60800@oracle.com> References: <4295855A5C1DE049A61835A1887419CC41E9F0AA@DEWDFEMB12A.global.corp.sap> <563840B0.10401@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB1746@DEWDFEMB12A.global.corp.sap> <56388760.60800@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC41EB1875@DEWDFEMB12A.global.corp.sap> Hi David, the new scan is already through. I made a new webrev: http://cr.openjdk.java.net/~goetz/webrevs/8140482-covRt/webrev.01/ attachListener_linux.cpp: I had moved the string termination out of the loop, and read one less char from the file. The scan still claims "Passing unterminated string buf to strlen" in line 274. So I will undo it again for the webrev. codeBuffer.cpp Doing memset is fine. I'll use memset(). > >>> ps_core.c: > >>> Pread not necessarily terminates interp_name which is printed > >>> thereafter. Increase buffer size by 1 and add '\0'. > >> > >> Given: > >> #define BUF_SIZE (PATH_MAX + NAME_MAX + 1) > >> isn't it impossible to encounter that problem? > > As I understand, pread does not null-terminate what it read. So the > > null must come from the file. This protects against a corrupted file. > > So are you saying the nul is not present in the file? I'm not familiar > with the ELF format. There should be a nul in the file, else the code would fail more obviously. The problem is if the file is corrupted. Best regards, Goetz. > -----Original Message----- > From: David Holmes [mailto:david.holmes at oracle.com] > Sent: Dienstag, 3. November 2015 11:07 > To: Lindenmaier, Goetz; hotspot-runtime-dev at openjdk.java.net; > serviceability-dev > Subject: Re: RFR(M): 8140482: Various minor code improvements (runtime) > > Hi Goetz, > > Quick follow up on a couple of things ... > > On 3/11/2015 7:33 PM, Lindenmaier, Goetz wrote: > > Hi David, > > > >> Sorry, lots of folks very busy at the moment as we try to get features > >> finalized before the deadline. > > thanks for looking at this! I know the Dec. 10 deadline, but I guess that also > > holds for us ... at least J1 is over now. (Unfortunately we could not attend > > this year.) > > Me neither :) > > >>> ps_core.c: > >>> Pread not necessarily terminates interp_name which is printed > thereafter. > >>> Increase buffer size by 1 and add '\0'. > >> > >> Given: > >> #define BUF_SIZE (PATH_MAX + NAME_MAX + 1) > >> isn't it impossible to encounter that problem? > > As I understand, pread does not null-terminate what it read. So the > > null must come from the file. This protects against a corrupted file. > > So are you saying the nul is not present in the file? I'm not familiar > with the ELF format. > > >>> stubRoutines_x86.cpp: > >>> Cast to proper type. This way, left and right of '&' have the same type. > >> > >> I think you could just have changed the uint64_t to uint32_t as applied > >> to the shift rather than casting the 1 to uint_32t. End result is the > >> same though. > > What you propose did not work. It was my first fix, too. > > Hmm okay. The result of the shift must be an unsigned type and the > constant 1 is signed, so needs the cast (or use the unsigned constant > form - 1ud? ) > > >>> attachListener_linux.cpp: > >>> Read does not terminate buf. Size for '\0' is already considered. > >> > >> Looks a little odd being done on each iteration, but okay I guess. > > I'll try to move it out of the loop. Better: I'll check whether the > > scan groks it if I move it out of the loop :) > > > >>> os_linux.cpp: > >>> Array sigflags[] has size MAXSIGNUM==32. _NSIG is bigger than > >>> MAXSIGNUM (_NSIG == 65 on my machine). > >>> sig is checked to be smaller than _NSIG. Later, in set_our_sigflags(), > >>> sig is used to access sigflags[MAXSIGNUM] which can overflow the array. > >>> Should we also increase MAXSIGNUM? > >> > >> Need to let the SR folk comment here as something definitely seems > >> wrong, but I'm not 100% sure the what the correct answer is. If > >> _JAVA_SR_SIGNUM is too big it should be validated somewhere and an > error > >> or warning reported. > > I'm also not sure how to best handle this. Might even require a fix > > exceeding this change. But I think this is the best finding. > > > >>> codeBuffer.cpp: > >>> New_capacity is not initialized. Figure_expanded_capacities() handles > this > >>> correctly, but initializing this is cheap and safe. > >> > >> Hmmm ... I hate redundancy - this is pure wasted cycles. If we had to do > >> it would memset not be better? Or would the code-checker not realize > >> what memset was doing? > > I guess it would work with memset, too. But I thought the 3-deep loop > > will be unrolled completely so that only three stores remain. > > I tend not to try and imagine what the compiler may or may not do. Happy > to take other opinions. Though again I'd prefer if the checker could be > shown that there is no missing initialization. > > >>> dict.cpp: > >>> If j-- is executed for j==0, the loop aborts because j is unsigned (0-- >= b- > >>> _cnt). > >>> Instead, only do j++ if necessary. > >> > >> Not at all obvious to me that it is possible to do j-- when j==0, but > >> the change seems reasonable. > > Yes, the scan does not understand there is j++ right after j-- because > > of the loop iteration. I saw it complaining about this pattern several times. > > > >> Lots of spacing changes in that code make it hard to see the real changes. > > Before, I was asked to fix indentation issues in a function I touch. > > Does that only hold for compiler files? > > Yes/no/maybe :) Fixing up bad formatting when you are touching an area > can be convenient, however it can also obscure the real changes, so it > depends on the ratio of functional changes to format changes. > > >> 145 // SAPJVM GL j-- can underflow, which will cause the loop to abort. > >> Seems unnecessary with the code change as noone will understand what > j-- > >> you are referring to. > > Didn't mean to leave this in here. Removed. > > > >> 150 nb->_keyvals[nbcnt + nbcnt ] = key; > >> 151 nb->_keyvals[nbcnt + nbcnt + 1] = b->_keyvals[j+j+1]; > >> hotspot-style doesn't align array index expressions like that. Ditto > >> 154/155. > > Fixed. > > > >>> generateOopMap.cpp: > >>> Idx is read from String. This is only called with constant strings, so > compare > >>> should be folded away by optimizing compilers if inlined. > >> > >> Not a fan of adding conditions that should never be false (hence the > >> assert) and relying on the compiler to elide them. > > OK, removed. > > > >>> deoptimization.cpp: > >>> If buflen == 0, buf[-1] is accessed. > >> > >> Okay - but an assert(buflen>0) would be better I think as we should > >> never be calling with a zero-length buffer. > > Ok, I added the assert. As this isn't critical code, I would like to leave the > > check in there, still. > > > >>> task.cpp: > >>> Fatal can return if -XX:SuppressErrorAt is used. Just don't access the > >>> array in this case. > >> > >> Okay. I would not be surprised if we have a lot of potential errors if a > >> fatal/guarantee failure is suppressed. > >> > >>> attachListener.hpp: > >>> Do strncpy to not overflow buffer. Don't write more chars than before. > >> > >> Again we have the assert to catch an error in the caller using an > >> invalid name. > > Hmm, the command comes from outside of the VM. It's not checked > > very thoroughly, see, e.g., attachListener_windows.cpp:194. Arg0 is > > checked twice, arg1 and arg2 are not checked at all. > > The libattach code is still part of our codebase so should be doing the > right things. The linux and solaris code seems to be doing the expected > name length check. On Windows the name is set using cmd, which is also > subject to a length check: > > if (strlen(cmd) > AttachOperation::name_length_max) return > ATTACH_ERROR_ILLEGALARG; > > > I add fixes for attachListener_windows.cpp to this change. > > > >>> heapDumper.cpp: > >>> strncpy does not null terminate. > >> > >> 1973 if (total_length >= sizeof(base_path)) { > >> > >> total_length already adds +1 for the nul character so the == case is > >> fine AFAICS. > >> > >> strncpy wont nul-terminate if the string exceeds the buffer size. But we > >> have already established that total_length <= sizeof(base_path), and > >> total_path includes space for a bunch of stuff other than HeapDumpPath, > >> so the strncpy of HeapDumpPath has to copy the nul character. > > Ok, removed. > > > >> > src/share/vm/services/memoryService.cpp > >> > >> Ok. > >> > >> > src/share/vm/utilities/xmlstream.cpp > >> > >> Ok - I'm more concerned about the "magic" 10 in that piece of code. > > I assume the 10 is the space needed for the "_done" plus some waste ... > > > > I'll do another run of the scan. That takes a day. I'll post a new webrev > after > > that. > > Thanks, > David > > > Thank again for this thorough review, > > Goetz > > > > > > > > > >>> Some of these, as the issue in codeBuffer.cpp, are actually handled > correctly. > >>> Nevertheless this is not that obvious so that somebody changing the > code > >>> Could oversee he has to add the initialization. > >> > >> Not an argument I buy en-masse as it leads to a lot of redundancy > >> through the code paths. Plus these tools that are being run should show > >> if a code changes requires initialization that is not present. > >> > >> Thanks, > >> David > >> > >>> Some of these fixes are part of SAP JVM for a long time. This change has > >>> been tested with our nightly build of openJDK. > >>> > >>> Best regards, > >>> Goetz,. > >>> From yasuenag at gmail.com Tue Nov 3 14:45:15 2015 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Tue, 3 Nov 2015 23:45:15 +0900 Subject: RFR: JDK-8140556: Add force rotation option to VM.log jcmd In-Reply-To: <56384824.4080904@oracle.com> References: <562101D8.5080909@gmail.com> <562DF8D4.40501@oracle.com> <562EBF4B.4020607@gmail.com> <562F6B46.5050500@oracle.com> <563226AB.2030206@gmail.com> <5632341F.4060400@oracle.com> <56338125.9030802@gmail.com> <56338D58.9090709@oracle.com> <5634D9FC.6080502@gmail.com> <5636C66A.6020304@oracle.com> <563765FC.2090804@gmail.com> <56384824.4080904@oracle.com> Message-ID: <5638C87B.7080206@gmail.com> David, Marcus, > I'm unclear of the context in which this could be called by an unattached thread (ie during early VM initialization). Could you clarify the code-path for me please? It seems an odd check to make on all rotate() requests rather then doing something only in the problematic case. I changed a little current implementation and built fastdebug. Changes: ------------------------ diff -r 4338b5465f50 src/share/vm/logging/logFileOutput.cpp --- a/src/share/vm/logging/logFileOutput.cpp Fri Oct 30 12:36:54 2015 +0100 +++ b/src/share/vm/logging/logFileOutput.cpp Tue Nov 03 23:34:33 2015 +0900 @@ -123,7 +123,8 @@ success = false; break; } - _rotate_size = value * K; + //_rotate_size = value * K; + _rotate_size = value; } else { success = false; break; @@ -155,12 +156,12 @@ int written = LogFileStreamOutput::write(decorations, msg); _current_size += written; - if (should_rotate()) { +// if (should_rotate()) { MutexLockerEx ml(&_rotation_lock, true /* no safepoint check */); if (should_rotate()) { rotate(); } - } +// } return written; } ------------------------ Results: - from hs_err log ------------------------ # A fatal error has been detected by the Java Runtime Environment: # # Internal Error (/home/ysuenaga/OpenJDK/hs-rt/hotspot/src/share/vm/runtime/thread.hpp:669), pid=61982, tid=61983 # assert(thread != __null) failed: just checking : Command Line: -Xlog:all=trace:file.txt::filesize=10,filecount=2 -Xlog:invalid : V [libjvm.so+0xeb6481] Monitor::lock_without_safepoint_check()+0xe1 V [libjvm.so+0xd4c118] LogFileOutput::write(LogDecorations const&, char const*)+0x38 V [libjvm.so+0xd4e5d0] LogTagSet::log(LogLevel::type, char const*)+0x60 V [libjvm.so+0xd499b7] void Log<(LogTag::type)2, (LogTag::type)0, (LogTag::type)0, (LogTag::type)0, (LogTag::type)0, (LogTag::type)0>::vwrite<(LogLevel::type)6>(char const*, __va_list_tag*)+0x67 V [libjvm.so+0xd49a5b] void Log<(LogTag::type)2, (LogTag::type)0, (LogTag::type)0, (LogTag::type)0, (LogTag::type)0, (LogTag::type)0>::write<(LogLevel::type)6>(char const*, ...)+0x8b V [libjvm.so+0xd49676] LogConfiguration::parse_command_line_arguments(char const*)+0x146 V [libjvm.so+0x489530] Arguments::parse_each_vm_init_arg(JavaVMInitArgs const*, SysClassPath*, bool*, Flag::Flags)+0x1d40 V [libjvm.so+0x48a354] Arguments::parse_vm_init_args(JavaVMInitArgs const*, JavaVMInitArgs const*, JavaVMInitArgs const*)+0x244 V [libjvm.so+0x48c286] Arguments::parse(JavaVMInitArgs const*)+0xa46 V [libjvm.so+0x1173052] Threads::create_vm(JavaVMInitArgs*, bool*)+0xb2 V [libjvm.so+0xb502af] JNI_CreateJavaVM+0x8f C [libjli.so+0x72a3] JavaMain+0x83 ------------------------ LogConfiguration::parse_command_line_arguments() calls before initialization of main thread. So I think that Thread::current() returns NULL and crash at assert(). Thanks, Yasumasa On 2015/11/03 14:37, David Holmes wrote: > On 2/11/2015 11:32 PM, Yasumasa Suenaga wrote: >> Marcus, David, >> >> I've uploaded new webrev: >> http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.02/ >> >> Could you review it? >> >> >>>>>>> Still missing the renaming of rotate_all_logfile to >>>>>>> rotate_all_outputs. >> >> I've fixed. >> >> >>>>> Small typo, please make that is_rotatable(). >> >> I've added, and have used at LogConfiguration::rotate_all_outputs(). >> >> >>> You can't safely lock a mutex until the current thread has attached to >>> the VM and Thread::current() returns non-null (that is what >>> Thread::initialize_thread_local_storage() currently does). You may get >>> away with it, particularly in product builds, during VM >>> initialization, as the fast-lock path doesn't require access to >>> Thread::current(). >> >> I will check whether Thread::current() is NULL or not at >> LogFileOutput::is_rotatable() >> is_rotatable() is called at LogFileOutput::rotate() before locking >> _rotation_lock. >> So we can avoid crash/assert which is related to TLS. > > I'm unclear of the context in which this could be called by an unattached thread (ie during early VM initialization). Could you clarify the code-path for me please? It seems an odd check to make on all rotate() requests rather then doing something only in the problematic case. > > Note I am not reviewing this in general as I am not familiar with the logging code, I am just commenting on the thread-safety issue. > > Thanks, > David > >> >> Thanks, >> >> Yasumasa >> >> >> On 2015/11/02 11:11, David Holmes wrote: >>> On 1/11/2015 1:10 AM, Yasumasa Suenaga wrote: >>>>> This in turn means that we are logging before the thread local storage >>>>> has been initialized, and the JVM will crash/hit the assert because we >>>>> are trying to take the rotation lock. >>>> >>>> I guess that you are saying about class member (not TLS - e.g. >>>> pthread_setspecific()). >>> >>> You can't safely lock a mutex until the current thread has attached to >>> the VM and Thread::current() returns non-null (that is what >>> Thread::initialize_thread_local_storage() currently does). You may get >>> away with it, particularly in product builds, during VM >>> initialization, as the fast-lock path doesn't require access to >>> Thread::current(). >>> >>> David >>> >>>> Most of fields (include _rotation_lock) in LogFileOutput are initialized >>>> at c'tor. >>>> Other field (_stream) is initialized at LogFileOutput::initialize(). >>>> >>>> We can avoid problem if we can move initialize() to c'tor because >>>> LogFileOutput >>>> will be instantiated at LogConfiguration::new_output(). >>>> >>>> Currently, LogFileOutput will be added to LogConfiguration::_outputs and >>>> to LogTagSet after calling LogFileOutput::initialize(). >>>> I wonder why we cannot avoid crash/assert with current implementation... >>>> >>>> >>>> I will keep current implementation about _rotation_lock if the above is >>>> incorrect. >>>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> On 2015/10/31 0:31, Marcus Larsson wrote: >>>>> Hey, >>>>> >>>>> On 2015-10-30 15:39, Yasumasa Suenaga wrote: >>>>>> Hi Marcus, >>>>>> >>>>>> Thank you for your comment. >>>>>> >>>>>> >>>>>>> Still missing the renaming of rotate_all_logfile to >>>>>>> rotate_all_outputs. >>>>>> >>>>>> Sorry, I will fix. >>>>>> >>>>> >>>>> Thanks! >>>>> >>>>>> >>>>>>> This rotate function works I guess, but it will probably need some >>>>>>> rework once there are other types of log outputs (rotate doesn't >>>>>>> make sense on all types of log outputs). >>>>>> >>>>>> I will add is_rotetable() to LogOutput as virtual function. >>>>>> This function return false (can not rotate) by default, return true >>>>>> if _file_count is >>>>>> greater than 0 in LogFileOutput. >>>>>> >>>>>> LogConfiguration::rotate_all_outputs() will rotate if this function >>>>>> returns true. >>>>>> >>>>> >>>>> Small typo, please make that is_rotatable(). >>>>> >>>>>> >>>>>>> If we configure a rotated file output and then log something on that >>>>>>> output before thread local storage initialization is complete, we >>>>>>> will crash/hit an assert. The previous implementation avoided this >>>>>>> as long as no rotation was needed before this initialization was >>>>>>> complete. (This can be triggered using the following arguments >>>>>>> "-Xlog:all=trace:file.txt::filesize=10,filecount=2 -Xlog:invalid", >>>>>>> for example.) >>>>>> >>>>>> I do not think so. >>>>>> Each -Xlog option creates unique instance of LogOutput (and subclass). >>>>>> So they are isolated. >>>>> >>>>> My example includes a second (and intentionally incorrect) -Xlog >>>>> option only to trigger logging on the newly configured rotating >>>>> LogFileOutput. This will cause some logging about the invalid -Xlog >>>>> argument, and this logging happens during the argument parsing. This >>>>> in turn means that we are logging before the thread local storage has >>>>> been initialized, and the JVM will crash/hit the assert because we are >>>>> trying to take the rotation lock. This has nothing to do with >>>>> concurrent fprintfs or freopens, even if that is a problem. >>>>> >>>>>> >>>>>> However, we might have to rotate at safepoint. >>>>> >>>>> Doing stuff at a safepoint will not give us much in this case. There >>>>> are threads that continue execution through a safepoint, which means >>>>> we will see log calls being made even during this time. >>>>> >>>>>> Currently, LogOutput might be used by multiple threads concurrently. >>>>>> If we rotate log when another thread is writing string to output, we >>>>>> encounter unexpected behavior. >>>>>> >>>>>> LogFileOutput::rotate() uses freopen(). >>>>>> LogFileStreamOutput::write() uses vfprintf() through jio_fprintf(). >>>>>> freopen() and vfprintf() are not atomic. >>>>> >>>>> You're right. There seems to be an overlooked concurrency issue with >>>>> vfprintfs during freopen. I don't think we should resolve this as part >>>>> of this particular fix, so instead I'll create a separate issue for it. >>>>> >>>>>> >>>>>> I think that cause of crash/assertion which you say is it. >>>>>> (GC log will be rotated at safepoint.) >>>>>> >>>>> >>>>> See my comment above for what crash/assert I'm talking about. >>>>> >>>>> Thanks, >>>>> Marcus >>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2015/10/29 23:58, Marcus Larsson wrote: >>>>>>> Hi, >>>>>>> >>>>>>> On 2015-10-29 15:01, Yasumasa Suenaga wrote: >>>>>>>> Hi Marcus, >>>>>>>> >>>>>>>> Thank you for your comment. >>>>>>>> >>>>>>>>> I'll sponsor it. >>>>>>>> >>>>>>>> Thank you so much! >>>>>>>> I've uploaded new webrev: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.01/ >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> Still missing the renaming of rotate_all_logfile to >>>>>>> rotate_all_outputs. >>>>>>> >>>>>>>>> logConfiguration.cpp/hpp: >>>>>>>>> >>>>>>>>> * I don't think we should rely on the archive_name or the output's >>>>>>>>> name to decide whether or not an output should be rotated. It >>>>>>>>> would be better to introduce an is_rotated() test function in >>>>>>>>> LogOutput that could be used here. >>>>>>>>> >>>>>>>>> * rotate_all_logfile() should be renamed to rotate_all_outputs(). >>>>>>>>> Currently there are only LogFileOutputs (other than >>>>>>>>> stdout/stderr), but in the future there might be other types of >>>>>>>>> outputs so I prefer having a more general name. >>>>>>>>> Also, please use static_cast instead of the >>>>>>>>> C-style cast. (Additional logic will be required here once more >>>>>>>>> types of log outputs are added, but I don't think we need to worry >>>>>>>>> about this right now.) >>>>>>>>> >>>>>>>>> * Don't print an error if an output is not rotatable, since it's >>>>>>>>> not an error to have some log outputs rotated while others are >>>>>>>>> not. If you really want some traceability here I suggest adding >>>>>>>>> log messages on trace level, tagged with 'logging'. >>>>>>>> >>>>>>>> I added LogOutput::rotate(bool) as virtual function. >>>>>>>> This function works nothing by default, but will rotate all logs at >>>>>>>> LogFileOutput. >>>>>>> >>>>>>> This rotate function works I guess, but it will probably need some >>>>>>> rework once there are other types of log outputs (rotate doesn't >>>>>>> make sense on all types of log outputs). >>>>>>> >>>>>>>> >>>>>>>> If true is passed to this function, all files will be rotated. >>>>>>>> >>>>>>>> >>>>>>>>> logDiagnosticCommand.cpp/hpp: >>>>>>>>> >>>>>>>>> * I think the description could be improved to something like >>>>>>>>> "Lists current log configuration, enables/disables/configures a >>>>>>>>> log output, or disables/rotates all logs." >>>>>>>>> >>>>>>>>> * The rotate option description should clarify that all logs will >>>>>>>>> be rotated ("current log" is too ambiguous). >>>>>>>> >>>>>>>> I've fixed. >>>>>>> >>>>>>> Great, thanks! >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> logFileOutput.cpp/hpp: >>>>>>>>> >>>>>>>>> * Moving the MutexLocker like this introduces a race condition >>>>>>>>> where a log might be rotated multiple times by different threads, >>>>>>>>> instead of just once. >>>>>>>>> Instead of making the rotate() function public and moving the >>>>>>>>> mutexlocker, I suggest adding something like a public >>>>>>>>> force_rotation() function that grabs the lock and calls the >>>>>>>>> private rotate(). >>>>>>>>> >>>>>>>>> * Given the addition of is_rotated() in LogOutput, then >>>>>>>>> get_archive_name() should be removed. >>>>>>>> >>>>>>>> I moved MutexLocker and should_rotate() to rotate(). >>>>>>>> I think this change can avoid race condition. >>>>>>>> >>>>>>> >>>>>>> There is a subtle problem with taking the lock to check if we should >>>>>>> rotate. If we configure a rotated file output and then log something >>>>>>> on that output before thread local storage initialization is >>>>>>> complete, we will crash/hit an assert. The previous implementation >>>>>>> avoided this as long as no rotation was needed before this >>>>>>> initialization was complete. (This can be triggered using the >>>>>>> following arguments >>>>>>> "-Xlog:all=trace:file.txt::filesize=10,filecount=2 -Xlog:invalid", >>>>>>> for example.) >>>>>>> >>>>>>> Thanks, >>>>>>> Marcus >>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> On 2015/10/27 21:17, Marcus Larsson wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> On 2015-10-27 01:03, Yasumasa Suenaga wrote: >>>>>>>>>> Hi Marcus, >>>>>>>>>> >>>>>>>>>> Thank you for replying. >>>>>>>>>> >>>>>>>>>> I filed this feature to JBS and change subject of this email. >>>>>>>>>> Could you be a sponsor and review it? >>>>>>>>> >>>>>>>>> I'll sponsor it. >>>>>>>>> >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.00/ >>>>>>>>> >>>>>>>>> logConfiguration.cpp/hpp: >>>>>>>>> >>>>>>>>> * I don't think we should rely on the archive_name or the output's >>>>>>>>> name to decide whether or not an output should be rotated. It >>>>>>>>> would be better to introduce an is_rotated() test function in >>>>>>>>> LogOutput that could be used here. >>>>>>>>> >>>>>>>>> * rotate_all_logfile() should be renamed to rotate_all_outputs(). >>>>>>>>> Currently there are only LogFileOutputs (other than >>>>>>>>> stdout/stderr), but in the future there might be other types of >>>>>>>>> outputs so I prefer having a more general name. >>>>>>>>> Also, please use static_cast instead of the >>>>>>>>> C-style cast. (Additional logic will be required here once more >>>>>>>>> types of log outputs are added, but I don't think we need to worry >>>>>>>>> about this right now.) >>>>>>>>> >>>>>>>>> * Don't print an error if an output is not rotatable, since it's >>>>>>>>> not an error to have some log outputs rotated while others are >>>>>>>>> not. If you really want some traceability here I suggest adding >>>>>>>>> log messages on trace level, tagged with 'logging'. >>>>>>>>> >>>>>>>>> >>>>>>>>> logDiagnosticCommand.cpp/hpp: >>>>>>>>> >>>>>>>>> * I think the description could be improved to something like >>>>>>>>> "Lists current log configuration, enables/disables/configures a >>>>>>>>> log output, or disables/rotates all logs." >>>>>>>>> >>>>>>>>> * The rotate option description should clarify that all logs will >>>>>>>>> be rotated ("current log" is too ambiguous). >>>>>>>>> >>>>>>>>> >>>>>>>>> logFileOutput.cpp/hpp: >>>>>>>>> >>>>>>>>> * Moving the MutexLocker like this introduces a race condition >>>>>>>>> where a log might be rotated multiple times by different threads, >>>>>>>>> instead of just once. >>>>>>>>> Instead of making the rotate() function public and moving the >>>>>>>>> mutexlocker, I suggest adding something like a public >>>>>>>>> force_rotation() function that grabs the lock and calls the >>>>>>>>> private rotate(). >>>>>>>>> >>>>>>>>> * Given the addition of is_rotated() in LogOutput, then >>>>>>>>> get_archive_name() should be removed. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Marcus >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 2015/10/26 18:56, Marcus Larsson wrote: >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> Sorry for my late reply. >>>>>>>>>>> >>>>>>>>>>> I think being able to force rotation via jcmd seems like a good >>>>>>>>>>> feature. Files are currently opened in append mode so it should >>>>>>>>>>> already be possible to use external log rotation tools by >>>>>>>>>>> copying and truncating the files. Still I think it would be nice >>>>>>>>>>> to provide the jcmd for rotation as well. >>>>>>>>>>> >>>>>>>>>>> I can see some small issues with the implementation, but we can >>>>>>>>>>> deal with that during the review. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Marcus >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 2015-10-26 00:26, Yasumasa Suenaga wrote: >>>>>>>>>>>> Hi all, >>>>>>>>>>>> >>>>>>>>>>>> Have you ever seen my change? >>>>>>>>>>>> I (and my customers) need log rotation function via external >>>>>>>>>>>> tool. >>>>>>>>>>>> >>>>>>>>>>>> I want to merge it by Feature Complete. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Yasumasa >>>>>>>>>>>> 2015/10/16 22:55 "Yasumasa Suenaga" : >>>>>>>>>>>> >>>>>>>>>>>>> Hi all, >>>>>>>>>>>>> >>>>>>>>>>>>> I contributed JDK-7090324: gclog rotation via external tool >>>>>>>>>>>>> to be >>>>>>>>>>>>> synchronized with >>>>>>>>>>>>> logrotated tool on Linux. >>>>>>>>>>>>> >>>>>>>>>>>>> I think JEP 158 is in progress. >>>>>>>>>>>>> However, this JEP does not contain log rotation via external >>>>>>>>>>>>> tool in the >>>>>>>>>>>>> spec. >>>>>>>>>>>>> I want to rotate logs via jcmd in this JEP. >>>>>>>>>>>>> >>>>>>>>>>>>> I've updated a patch for it: >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/jvmlogging-logrotate/ >>>>>>>>>>>>> >>>>>>>>>>>>> This patch provides new option "rotate" in VM.log command. >>>>>>>>>>>>> If this change can be accepted, I will file it to JBS and send >>>>>>>>>>>>> RFR. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> >>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>> >>>>> From jaroslav.bachorik at oracle.com Tue Nov 3 14:45:51 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Tue, 3 Nov 2015 15:45:51 +0100 Subject: jmx-dev RFR 6425769: jmx remote bind address In-Reply-To: <1446487615.10865.57.camel@redhat.com> References: <1446460682.10865.20.camel@redhat.com> <56377F17.4050006@oracle.com> <5637871B.30705@oracle.com> <1446487615.10865.57.camel@redhat.com> Message-ID: <5638C89F.2090406@oracle.com> On 2.11.2015 19:06, Severin Gehwolf wrote: > Hi, > > Thanks Jaroslav and Daniel for the reviews! Comments inline. > > On Mon, 2015-11-02 at 16:54 +0100, Jaroslav Bachorik wrote: > > Hi, > > > > On 2.11.2015 16:19, Daniel Fuchs wrote: > >> Hi Severin, > >> > >> Adding serviceability-dev at openjdk.java.net into the loop - that's > >> a better alias than hotspot-dev for this kind of changes - maybe > >> someone from serviceability-dev will offer to sponsor :-) > >> > >> I will let serviceability team members comment on the hotspot > >> changes. > >> > >> ConnectorBootstrap.java > >> > >> I have one suggestion and one concern: > >> > >> Suggestion: I would suggest to reuse 'csf' (Client Socket Factory) > >> and > >> ssf (Server Socket Factory) variables rather than introduce the > >> two > >> new variables rmiServerSocketFactory and rmiClientSocketFactory. > >> You might want to create a new boolean 'useSocketFactory' variable, > >> if that makes the code more readable. > >> > >> Concern: If I understand correctly how RMI socket factories work, > >> the client socket factory will be serialized and sent to the client > >> side. This is problematic for interoperability, as the class may > >> not > >> present in the remote client - if the remote client is e.g. jdk 8. > >> > >> As far as I can see, your new DefaultClientSocketFactory doesn't do > >> anything useful - so I would suggest to simply get rid of it, and > >> only > >> set the Server Socket Factory when SSL is not involved. > > Thanks. Fixed in updated webrev. > > >> Tests: > >> > >> Concerning the tests - we're trying to get rid of shell scripts > >> rather than introducing new ones :-) > >> Could the test be rewritten in pure java using the Process API? > >> > >> I believe there's even a test library that will let you do that > >> easily jdk/test/lib/testlibrary/jdk/testlibrary/ > >> (see ProcessTools.java) > > It'll take me a bit to rewrite the test in pure Java, but should be > fine. This is not yet fixed in the updated webrev. > > >> Other: > >> > >> Also - I believe the new option should be documented in: > >> src/java.management/share/conf/management.properties > > Docs have been updated > in src/java.management/share/conf/management.properties. > > > I share Daniel's concerns. Also, the part of the changeset is related > > to javax.rmi.ssl - someone maintaining this library should also > > comment here. > > > > Also, the change is introducing new API (system property) and > > changing the existing one (adding SslRmiServerSocketFactory public > > constructors) so compatibility review process will have to be > > involved. > > OK. What exactly is there for me to do? I'm not familiar with this > process. Note that the intent would be to get this backported to JDK 8. Not much for you. But for the potential Oracle sponsor this means she will have to remember to go through some extra hoops before integrating your patch. -JB- > > New webrev at: > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-6425769/01/ > > Thanks, > Severin > > > -JB- > >> > >> best regards, > >> > >> -- daniel > >> > >> On 02/11/15 11:38, Severin Gehwolf wrote: > >>> Hi, > >>> > >>> Here is a patch addressing JDK-6425769. The issue is that the JMX > >>> agent > >>> binds to the wildcard address by default, preventing users to use > >>> system properties for JMX agents on multi-homed hosts. Given a > >>> host > >>> with local network interfaces, 192.168.0.1 and 192.168.0.2 say, > >>> it's > >>> impossible to start two JMX agents binding to fixed ports but to > >>> different network interfaces, 192.168.0.1:{9111,9112} and > >>> 192.168.0.2:{9111,9112} say. > >>> > >>> The JDK would bind to all local interfaces by default. In the > >>> above > >>> example to 192.168.0.1 *and* 192.168.0.2. The effect is that the > >>> second > >>> Java process would get a "Connection refused" error because > >>> another > >>> process has already been bound to the specified JMX/RMI port > >>> pairs. > >>> > >>> Bug: https://bugs.openjdk.java.net/browse/JDK-6425769 > >>> webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-6425769/ > >>> 00/ > >>> > >>> Testing done: > >>> jdk_jmx and jdk_management tests all pass after this change (no > >>> regressions). There is also a new JTREG test which fails prior > >>> this > >>> change and passes after. Updates to the diagnostic command have > >>> been > >>> tested manually. > >>> > >>> I understand that, if approved, the JDK and hotspot changes > >>> should get > >>> pushed together? Hotspot changes are fairly trivial since it's > >>> only a > >>> doc-update for the new JDK property in the relevant diagnostic > >>> command. > >>> > >>> Could someone please review and sponsor this change? Please let > >>> me know > >>> if there are questions. > >>> > >>> Thanks, > >>> Severin > >>> > >> > > > From david.holmes at oracle.com Wed Nov 4 07:14:32 2015 From: david.holmes at oracle.com (David Holmes) Date: Wed, 4 Nov 2015 17:14:32 +1000 Subject: RFR(M): 8140482: Various minor code improvements (runtime) In-Reply-To: <4295855A5C1DE049A61835A1887419CC41EB1875@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC41E9F0AA@DEWDFEMB12A.global.corp.sap> <563840B0.10401@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB1746@DEWDFEMB12A.global.corp.sap> <56388760.60800@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB1875@DEWDFEMB12A.global.corp.sap> Message-ID: <5639B058.9090609@oracle.com> Hi Goetz, On 4/11/2015 12:10 AM, Lindenmaier, Goetz wrote: > Hi David, > > the new scan is already through. I made a new webrev: > http://cr.openjdk.java.net/~goetz/webrevs/8140482-covRt/webrev.01/ > > attachListener_linux.cpp: > I had moved the string termination out of the loop, and read one > less char from the file. The scan still claims "Passing unterminated > string buf to strlen" in line 274. So I will undo it again for the > webrev. > > codeBuffer.cpp > Doing memset is fine. I'll use memset(). > >>>>> ps_core.c: >>>>> Pread not necessarily terminates interp_name which is printed >>>>> thereafter. Increase buffer size by 1 and add '\0'. >>>> >>>> Given: >>>> #define BUF_SIZE (PATH_MAX + NAME_MAX + 1) >>>> isn't it impossible to encounter that problem? >>> As I understand, pread does not null-terminate what it read. So the >>> null must come from the file. This protects against a corrupted file. >> >> So are you saying the nul is not present in the file? I'm not familiar >> with the ELF format. > There should be a nul in the file, else the code would fail more > obviously. The problem is if the file is corrupted. Thanks for clarifying. I'm still unclear why the attachListener.hpp changes are needed if we have validated the entry points in the attachListener_.cpp files? Also we still need someone from serviceability to comment on the _NSIG issue. Thanks, David > Best regards, > Goetz. > > >> -----Original Message----- >> From: David Holmes [mailto:david.holmes at oracle.com] >> Sent: Dienstag, 3. November 2015 11:07 >> To: Lindenmaier, Goetz; hotspot-runtime-dev at openjdk.java.net; >> serviceability-dev >> Subject: Re: RFR(M): 8140482: Various minor code improvements (runtime) >> >> Hi Goetz, >> >> Quick follow up on a couple of things ... >> >> On 3/11/2015 7:33 PM, Lindenmaier, Goetz wrote: >>> Hi David, >>> >>>> Sorry, lots of folks very busy at the moment as we try to get features >>>> finalized before the deadline. >>> thanks for looking at this! I know the Dec. 10 deadline, but I guess that also >>> holds for us ... at least J1 is over now. (Unfortunately we could not attend >>> this year.) >> >> Me neither :) >> >>>>> ps_core.c: >>>>> Pread not necessarily terminates interp_name which is printed >> thereafter. >>>>> Increase buffer size by 1 and add '\0'. >>>> >>>> Given: >>>> #define BUF_SIZE (PATH_MAX + NAME_MAX + 1) >>>> isn't it impossible to encounter that problem? >>> As I understand, pread does not null-terminate what it read. So the >>> null must come from the file. This protects against a corrupted file. >> >> So are you saying the nul is not present in the file? I'm not familiar >> with the ELF format. >> >>>>> stubRoutines_x86.cpp: >>>>> Cast to proper type. This way, left and right of '&' have the same type. >>>> >>>> I think you could just have changed the uint64_t to uint32_t as applied >>>> to the shift rather than casting the 1 to uint_32t. End result is the >>>> same though. >>> What you propose did not work. It was my first fix, too. >> >> Hmm okay. The result of the shift must be an unsigned type and the >> constant 1 is signed, so needs the cast (or use the unsigned constant >> form - 1ud? ) >> >>>>> attachListener_linux.cpp: >>>>> Read does not terminate buf. Size for '\0' is already considered. >>>> >>>> Looks a little odd being done on each iteration, but okay I guess. >>> I'll try to move it out of the loop. Better: I'll check whether the >>> scan groks it if I move it out of the loop :) >>> >>>>> os_linux.cpp: >>>>> Array sigflags[] has size MAXSIGNUM==32. _NSIG is bigger than >>>>> MAXSIGNUM (_NSIG == 65 on my machine). >>>>> sig is checked to be smaller than _NSIG. Later, in set_our_sigflags(), >>>>> sig is used to access sigflags[MAXSIGNUM] which can overflow the array. >>>>> Should we also increase MAXSIGNUM? >>>> >>>> Need to let the SR folk comment here as something definitely seems >>>> wrong, but I'm not 100% sure the what the correct answer is. If >>>> _JAVA_SR_SIGNUM is too big it should be validated somewhere and an >> error >>>> or warning reported. >>> I'm also not sure how to best handle this. Might even require a fix >>> exceeding this change. But I think this is the best finding. >>> >>>>> codeBuffer.cpp: >>>>> New_capacity is not initialized. Figure_expanded_capacities() handles >> this >>>>> correctly, but initializing this is cheap and safe. >>>> >>>> Hmmm ... I hate redundancy - this is pure wasted cycles. If we had to do >>>> it would memset not be better? Or would the code-checker not realize >>>> what memset was doing? >>> I guess it would work with memset, too. But I thought the 3-deep loop >>> will be unrolled completely so that only three stores remain. >> >> I tend not to try and imagine what the compiler may or may not do. Happy >> to take other opinions. Though again I'd prefer if the checker could be >> shown that there is no missing initialization. >> >>>>> dict.cpp: >>>>> If j-- is executed for j==0, the loop aborts because j is unsigned (0-- >= b- >>>>> _cnt). >>>>> Instead, only do j++ if necessary. >>>> >>>> Not at all obvious to me that it is possible to do j-- when j==0, but >>>> the change seems reasonable. >>> Yes, the scan does not understand there is j++ right after j-- because >>> of the loop iteration. I saw it complaining about this pattern several times. >>> >>>> Lots of spacing changes in that code make it hard to see the real changes. >>> Before, I was asked to fix indentation issues in a function I touch. >>> Does that only hold for compiler files? >> >> Yes/no/maybe :) Fixing up bad formatting when you are touching an area >> can be convenient, however it can also obscure the real changes, so it >> depends on the ratio of functional changes to format changes. >> >>>> 145 // SAPJVM GL j-- can underflow, which will cause the loop to abort. >>>> Seems unnecessary with the code change as noone will understand what >> j-- >>>> you are referring to. >>> Didn't mean to leave this in here. Removed. >>> >>>> 150 nb->_keyvals[nbcnt + nbcnt ] = key; >>>> 151 nb->_keyvals[nbcnt + nbcnt + 1] = b->_keyvals[j+j+1]; >>>> hotspot-style doesn't align array index expressions like that. Ditto >>>> 154/155. >>> Fixed. >>> >>>>> generateOopMap.cpp: >>>>> Idx is read from String. This is only called with constant strings, so >> compare >>>>> should be folded away by optimizing compilers if inlined. >>>> >>>> Not a fan of adding conditions that should never be false (hence the >>>> assert) and relying on the compiler to elide them. >>> OK, removed. >>> >>>>> deoptimization.cpp: >>>>> If buflen == 0, buf[-1] is accessed. >>>> >>>> Okay - but an assert(buflen>0) would be better I think as we should >>>> never be calling with a zero-length buffer. >>> Ok, I added the assert. As this isn't critical code, I would like to leave the >>> check in there, still. >>> >>>>> task.cpp: >>>>> Fatal can return if -XX:SuppressErrorAt is used. Just don't access the >>>>> array in this case. >>>> >>>> Okay. I would not be surprised if we have a lot of potential errors if a >>>> fatal/guarantee failure is suppressed. >>>> >>>>> attachListener.hpp: >>>>> Do strncpy to not overflow buffer. Don't write more chars than before. >>>> >>>> Again we have the assert to catch an error in the caller using an >>>> invalid name. >>> Hmm, the command comes from outside of the VM. It's not checked >>> very thoroughly, see, e.g., attachListener_windows.cpp:194. Arg0 is >>> checked twice, arg1 and arg2 are not checked at all. >> >> The libattach code is still part of our codebase so should be doing the >> right things. The linux and solaris code seems to be doing the expected >> name length check. On Windows the name is set using cmd, which is also >> subject to a length check: >> >> if (strlen(cmd) > AttachOperation::name_length_max) return >> ATTACH_ERROR_ILLEGALARG; >> >>> I add fixes for attachListener_windows.cpp to this change. >>> >>>>> heapDumper.cpp: >>>>> strncpy does not null terminate. >>>> >>>> 1973 if (total_length >= sizeof(base_path)) { >>>> >>>> total_length already adds +1 for the nul character so the == case is >>>> fine AFAICS. >>>> >>>> strncpy wont nul-terminate if the string exceeds the buffer size. But we >>>> have already established that total_length <= sizeof(base_path), and >>>> total_path includes space for a bunch of stuff other than HeapDumpPath, >>>> so the strncpy of HeapDumpPath has to copy the nul character. >>> Ok, removed. >>> >>>> > src/share/vm/services/memoryService.cpp >>>> >>>> Ok. >>>> >>>> > src/share/vm/utilities/xmlstream.cpp >>>> >>>> Ok - I'm more concerned about the "magic" 10 in that piece of code. >>> I assume the 10 is the space needed for the "_done" plus some waste ... >>> >>> I'll do another run of the scan. That takes a day. I'll post a new webrev >> after >>> that. >> >> Thanks, >> David >> >>> Thank again for this thorough review, >>> Goetz >>> >>> >>> >>> >>>>> Some of these, as the issue in codeBuffer.cpp, are actually handled >> correctly. >>>>> Nevertheless this is not that obvious so that somebody changing the >> code >>>>> Could oversee he has to add the initialization. >>>> >>>> Not an argument I buy en-masse as it leads to a lot of redundancy >>>> through the code paths. Plus these tools that are being run should show >>>> if a code changes requires initialization that is not present. >>>> >>>> Thanks, >>>> David >>>> >>>>> Some of these fixes are part of SAP JVM for a long time. This change has >>>>> been tested with our nightly build of openJDK. >>>>> >>>>> Best regards, >>>>> Goetz,. >>>>> From david.holmes at oracle.com Wed Nov 4 07:31:26 2015 From: david.holmes at oracle.com (David Holmes) Date: Wed, 4 Nov 2015 17:31:26 +1000 Subject: RFR: JDK-8134030: test/serviceability/dcmd/gc/HeapDumpTest fails to verify the dump In-Reply-To: <56389632.3080805@oracle.com> References: <563378B1.6080607@oracle.com> <563853E9.6090901@oracle.com> <56389632.3080805@oracle.com> Message-ID: <5639B44E.9090609@oracle.com> On 3/11/2015 9:10 PM, Andreas Eriksson wrote: > > > On 2015-11-03 07:27, David Holmes wrote: >> Hi Andreas, >> >> On 31/10/2015 12:03 AM, Andreas Eriksson wrote: >>> Hi, >>> >>> Please review this change to heap dump logic. >>> Including runtime list, since this is related to class loading. >>> >>> Bug: >>> JDK-8134030: test/serviceability/dcmd/gc/HeapDumpTest fails to verify >>> the dump >>> https://bugs.openjdk.java.net/browse/JDK-8134030 >>> >>> Webrev: http://cr.openjdk.java.net/~aeriksso/8134030/webrev.00/ >>> >>> A heap dump can occur at a point where an InstanceKlass doesn't have a >>> Java class mirror yet. >>> To avoid a crash because of this JDK-8131600 [1] excluded classes that >>> are not linked yet from the dump. >>> >>> The problem here is that since a class that has been loaded is already >>> exposed to the Java side, another class can have a reference to the >>> excluded class. >>> So, if it has been *loaded* but not *linked* it will be excluded, but >>> can still be referenced by other classes in the dump. >> >> I'm having trouble seeing exactly when the class is marked as "loaded" >> but it certainly seems to be after the mirror has been set. However >> linking happens even later than that - so if you still see a crash >> excluding linked classes then I fear you will also see problems >> excluding loaded classes! >> >> David >> ----- >> > > Hi, > > The crash does not happen when excluding linked classes, sorry if that > was unclear. > > Let me try to explain it again: > The crash was fixed by excluding non-linked classes (in JDK-8131600). > But that excludes some classes that are already referenced in the heap > dump by other classes. > By changing to excluding non-loaded classes, the crash can still not > happen, and all classes that are referenced in the dump are also included. Thanks for clarifying, I had misunderstood. Change seems fine in that case. Thanks, David > Not dumping these classes is only a problem when we try to look at the > heap dump at a later point (and even then it is only a problem of > missing information, not a crash). > For example, HeapDumpTest gives the following warning when it tries to > follow a reference to a class that was excluded from the dump: WARNING: > Failed to resolve object id 0x6c003d348 [...]. > VisualVM on the other hand will ignore the value, and tell the user the > field was null (which is very misleading). > > I hope that made sense. > > Thanks, > Andreas > >> >>> This gives the following warning in HeapDumpTest: >>> WARNING: Failed to resolve object id 0x6c003d348 [...]. >>> >>> This fix changes heapDumper.cpp to only exclude classes that are not yet >>> loaded instead. >>> >>> I'd like confirmation on this from someone who knows more about loaded >>> vs linked, but I think that this is correct: >>> When a class has been loaded, we are guaranteed that a Java class mirror >>> is setup, so the crash seen in JDK-8131600 still cannot happen. >>> This seems to be confirmed by the manual reproducer from JDK-8131600, >>> which still succeeds with this fix. >>> >>> Also in this change is the re-addition of the actual heap dump >>> verification call in HeapDumpTest, which was accidentally removed when >>> the test was re-factored. >>> >>> Regards, >>> Andreas >>> >>> [1]: https://bugs.openjdk.java.net/browse/JDK-8131600 > From david.holmes at oracle.com Wed Nov 4 07:34:21 2015 From: david.holmes at oracle.com (David Holmes) Date: Wed, 4 Nov 2015 17:34:21 +1000 Subject: RFR: JDK-8140556: Add force rotation option to VM.log jcmd In-Reply-To: <5638C87B.7080206@gmail.com> References: <562101D8.5080909@gmail.com> <562DF8D4.40501@oracle.com> <562EBF4B.4020607@gmail.com> <562F6B46.5050500@oracle.com> <563226AB.2030206@gmail.com> <5632341F.4060400@oracle.com> <56338125.9030802@gmail.com> <56338D58.9090709@oracle.com> <5634D9FC.6080502@gmail.com> <5636C66A.6020304@oracle.com> <563765FC.2090804@gmail.com> <56384824.4080904@oracle.com> <5638C87B.7080206@gmail.com> Message-ID: <5639B4FD.10205@oracle.com> On 4/11/2015 12:45 AM, Yasumasa Suenaga wrote: > David, Marcus, > >> I'm unclear of the context in which this could be called by an >> unattached thread (ie during early VM initialization). Could you >> clarify the code-path for me please? It seems an odd check to make on >> all rotate() requests rather then doing something only in the >> problematic case. > > I changed a little current implementation and built fastdebug. Thanks, now I see the problem and don't have a better solution. David ----- > Changes: > ------------------------ > diff -r 4338b5465f50 src/share/vm/logging/logFileOutput.cpp > --- a/src/share/vm/logging/logFileOutput.cpp Fri Oct 30 12:36:54 2015 > +0100 > +++ b/src/share/vm/logging/logFileOutput.cpp Tue Nov 03 23:34:33 2015 > +0900 > @@ -123,7 +123,8 @@ > success = false; > break; > } > - _rotate_size = value * K; > + //_rotate_size = value * K; > + _rotate_size = value; > } else { > success = false; > break; > @@ -155,12 +156,12 @@ > int written = LogFileStreamOutput::write(decorations, msg); > _current_size += written; > > - if (should_rotate()) { > +// if (should_rotate()) { > MutexLockerEx ml(&_rotation_lock, true /* no safepoint check */); > if (should_rotate()) { > rotate(); > } > - } > +// } > > return written; > } > ------------------------ > > Results: - from hs_err log > ------------------------ > # A fatal error has been detected by the Java Runtime Environment: > # > # Internal Error > (/home/ysuenaga/OpenJDK/hs-rt/hotspot/src/share/vm/runtime/thread.hpp:669), > pid=61982, tid=61983 > # assert(thread != __null) failed: just checking > : > Command Line: -Xlog:all=trace:file.txt::filesize=10,filecount=2 > -Xlog:invalid > : > V [libjvm.so+0xeb6481] Monitor::lock_without_safepoint_check()+0xe1 > V [libjvm.so+0xd4c118] LogFileOutput::write(LogDecorations const&, > char const*)+0x38 > V [libjvm.so+0xd4e5d0] LogTagSet::log(LogLevel::type, char const*)+0x60 > V [libjvm.so+0xd499b7] void Log<(LogTag::type)2, (LogTag::type)0, > (LogTag::type)0, (LogTag::type)0, (LogTag::type)0, > (LogTag::type)0>::vwrite<(LogLevel::type)6>(char const*, > __va_list_tag*)+0x67 > V [libjvm.so+0xd49a5b] void Log<(LogTag::type)2, (LogTag::type)0, > (LogTag::type)0, (LogTag::type)0, (LogTag::type)0, > (LogTag::type)0>::write<(LogLevel::type)6>(char const*, ...)+0x8b > V [libjvm.so+0xd49676] > LogConfiguration::parse_command_line_arguments(char const*)+0x146 > V [libjvm.so+0x489530] > Arguments::parse_each_vm_init_arg(JavaVMInitArgs const*, SysClassPath*, > bool*, Flag::Flags)+0x1d40 > V [libjvm.so+0x48a354] Arguments::parse_vm_init_args(JavaVMInitArgs > const*, JavaVMInitArgs const*, JavaVMInitArgs const*)+0x244 > V [libjvm.so+0x48c286] Arguments::parse(JavaVMInitArgs const*)+0xa46 > V [libjvm.so+0x1173052] Threads::create_vm(JavaVMInitArgs*, bool*)+0xb2 > V [libjvm.so+0xb502af] JNI_CreateJavaVM+0x8f > C [libjli.so+0x72a3] JavaMain+0x83 > ------------------------ > > > LogConfiguration::parse_command_line_arguments() calls before > initialization of main thread. > So I think that Thread::current() returns NULL and crash at assert(). > > > Thanks, > > Yasumasa > > > On 2015/11/03 14:37, David Holmes wrote: >> On 2/11/2015 11:32 PM, Yasumasa Suenaga wrote: >>> Marcus, David, >>> >>> I've uploaded new webrev: >>> http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.02/ >>> >>> Could you review it? >>> >>> >>>>>>>> Still missing the renaming of rotate_all_logfile to >>>>>>>> rotate_all_outputs. >>> >>> I've fixed. >>> >>> >>>>>> Small typo, please make that is_rotatable(). >>> >>> I've added, and have used at LogConfiguration::rotate_all_outputs(). >>> >>> >>>> You can't safely lock a mutex until the current thread has attached to >>>> the VM and Thread::current() returns non-null (that is what >>>> Thread::initialize_thread_local_storage() currently does). You may get >>>> away with it, particularly in product builds, during VM >>>> initialization, as the fast-lock path doesn't require access to >>>> Thread::current(). >>> >>> I will check whether Thread::current() is NULL or not at >>> LogFileOutput::is_rotatable() >>> is_rotatable() is called at LogFileOutput::rotate() before locking >>> _rotation_lock. >>> So we can avoid crash/assert which is related to TLS. >> >> I'm unclear of the context in which this could be called by an >> unattached thread (ie during early VM initialization). Could you >> clarify the code-path for me please? It seems an odd check to make on >> all rotate() requests rather then doing something only in the >> problematic case. >> >> Note I am not reviewing this in general as I am not familiar with the >> logging code, I am just commenting on the thread-safety issue. >> >> Thanks, >> David >> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> On 2015/11/02 11:11, David Holmes wrote: >>>> On 1/11/2015 1:10 AM, Yasumasa Suenaga wrote: >>>>>> This in turn means that we are logging before the thread local >>>>>> storage >>>>>> has been initialized, and the JVM will crash/hit the assert >>>>>> because we >>>>>> are trying to take the rotation lock. >>>>> >>>>> I guess that you are saying about class member (not TLS - e.g. >>>>> pthread_setspecific()). >>>> >>>> You can't safely lock a mutex until the current thread has attached to >>>> the VM and Thread::current() returns non-null (that is what >>>> Thread::initialize_thread_local_storage() currently does). You may get >>>> away with it, particularly in product builds, during VM >>>> initialization, as the fast-lock path doesn't require access to >>>> Thread::current(). >>>> >>>> David >>>> >>>>> Most of fields (include _rotation_lock) in LogFileOutput are >>>>> initialized >>>>> at c'tor. >>>>> Other field (_stream) is initialized at LogFileOutput::initialize(). >>>>> >>>>> We can avoid problem if we can move initialize() to c'tor because >>>>> LogFileOutput >>>>> will be instantiated at LogConfiguration::new_output(). >>>>> >>>>> Currently, LogFileOutput will be added to >>>>> LogConfiguration::_outputs and >>>>> to LogTagSet after calling LogFileOutput::initialize(). >>>>> I wonder why we cannot avoid crash/assert with current >>>>> implementation... >>>>> >>>>> >>>>> I will keep current implementation about _rotation_lock if the >>>>> above is >>>>> incorrect. >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2015/10/31 0:31, Marcus Larsson wrote: >>>>>> Hey, >>>>>> >>>>>> On 2015-10-30 15:39, Yasumasa Suenaga wrote: >>>>>>> Hi Marcus, >>>>>>> >>>>>>> Thank you for your comment. >>>>>>> >>>>>>> >>>>>>>> Still missing the renaming of rotate_all_logfile to >>>>>>>> rotate_all_outputs. >>>>>>> >>>>>>> Sorry, I will fix. >>>>>>> >>>>>> >>>>>> Thanks! >>>>>> >>>>>>> >>>>>>>> This rotate function works I guess, but it will probably need some >>>>>>>> rework once there are other types of log outputs (rotate doesn't >>>>>>>> make sense on all types of log outputs). >>>>>>> >>>>>>> I will add is_rotetable() to LogOutput as virtual function. >>>>>>> This function return false (can not rotate) by default, return true >>>>>>> if _file_count is >>>>>>> greater than 0 in LogFileOutput. >>>>>>> >>>>>>> LogConfiguration::rotate_all_outputs() will rotate if this function >>>>>>> returns true. >>>>>>> >>>>>> >>>>>> Small typo, please make that is_rotatable(). >>>>>> >>>>>>> >>>>>>>> If we configure a rotated file output and then log something on >>>>>>>> that >>>>>>>> output before thread local storage initialization is complete, we >>>>>>>> will crash/hit an assert. The previous implementation avoided this >>>>>>>> as long as no rotation was needed before this initialization was >>>>>>>> complete. (This can be triggered using the following arguments >>>>>>>> "-Xlog:all=trace:file.txt::filesize=10,filecount=2 -Xlog:invalid", >>>>>>>> for example.) >>>>>>> >>>>>>> I do not think so. >>>>>>> Each -Xlog option creates unique instance of LogOutput (and >>>>>>> subclass). >>>>>>> So they are isolated. >>>>>> >>>>>> My example includes a second (and intentionally incorrect) -Xlog >>>>>> option only to trigger logging on the newly configured rotating >>>>>> LogFileOutput. This will cause some logging about the invalid -Xlog >>>>>> argument, and this logging happens during the argument parsing. This >>>>>> in turn means that we are logging before the thread local storage has >>>>>> been initialized, and the JVM will crash/hit the assert because we >>>>>> are >>>>>> trying to take the rotation lock. This has nothing to do with >>>>>> concurrent fprintfs or freopens, even if that is a problem. >>>>>> >>>>>>> >>>>>>> However, we might have to rotate at safepoint. >>>>>> >>>>>> Doing stuff at a safepoint will not give us much in this case. There >>>>>> are threads that continue execution through a safepoint, which means >>>>>> we will see log calls being made even during this time. >>>>>> >>>>>>> Currently, LogOutput might be used by multiple threads concurrently. >>>>>>> If we rotate log when another thread is writing string to output, we >>>>>>> encounter unexpected behavior. >>>>>>> >>>>>>> LogFileOutput::rotate() uses freopen(). >>>>>>> LogFileStreamOutput::write() uses vfprintf() through jio_fprintf(). >>>>>>> freopen() and vfprintf() are not atomic. >>>>>> >>>>>> You're right. There seems to be an overlooked concurrency issue with >>>>>> vfprintfs during freopen. I don't think we should resolve this as >>>>>> part >>>>>> of this particular fix, so instead I'll create a separate issue >>>>>> for it. >>>>>> >>>>>>> >>>>>>> I think that cause of crash/assertion which you say is it. >>>>>>> (GC log will be rotated at safepoint.) >>>>>>> >>>>>> >>>>>> See my comment above for what crash/assert I'm talking about. >>>>>> >>>>>> Thanks, >>>>>> Marcus >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2015/10/29 23:58, Marcus Larsson wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> On 2015-10-29 15:01, Yasumasa Suenaga wrote: >>>>>>>>> Hi Marcus, >>>>>>>>> >>>>>>>>> Thank you for your comment. >>>>>>>>> >>>>>>>>>> I'll sponsor it. >>>>>>>>> >>>>>>>>> Thank you so much! >>>>>>>>> I've uploaded new webrev: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.01/ >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> Still missing the renaming of rotate_all_logfile to >>>>>>>> rotate_all_outputs. >>>>>>>> >>>>>>>>>> logConfiguration.cpp/hpp: >>>>>>>>>> >>>>>>>>>> * I don't think we should rely on the archive_name or the >>>>>>>>>> output's >>>>>>>>>> name to decide whether or not an output should be rotated. It >>>>>>>>>> would be better to introduce an is_rotated() test function in >>>>>>>>>> LogOutput that could be used here. >>>>>>>>>> >>>>>>>>>> * rotate_all_logfile() should be renamed to rotate_all_outputs(). >>>>>>>>>> Currently there are only LogFileOutputs (other than >>>>>>>>>> stdout/stderr), but in the future there might be other types of >>>>>>>>>> outputs so I prefer having a more general name. >>>>>>>>>> Also, please use static_cast instead of the >>>>>>>>>> C-style cast. (Additional logic will be required here once more >>>>>>>>>> types of log outputs are added, but I don't think we need to >>>>>>>>>> worry >>>>>>>>>> about this right now.) >>>>>>>>>> >>>>>>>>>> * Don't print an error if an output is not rotatable, since it's >>>>>>>>>> not an error to have some log outputs rotated while others are >>>>>>>>>> not. If you really want some traceability here I suggest adding >>>>>>>>>> log messages on trace level, tagged with 'logging'. >>>>>>>>> >>>>>>>>> I added LogOutput::rotate(bool) as virtual function. >>>>>>>>> This function works nothing by default, but will rotate all >>>>>>>>> logs at >>>>>>>>> LogFileOutput. >>>>>>>> >>>>>>>> This rotate function works I guess, but it will probably need some >>>>>>>> rework once there are other types of log outputs (rotate doesn't >>>>>>>> make sense on all types of log outputs). >>>>>>>> >>>>>>>>> >>>>>>>>> If true is passed to this function, all files will be rotated. >>>>>>>>> >>>>>>>>> >>>>>>>>>> logDiagnosticCommand.cpp/hpp: >>>>>>>>>> >>>>>>>>>> * I think the description could be improved to something like >>>>>>>>>> "Lists current log configuration, enables/disables/configures a >>>>>>>>>> log output, or disables/rotates all logs." >>>>>>>>>> >>>>>>>>>> * The rotate option description should clarify that all logs will >>>>>>>>>> be rotated ("current log" is too ambiguous). >>>>>>>>> >>>>>>>>> I've fixed. >>>>>>>> >>>>>>>> Great, thanks! >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> logFileOutput.cpp/hpp: >>>>>>>>>> >>>>>>>>>> * Moving the MutexLocker like this introduces a race condition >>>>>>>>>> where a log might be rotated multiple times by different threads, >>>>>>>>>> instead of just once. >>>>>>>>>> Instead of making the rotate() function public and moving the >>>>>>>>>> mutexlocker, I suggest adding something like a public >>>>>>>>>> force_rotation() function that grabs the lock and calls the >>>>>>>>>> private rotate(). >>>>>>>>>> >>>>>>>>>> * Given the addition of is_rotated() in LogOutput, then >>>>>>>>>> get_archive_name() should be removed. >>>>>>>>> >>>>>>>>> I moved MutexLocker and should_rotate() to rotate(). >>>>>>>>> I think this change can avoid race condition. >>>>>>>>> >>>>>>>> >>>>>>>> There is a subtle problem with taking the lock to check if we >>>>>>>> should >>>>>>>> rotate. If we configure a rotated file output and then log >>>>>>>> something >>>>>>>> on that output before thread local storage initialization is >>>>>>>> complete, we will crash/hit an assert. The previous implementation >>>>>>>> avoided this as long as no rotation was needed before this >>>>>>>> initialization was complete. (This can be triggered using the >>>>>>>> following arguments >>>>>>>> "-Xlog:all=trace:file.txt::filesize=10,filecount=2 -Xlog:invalid", >>>>>>>> for example.) >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Marcus >>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2015/10/27 21:17, Marcus Larsson wrote: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> On 2015-10-27 01:03, Yasumasa Suenaga wrote: >>>>>>>>>>> Hi Marcus, >>>>>>>>>>> >>>>>>>>>>> Thank you for replying. >>>>>>>>>>> >>>>>>>>>>> I filed this feature to JBS and change subject of this email. >>>>>>>>>>> Could you be a sponsor and review it? >>>>>>>>>> >>>>>>>>>> I'll sponsor it. >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.00/ >>>>>>>>>> >>>>>>>>>> logConfiguration.cpp/hpp: >>>>>>>>>> >>>>>>>>>> * I don't think we should rely on the archive_name or the >>>>>>>>>> output's >>>>>>>>>> name to decide whether or not an output should be rotated. It >>>>>>>>>> would be better to introduce an is_rotated() test function in >>>>>>>>>> LogOutput that could be used here. >>>>>>>>>> >>>>>>>>>> * rotate_all_logfile() should be renamed to rotate_all_outputs(). >>>>>>>>>> Currently there are only LogFileOutputs (other than >>>>>>>>>> stdout/stderr), but in the future there might be other types of >>>>>>>>>> outputs so I prefer having a more general name. >>>>>>>>>> Also, please use static_cast instead of the >>>>>>>>>> C-style cast. (Additional logic will be required here once more >>>>>>>>>> types of log outputs are added, but I don't think we need to >>>>>>>>>> worry >>>>>>>>>> about this right now.) >>>>>>>>>> >>>>>>>>>> * Don't print an error if an output is not rotatable, since it's >>>>>>>>>> not an error to have some log outputs rotated while others are >>>>>>>>>> not. If you really want some traceability here I suggest adding >>>>>>>>>> log messages on trace level, tagged with 'logging'. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> logDiagnosticCommand.cpp/hpp: >>>>>>>>>> >>>>>>>>>> * I think the description could be improved to something like >>>>>>>>>> "Lists current log configuration, enables/disables/configures a >>>>>>>>>> log output, or disables/rotates all logs." >>>>>>>>>> >>>>>>>>>> * The rotate option description should clarify that all logs will >>>>>>>>>> be rotated ("current log" is too ambiguous). >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> logFileOutput.cpp/hpp: >>>>>>>>>> >>>>>>>>>> * Moving the MutexLocker like this introduces a race condition >>>>>>>>>> where a log might be rotated multiple times by different threads, >>>>>>>>>> instead of just once. >>>>>>>>>> Instead of making the rotate() function public and moving the >>>>>>>>>> mutexlocker, I suggest adding something like a public >>>>>>>>>> force_rotation() function that grabs the lock and calls the >>>>>>>>>> private rotate(). >>>>>>>>>> >>>>>>>>>> * Given the addition of is_rotated() in LogOutput, then >>>>>>>>>> get_archive_name() should be removed. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Marcus >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 2015/10/26 18:56, Marcus Larsson wrote: >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> Sorry for my late reply. >>>>>>>>>>>> >>>>>>>>>>>> I think being able to force rotation via jcmd seems like a good >>>>>>>>>>>> feature. Files are currently opened in append mode so it should >>>>>>>>>>>> already be possible to use external log rotation tools by >>>>>>>>>>>> copying and truncating the files. Still I think it would be >>>>>>>>>>>> nice >>>>>>>>>>>> to provide the jcmd for rotation as well. >>>>>>>>>>>> >>>>>>>>>>>> I can see some small issues with the implementation, but we can >>>>>>>>>>>> deal with that during the review. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Marcus >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 2015-10-26 00:26, Yasumasa Suenaga wrote: >>>>>>>>>>>>> Hi all, >>>>>>>>>>>>> >>>>>>>>>>>>> Have you ever seen my change? >>>>>>>>>>>>> I (and my customers) need log rotation function via external >>>>>>>>>>>>> tool. >>>>>>>>>>>>> >>>>>>>>>>>>> I want to merge it by Feature Complete. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> >>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>> 2015/10/16 22:55 "Yasumasa Suenaga" : >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>> >>>>>>>>>>>>>> I contributed JDK-7090324: gclog rotation via external tool >>>>>>>>>>>>>> to be >>>>>>>>>>>>>> synchronized with >>>>>>>>>>>>>> logrotated tool on Linux. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I think JEP 158 is in progress. >>>>>>>>>>>>>> However, this JEP does not contain log rotation via external >>>>>>>>>>>>>> tool in the >>>>>>>>>>>>>> spec. >>>>>>>>>>>>>> I want to rotate logs via jcmd in this JEP. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I've updated a patch for it: >>>>>>>>>>>>>> >>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/jvmlogging-logrotate/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> This patch provides new option "rotate" in VM.log command. >>>>>>>>>>>>>> If this change can be accepted, I will file it to JBS and >>>>>>>>>>>>>> send >>>>>>>>>>>>>> RFR. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>> From goetz.lindenmaier at sap.com Wed Nov 4 08:01:28 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 4 Nov 2015 08:01:28 +0000 Subject: RFR(M): 8140482: Various minor code improvements (runtime) In-Reply-To: <5639B058.9090609@oracle.com> References: <4295855A5C1DE049A61835A1887419CC41E9F0AA@DEWDFEMB12A.global.corp.sap> <563840B0.10401@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB1746@DEWDFEMB12A.global.corp.sap> <56388760.60800@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB1875@DEWDFEMB12A.global.corp.sap> <5639B058.9090609@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC41EB1B57@DEWDFEMB12A.global.corp.sap> Hi David, attachListener.hpp: I agree that I can not find another possible issue with the strcpy. Still I think it's better to have the strncpy, as it would have protected against the bug in attachListener_windows.cpp. But if you insist I'll just remove it. Should I remove the _NSIG issue from this change and open an issue of it's own discussed on the serviceability list? Best regards, Goetz. > -----Original Message----- > From: David Holmes [mailto:david.holmes at oracle.com] > Sent: Mittwoch, 4. November 2015 08:15 > To: Lindenmaier, Goetz; hotspot-runtime-dev at openjdk.java.net; > serviceability-dev > Subject: Re: RFR(M): 8140482: Various minor code improvements (runtime) > > Hi Goetz, > > On 4/11/2015 12:10 AM, Lindenmaier, Goetz wrote: > > Hi David, > > > > the new scan is already through. I made a new webrev: > > http://cr.openjdk.java.net/~goetz/webrevs/8140482-covRt/webrev.01/ > > > > attachListener_linux.cpp: > > I had moved the string termination out of the loop, and read one > > less char from the file. The scan still claims "Passing unterminated > > string buf to strlen" in line 274. So I will undo it again for the > > webrev. > > > > codeBuffer.cpp > > Doing memset is fine. I'll use memset(). > > > >>>>> ps_core.c: > >>>>> Pread not necessarily terminates interp_name which is printed > >>>>> thereafter. Increase buffer size by 1 and add '\0'. > >>>> > >>>> Given: > >>>> #define BUF_SIZE (PATH_MAX + NAME_MAX + 1) > >>>> isn't it impossible to encounter that problem? > >>> As I understand, pread does not null-terminate what it read. So the > >>> null must come from the file. This protects against a corrupted file. > >> > >> So are you saying the nul is not present in the file? I'm not familiar > >> with the ELF format. > > There should be a nul in the file, else the code would fail more > > obviously. The problem is if the file is corrupted. > > Thanks for clarifying. > > I'm still unclear why the attachListener.hpp changes are needed if we > have validated the entry points in the attachListener_.cpp files? > > Also we still need someone from serviceability to comment on the _NSIG > issue. > > Thanks, > David > > > > Best regards, > > Goetz. > > > > > >> -----Original Message----- > >> From: David Holmes [mailto:david.holmes at oracle.com] > >> Sent: Dienstag, 3. November 2015 11:07 > >> To: Lindenmaier, Goetz; hotspot-runtime-dev at openjdk.java.net; > >> serviceability-dev > >> Subject: Re: RFR(M): 8140482: Various minor code improvements > (runtime) > >> > >> Hi Goetz, > >> > >> Quick follow up on a couple of things ... > >> > >> On 3/11/2015 7:33 PM, Lindenmaier, Goetz wrote: > >>> Hi David, > >>> > >>>> Sorry, lots of folks very busy at the moment as we try to get features > >>>> finalized before the deadline. > >>> thanks for looking at this! I know the Dec. 10 deadline, but I guess that > also > >>> holds for us ... at least J1 is over now. (Unfortunately we could not > attend > >>> this year.) > >> > >> Me neither :) > >> > >>>>> ps_core.c: > >>>>> Pread not necessarily terminates interp_name which is printed > >> thereafter. > >>>>> Increase buffer size by 1 and add '\0'. > >>>> > >>>> Given: > >>>> #define BUF_SIZE (PATH_MAX + NAME_MAX + 1) > >>>> isn't it impossible to encounter that problem? > >>> As I understand, pread does not null-terminate what it read. So the > >>> null must come from the file. This protects against a corrupted file. > >> > >> So are you saying the nul is not present in the file? I'm not familiar > >> with the ELF format. > >> > >>>>> stubRoutines_x86.cpp: > >>>>> Cast to proper type. This way, left and right of '&' have the same type. > >>>> > >>>> I think you could just have changed the uint64_t to uint32_t as applied > >>>> to the shift rather than casting the 1 to uint_32t. End result is the > >>>> same though. > >>> What you propose did not work. It was my first fix, too. > >> > >> Hmm okay. The result of the shift must be an unsigned type and the > >> constant 1 is signed, so needs the cast (or use the unsigned constant > >> form - 1ud? ) > >> > >>>>> attachListener_linux.cpp: > >>>>> Read does not terminate buf. Size for '\0' is already considered. > >>>> > >>>> Looks a little odd being done on each iteration, but okay I guess. > >>> I'll try to move it out of the loop. Better: I'll check whether the > >>> scan groks it if I move it out of the loop :) > >>> > >>>>> os_linux.cpp: > >>>>> Array sigflags[] has size MAXSIGNUM==32. _NSIG is bigger than > >>>>> MAXSIGNUM (_NSIG == 65 on my machine). > >>>>> sig is checked to be smaller than _NSIG. Later, in set_our_sigflags(), > >>>>> sig is used to access sigflags[MAXSIGNUM] which can overflow the > array. > >>>>> Should we also increase MAXSIGNUM? > >>>> > >>>> Need to let the SR folk comment here as something definitely seems > >>>> wrong, but I'm not 100% sure the what the correct answer is. If > >>>> _JAVA_SR_SIGNUM is too big it should be validated somewhere and > an > >> error > >>>> or warning reported. > >>> I'm also not sure how to best handle this. Might even require a fix > >>> exceeding this change. But I think this is the best finding. > >>> > >>>>> codeBuffer.cpp: > >>>>> New_capacity is not initialized. Figure_expanded_capacities() handles > >> this > >>>>> correctly, but initializing this is cheap and safe. > >>>> > >>>> Hmmm ... I hate redundancy - this is pure wasted cycles. If we had to > do > >>>> it would memset not be better? Or would the code-checker not realize > >>>> what memset was doing? > >>> I guess it would work with memset, too. But I thought the 3-deep loop > >>> will be unrolled completely so that only three stores remain. > >> > >> I tend not to try and imagine what the compiler may or may not do. Happy > >> to take other opinions. Though again I'd prefer if the checker could be > >> shown that there is no missing initialization. > >> > >>>>> dict.cpp: > >>>>> If j-- is executed for j==0, the loop aborts because j is unsigned (0-- >= > b- > >>>>> _cnt). > >>>>> Instead, only do j++ if necessary. > >>>> > >>>> Not at all obvious to me that it is possible to do j-- when j==0, but > >>>> the change seems reasonable. > >>> Yes, the scan does not understand there is j++ right after j-- because > >>> of the loop iteration. I saw it complaining about this pattern several > times. > >>> > >>>> Lots of spacing changes in that code make it hard to see the real > changes. > >>> Before, I was asked to fix indentation issues in a function I touch. > >>> Does that only hold for compiler files? > >> > >> Yes/no/maybe :) Fixing up bad formatting when you are touching an area > >> can be convenient, however it can also obscure the real changes, so it > >> depends on the ratio of functional changes to format changes. > >> > >>>> 145 // SAPJVM GL j-- can underflow, which will cause the loop to > abort. > >>>> Seems unnecessary with the code change as noone will understand > what > >> j-- > >>>> you are referring to. > >>> Didn't mean to leave this in here. Removed. > >>> > >>>> 150 nb->_keyvals[nbcnt + nbcnt ] = key; > >>>> 151 nb->_keyvals[nbcnt + nbcnt + 1] = b->_keyvals[j+j+1]; > >>>> hotspot-style doesn't align array index expressions like that. Ditto > >>>> 154/155. > >>> Fixed. > >>> > >>>>> generateOopMap.cpp: > >>>>> Idx is read from String. This is only called with constant strings, so > >> compare > >>>>> should be folded away by optimizing compilers if inlined. > >>>> > >>>> Not a fan of adding conditions that should never be false (hence the > >>>> assert) and relying on the compiler to elide them. > >>> OK, removed. > >>> > >>>>> deoptimization.cpp: > >>>>> If buflen == 0, buf[-1] is accessed. > >>>> > >>>> Okay - but an assert(buflen>0) would be better I think as we should > >>>> never be calling with a zero-length buffer. > >>> Ok, I added the assert. As this isn't critical code, I would like to leave the > >>> check in there, still. > >>> > >>>>> task.cpp: > >>>>> Fatal can return if -XX:SuppressErrorAt is used. Just don't access the > >>>>> array in this case. > >>>> > >>>> Okay. I would not be surprised if we have a lot of potential errors if a > >>>> fatal/guarantee failure is suppressed. > >>>> > >>>>> attachListener.hpp: > >>>>> Do strncpy to not overflow buffer. Don't write more chars than > before. > >>>> > >>>> Again we have the assert to catch an error in the caller using an > >>>> invalid name. > >>> Hmm, the command comes from outside of the VM. It's not checked > >>> very thoroughly, see, e.g., attachListener_windows.cpp:194. Arg0 is > >>> checked twice, arg1 and arg2 are not checked at all. > >> > >> The libattach code is still part of our codebase so should be doing the > >> right things. The linux and solaris code seems to be doing the expected > >> name length check. On Windows the name is set using cmd, which is also > >> subject to a length check: > >> > >> if (strlen(cmd) > AttachOperation::name_length_max) return > >> ATTACH_ERROR_ILLEGALARG; > >> > >>> I add fixes for attachListener_windows.cpp to this change. > >>> > >>>>> heapDumper.cpp: > >>>>> strncpy does not null terminate. > >>>> > >>>> 1973 if (total_length >= sizeof(base_path)) { > >>>> > >>>> total_length already adds +1 for the nul character so the == case is > >>>> fine AFAICS. > >>>> > >>>> strncpy wont nul-terminate if the string exceeds the buffer size. But > we > >>>> have already established that total_length <= sizeof(base_path), and > >>>> total_path includes space for a bunch of stuff other than > HeapDumpPath, > >>>> so the strncpy of HeapDumpPath has to copy the nul character. > >>> Ok, removed. > >>> > >>>> > src/share/vm/services/memoryService.cpp > >>>> > >>>> Ok. > >>>> > >>>> > src/share/vm/utilities/xmlstream.cpp > >>>> > >>>> Ok - I'm more concerned about the "magic" 10 in that piece of code. > >>> I assume the 10 is the space needed for the "_done" plus some waste ... > >>> > >>> I'll do another run of the scan. That takes a day. I'll post a new webrev > >> after > >>> that. > >> > >> Thanks, > >> David > >> > >>> Thank again for this thorough review, > >>> Goetz > >>> > >>> > >>> > >>> > >>>>> Some of these, as the issue in codeBuffer.cpp, are actually handled > >> correctly. > >>>>> Nevertheless this is not that obvious so that somebody changing the > >> code > >>>>> Could oversee he has to add the initialization. > >>>> > >>>> Not an argument I buy en-masse as it leads to a lot of redundancy > >>>> through the code paths. Plus these tools that are being run should > show > >>>> if a code changes requires initialization that is not present. > >>>> > >>>> Thanks, > >>>> David > >>>> > >>>>> Some of these fixes are part of SAP JVM for a long time. This change > has > >>>>> been tested with our nightly build of openJDK. > >>>>> > >>>>> Best regards, > >>>>> Goetz,. > >>>>> From david.holmes at oracle.com Wed Nov 4 09:29:02 2015 From: david.holmes at oracle.com (David Holmes) Date: Wed, 4 Nov 2015 19:29:02 +1000 Subject: RFR(M): 8140482: Various minor code improvements (runtime) In-Reply-To: <4295855A5C1DE049A61835A1887419CC41EB1B57@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC41E9F0AA@DEWDFEMB12A.global.corp.sap> <563840B0.10401@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB1746@DEWDFEMB12A.global.corp.sap> <56388760.60800@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB1875@DEWDFEMB12A.global.corp.sap> <5639B058.9090609@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB1B57@DEWDFEMB12A.global.corp.sap> Message-ID: <5639CFDE.1010302@oracle.com> On 4/11/2015 6:01 PM, Lindenmaier, Goetz wrote: > Hi David, > > attachListener.hpp: > I agree that I can not find another possible issue > with the strcpy. > Still I think it's better to have the strncpy, as it would have > protected against the bug in attachListener_windows.cpp. > But if you insist I'll just remove it. I don't insist, but I do prefer to place all the guards at the "boundary" of the VM rather than at every level when possible. > Should I remove the _NSIG issue from this change and > open an issue of it's own discussed on the serviceability list? Let's give them a chance to respond. I'll ping them on the hotline ;-) Thanks, David > Best regards, > Goetz. > > >> -----Original Message----- >> From: David Holmes [mailto:david.holmes at oracle.com] >> Sent: Mittwoch, 4. November 2015 08:15 >> To: Lindenmaier, Goetz; hotspot-runtime-dev at openjdk.java.net; >> serviceability-dev >> Subject: Re: RFR(M): 8140482: Various minor code improvements (runtime) >> >> Hi Goetz, >> >> On 4/11/2015 12:10 AM, Lindenmaier, Goetz wrote: >>> Hi David, >>> >>> the new scan is already through. I made a new webrev: >>> http://cr.openjdk.java.net/~goetz/webrevs/8140482-covRt/webrev.01/ >>> >>> attachListener_linux.cpp: >>> I had moved the string termination out of the loop, and read one >>> less char from the file. The scan still claims "Passing unterminated >>> string buf to strlen" in line 274. So I will undo it again for the >>> webrev. >>> >>> codeBuffer.cpp >>> Doing memset is fine. I'll use memset(). >>> >>>>>>> ps_core.c: >>>>>>> Pread not necessarily terminates interp_name which is printed >>>>>>> thereafter. Increase buffer size by 1 and add '\0'. >>>>>> >>>>>> Given: >>>>>> #define BUF_SIZE (PATH_MAX + NAME_MAX + 1) >>>>>> isn't it impossible to encounter that problem? >>>>> As I understand, pread does not null-terminate what it read. So the >>>>> null must come from the file. This protects against a corrupted file. >>>> >>>> So are you saying the nul is not present in the file? I'm not familiar >>>> with the ELF format. >>> There should be a nul in the file, else the code would fail more >>> obviously. The problem is if the file is corrupted. >> >> Thanks for clarifying. >> >> I'm still unclear why the attachListener.hpp changes are needed if we >> have validated the entry points in the attachListener_.cpp files? >> >> Also we still need someone from serviceability to comment on the _NSIG >> issue. >> >> Thanks, >> David >> >> >>> Best regards, >>> Goetz. >>> >>> >>>> -----Original Message----- >>>> From: David Holmes [mailto:david.holmes at oracle.com] >>>> Sent: Dienstag, 3. November 2015 11:07 >>>> To: Lindenmaier, Goetz; hotspot-runtime-dev at openjdk.java.net; >>>> serviceability-dev >>>> Subject: Re: RFR(M): 8140482: Various minor code improvements >> (runtime) >>>> >>>> Hi Goetz, >>>> >>>> Quick follow up on a couple of things ... >>>> >>>> On 3/11/2015 7:33 PM, Lindenmaier, Goetz wrote: >>>>> Hi David, >>>>> >>>>>> Sorry, lots of folks very busy at the moment as we try to get features >>>>>> finalized before the deadline. >>>>> thanks for looking at this! I know the Dec. 10 deadline, but I guess that >> also >>>>> holds for us ... at least J1 is over now. (Unfortunately we could not >> attend >>>>> this year.) >>>> >>>> Me neither :) >>>> >>>>>>> ps_core.c: >>>>>>> Pread not necessarily terminates interp_name which is printed >>>> thereafter. >>>>>>> Increase buffer size by 1 and add '\0'. >>>>>> >>>>>> Given: >>>>>> #define BUF_SIZE (PATH_MAX + NAME_MAX + 1) >>>>>> isn't it impossible to encounter that problem? >>>>> As I understand, pread does not null-terminate what it read. So the >>>>> null must come from the file. This protects against a corrupted file. >>>> >>>> So are you saying the nul is not present in the file? I'm not familiar >>>> with the ELF format. >>>> >>>>>>> stubRoutines_x86.cpp: >>>>>>> Cast to proper type. This way, left and right of '&' have the same type. >>>>>> >>>>>> I think you could just have changed the uint64_t to uint32_t as applied >>>>>> to the shift rather than casting the 1 to uint_32t. End result is the >>>>>> same though. >>>>> What you propose did not work. It was my first fix, too. >>>> >>>> Hmm okay. The result of the shift must be an unsigned type and the >>>> constant 1 is signed, so needs the cast (or use the unsigned constant >>>> form - 1ud? ) >>>> >>>>>>> attachListener_linux.cpp: >>>>>>> Read does not terminate buf. Size for '\0' is already considered. >>>>>> >>>>>> Looks a little odd being done on each iteration, but okay I guess. >>>>> I'll try to move it out of the loop. Better: I'll check whether the >>>>> scan groks it if I move it out of the loop :) >>>>> >>>>>>> os_linux.cpp: >>>>>>> Array sigflags[] has size MAXSIGNUM==32. _NSIG is bigger than >>>>>>> MAXSIGNUM (_NSIG == 65 on my machine). >>>>>>> sig is checked to be smaller than _NSIG. Later, in set_our_sigflags(), >>>>>>> sig is used to access sigflags[MAXSIGNUM] which can overflow the >> array. >>>>>>> Should we also increase MAXSIGNUM? >>>>>> >>>>>> Need to let the SR folk comment here as something definitely seems >>>>>> wrong, but I'm not 100% sure the what the correct answer is. If >>>>>> _JAVA_SR_SIGNUM is too big it should be validated somewhere and >> an >>>> error >>>>>> or warning reported. >>>>> I'm also not sure how to best handle this. Might even require a fix >>>>> exceeding this change. But I think this is the best finding. >>>>> >>>>>>> codeBuffer.cpp: >>>>>>> New_capacity is not initialized. Figure_expanded_capacities() handles >>>> this >>>>>>> correctly, but initializing this is cheap and safe. >>>>>> >>>>>> Hmmm ... I hate redundancy - this is pure wasted cycles. If we had to >> do >>>>>> it would memset not be better? Or would the code-checker not realize >>>>>> what memset was doing? >>>>> I guess it would work with memset, too. But I thought the 3-deep loop >>>>> will be unrolled completely so that only three stores remain. >>>> >>>> I tend not to try and imagine what the compiler may or may not do. Happy >>>> to take other opinions. Though again I'd prefer if the checker could be >>>> shown that there is no missing initialization. >>>> >>>>>>> dict.cpp: >>>>>>> If j-- is executed for j==0, the loop aborts because j is unsigned (0-- >= >> b- >>>>>>> _cnt). >>>>>>> Instead, only do j++ if necessary. >>>>>> >>>>>> Not at all obvious to me that it is possible to do j-- when j==0, but >>>>>> the change seems reasonable. >>>>> Yes, the scan does not understand there is j++ right after j-- because >>>>> of the loop iteration. I saw it complaining about this pattern several >> times. >>>>> >>>>>> Lots of spacing changes in that code make it hard to see the real >> changes. >>>>> Before, I was asked to fix indentation issues in a function I touch. >>>>> Does that only hold for compiler files? >>>> >>>> Yes/no/maybe :) Fixing up bad formatting when you are touching an area >>>> can be convenient, however it can also obscure the real changes, so it >>>> depends on the ratio of functional changes to format changes. >>>> >>>>>> 145 // SAPJVM GL j-- can underflow, which will cause the loop to >> abort. >>>>>> Seems unnecessary with the code change as noone will understand >> what >>>> j-- >>>>>> you are referring to. >>>>> Didn't mean to leave this in here. Removed. >>>>> >>>>>> 150 nb->_keyvals[nbcnt + nbcnt ] = key; >>>>>> 151 nb->_keyvals[nbcnt + nbcnt + 1] = b->_keyvals[j+j+1]; >>>>>> hotspot-style doesn't align array index expressions like that. Ditto >>>>>> 154/155. >>>>> Fixed. >>>>> >>>>>>> generateOopMap.cpp: >>>>>>> Idx is read from String. This is only called with constant strings, so >>>> compare >>>>>>> should be folded away by optimizing compilers if inlined. >>>>>> >>>>>> Not a fan of adding conditions that should never be false (hence the >>>>>> assert) and relying on the compiler to elide them. >>>>> OK, removed. >>>>> >>>>>>> deoptimization.cpp: >>>>>>> If buflen == 0, buf[-1] is accessed. >>>>>> >>>>>> Okay - but an assert(buflen>0) would be better I think as we should >>>>>> never be calling with a zero-length buffer. >>>>> Ok, I added the assert. As this isn't critical code, I would like to leave the >>>>> check in there, still. >>>>> >>>>>>> task.cpp: >>>>>>> Fatal can return if -XX:SuppressErrorAt is used. Just don't access the >>>>>>> array in this case. >>>>>> >>>>>> Okay. I would not be surprised if we have a lot of potential errors if a >>>>>> fatal/guarantee failure is suppressed. >>>>>> >>>>>>> attachListener.hpp: >>>>>>> Do strncpy to not overflow buffer. Don't write more chars than >> before. >>>>>> >>>>>> Again we have the assert to catch an error in the caller using an >>>>>> invalid name. >>>>> Hmm, the command comes from outside of the VM. It's not checked >>>>> very thoroughly, see, e.g., attachListener_windows.cpp:194. Arg0 is >>>>> checked twice, arg1 and arg2 are not checked at all. >>>> >>>> The libattach code is still part of our codebase so should be doing the >>>> right things. The linux and solaris code seems to be doing the expected >>>> name length check. On Windows the name is set using cmd, which is also >>>> subject to a length check: >>>> >>>> if (strlen(cmd) > AttachOperation::name_length_max) return >>>> ATTACH_ERROR_ILLEGALARG; >>>> >>>>> I add fixes for attachListener_windows.cpp to this change. >>>>> >>>>>>> heapDumper.cpp: >>>>>>> strncpy does not null terminate. >>>>>> >>>>>> 1973 if (total_length >= sizeof(base_path)) { >>>>>> >>>>>> total_length already adds +1 for the nul character so the == case is >>>>>> fine AFAICS. >>>>>> >>>>>> strncpy wont nul-terminate if the string exceeds the buffer size. But >> we >>>>>> have already established that total_length <= sizeof(base_path), and >>>>>> total_path includes space for a bunch of stuff other than >> HeapDumpPath, >>>>>> so the strncpy of HeapDumpPath has to copy the nul character. >>>>> Ok, removed. >>>>> >>>>>> > src/share/vm/services/memoryService.cpp >>>>>> >>>>>> Ok. >>>>>> >>>>>> > src/share/vm/utilities/xmlstream.cpp >>>>>> >>>>>> Ok - I'm more concerned about the "magic" 10 in that piece of code. >>>>> I assume the 10 is the space needed for the "_done" plus some waste ... >>>>> >>>>> I'll do another run of the scan. That takes a day. I'll post a new webrev >>>> after >>>>> that. >>>> >>>> Thanks, >>>> David >>>> >>>>> Thank again for this thorough review, >>>>> Goetz >>>>> >>>>> >>>>> >>>>> >>>>>>> Some of these, as the issue in codeBuffer.cpp, are actually handled >>>> correctly. >>>>>>> Nevertheless this is not that obvious so that somebody changing the >>>> code >>>>>>> Could oversee he has to add the initialization. >>>>>> >>>>>> Not an argument I buy en-masse as it leads to a lot of redundancy >>>>>> through the code paths. Plus these tools that are being run should >> show >>>>>> if a code changes requires initialization that is not present. >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> >>>>>>> Some of these fixes are part of SAP JVM for a long time. This change >> has >>>>>>> been tested with our nightly build of openJDK. >>>>>>> >>>>>>> Best regards, >>>>>>> Goetz,. >>>>>>> From marcus.larsson at oracle.com Wed Nov 4 09:49:41 2015 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Wed, 4 Nov 2015 10:49:41 +0100 Subject: RFR: 8138916: Logging write function does not allow for long enough messages In-Reply-To: <5637A57E.4050103@oracle.com> References: <562E9406.5040207@oracle.com> <562F6567.6040707@oracle.com> <5630E029.5040500@oracle.com> <5630EC54.4020402@oracle.com> <5630EF9E.6090401@oracle.com> <5630F328.8@oracle.com> <5630F77D.1050907@oracle.com> <56310AFC.10609@oracle.com> <56312A94.1060202@oracle.com> <56312B2B.5090305@oracle.com> <56313726.3080507@oracle.com> <5631F66E.8090601@oracle.com> <5637A57E.4050103@oracle.com> Message-ID: <5639D4B5.7080809@oracle.com> Looks good, thank you for fixing this! Regards, Marcus On 2015-11-02 19:03, Rachel Protacio wrote: > Thanks for the comments, Marcus. I've undone the "logtest" tag and > instead created an InternalVMTest in log.cpp that writes the > 518-character string to a temporary file and looks for it there. Using > "logging" tag in the "develop" level. Testing succeeds. > > Webrev: http://cr.openjdk.java.net/~rprotacio/8139564.03/ > > Thank you, > Rachel > > On 10/29/2015 6:35 AM, Marcus Larsson wrote: >> Hi, >> >> On 2015-10-28 21:59, Rachel Protacio wrote: >>> Including serviceability-dev. Recap: >>> >>> Summary: In logging/log.hpp, the logging vwrite function previously >>> asserted that the buffer remains within 512 characters, which is >>> too short for logging message of non-pre-determined length, e.g. for >>> vtable and itable function names. Now, the function resizes the buffer >>> to the appropriate length, asserting only that the resulting buffer >>> is valid. Includes tag "logtest" to test that logging can print an >>> output >>> of 518 characters. >>> >>> Open webrev: http://cr.openjdk.java.net/~rprotacio/8138916.02 >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8138916 >> >> I don't like that the test log message is included in the product, or >> that a completely new tag is introduced for it. >> >> I think a much better approach would be to do the test logging in an >> internal VM test, reusing the logging tag. This way it won't be >> logged unless we are actually testing stuff, and we don't introduce a >> new tag. Also it would look better if it were logged on develop level. >> >> Thanks, >> Marcus >> >>> >>> Includes own acceptance test, and passes JPRT and remote >>> hotspot/test tests. >>> >>> Thank you, >>> Rachel >>> >>> On 10/28/2015 4:08 PM, Rachel Protacio wrote: >>>> Thanks for the review and all the feedback! >>>> Rachel >>>> >>>> On 10/28/2015 4:05 PM, Dmitry Samersoff wrote: >>>>> Rachel, >>>>> >>>>> Looks good for me! >>>>> >>>>> -Dmitry >>>>> >>>>> On 2015-10-28 20:50, Rachel Protacio wrote: >>>>>> Thank you both. I've removed the re-declaration and added >>>>>> prefix_len to >>>>>> newbuf_len. As for line 115, that line is not for the sake of >>>>>> recalculating prefix_len, but for writing the prefix to the buffer. >>>>>> >>>>>> http://cr.openjdk.java.net/~rprotacio/8138916.02/ >>>>>> >>>>>> Thanks, >>>>>> Rachel >>>>>> >>>>>> On 10/28/2015 12:27 PM, Coleen Phillimore wrote: >>>>>>> Dmitry, Thank you for reviewing and providing comments. The second >>>>>>> version is better because of these things that I didn't notice >>>>>>> until >>>>>>> you pointed it out. >>>>>>> >>>>>>> The malloc won't be frequent. I think it'll be okay and follows >>>>>>> other >>>>>>> coding patterns in the jvm. >>>>>>> >>>>>>> Thanks! >>>>>>> Coleen >>>>>>> >>>>>>> On 10/28/15 12:09 PM, Dmitry Samersoff wrote: >>>>>>>> Coleen, >>>>>>>> >>>>>>>> OK. I'm leaving it in good hands ;) >>>>>>>> >>>>>>>> -Dmitry >>>>>>>> >>>>>>>> On 2015-10-28 18:54, Coleen Phillimore wrote: >>>>>>>>> Hi, I'm also reviewing this. I think this looks good. I >>>>>>>>> don't think >>>>>>>>> the solution to allocate a static buffer is very nice because >>>>>>>>> it still >>>>>>>>> has limits and we don't want to allocate something large on >>>>>>>>> the stack. >>>>>>>>> The malloc case is the infrequent case. If it becomes >>>>>>>>> frequent we can >>>>>>>>> increase LogBufferSize from 512. >>>>>>>>> >>>>>>>>> I think 111 the assert is good because if the windows code is >>>>>>>>> wrong (or >>>>>>>>> the vscprintf does something wrong an unexpected) we can catch >>>>>>>>> the >>>>>>>>> error >>>>>>>>> with the assert before writing over memory. >>>>>>>>> >>>>>>>>> I agree 116 shouldn't redeclare 'ret'. >>>>>>>>> >>>>>>>>> Otherwise, I think it looks really good and I don't need to >>>>>>>>> see another >>>>>>>>> revision. >>>>>>>>> >>>>>>>>> Thanks! >>>>>>>>> Coleen >>>>>>>>> >>>>>>>>> >>>>>>>>> On 10/28/15 11:40 AM, Dmitry Samersoff wrote: >>>>>>>>>> Rachel, >>>>>>>>>> >>>>>>>>>> On 2015-10-28 17:48, Rachel Protacio wrote: >>>>>>>>>>> Thank you for the comments, both Dmitry and Mikael. Replies >>>>>>>>>>> inline. >>>>>>>>>>> >>>>>>>>>>> Modified webrev: >>>>>>>>>>> http://cr.openjdk.java.net/~rprotacio/8138916.01/ >>>>>>>>>> 111 Assert is never happens on Windows, so it's better to >>>>>>>>>> move it to >>>>>>>>>> os_posix >>>>>>>>>> >>>>>>>>>> 113 newbuf_len doesn't account prefix_len >>>>>>>>>> >>>>>>>>>> 115 you may consider to use pre-calculated at ll. 108 prefix >>>>>>>>>> rather >>>>>>>>>> than >>>>>>>>>> calculate it again. >>>>>>>>>> >>>>>>>>>> 116 int ret redefined it causes a warning. >>>>>>>>>> >>>>>>>>>> -Dmitry >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> (Retested and works fine.) >>>>>>>>>>> >>>>>>>>>>> On 10/27/2015 7:52 AM, Dmitry Samersoff wrote: >>>>>>>>>>>> Rachel, >>>>>>>>>>>> >>>>>>>>>>>> In worst case (windows, overflow) the code would parse >>>>>>>>>>>> format string >>>>>>>>>>>> four times. >>>>>>>>>>> You're right. I've added a check. New amount of times >>>>>>>>>>> vnsprintf() is >>>>>>>>>>> called: >>>>>>>>>>> posix, fits => 1 >>>>>>>>>>> posix, overflow => 2 >>>>>>>>>>> windows, fits => 1 >>>>>>>>>>> windows, overflow => 3 (the best performance we can get) >>>>>>>>>>> >>>>>>>>>>>> Also NEW_C_HEAP_ARRAY takes a lock and might become a >>>>>>>>>>>> performance bottleneck. >>>>>>>>>>>> >>>>>>>>>>>> POSIX equivalent of _vscprintf(fmt, args) is >>>>>>>>>>>> vsnprintf(NULL, 0, fmt, >>>>>>>>>>>> args) >>>>>>>>>>>> >>>>>>>>>>>> So it might be better to create os::log_vcprintf(fmt, >>>>>>>>>>>> args), then >>>>>>>>>>>> write >>>>>>>>>>>> code at log.hpp as: >>>>>>>>>>>> >>>>>>>>>>>> int buf_len = os::log_vcprintf(fmt, args); >>>>>>>>>>>> assert(buf_len < 2*page_size, "Stack overflow in logging"); >>>>>>>>>>>> char buf[buf_len]; >>>>>>>>>>>> vsprintf(buf, fmt, args); >>>>>>>>>>> I don't think this would work anyway. A char[] needs a >>>>>>>>>>> static size. >>>>>>>>>>> Thanks, >>>>>>>>>>> Rachel >>>>>>>>>>>> -Dmitry >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 2015-10-26 23:58, Rachel Protacio wrote: >>>>>>>>>>>>> Hello, all, >>>>>>>>>>>>> >>>>>>>>>>>>> Please review this fix to the faulty write function from the >>>>>>>>>>>>> Unified >>>>>>>>>>>>> Logging framework. >>>>>>>>>>>>> >>>>>>>>>>>>> Summary: In logging/log.hpp, the logging vwrite function >>>>>>>>>>>>> previously >>>>>>>>>>>>> asserted that the buffer remains within 512 characters, >>>>>>>>>>>>> which is >>>>>>>>>>>>> too >>>>>>>>>>>>> short for logging message of non-pre-determined length, >>>>>>>>>>>>> e.g. for >>>>>>>>>>>>> vtable >>>>>>>>>>>>> and itable function names. Now, the function resizes the >>>>>>>>>>>>> buffer >>>>>>>>>>>>> to the >>>>>>>>>>>>> appropriate length, asserting only that the resulting >>>>>>>>>>>>> buffer is >>>>>>>>>>>>> valid. >>>>>>>>>>>>> Includes tag "logtest" to test that logging can print an >>>>>>>>>>>>> output >>>>>>>>>>>>> of 518 >>>>>>>>>>>>> characters. >>>>>>>>>>>>> >>>>>>>>>>>>> Open webrev:http://cr.openjdk.java.net/~rprotacio/8138916/ >>>>>>>>>>>>> Bug:https://bugs.openjdk.java.net/browse/JDK-8138916 >>>>>>>>>>>>> >>>>>>>>>>>>> Includes own acceptance test, and passes JPRT and remote >>>>>>>>>>>>> hotspot/test >>>>>>>>>>>>> tests. >>>>>>>>>>>>> >>>>>>>>>>>>> Thank you, >>>>>>>>>>>>> Rachel >>>>> >>>> >>> >> > From marcus.larsson at oracle.com Wed Nov 4 09:59:44 2015 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Wed, 4 Nov 2015 10:59:44 +0100 Subject: RFR: 8138916: Logging write function does not allow for long enough messages In-Reply-To: <5639D4B5.7080809@oracle.com> References: <562E9406.5040207@oracle.com> <562F6567.6040707@oracle.com> <5630E029.5040500@oracle.com> <5630EC54.4020402@oracle.com> <5630EF9E.6090401@oracle.com> <5630F328.8@oracle.com> <5630F77D.1050907@oracle.com> <56310AFC.10609@oracle.com> <56312A94.1060202@oracle.com> <56312B2B.5090305@oracle.com> <56313726.3080507@oracle.com> <5631F66E.8090601@oracle.com> <5637A57E.4050103@oracle.com> <5639D4B5.7080809@oracle.com> Message-ID: <5639D710.2070100@oracle.com> Hmm, I just noticed that the test doesn't null terminate the string it reads from file. Replacing the fgetc loop with a call to fgets should fix this. Regards, Marcus On 2015-11-04 10:49, Marcus Larsson wrote: > Looks good, thank you for fixing this! > > Regards, > Marcus > > > On 2015-11-02 19:03, Rachel Protacio wrote: >> Thanks for the comments, Marcus. I've undone the "logtest" tag and >> instead created an InternalVMTest in log.cpp that writes the >> 518-character string to a temporary file and looks for it there. >> Using "logging" tag in the "develop" level. Testing succeeds. >> >> Webrev: http://cr.openjdk.java.net/~rprotacio/8139564.03/ >> >> Thank you, >> Rachel >> >> On 10/29/2015 6:35 AM, Marcus Larsson wrote: >>> Hi, >>> >>> On 2015-10-28 21:59, Rachel Protacio wrote: >>>> Including serviceability-dev. Recap: >>>> >>>> Summary: In logging/log.hpp, the logging vwrite function previously >>>> asserted that the buffer remains within 512 characters, which is >>>> too short for logging message of non-pre-determined length, e.g. >>>> for vtable and itable function names. Now, the function resizes the >>>> buffer >>>> to the appropriate length, asserting only that the resulting buffer >>>> is valid. Includes tag "logtest" to test that logging can print an >>>> output >>>> of 518 characters. >>>> >>>> Open webrev: http://cr.openjdk.java.net/~rprotacio/8138916.02 >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8138916 >>> >>> I don't like that the test log message is included in the product, >>> or that a completely new tag is introduced for it. >>> >>> I think a much better approach would be to do the test logging in an >>> internal VM test, reusing the logging tag. This way it won't be >>> logged unless we are actually testing stuff, and we don't introduce >>> a new tag. Also it would look better if it were logged on develop >>> level. >>> >>> Thanks, >>> Marcus >>> >>>> >>>> Includes own acceptance test, and passes JPRT and remote >>>> hotspot/test tests. >>>> >>>> Thank you, >>>> Rachel >>>> >>>> On 10/28/2015 4:08 PM, Rachel Protacio wrote: >>>>> Thanks for the review and all the feedback! >>>>> Rachel >>>>> >>>>> On 10/28/2015 4:05 PM, Dmitry Samersoff wrote: >>>>>> Rachel, >>>>>> >>>>>> Looks good for me! >>>>>> >>>>>> -Dmitry >>>>>> >>>>>> On 2015-10-28 20:50, Rachel Protacio wrote: >>>>>>> Thank you both. I've removed the re-declaration and added >>>>>>> prefix_len to >>>>>>> newbuf_len. As for line 115, that line is not for the sake of >>>>>>> recalculating prefix_len, but for writing the prefix to the buffer. >>>>>>> >>>>>>> http://cr.openjdk.java.net/~rprotacio/8138916.02/ >>>>>>> >>>>>>> Thanks, >>>>>>> Rachel >>>>>>> >>>>>>> On 10/28/2015 12:27 PM, Coleen Phillimore wrote: >>>>>>>> Dmitry, Thank you for reviewing and providing comments. The >>>>>>>> second >>>>>>>> version is better because of these things that I didn't notice >>>>>>>> until >>>>>>>> you pointed it out. >>>>>>>> >>>>>>>> The malloc won't be frequent. I think it'll be okay and >>>>>>>> follows other >>>>>>>> coding patterns in the jvm. >>>>>>>> >>>>>>>> Thanks! >>>>>>>> Coleen >>>>>>>> >>>>>>>> On 10/28/15 12:09 PM, Dmitry Samersoff wrote: >>>>>>>>> Coleen, >>>>>>>>> >>>>>>>>> OK. I'm leaving it in good hands ;) >>>>>>>>> >>>>>>>>> -Dmitry >>>>>>>>> >>>>>>>>> On 2015-10-28 18:54, Coleen Phillimore wrote: >>>>>>>>>> Hi, I'm also reviewing this. I think this looks good. I >>>>>>>>>> don't think >>>>>>>>>> the solution to allocate a static buffer is very nice because >>>>>>>>>> it still >>>>>>>>>> has limits and we don't want to allocate something large on >>>>>>>>>> the stack. >>>>>>>>>> The malloc case is the infrequent case. If it becomes >>>>>>>>>> frequent we can >>>>>>>>>> increase LogBufferSize from 512. >>>>>>>>>> >>>>>>>>>> I think 111 the assert is good because if the windows code is >>>>>>>>>> wrong (or >>>>>>>>>> the vscprintf does something wrong an unexpected) we can >>>>>>>>>> catch the >>>>>>>>>> error >>>>>>>>>> with the assert before writing over memory. >>>>>>>>>> >>>>>>>>>> I agree 116 shouldn't redeclare 'ret'. >>>>>>>>>> >>>>>>>>>> Otherwise, I think it looks really good and I don't need to >>>>>>>>>> see another >>>>>>>>>> revision. >>>>>>>>>> >>>>>>>>>> Thanks! >>>>>>>>>> Coleen >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 10/28/15 11:40 AM, Dmitry Samersoff wrote: >>>>>>>>>>> Rachel, >>>>>>>>>>> >>>>>>>>>>> On 2015-10-28 17:48, Rachel Protacio wrote: >>>>>>>>>>>> Thank you for the comments, both Dmitry and Mikael. Replies >>>>>>>>>>>> inline. >>>>>>>>>>>> >>>>>>>>>>>> Modified webrev: >>>>>>>>>>>> http://cr.openjdk.java.net/~rprotacio/8138916.01/ >>>>>>>>>>> 111 Assert is never happens on Windows, so it's better to >>>>>>>>>>> move it to >>>>>>>>>>> os_posix >>>>>>>>>>> >>>>>>>>>>> 113 newbuf_len doesn't account prefix_len >>>>>>>>>>> >>>>>>>>>>> 115 you may consider to use pre-calculated at ll. 108 prefix >>>>>>>>>>> rather >>>>>>>>>>> than >>>>>>>>>>> calculate it again. >>>>>>>>>>> >>>>>>>>>>> 116 int ret redefined it causes a warning. >>>>>>>>>>> >>>>>>>>>>> -Dmitry >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> (Retested and works fine.) >>>>>>>>>>>> >>>>>>>>>>>> On 10/27/2015 7:52 AM, Dmitry Samersoff wrote: >>>>>>>>>>>>> Rachel, >>>>>>>>>>>>> >>>>>>>>>>>>> In worst case (windows, overflow) the code would parse >>>>>>>>>>>>> format string >>>>>>>>>>>>> four times. >>>>>>>>>>>> You're right. I've added a check. New amount of times >>>>>>>>>>>> vnsprintf() is >>>>>>>>>>>> called: >>>>>>>>>>>> posix, fits => 1 >>>>>>>>>>>> posix, overflow => 2 >>>>>>>>>>>> windows, fits => 1 >>>>>>>>>>>> windows, overflow => 3 (the best performance we can get) >>>>>>>>>>>> >>>>>>>>>>>>> Also NEW_C_HEAP_ARRAY takes a lock and might become a >>>>>>>>>>>>> performance bottleneck. >>>>>>>>>>>>> >>>>>>>>>>>>> POSIX equivalent of _vscprintf(fmt, args) is >>>>>>>>>>>>> vsnprintf(NULL, 0, fmt, >>>>>>>>>>>>> args) >>>>>>>>>>>>> >>>>>>>>>>>>> So it might be better to create os::log_vcprintf(fmt, >>>>>>>>>>>>> args), then >>>>>>>>>>>>> write >>>>>>>>>>>>> code at log.hpp as: >>>>>>>>>>>>> >>>>>>>>>>>>> int buf_len = os::log_vcprintf(fmt, args); >>>>>>>>>>>>> assert(buf_len < 2*page_size, "Stack overflow in logging"); >>>>>>>>>>>>> char buf[buf_len]; >>>>>>>>>>>>> vsprintf(buf, fmt, args); >>>>>>>>>>>> I don't think this would work anyway. A char[] needs a >>>>>>>>>>>> static size. >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Rachel >>>>>>>>>>>>> -Dmitry >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 2015-10-26 23:58, Rachel Protacio wrote: >>>>>>>>>>>>>> Hello, all, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Please review this fix to the faulty write function from the >>>>>>>>>>>>>> Unified >>>>>>>>>>>>>> Logging framework. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Summary: In logging/log.hpp, the logging vwrite function >>>>>>>>>>>>>> previously >>>>>>>>>>>>>> asserted that the buffer remains within 512 characters, >>>>>>>>>>>>>> which is >>>>>>>>>>>>>> too >>>>>>>>>>>>>> short for logging message of non-pre-determined length, >>>>>>>>>>>>>> e.g. for >>>>>>>>>>>>>> vtable >>>>>>>>>>>>>> and itable function names. Now, the function resizes the >>>>>>>>>>>>>> buffer >>>>>>>>>>>>>> to the >>>>>>>>>>>>>> appropriate length, asserting only that the resulting >>>>>>>>>>>>>> buffer is >>>>>>>>>>>>>> valid. >>>>>>>>>>>>>> Includes tag "logtest" to test that logging can print an >>>>>>>>>>>>>> output >>>>>>>>>>>>>> of 518 >>>>>>>>>>>>>> characters. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Open webrev:http://cr.openjdk.java.net/~rprotacio/8138916/ >>>>>>>>>>>>>> Bug:https://bugs.openjdk.java.net/browse/JDK-8138916 >>>>>>>>>>>>>> >>>>>>>>>>>>>> Includes own acceptance test, and passes JPRT and remote >>>>>>>>>>>>>> hotspot/test >>>>>>>>>>>>>> tests. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thank you, >>>>>>>>>>>>>> Rachel >>>>>> >>>>> >>>> >>> >> > From sgehwolf at redhat.com Wed Nov 4 10:54:36 2015 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Wed, 04 Nov 2015 11:54:36 +0100 Subject: jmx-dev RFR 6425769: jmx remote bind address In-Reply-To: <5638C89F.2090406@oracle.com> References: <1446460682.10865.20.camel@redhat.com> <56377F17.4050006@oracle.com> <5637871B.30705@oracle.com> <1446487615.10865.57.camel@redhat.com> <5638C89F.2090406@oracle.com> Message-ID: <1446634476.3554.8.camel@redhat.com> Hi, Updated webrev with jtreg test in Java: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-6425769/02/ bug:?https://bugs.openjdk.java.net/browse/JDK-6425769 I believe this updated webrev addresses all concerns and incorporates suggestions brought up by Jaroslav and Daniel. I'm still looking for a sponsor and a hotspot/servicability-dev reviewer. Could somebody maintaining javax.rmi.ssl have a look at this as well? Thank you! Cheers, Severin On Tue, 2015-11-03 at 15:45 +0100, Jaroslav Bachorik wrote: > On 2.11.2015 19:06, Severin Gehwolf wrote: > > Hi, > > > > Thanks Jaroslav and Daniel for the reviews! Comments inline. > > > > On Mon, 2015-11-02 at 16:54 +0100, Jaroslav Bachorik wrote: > > > Hi, > > > > > > On 2.11.2015 16:19, Daniel Fuchs wrote: > > > > Hi Severin, > > > > > > > > Adding serviceability-dev at openjdk.java.net into the loop - > > > > that's > > > > a better alias than hotspot-dev for this kind of changes - > > > > maybe > > > > someone from serviceability-dev will offer to sponsor :-) > > > > > > > > I will let serviceability team members comment on the hotspot > > > > changes. > > > > > > > > ConnectorBootstrap.java > > > > > > > > I have one suggestion and one concern: > > > > > > > > Suggestion: I would suggest to reuse 'csf' (Client Socket > > > > Factory) > > > > and > > > > ssf??(Server Socket Factory) variables rather than introduce > > > > the > > > > two > > > > new variables rmiServerSocketFactory and > > > > rmiClientSocketFactory. > > > > You might want to create a new boolean 'useSocketFactory' > > > > variable, > > > > if that makes the code more readable. > > > > > > > > Concern: If I understand correctly how RMI socket factories > > > > work, > > > > the client socket factory will be serialized and sent to the > > > > client > > > > side. This is problematic for interoperability, as the class > > > > may > > > > not > > > > present in the remote client - if the remote client is e.g. jdk > > > > 8. > > > > > > > > As far as I can see, your new DefaultClientSocketFactory > > > > doesn't do > > > > anything useful - so I would suggest to simply get rid of it, > > > > and > > > > only > > > > set the Server Socket Factory when SSL is not involved. > > > > Thanks. Fixed in updated webrev. > > > > > > Tests: > > > > > > > > Concerning the tests - we're trying to get rid of shell scripts > > > > rather than introducing new ones :-) > > > > Could the test be rewritten in pure java using the Process API? > > > > > > > > I believe there's even a test library that will let you do that > > > > easily jdk/test/lib/testlibrary/jdk/testlibrary/ > > > > (see ProcessTools.java) > > > > It'll take me a bit to rewrite the test in pure Java, but should be > > fine. This is not yet fixed in the updated webrev. > > > > > > Other: > > > > > > > > Also - I believe the new option should be documented in: > > > > src/java.management/share/conf/management.properties > > > > Docs have been updated > > in src/java.management/share/conf/management.properties. > > > > > I share Daniel's concerns. Also, the part of the changeset is > > > related > > > to javax.rmi.ssl - someone maintaining this library should also > > > comment here. > > > > > > Also, the change is introducing new API (system property) and > > > changing the existing one (adding SslRmiServerSocketFactory > > > public > > > constructors) so compatibility review process will have to be > > > involved. > > > > OK. What exactly is there for me to do? I'm not familiar with this > > process. Note that the intent would be to get this backported to > > JDK 8. > Not much for you. But for the potential Oracle sponsor this means she > will have to remember to go through some extra hoops before > integrating your patch. I see. Thanks for clarifying it. > -JB- > > > > > New webrev at: > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-6425769/01/ > > > > Thanks, > > Severin > > > > > -JB- > > > > > > > > best regards, > > > > > > > > -- daniel > > > > > > > > On 02/11/15 11:38, Severin Gehwolf wrote: > > > > > Hi, > > > > > > > > > > Here is a patch addressing JDK-6425769. The issue is that the > > > > > JMX > > > > > agent > > > > > binds to the wildcard address by default, preventing users to > > > > > use > > > > > system properties for JMX agents on multi-homed hosts. Given > > > > > a > > > > > host > > > > > with local network interfaces, 192.168.0.1 and 192.168.0.2 > > > > > say, > > > > > it's > > > > > impossible to start two JMX agents binding to fixed ports but > > > > > to > > > > > different network interfaces, 192.168.0.1:{9111,9112} and > > > > > 192.168.0.2:{9111,9112} say. > > > > > > > > > > The JDK would bind to all local interfaces by default. In the > > > > > above > > > > > example to 192.168.0.1 *and* 192.168.0.2. The effect is that > > > > > the > > > > > second > > > > > Java process would get a "Connection refused" error because > > > > > another > > > > > process has already been bound to the specified JMX/RMI port > > > > > pairs. > > > > > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-6425769 > > > > > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-6425 > > > > > 769/ > > > > > 00/ > > > > > > > > > > Testing done: > > > > > jdk_jmx and jdk_management tests all pass after this change > > > > > (no > > > > > regressions). There is also a new JTREG test which fails > > > > > prior > > > > > this > > > > > change and passes after. Updates to the diagnostic command > > > > > have > > > > > been > > > > > tested manually. > > > > > > > > > > I understand that, if approved, the JDK and hotspot changes > > > > > should get > > > > > pushed together? Hotspot changes are fairly trivial since > > > > > it's > > > > > only a > > > > > doc-update for the new JDK property in the relevant > > > > > diagnostic > > > > > command. > > > > > > > > > > Could someone please review and sponsor this change? Please > > > > > let > > > > > me know > > > > > if there are questions. > > > > > > > > > > Thanks, > > > > > Severin > > > > > > > > > > > > > > > From andreas.eriksson at oracle.com Wed Nov 4 10:59:21 2015 From: andreas.eriksson at oracle.com (Andreas Eriksson) Date: Wed, 4 Nov 2015 11:59:21 +0100 Subject: RFR: JDK-8134030: test/serviceability/dcmd/gc/HeapDumpTest fails to verify the dump In-Reply-To: <5639B44E.9090609@oracle.com> References: <563378B1.6080607@oracle.com> <563853E9.6090901@oracle.com> <56389632.3080805@oracle.com> <5639B44E.9090609@oracle.com> Message-ID: <5639E509.30900@oracle.com> On 2015-11-04 08:31, David Holmes wrote: > On 3/11/2015 9:10 PM, Andreas Eriksson wrote: >> >> >> On 2015-11-03 07:27, David Holmes wrote: >>> Hi Andreas, >>> >>> On 31/10/2015 12:03 AM, Andreas Eriksson wrote: >>>> Hi, >>>> >>>> Please review this change to heap dump logic. >>>> Including runtime list, since this is related to class loading. >>>> >>>> Bug: >>>> JDK-8134030: test/serviceability/dcmd/gc/HeapDumpTest fails to verify >>>> the dump >>>> https://bugs.openjdk.java.net/browse/JDK-8134030 >>>> >>>> Webrev: http://cr.openjdk.java.net/~aeriksso/8134030/webrev.00/ >>>> >>>> A heap dump can occur at a point where an InstanceKlass doesn't have a >>>> Java class mirror yet. >>>> To avoid a crash because of this JDK-8131600 [1] excluded classes that >>>> are not linked yet from the dump. >>>> >>>> The problem here is that since a class that has been loaded is already >>>> exposed to the Java side, another class can have a reference to the >>>> excluded class. >>>> So, if it has been *loaded* but not *linked* it will be excluded, but >>>> can still be referenced by other classes in the dump. >>> >>> I'm having trouble seeing exactly when the class is marked as "loaded" >>> but it certainly seems to be after the mirror has been set. However >>> linking happens even later than that - so if you still see a crash >>> excluding linked classes then I fear you will also see problems >>> excluding loaded classes! >>> >>> David >>> ----- >>> >> >> Hi, >> >> The crash does not happen when excluding linked classes, sorry if that >> was unclear. >> >> Let me try to explain it again: >> The crash was fixed by excluding non-linked classes (in JDK-8131600). >> But that excludes some classes that are already referenced in the heap >> dump by other classes. >> By changing to excluding non-loaded classes, the crash can still not >> happen, and all classes that are referenced in the dump are also >> included. > > Thanks for clarifying, I had misunderstood. > > Change seems fine in that case. > Thanks. - Andreas > Thanks, > David > > >> Not dumping these classes is only a problem when we try to look at the >> heap dump at a later point (and even then it is only a problem of >> missing information, not a crash). >> For example, HeapDumpTest gives the following warning when it tries to >> follow a reference to a class that was excluded from the dump: WARNING: >> Failed to resolve object id 0x6c003d348 [...]. >> VisualVM on the other hand will ignore the value, and tell the user the >> field was null (which is very misleading). >> >> I hope that made sense. >> >> Thanks, >> Andreas >> >>> >>>> This gives the following warning in HeapDumpTest: >>>> WARNING: Failed to resolve object id 0x6c003d348 [...]. >>>> >>>> This fix changes heapDumper.cpp to only exclude classes that are >>>> not yet >>>> loaded instead. >>>> >>>> I'd like confirmation on this from someone who knows more about loaded >>>> vs linked, but I think that this is correct: >>>> When a class has been loaded, we are guaranteed that a Java class >>>> mirror >>>> is setup, so the crash seen in JDK-8131600 still cannot happen. >>>> This seems to be confirmed by the manual reproducer from JDK-8131600, >>>> which still succeeds with this fix. >>>> >>>> Also in this change is the re-addition of the actual heap dump >>>> verification call in HeapDumpTest, which was accidentally removed when >>>> the test was re-factored. >>>> >>>> Regards, >>>> Andreas >>>> >>>> [1]: https://bugs.openjdk.java.net/browse/JDK-8131600 >> From rachel.protacio at oracle.com Wed Nov 4 16:19:04 2015 From: rachel.protacio at oracle.com (Rachel Protacio) Date: Wed, 4 Nov 2015 11:19:04 -0500 Subject: RFR: 8138916: Logging write function does not allow for long enough messages In-Reply-To: <5639D710.2070100@oracle.com> References: <562E9406.5040207@oracle.com> <562F6567.6040707@oracle.com> <5630E029.5040500@oracle.com> <5630EC54.4020402@oracle.com> <5630EF9E.6090401@oracle.com> <5630F328.8@oracle.com> <5630F77D.1050907@oracle.com> <56310AFC.10609@oracle.com> <56312A94.1060202@oracle.com> <56312B2B.5090305@oracle.com> <56313726.3080507@oracle.com> <5631F66E.8090601@oracle.com> <5637A57E.4050103@oracle.com> <5639D4B5.7080809@oracle.com> <5639D710.2070100@oracle.com> Message-ID: <563A2FF8.50606@oracle.com> Thanks for the review, Marcus. I'll make that change. Rachel On 11/4/2015 4:59 AM, Marcus Larsson wrote: > Hmm, I just noticed that the test doesn't null terminate the string it > reads from file. Replacing the fgetc loop with a call to fgets should > fix this. > > Regards, > Marcus > > > On 2015-11-04 10:49, Marcus Larsson wrote: >> Looks good, thank you for fixing this! >> >> Regards, >> Marcus >> >> >> On 2015-11-02 19:03, Rachel Protacio wrote: >>> Thanks for the comments, Marcus. I've undone the "logtest" tag and >>> instead created an InternalVMTest in log.cpp that writes the >>> 518-character string to a temporary file and looks for it there. >>> Using "logging" tag in the "develop" level. Testing succeeds. >>> >>> Webrev: http://cr.openjdk.java.net/~rprotacio/8139564.03/ >>> >>> Thank you, >>> Rachel >>> >>> On 10/29/2015 6:35 AM, Marcus Larsson wrote: >>>> Hi, >>>> >>>> On 2015-10-28 21:59, Rachel Protacio wrote: >>>>> Including serviceability-dev. Recap: >>>>> >>>>> Summary: In logging/log.hpp, the logging vwrite function >>>>> previously asserted that the buffer remains within 512 characters, >>>>> which is >>>>> too short for logging message of non-pre-determined length, e.g. >>>>> for vtable and itable function names. Now, the function resizes >>>>> the buffer >>>>> to the appropriate length, asserting only that the resulting >>>>> buffer is valid. Includes tag "logtest" to test that logging can >>>>> print an output >>>>> of 518 characters. >>>>> >>>>> Open webrev: http://cr.openjdk.java.net/~rprotacio/8138916.02 >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8138916 >>>> >>>> I don't like that the test log message is included in the product, >>>> or that a completely new tag is introduced for it. >>>> >>>> I think a much better approach would be to do the test logging in >>>> an internal VM test, reusing the logging tag. This way it won't be >>>> logged unless we are actually testing stuff, and we don't introduce >>>> a new tag. Also it would look better if it were logged on develop >>>> level. >>>> >>>> Thanks, >>>> Marcus >>>> >>>>> >>>>> Includes own acceptance test, and passes JPRT and remote >>>>> hotspot/test tests. >>>>> >>>>> Thank you, >>>>> Rachel >>>>> >>>>> On 10/28/2015 4:08 PM, Rachel Protacio wrote: >>>>>> Thanks for the review and all the feedback! >>>>>> Rachel >>>>>> >>>>>> On 10/28/2015 4:05 PM, Dmitry Samersoff wrote: >>>>>>> Rachel, >>>>>>> >>>>>>> Looks good for me! >>>>>>> >>>>>>> -Dmitry >>>>>>> >>>>>>> On 2015-10-28 20:50, Rachel Protacio wrote: >>>>>>>> Thank you both. I've removed the re-declaration and added >>>>>>>> prefix_len to >>>>>>>> newbuf_len. As for line 115, that line is not for the sake of >>>>>>>> recalculating prefix_len, but for writing the prefix to the >>>>>>>> buffer. >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~rprotacio/8138916.02/ >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Rachel >>>>>>>> >>>>>>>> On 10/28/2015 12:27 PM, Coleen Phillimore wrote: >>>>>>>>> Dmitry, Thank you for reviewing and providing comments. The >>>>>>>>> second >>>>>>>>> version is better because of these things that I didn't notice >>>>>>>>> until >>>>>>>>> you pointed it out. >>>>>>>>> >>>>>>>>> The malloc won't be frequent. I think it'll be okay and >>>>>>>>> follows other >>>>>>>>> coding patterns in the jvm. >>>>>>>>> >>>>>>>>> Thanks! >>>>>>>>> Coleen >>>>>>>>> >>>>>>>>> On 10/28/15 12:09 PM, Dmitry Samersoff wrote: >>>>>>>>>> Coleen, >>>>>>>>>> >>>>>>>>>> OK. I'm leaving it in good hands ;) >>>>>>>>>> >>>>>>>>>> -Dmitry >>>>>>>>>> >>>>>>>>>> On 2015-10-28 18:54, Coleen Phillimore wrote: >>>>>>>>>>> Hi, I'm also reviewing this. I think this looks good. I >>>>>>>>>>> don't think >>>>>>>>>>> the solution to allocate a static buffer is very nice >>>>>>>>>>> because it still >>>>>>>>>>> has limits and we don't want to allocate something large on >>>>>>>>>>> the stack. >>>>>>>>>>> The malloc case is the infrequent case. If it becomes >>>>>>>>>>> frequent we can >>>>>>>>>>> increase LogBufferSize from 512. >>>>>>>>>>> >>>>>>>>>>> I think 111 the assert is good because if the windows code >>>>>>>>>>> is wrong (or >>>>>>>>>>> the vscprintf does something wrong an unexpected) we can >>>>>>>>>>> catch the >>>>>>>>>>> error >>>>>>>>>>> with the assert before writing over memory. >>>>>>>>>>> >>>>>>>>>>> I agree 116 shouldn't redeclare 'ret'. >>>>>>>>>>> >>>>>>>>>>> Otherwise, I think it looks really good and I don't need to >>>>>>>>>>> see another >>>>>>>>>>> revision. >>>>>>>>>>> >>>>>>>>>>> Thanks! >>>>>>>>>>> Coleen >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 10/28/15 11:40 AM, Dmitry Samersoff wrote: >>>>>>>>>>>> Rachel, >>>>>>>>>>>> >>>>>>>>>>>> On 2015-10-28 17:48, Rachel Protacio wrote: >>>>>>>>>>>>> Thank you for the comments, both Dmitry and Mikael. >>>>>>>>>>>>> Replies inline. >>>>>>>>>>>>> >>>>>>>>>>>>> Modified webrev: >>>>>>>>>>>>> http://cr.openjdk.java.net/~rprotacio/8138916.01/ >>>>>>>>>>>> 111 Assert is never happens on Windows, so it's better to >>>>>>>>>>>> move it to >>>>>>>>>>>> os_posix >>>>>>>>>>>> >>>>>>>>>>>> 113 newbuf_len doesn't account prefix_len >>>>>>>>>>>> >>>>>>>>>>>> 115 you may consider to use pre-calculated at ll. 108 >>>>>>>>>>>> prefix rather >>>>>>>>>>>> than >>>>>>>>>>>> calculate it again. >>>>>>>>>>>> >>>>>>>>>>>> 116 int ret redefined it causes a warning. >>>>>>>>>>>> >>>>>>>>>>>> -Dmitry >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> (Retested and works fine.) >>>>>>>>>>>>> >>>>>>>>>>>>> On 10/27/2015 7:52 AM, Dmitry Samersoff wrote: >>>>>>>>>>>>>> Rachel, >>>>>>>>>>>>>> >>>>>>>>>>>>>> In worst case (windows, overflow) the code would parse >>>>>>>>>>>>>> format string >>>>>>>>>>>>>> four times. >>>>>>>>>>>>> You're right. I've added a check. New amount of times >>>>>>>>>>>>> vnsprintf() is >>>>>>>>>>>>> called: >>>>>>>>>>>>> posix, fits => 1 >>>>>>>>>>>>> posix, overflow => 2 >>>>>>>>>>>>> windows, fits => 1 >>>>>>>>>>>>> windows, overflow => 3 (the best performance we can get) >>>>>>>>>>>>> >>>>>>>>>>>>>> Also NEW_C_HEAP_ARRAY takes a lock and might become a >>>>>>>>>>>>>> performance bottleneck. >>>>>>>>>>>>>> >>>>>>>>>>>>>> POSIX equivalent of _vscprintf(fmt, args) is >>>>>>>>>>>>>> vsnprintf(NULL, 0, fmt, >>>>>>>>>>>>>> args) >>>>>>>>>>>>>> >>>>>>>>>>>>>> So it might be better to create os::log_vcprintf(fmt, >>>>>>>>>>>>>> args), then >>>>>>>>>>>>>> write >>>>>>>>>>>>>> code at log.hpp as: >>>>>>>>>>>>>> >>>>>>>>>>>>>> int buf_len = os::log_vcprintf(fmt, args); >>>>>>>>>>>>>> assert(buf_len < 2*page_size, "Stack overflow in logging"); >>>>>>>>>>>>>> char buf[buf_len]; >>>>>>>>>>>>>> vsprintf(buf, fmt, args); >>>>>>>>>>>>> I don't think this would work anyway. A char[] needs a >>>>>>>>>>>>> static size. >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Rachel >>>>>>>>>>>>>> -Dmitry >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 2015-10-26 23:58, Rachel Protacio wrote: >>>>>>>>>>>>>>> Hello, all, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Please review this fix to the faulty write function from >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>> Unified >>>>>>>>>>>>>>> Logging framework. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Summary: In logging/log.hpp, the logging vwrite function >>>>>>>>>>>>>>> previously >>>>>>>>>>>>>>> asserted that the buffer remains within 512 characters, >>>>>>>>>>>>>>> which is >>>>>>>>>>>>>>> too >>>>>>>>>>>>>>> short for logging message of non-pre-determined length, >>>>>>>>>>>>>>> e.g. for >>>>>>>>>>>>>>> vtable >>>>>>>>>>>>>>> and itable function names. Now, the function resizes the >>>>>>>>>>>>>>> buffer >>>>>>>>>>>>>>> to the >>>>>>>>>>>>>>> appropriate length, asserting only that the resulting >>>>>>>>>>>>>>> buffer is >>>>>>>>>>>>>>> valid. >>>>>>>>>>>>>>> Includes tag "logtest" to test that logging can print an >>>>>>>>>>>>>>> output >>>>>>>>>>>>>>> of 518 >>>>>>>>>>>>>>> characters. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Open webrev:http://cr.openjdk.java.net/~rprotacio/8138916/ >>>>>>>>>>>>>>> Bug:https://bugs.openjdk.java.net/browse/JDK-8138916 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Includes own acceptance test, and passes JPRT and remote >>>>>>>>>>>>>>> hotspot/test >>>>>>>>>>>>>>> tests. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thank you, >>>>>>>>>>>>>>> Rachel >>>>>>> >>>>>> >>>>> >>>> >>> >> > From rachel.protacio at oracle.com Wed Nov 4 19:49:38 2015 From: rachel.protacio at oracle.com (Rachel Protacio) Date: Wed, 4 Nov 2015 14:49:38 -0500 Subject: RFR: 8138916: Logging write function does not allow for long enough messages In-Reply-To: <5639D710.2070100@oracle.com> References: <562E9406.5040207@oracle.com> <562F6567.6040707@oracle.com> <5630E029.5040500@oracle.com> <5630EC54.4020402@oracle.com> <5630EF9E.6090401@oracle.com> <5630F328.8@oracle.com> <5630F77D.1050907@oracle.com> <56310AFC.10609@oracle.com> <56312A94.1060202@oracle.com> <56312B2B.5090305@oracle.com> <56313726.3080507@oracle.com> <5631F66E.8090601@oracle.com> <5637A57E.4050103@oracle.com> <5639D4B5.7080809@oracle.com> <5639D710.2070100@oracle.com> Message-ID: <563A6152.2040509@oracle.com> Updated webrev with this fgets change: http://cr.openjdk.java.net/~rprotacio/8139564.04/ Passes JPRT tests. Thank you, Rachel On 11/4/2015 4:59 AM, Marcus Larsson wrote: > Hmm, I just noticed that the test doesn't null terminate the string it > reads from file. Replacing the fgetc loop with a call to fgets should > fix this. > > Regards, > Marcus > > > On 2015-11-04 10:49, Marcus Larsson wrote: >> Looks good, thank you for fixing this! >> >> Regards, >> Marcus >> >> >> On 2015-11-02 19:03, Rachel Protacio wrote: >>> Thanks for the comments, Marcus. I've undone the "logtest" tag and >>> instead created an InternalVMTest in log.cpp that writes the >>> 518-character string to a temporary file and looks for it there. >>> Using "logging" tag in the "develop" level. Testing succeeds. >>> >>> Webrev: http://cr.openjdk.java.net/~rprotacio/8139564.03/ >>> >>> Thank you, >>> Rachel >>> >>> On 10/29/2015 6:35 AM, Marcus Larsson wrote: >>>> Hi, >>>> >>>> On 2015-10-28 21:59, Rachel Protacio wrote: >>>>> Including serviceability-dev. Recap: >>>>> >>>>> Summary: In logging/log.hpp, the logging vwrite function >>>>> previously asserted that the buffer remains within 512 characters, >>>>> which is >>>>> too short for logging message of non-pre-determined length, e.g. >>>>> for vtable and itable function names. Now, the function resizes >>>>> the buffer >>>>> to the appropriate length, asserting only that the resulting >>>>> buffer is valid. Includes tag "logtest" to test that logging can >>>>> print an output >>>>> of 518 characters. >>>>> >>>>> Open webrev: http://cr.openjdk.java.net/~rprotacio/8138916.02 >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8138916 >>>> >>>> I don't like that the test log message is included in the product, >>>> or that a completely new tag is introduced for it. >>>> >>>> I think a much better approach would be to do the test logging in >>>> an internal VM test, reusing the logging tag. This way it won't be >>>> logged unless we are actually testing stuff, and we don't introduce >>>> a new tag. Also it would look better if it were logged on develop >>>> level. >>>> >>>> Thanks, >>>> Marcus >>>> >>>>> >>>>> Includes own acceptance test, and passes JPRT and remote >>>>> hotspot/test tests. >>>>> >>>>> Thank you, >>>>> Rachel >>>>> >>>>> On 10/28/2015 4:08 PM, Rachel Protacio wrote: >>>>>> Thanks for the review and all the feedback! >>>>>> Rachel >>>>>> >>>>>> On 10/28/2015 4:05 PM, Dmitry Samersoff wrote: >>>>>>> Rachel, >>>>>>> >>>>>>> Looks good for me! >>>>>>> >>>>>>> -Dmitry >>>>>>> >>>>>>> On 2015-10-28 20:50, Rachel Protacio wrote: >>>>>>>> Thank you both. I've removed the re-declaration and added >>>>>>>> prefix_len to >>>>>>>> newbuf_len. As for line 115, that line is not for the sake of >>>>>>>> recalculating prefix_len, but for writing the prefix to the >>>>>>>> buffer. >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~rprotacio/8138916.02/ >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Rachel >>>>>>>> >>>>>>>> On 10/28/2015 12:27 PM, Coleen Phillimore wrote: >>>>>>>>> Dmitry, Thank you for reviewing and providing comments. The >>>>>>>>> second >>>>>>>>> version is better because of these things that I didn't notice >>>>>>>>> until >>>>>>>>> you pointed it out. >>>>>>>>> >>>>>>>>> The malloc won't be frequent. I think it'll be okay and >>>>>>>>> follows other >>>>>>>>> coding patterns in the jvm. >>>>>>>>> >>>>>>>>> Thanks! >>>>>>>>> Coleen >>>>>>>>> >>>>>>>>> On 10/28/15 12:09 PM, Dmitry Samersoff wrote: >>>>>>>>>> Coleen, >>>>>>>>>> >>>>>>>>>> OK. I'm leaving it in good hands ;) >>>>>>>>>> >>>>>>>>>> -Dmitry >>>>>>>>>> >>>>>>>>>> On 2015-10-28 18:54, Coleen Phillimore wrote: >>>>>>>>>>> Hi, I'm also reviewing this. I think this looks good. I >>>>>>>>>>> don't think >>>>>>>>>>> the solution to allocate a static buffer is very nice >>>>>>>>>>> because it still >>>>>>>>>>> has limits and we don't want to allocate something large on >>>>>>>>>>> the stack. >>>>>>>>>>> The malloc case is the infrequent case. If it becomes >>>>>>>>>>> frequent we can >>>>>>>>>>> increase LogBufferSize from 512. >>>>>>>>>>> >>>>>>>>>>> I think 111 the assert is good because if the windows code >>>>>>>>>>> is wrong (or >>>>>>>>>>> the vscprintf does something wrong an unexpected) we can >>>>>>>>>>> catch the >>>>>>>>>>> error >>>>>>>>>>> with the assert before writing over memory. >>>>>>>>>>> >>>>>>>>>>> I agree 116 shouldn't redeclare 'ret'. >>>>>>>>>>> >>>>>>>>>>> Otherwise, I think it looks really good and I don't need to >>>>>>>>>>> see another >>>>>>>>>>> revision. >>>>>>>>>>> >>>>>>>>>>> Thanks! >>>>>>>>>>> Coleen >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 10/28/15 11:40 AM, Dmitry Samersoff wrote: >>>>>>>>>>>> Rachel, >>>>>>>>>>>> >>>>>>>>>>>> On 2015-10-28 17:48, Rachel Protacio wrote: >>>>>>>>>>>>> Thank you for the comments, both Dmitry and Mikael. >>>>>>>>>>>>> Replies inline. >>>>>>>>>>>>> >>>>>>>>>>>>> Modified webrev: >>>>>>>>>>>>> http://cr.openjdk.java.net/~rprotacio/8138916.01/ >>>>>>>>>>>> 111 Assert is never happens on Windows, so it's better to >>>>>>>>>>>> move it to >>>>>>>>>>>> os_posix >>>>>>>>>>>> >>>>>>>>>>>> 113 newbuf_len doesn't account prefix_len >>>>>>>>>>>> >>>>>>>>>>>> 115 you may consider to use pre-calculated at ll. 108 >>>>>>>>>>>> prefix rather >>>>>>>>>>>> than >>>>>>>>>>>> calculate it again. >>>>>>>>>>>> >>>>>>>>>>>> 116 int ret redefined it causes a warning. >>>>>>>>>>>> >>>>>>>>>>>> -Dmitry >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> (Retested and works fine.) >>>>>>>>>>>>> >>>>>>>>>>>>> On 10/27/2015 7:52 AM, Dmitry Samersoff wrote: >>>>>>>>>>>>>> Rachel, >>>>>>>>>>>>>> >>>>>>>>>>>>>> In worst case (windows, overflow) the code would parse >>>>>>>>>>>>>> format string >>>>>>>>>>>>>> four times. >>>>>>>>>>>>> You're right. I've added a check. New amount of times >>>>>>>>>>>>> vnsprintf() is >>>>>>>>>>>>> called: >>>>>>>>>>>>> posix, fits => 1 >>>>>>>>>>>>> posix, overflow => 2 >>>>>>>>>>>>> windows, fits => 1 >>>>>>>>>>>>> windows, overflow => 3 (the best performance we can get) >>>>>>>>>>>>> >>>>>>>>>>>>>> Also NEW_C_HEAP_ARRAY takes a lock and might become a >>>>>>>>>>>>>> performance bottleneck. >>>>>>>>>>>>>> >>>>>>>>>>>>>> POSIX equivalent of _vscprintf(fmt, args) is >>>>>>>>>>>>>> vsnprintf(NULL, 0, fmt, >>>>>>>>>>>>>> args) >>>>>>>>>>>>>> >>>>>>>>>>>>>> So it might be better to create os::log_vcprintf(fmt, >>>>>>>>>>>>>> args), then >>>>>>>>>>>>>> write >>>>>>>>>>>>>> code at log.hpp as: >>>>>>>>>>>>>> >>>>>>>>>>>>>> int buf_len = os::log_vcprintf(fmt, args); >>>>>>>>>>>>>> assert(buf_len < 2*page_size, "Stack overflow in logging"); >>>>>>>>>>>>>> char buf[buf_len]; >>>>>>>>>>>>>> vsprintf(buf, fmt, args); >>>>>>>>>>>>> I don't think this would work anyway. A char[] needs a >>>>>>>>>>>>> static size. >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Rachel >>>>>>>>>>>>>> -Dmitry >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 2015-10-26 23:58, Rachel Protacio wrote: >>>>>>>>>>>>>>> Hello, all, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Please review this fix to the faulty write function from >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>> Unified >>>>>>>>>>>>>>> Logging framework. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Summary: In logging/log.hpp, the logging vwrite function >>>>>>>>>>>>>>> previously >>>>>>>>>>>>>>> asserted that the buffer remains within 512 characters, >>>>>>>>>>>>>>> which is >>>>>>>>>>>>>>> too >>>>>>>>>>>>>>> short for logging message of non-pre-determined length, >>>>>>>>>>>>>>> e.g. for >>>>>>>>>>>>>>> vtable >>>>>>>>>>>>>>> and itable function names. Now, the function resizes the >>>>>>>>>>>>>>> buffer >>>>>>>>>>>>>>> to the >>>>>>>>>>>>>>> appropriate length, asserting only that the resulting >>>>>>>>>>>>>>> buffer is >>>>>>>>>>>>>>> valid. >>>>>>>>>>>>>>> Includes tag "logtest" to test that logging can print an >>>>>>>>>>>>>>> output >>>>>>>>>>>>>>> of 518 >>>>>>>>>>>>>>> characters. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Open webrev:http://cr.openjdk.java.net/~rprotacio/8138916/ >>>>>>>>>>>>>>> Bug:https://bugs.openjdk.java.net/browse/JDK-8138916 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Includes own acceptance test, and passes JPRT and remote >>>>>>>>>>>>>>> hotspot/test >>>>>>>>>>>>>>> tests. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thank you, >>>>>>>>>>>>>>> Rachel >>>>>>> >>>>>> >>>>> >>>> >>> >> > From marcus.larsson at oracle.com Wed Nov 4 19:59:12 2015 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Wed, 4 Nov 2015 20:59:12 +0100 Subject: RFR: 8138916: Logging write function does not allow for long enough messages In-Reply-To: <563A6152.2040509@oracle.com> References: <562E9406.5040207@oracle.com> <562F6567.6040707@oracle.com> <5630E029.5040500@oracle.com> <5630EC54.4020402@oracle.com> <5630EF9E.6090401@oracle.com> <5630F328.8@oracle.com> <5630F77D.1050907@oracle.com> <56310AFC.10609@oracle.com> <56312A94.1060202@oracle.com> <56312B2B.5090305@oracle.com> <56313726.3080507@oracle.com> <5631F66E.8090601@oracle.com> <5637A57E.4050103@oracle.com> <5639D4B5.7080809@oracle.com> <5639D710.2070100@oracle.com> <563A6152.2040509@oracle.com> Message-ID: <563A6390.5040706@oracle.com> Looks good! Regards, Marcus On 2015-11-04 20:49, Rachel Protacio wrote: > Updated webrev with this fgets change: > http://cr.openjdk.java.net/~rprotacio/8139564.04/ > > Passes JPRT tests. > > Thank you, > Rachel > > On 11/4/2015 4:59 AM, Marcus Larsson wrote: >> Hmm, I just noticed that the test doesn't null terminate the string >> it reads from file. Replacing the fgetc loop with a call to fgets >> should fix this. >> >> Regards, >> Marcus >> >> >> On 2015-11-04 10:49, Marcus Larsson wrote: >>> Looks good, thank you for fixing this! >>> >>> Regards, >>> Marcus >>> >>> >>> On 2015-11-02 19:03, Rachel Protacio wrote: >>>> Thanks for the comments, Marcus. I've undone the "logtest" tag and >>>> instead created an InternalVMTest in log.cpp that writes the >>>> 518-character string to a temporary file and looks for it there. >>>> Using "logging" tag in the "develop" level. Testing succeeds. >>>> >>>> Webrev: http://cr.openjdk.java.net/~rprotacio/8139564.03/ >>>> >>>> Thank you, >>>> Rachel >>>> >>>> On 10/29/2015 6:35 AM, Marcus Larsson wrote: >>>>> Hi, >>>>> >>>>> On 2015-10-28 21:59, Rachel Protacio wrote: >>>>>> Including serviceability-dev. Recap: >>>>>> >>>>>> Summary: In logging/log.hpp, the logging vwrite function >>>>>> previously asserted that the buffer remains within 512 >>>>>> characters, which is >>>>>> too short for logging message of non-pre-determined length, e.g. >>>>>> for vtable and itable function names. Now, the function resizes >>>>>> the buffer >>>>>> to the appropriate length, asserting only that the resulting >>>>>> buffer is valid. Includes tag "logtest" to test that logging can >>>>>> print an output >>>>>> of 518 characters. >>>>>> >>>>>> Open webrev: http://cr.openjdk.java.net/~rprotacio/8138916.02 >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8138916 >>>>> >>>>> I don't like that the test log message is included in the product, >>>>> or that a completely new tag is introduced for it. >>>>> >>>>> I think a much better approach would be to do the test logging in >>>>> an internal VM test, reusing the logging tag. This way it won't be >>>>> logged unless we are actually testing stuff, and we don't >>>>> introduce a new tag. Also it would look better if it were logged >>>>> on develop level. >>>>> >>>>> Thanks, >>>>> Marcus >>>>> >>>>>> >>>>>> Includes own acceptance test, and passes JPRT and remote >>>>>> hotspot/test tests. >>>>>> >>>>>> Thank you, >>>>>> Rachel >>>>>> >>>>>> On 10/28/2015 4:08 PM, Rachel Protacio wrote: >>>>>>> Thanks for the review and all the feedback! >>>>>>> Rachel >>>>>>> >>>>>>> On 10/28/2015 4:05 PM, Dmitry Samersoff wrote: >>>>>>>> Rachel, >>>>>>>> >>>>>>>> Looks good for me! >>>>>>>> >>>>>>>> -Dmitry >>>>>>>> >>>>>>>> On 2015-10-28 20:50, Rachel Protacio wrote: >>>>>>>>> Thank you both. I've removed the re-declaration and added >>>>>>>>> prefix_len to >>>>>>>>> newbuf_len. As for line 115, that line is not for the sake of >>>>>>>>> recalculating prefix_len, but for writing the prefix to the >>>>>>>>> buffer. >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~rprotacio/8138916.02/ >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Rachel >>>>>>>>> >>>>>>>>> On 10/28/2015 12:27 PM, Coleen Phillimore wrote: >>>>>>>>>> Dmitry, Thank you for reviewing and providing comments. The >>>>>>>>>> second >>>>>>>>>> version is better because of these things that I didn't >>>>>>>>>> notice until >>>>>>>>>> you pointed it out. >>>>>>>>>> >>>>>>>>>> The malloc won't be frequent. I think it'll be okay and >>>>>>>>>> follows other >>>>>>>>>> coding patterns in the jvm. >>>>>>>>>> >>>>>>>>>> Thanks! >>>>>>>>>> Coleen >>>>>>>>>> >>>>>>>>>> On 10/28/15 12:09 PM, Dmitry Samersoff wrote: >>>>>>>>>>> Coleen, >>>>>>>>>>> >>>>>>>>>>> OK. I'm leaving it in good hands ;) >>>>>>>>>>> >>>>>>>>>>> -Dmitry >>>>>>>>>>> >>>>>>>>>>> On 2015-10-28 18:54, Coleen Phillimore wrote: >>>>>>>>>>>> Hi, I'm also reviewing this. I think this looks good. I >>>>>>>>>>>> don't think >>>>>>>>>>>> the solution to allocate a static buffer is very nice >>>>>>>>>>>> because it still >>>>>>>>>>>> has limits and we don't want to allocate something large on >>>>>>>>>>>> the stack. >>>>>>>>>>>> The malloc case is the infrequent case. If it becomes >>>>>>>>>>>> frequent we can >>>>>>>>>>>> increase LogBufferSize from 512. >>>>>>>>>>>> >>>>>>>>>>>> I think 111 the assert is good because if the windows code >>>>>>>>>>>> is wrong (or >>>>>>>>>>>> the vscprintf does something wrong an unexpected) we can >>>>>>>>>>>> catch the >>>>>>>>>>>> error >>>>>>>>>>>> with the assert before writing over memory. >>>>>>>>>>>> >>>>>>>>>>>> I agree 116 shouldn't redeclare 'ret'. >>>>>>>>>>>> >>>>>>>>>>>> Otherwise, I think it looks really good and I don't need to >>>>>>>>>>>> see another >>>>>>>>>>>> revision. >>>>>>>>>>>> >>>>>>>>>>>> Thanks! >>>>>>>>>>>> Coleen >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 10/28/15 11:40 AM, Dmitry Samersoff wrote: >>>>>>>>>>>>> Rachel, >>>>>>>>>>>>> >>>>>>>>>>>>> On 2015-10-28 17:48, Rachel Protacio wrote: >>>>>>>>>>>>>> Thank you for the comments, both Dmitry and Mikael. >>>>>>>>>>>>>> Replies inline. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Modified webrev: >>>>>>>>>>>>>> http://cr.openjdk.java.net/~rprotacio/8138916.01/ >>>>>>>>>>>>> 111 Assert is never happens on Windows, so it's better to >>>>>>>>>>>>> move it to >>>>>>>>>>>>> os_posix >>>>>>>>>>>>> >>>>>>>>>>>>> 113 newbuf_len doesn't account prefix_len >>>>>>>>>>>>> >>>>>>>>>>>>> 115 you may consider to use pre-calculated at ll. 108 >>>>>>>>>>>>> prefix rather >>>>>>>>>>>>> than >>>>>>>>>>>>> calculate it again. >>>>>>>>>>>>> >>>>>>>>>>>>> 116 int ret redefined it causes a warning. >>>>>>>>>>>>> >>>>>>>>>>>>> -Dmitry >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> (Retested and works fine.) >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 10/27/2015 7:52 AM, Dmitry Samersoff wrote: >>>>>>>>>>>>>>> Rachel, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> In worst case (windows, overflow) the code would parse >>>>>>>>>>>>>>> format string >>>>>>>>>>>>>>> four times. >>>>>>>>>>>>>> You're right. I've added a check. New amount of times >>>>>>>>>>>>>> vnsprintf() is >>>>>>>>>>>>>> called: >>>>>>>>>>>>>> posix, fits => 1 >>>>>>>>>>>>>> posix, overflow => 2 >>>>>>>>>>>>>> windows, fits => 1 >>>>>>>>>>>>>> windows, overflow => 3 (the best performance we can get) >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Also NEW_C_HEAP_ARRAY takes a lock and might become a >>>>>>>>>>>>>>> performance bottleneck. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> POSIX equivalent of _vscprintf(fmt, args) is >>>>>>>>>>>>>>> vsnprintf(NULL, 0, fmt, >>>>>>>>>>>>>>> args) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> So it might be better to create os::log_vcprintf(fmt, >>>>>>>>>>>>>>> args), then >>>>>>>>>>>>>>> write >>>>>>>>>>>>>>> code at log.hpp as: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> int buf_len = os::log_vcprintf(fmt, args); >>>>>>>>>>>>>>> assert(buf_len < 2*page_size, "Stack overflow in logging"); >>>>>>>>>>>>>>> char buf[buf_len]; >>>>>>>>>>>>>>> vsprintf(buf, fmt, args); >>>>>>>>>>>>>> I don't think this would work anyway. A char[] needs a >>>>>>>>>>>>>> static size. >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Rachel >>>>>>>>>>>>>>> -Dmitry >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 2015-10-26 23:58, Rachel Protacio wrote: >>>>>>>>>>>>>>>> Hello, all, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Please review this fix to the faulty write function >>>>>>>>>>>>>>>> from the >>>>>>>>>>>>>>>> Unified >>>>>>>>>>>>>>>> Logging framework. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Summary: In logging/log.hpp, the logging vwrite >>>>>>>>>>>>>>>> function previously >>>>>>>>>>>>>>>> asserted that the buffer remains within 512 characters, >>>>>>>>>>>>>>>> which is >>>>>>>>>>>>>>>> too >>>>>>>>>>>>>>>> short for logging message of non-pre-determined length, >>>>>>>>>>>>>>>> e.g. for >>>>>>>>>>>>>>>> vtable >>>>>>>>>>>>>>>> and itable function names. Now, the function resizes >>>>>>>>>>>>>>>> the buffer >>>>>>>>>>>>>>>> to the >>>>>>>>>>>>>>>> appropriate length, asserting only that the resulting >>>>>>>>>>>>>>>> buffer is >>>>>>>>>>>>>>>> valid. >>>>>>>>>>>>>>>> Includes tag "logtest" to test that logging can print >>>>>>>>>>>>>>>> an output >>>>>>>>>>>>>>>> of 518 >>>>>>>>>>>>>>>> characters. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Open webrev:http://cr.openjdk.java.net/~rprotacio/8138916/ >>>>>>>>>>>>>>>> Bug:https://bugs.openjdk.java.net/browse/JDK-8138916 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Includes own acceptance test, and passes JPRT and remote >>>>>>>>>>>>>>>> hotspot/test >>>>>>>>>>>>>>>> tests. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thank you, >>>>>>>>>>>>>>>> Rachel >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From rachel.protacio at oracle.com Wed Nov 4 22:27:59 2015 From: rachel.protacio at oracle.com (Rachel Protacio) Date: Wed, 4 Nov 2015 17:27:59 -0500 Subject: RFR: 8138916: Logging write function does not allow for long enough messages In-Reply-To: <563A6390.5040706@oracle.com> References: <562E9406.5040207@oracle.com> <562F6567.6040707@oracle.com> <5630E029.5040500@oracle.com> <5630EC54.4020402@oracle.com> <5630EF9E.6090401@oracle.com> <5630F328.8@oracle.com> <5630F77D.1050907@oracle.com> <56310AFC.10609@oracle.com> <56312A94.1060202@oracle.com> <56312B2B.5090305@oracle.com> <56313726.3080507@oracle.com> <5631F66E.8090601@oracle.com> <5637A57E.4050103@oracle.com> <5639D4B5.7080809@oracle.com> <5639D710.2070100@oracle.com> <563A6152.2040509@oracle.com> <563A6390.5040706@oracle.com> Message-ID: <563A866F.5080002@oracle.com> Great, thank you, Marcus! Rachel On 11/4/2015 2:59 PM, Marcus Larsson wrote: > Looks good! > > Regards, > Marcus > > On 2015-11-04 20:49, Rachel Protacio wrote: >> Updated webrev with this fgets change: >> http://cr.openjdk.java.net/~rprotacio/8139564.04/ >> >> Passes JPRT tests. >> >> Thank you, >> Rachel >> >> On 11/4/2015 4:59 AM, Marcus Larsson wrote: >>> Hmm, I just noticed that the test doesn't null terminate the string >>> it reads from file. Replacing the fgetc loop with a call to fgets >>> should fix this. >>> >>> Regards, >>> Marcus >>> >>> >>> On 2015-11-04 10:49, Marcus Larsson wrote: >>>> Looks good, thank you for fixing this! >>>> >>>> Regards, >>>> Marcus >>>> >>>> >>>> On 2015-11-02 19:03, Rachel Protacio wrote: >>>>> Thanks for the comments, Marcus. I've undone the "logtest" tag and >>>>> instead created an InternalVMTest in log.cpp that writes the >>>>> 518-character string to a temporary file and looks for it there. >>>>> Using "logging" tag in the "develop" level. Testing succeeds. >>>>> >>>>> Webrev: http://cr.openjdk.java.net/~rprotacio/8139564.03/ >>>>> >>>>> Thank you, >>>>> Rachel >>>>> >>>>> On 10/29/2015 6:35 AM, Marcus Larsson wrote: >>>>>> Hi, >>>>>> >>>>>> On 2015-10-28 21:59, Rachel Protacio wrote: >>>>>>> Including serviceability-dev. Recap: >>>>>>> >>>>>>> Summary: In logging/log.hpp, the logging vwrite function >>>>>>> previously asserted that the buffer remains within 512 >>>>>>> characters, which is >>>>>>> too short for logging message of non-pre-determined length, e.g. >>>>>>> for vtable and itable function names. Now, the function resizes >>>>>>> the buffer >>>>>>> to the appropriate length, asserting only that the resulting >>>>>>> buffer is valid. Includes tag "logtest" to test that logging can >>>>>>> print an output >>>>>>> of 518 characters. >>>>>>> >>>>>>> Open webrev: http://cr.openjdk.java.net/~rprotacio/8138916.02 >>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8138916 >>>>>> >>>>>> I don't like that the test log message is included in the >>>>>> product, or that a completely new tag is introduced for it. >>>>>> >>>>>> I think a much better approach would be to do the test logging in >>>>>> an internal VM test, reusing the logging tag. This way it won't >>>>>> be logged unless we are actually testing stuff, and we don't >>>>>> introduce a new tag. Also it would look better if it were logged >>>>>> on develop level. >>>>>> >>>>>> Thanks, >>>>>> Marcus >>>>>> >>>>>>> >>>>>>> Includes own acceptance test, and passes JPRT and remote >>>>>>> hotspot/test tests. >>>>>>> >>>>>>> Thank you, >>>>>>> Rachel >>>>>>> >>>>>>> On 10/28/2015 4:08 PM, Rachel Protacio wrote: >>>>>>>> Thanks for the review and all the feedback! >>>>>>>> Rachel >>>>>>>> >>>>>>>> On 10/28/2015 4:05 PM, Dmitry Samersoff wrote: >>>>>>>>> Rachel, >>>>>>>>> >>>>>>>>> Looks good for me! >>>>>>>>> >>>>>>>>> -Dmitry >>>>>>>>> >>>>>>>>> On 2015-10-28 20:50, Rachel Protacio wrote: >>>>>>>>>> Thank you both. I've removed the re-declaration and added >>>>>>>>>> prefix_len to >>>>>>>>>> newbuf_len. As for line 115, that line is not for the sake of >>>>>>>>>> recalculating prefix_len, but for writing the prefix to the >>>>>>>>>> buffer. >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~rprotacio/8138916.02/ >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Rachel >>>>>>>>>> >>>>>>>>>> On 10/28/2015 12:27 PM, Coleen Phillimore wrote: >>>>>>>>>>> Dmitry, Thank you for reviewing and providing comments. The >>>>>>>>>>> second >>>>>>>>>>> version is better because of these things that I didn't >>>>>>>>>>> notice until >>>>>>>>>>> you pointed it out. >>>>>>>>>>> >>>>>>>>>>> The malloc won't be frequent. I think it'll be okay and >>>>>>>>>>> follows other >>>>>>>>>>> coding patterns in the jvm. >>>>>>>>>>> >>>>>>>>>>> Thanks! >>>>>>>>>>> Coleen >>>>>>>>>>> >>>>>>>>>>> On 10/28/15 12:09 PM, Dmitry Samersoff wrote: >>>>>>>>>>>> Coleen, >>>>>>>>>>>> >>>>>>>>>>>> OK. I'm leaving it in good hands ;) >>>>>>>>>>>> >>>>>>>>>>>> -Dmitry >>>>>>>>>>>> >>>>>>>>>>>> On 2015-10-28 18:54, Coleen Phillimore wrote: >>>>>>>>>>>>> Hi, I'm also reviewing this. I think this looks good. I >>>>>>>>>>>>> don't think >>>>>>>>>>>>> the solution to allocate a static buffer is very nice >>>>>>>>>>>>> because it still >>>>>>>>>>>>> has limits and we don't want to allocate something large >>>>>>>>>>>>> on the stack. >>>>>>>>>>>>> The malloc case is the infrequent case. If it becomes >>>>>>>>>>>>> frequent we can >>>>>>>>>>>>> increase LogBufferSize from 512. >>>>>>>>>>>>> >>>>>>>>>>>>> I think 111 the assert is good because if the windows code >>>>>>>>>>>>> is wrong (or >>>>>>>>>>>>> the vscprintf does something wrong an unexpected) we can >>>>>>>>>>>>> catch the >>>>>>>>>>>>> error >>>>>>>>>>>>> with the assert before writing over memory. >>>>>>>>>>>>> >>>>>>>>>>>>> I agree 116 shouldn't redeclare 'ret'. >>>>>>>>>>>>> >>>>>>>>>>>>> Otherwise, I think it looks really good and I don't need >>>>>>>>>>>>> to see another >>>>>>>>>>>>> revision. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks! >>>>>>>>>>>>> Coleen >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 10/28/15 11:40 AM, Dmitry Samersoff wrote: >>>>>>>>>>>>>> Rachel, >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 2015-10-28 17:48, Rachel Protacio wrote: >>>>>>>>>>>>>>> Thank you for the comments, both Dmitry and Mikael. >>>>>>>>>>>>>>> Replies inline. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Modified webrev: >>>>>>>>>>>>>>> http://cr.openjdk.java.net/~rprotacio/8138916.01/ >>>>>>>>>>>>>> 111 Assert is never happens on Windows, so it's better to >>>>>>>>>>>>>> move it to >>>>>>>>>>>>>> os_posix >>>>>>>>>>>>>> >>>>>>>>>>>>>> 113 newbuf_len doesn't account prefix_len >>>>>>>>>>>>>> >>>>>>>>>>>>>> 115 you may consider to use pre-calculated at ll. 108 >>>>>>>>>>>>>> prefix rather >>>>>>>>>>>>>> than >>>>>>>>>>>>>> calculate it again. >>>>>>>>>>>>>> >>>>>>>>>>>>>> 116 int ret redefined it causes a warning. >>>>>>>>>>>>>> >>>>>>>>>>>>>> -Dmitry >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> (Retested and works fine.) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 10/27/2015 7:52 AM, Dmitry Samersoff wrote: >>>>>>>>>>>>>>>> Rachel, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> In worst case (windows, overflow) the code would parse >>>>>>>>>>>>>>>> format string >>>>>>>>>>>>>>>> four times. >>>>>>>>>>>>>>> You're right. I've added a check. New amount of times >>>>>>>>>>>>>>> vnsprintf() is >>>>>>>>>>>>>>> called: >>>>>>>>>>>>>>> posix, fits => 1 >>>>>>>>>>>>>>> posix, overflow => 2 >>>>>>>>>>>>>>> windows, fits => 1 >>>>>>>>>>>>>>> windows, overflow => 3 (the best performance we can get) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Also NEW_C_HEAP_ARRAY takes a lock and might become a >>>>>>>>>>>>>>>> performance bottleneck. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> POSIX equivalent of _vscprintf(fmt, args) is >>>>>>>>>>>>>>>> vsnprintf(NULL, 0, fmt, >>>>>>>>>>>>>>>> args) >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> So it might be better to create os::log_vcprintf(fmt, >>>>>>>>>>>>>>>> args), then >>>>>>>>>>>>>>>> write >>>>>>>>>>>>>>>> code at log.hpp as: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> int buf_len = os::log_vcprintf(fmt, args); >>>>>>>>>>>>>>>> assert(buf_len < 2*page_size, "Stack overflow in >>>>>>>>>>>>>>>> logging"); >>>>>>>>>>>>>>>> char buf[buf_len]; >>>>>>>>>>>>>>>> vsprintf(buf, fmt, args); >>>>>>>>>>>>>>> I don't think this would work anyway. A char[] needs a >>>>>>>>>>>>>>> static size. >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> Rachel >>>>>>>>>>>>>>>> -Dmitry >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 2015-10-26 23:58, Rachel Protacio wrote: >>>>>>>>>>>>>>>>> Hello, all, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Please review this fix to the faulty write function >>>>>>>>>>>>>>>>> from the >>>>>>>>>>>>>>>>> Unified >>>>>>>>>>>>>>>>> Logging framework. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Summary: In logging/log.hpp, the logging vwrite >>>>>>>>>>>>>>>>> function previously >>>>>>>>>>>>>>>>> asserted that the buffer remains within 512 >>>>>>>>>>>>>>>>> characters, which is >>>>>>>>>>>>>>>>> too >>>>>>>>>>>>>>>>> short for logging message of non-pre-determined >>>>>>>>>>>>>>>>> length, e.g. for >>>>>>>>>>>>>>>>> vtable >>>>>>>>>>>>>>>>> and itable function names. Now, the function resizes >>>>>>>>>>>>>>>>> the buffer >>>>>>>>>>>>>>>>> to the >>>>>>>>>>>>>>>>> appropriate length, asserting only that the resulting >>>>>>>>>>>>>>>>> buffer is >>>>>>>>>>>>>>>>> valid. >>>>>>>>>>>>>>>>> Includes tag "logtest" to test that logging can print >>>>>>>>>>>>>>>>> an output >>>>>>>>>>>>>>>>> of 518 >>>>>>>>>>>>>>>>> characters. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Open >>>>>>>>>>>>>>>>> webrev:http://cr.openjdk.java.net/~rprotacio/8138916/ >>>>>>>>>>>>>>>>> Bug:https://bugs.openjdk.java.net/browse/JDK-8138916 >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Includes own acceptance test, and passes JPRT and remote >>>>>>>>>>>>>>>>> hotspot/test >>>>>>>>>>>>>>>>> tests. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thank you, >>>>>>>>>>>>>>>>> Rachel >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From keimpe.bronkhorst at oracle.com Wed Nov 4 22:32:49 2015 From: keimpe.bronkhorst at oracle.com (Keimpe Bronkhorst) Date: Wed, 4 Nov 2015 15:32:49 -0700 Subject: Collect thread dumps of all Java process on localhost Message-ID: <563A8791.5060902@oracle.com> This concerns code we have in Oracle JDeveloper which collect thread dumps of all Java Processes on local host. This code uses the non-public JDK classes: import sun.jvmstat.monitor.MonitorException; import sun.jvmstat.monitor.MonitoredHost; import sun.jvmstat.monitor.MonitoredVm; import sun.jvmstat.monitor.MonitoredVmUtil; import sun.jvmstat.monitor.VmIdentifier; to identify the Java processes on local host, and import com.sun.tools.attach.VirtualMachine; import sun.tools.attach.HotSpotVirtualMachine; to attach to the processes and dump their threads. The trouble is that these classes are not accessible anymore in JDK9 unless -XaddExports: is used. Does anybody know how to do this functionality using public classes, or are there plans to add public classes to replace these non-public classes? Keimpe Bronkhorst Oracle JDeveloper From mandy.chung at oracle.com Thu Nov 5 00:02:02 2015 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 4 Nov 2015 16:02:02 -0800 Subject: Collect thread dumps of all Java process on localhost In-Reply-To: <563A8791.5060902@oracle.com> References: <563A8791.5060902@oracle.com> Message-ID: <5A834E81-3FDC-40A5-8B16-5BB9FC270B60@oracle.com> > On Nov 4, 2015, at 2:32 PM, Keimpe Bronkhorst wrote: > > This concerns code we have in Oracle JDeveloper which collect thread dumps of all Java Processes on local host. This code uses the non-public JDK classes: > import sun.jvmstat.monitor.MonitorException; > import sun.jvmstat.monitor.MonitoredHost; > import sun.jvmstat.monitor.MonitoredVm; > import sun.jvmstat.monitor.MonitoredVmUtil; > import sun.jvmstat.monitor.VmIdentifier; > > to identify the Java processes on local host, and > > import com.sun.tools.attach.VirtualMachine; > import sun.tools.attach.HotSpotVirtualMachine; > > to attach to the processes and dump their threads. > > The trouble is that these classes are not accessible anymore in JDK9 unless -XaddExports: is used. > > Does anybody know how to do this functionality using public classes, or are there plans to add public classes to replace these non-public classes? com.sun.tools.attach is supported API that you should use to list the VM processes on a local host: http://download.java.net/jdk9/docs/jdk/api/attach/spec/index.html I guess you are calling HotSpotVirtualMachine.remoteDataDump method to get thread dump. Can you use java.lang.management.ThreadMXBean.dumpAllThreads once you start a local JMX agent in the attached VM? Mandy From keimpe.bronkhorst at oracle.com Thu Nov 5 01:25:39 2015 From: keimpe.bronkhorst at oracle.com (Keimpe Bronkhorst) Date: Wed, 4 Nov 2015 18:25:39 -0700 Subject: Collect thread dumps of all Java process on localhost In-Reply-To: <5A834E81-3FDC-40A5-8B16-5BB9FC270B60@oracle.com> References: <563A8791.5060902@oracle.com> <5A834E81-3FDC-40A5-8B16-5BB9FC270B60@oracle.com> Message-ID: <563AB013.6030002@oracle.com> Mandy, Thanks, I got it to work using your suggestions. Keimpe Bronkhorst Oracle JDeveloper On 11/4/2015 5:02 PM, Mandy Chung wrote: >> On Nov 4, 2015, at 2:32 PM, Keimpe Bronkhorst wrote: >> >> This concerns code we have in Oracle JDeveloper which collect thread dumps of all Java Processes on local host. This code uses the non-public JDK classes: >> import sun.jvmstat.monitor.MonitorException; >> import sun.jvmstat.monitor.MonitoredHost; >> import sun.jvmstat.monitor.MonitoredVm; >> import sun.jvmstat.monitor.MonitoredVmUtil; >> import sun.jvmstat.monitor.VmIdentifier; >> >> to identify the Java processes on local host, and >> >> import com.sun.tools.attach.VirtualMachine; >> import sun.tools.attach.HotSpotVirtualMachine; >> >> to attach to the processes and dump their threads. >> >> The trouble is that these classes are not accessible anymore in JDK9 unless -XaddExports: is used. >> >> Does anybody know how to do this functionality using public classes, or are there plans to add public classes to replace these non-public classes? > > com.sun.tools.attach is supported API that you should use to list the VM processes on a local host: > http://download.java.net/jdk9/docs/jdk/api/attach/spec/index.html > > I guess you are calling HotSpotVirtualMachine.remoteDataDump method to get thread dump. Can you use java.lang.management.ThreadMXBean.dumpAllThreads once you start a local JMX agent in the attached VM? > > Mandy From yasuenag at gmail.com Thu Nov 5 03:30:55 2015 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Thu, 5 Nov 2015 12:30:55 +0900 Subject: RFR: JDK-8140556: Add force rotation option to VM.log jcmd In-Reply-To: <563765FC.2090804@gmail.com> References: <562101D8.5080909@gmail.com> <562DF8D4.40501@oracle.com> <562EBF4B.4020607@gmail.com> <562F6B46.5050500@oracle.com> <563226AB.2030206@gmail.com> <5632341F.4060400@oracle.com> <56338125.9030802@gmail.com> <56338D58.9090709@oracle.com> <5634D9FC.6080502@gmail.com> <5636C66A.6020304@oracle.com> <563765FC.2090804@gmail.com> Message-ID: <563ACD6F.1050600@gmail.com> I've uploaded new webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.03/ I added _initialized to LogConfiguration. LogConfiguration::post_initialize() will be called after main thread initialization. So _initialized set to true in this function, and we can check this flag at rotation. Could you review it? Thanks, Yasumasa On 2015/11/02 22:32, Yasumasa Suenaga wrote: > Marcus, David, > > I've uploaded new webrev: > http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.02/ > > Could you review it? > > >>>>>> Still missing the renaming of rotate_all_logfile to rotate_all_outputs. > > I've fixed. > > >>>> Small typo, please make that is_rotatable(). > > I've added, and have used at LogConfiguration::rotate_all_outputs(). > > >> You can't safely lock a mutex until the current thread has attached to the VM and Thread::current() returns non-null (that is what Thread::initialize_thread_local_storage() currently does). You may get away with it, particularly in product builds, during VM initialization, as the fast-lock path doesn't require access to Thread::current(). > > I will check whether Thread::current() is NULL or not at LogFileOutput::is_rotatable() > is_rotatable() is called at LogFileOutput::rotate() before locking _rotation_lock. > So we can avoid crash/assert which is related to TLS. > > > Thanks, > > Yasumasa > > > On 2015/11/02 11:11, David Holmes wrote: >> On 1/11/2015 1:10 AM, Yasumasa Suenaga wrote: >>>> This in turn means that we are logging before the thread local storage >>>> has been initialized, and the JVM will crash/hit the assert because we >>>> are trying to take the rotation lock. >>> >>> I guess that you are saying about class member (not TLS - e.g. >>> pthread_setspecific()). >> >> You can't safely lock a mutex until the current thread has attached to the VM and Thread::current() returns non-null (that is what Thread::initialize_thread_local_storage() currently does). You may get away with it, particularly in product builds, during VM initialization, as the fast-lock path doesn't require access to Thread::current(). >> >> David >> >>> Most of fields (include _rotation_lock) in LogFileOutput are initialized >>> at c'tor. >>> Other field (_stream) is initialized at LogFileOutput::initialize(). >>> >>> We can avoid problem if we can move initialize() to c'tor because >>> LogFileOutput >>> will be instantiated at LogConfiguration::new_output(). >>> >>> Currently, LogFileOutput will be added to LogConfiguration::_outputs and >>> to LogTagSet after calling LogFileOutput::initialize(). >>> I wonder why we cannot avoid crash/assert with current implementation... >>> >>> >>> I will keep current implementation about _rotation_lock if the above is >>> incorrect. >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> On 2015/10/31 0:31, Marcus Larsson wrote: >>>> Hey, >>>> >>>> On 2015-10-30 15:39, Yasumasa Suenaga wrote: >>>>> Hi Marcus, >>>>> >>>>> Thank you for your comment. >>>>> >>>>> >>>>>> Still missing the renaming of rotate_all_logfile to rotate_all_outputs. >>>>> >>>>> Sorry, I will fix. >>>>> >>>> >>>> Thanks! >>>> >>>>> >>>>>> This rotate function works I guess, but it will probably need some >>>>>> rework once there are other types of log outputs (rotate doesn't >>>>>> make sense on all types of log outputs). >>>>> >>>>> I will add is_rotetable() to LogOutput as virtual function. >>>>> This function return false (can not rotate) by default, return true >>>>> if _file_count is >>>>> greater than 0 in LogFileOutput. >>>>> >>>>> LogConfiguration::rotate_all_outputs() will rotate if this function >>>>> returns true. >>>>> >>>> >>>> Small typo, please make that is_rotatable(). >>>> >>>>> >>>>>> If we configure a rotated file output and then log something on that >>>>>> output before thread local storage initialization is complete, we >>>>>> will crash/hit an assert. The previous implementation avoided this >>>>>> as long as no rotation was needed before this initialization was >>>>>> complete. (This can be triggered using the following arguments >>>>>> "-Xlog:all=trace:file.txt::filesize=10,filecount=2 -Xlog:invalid", >>>>>> for example.) >>>>> >>>>> I do not think so. >>>>> Each -Xlog option creates unique instance of LogOutput (and subclass). >>>>> So they are isolated. >>>> >>>> My example includes a second (and intentionally incorrect) -Xlog >>>> option only to trigger logging on the newly configured rotating >>>> LogFileOutput. This will cause some logging about the invalid -Xlog >>>> argument, and this logging happens during the argument parsing. This >>>> in turn means that we are logging before the thread local storage has >>>> been initialized, and the JVM will crash/hit the assert because we are >>>> trying to take the rotation lock. This has nothing to do with >>>> concurrent fprintfs or freopens, even if that is a problem. >>>> >>>>> >>>>> However, we might have to rotate at safepoint. >>>> >>>> Doing stuff at a safepoint will not give us much in this case. There >>>> are threads that continue execution through a safepoint, which means >>>> we will see log calls being made even during this time. >>>> >>>>> Currently, LogOutput might be used by multiple threads concurrently. >>>>> If we rotate log when another thread is writing string to output, we >>>>> encounter unexpected behavior. >>>>> >>>>> LogFileOutput::rotate() uses freopen(). >>>>> LogFileStreamOutput::write() uses vfprintf() through jio_fprintf(). >>>>> freopen() and vfprintf() are not atomic. >>>> >>>> You're right. There seems to be an overlooked concurrency issue with >>>> vfprintfs during freopen. I don't think we should resolve this as part >>>> of this particular fix, so instead I'll create a separate issue for it. >>>> >>>>> >>>>> I think that cause of crash/assertion which you say is it. >>>>> (GC log will be rotated at safepoint.) >>>>> >>>> >>>> See my comment above for what crash/assert I'm talking about. >>>> >>>> Thanks, >>>> Marcus >>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2015/10/29 23:58, Marcus Larsson wrote: >>>>>> Hi, >>>>>> >>>>>> On 2015-10-29 15:01, Yasumasa Suenaga wrote: >>>>>>> Hi Marcus, >>>>>>> >>>>>>> Thank you for your comment. >>>>>>> >>>>>>>> I'll sponsor it. >>>>>>> >>>>>>> Thank you so much! >>>>>>> I've uploaded new webrev: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.01/ >>>>>>> >>>>>>> >>>>>> >>>>>> Still missing the renaming of rotate_all_logfile to rotate_all_outputs. >>>>>> >>>>>>>> logConfiguration.cpp/hpp: >>>>>>>> >>>>>>>> * I don't think we should rely on the archive_name or the output's >>>>>>>> name to decide whether or not an output should be rotated. It >>>>>>>> would be better to introduce an is_rotated() test function in >>>>>>>> LogOutput that could be used here. >>>>>>>> >>>>>>>> * rotate_all_logfile() should be renamed to rotate_all_outputs(). >>>>>>>> Currently there are only LogFileOutputs (other than >>>>>>>> stdout/stderr), but in the future there might be other types of >>>>>>>> outputs so I prefer having a more general name. >>>>>>>> Also, please use static_cast instead of the >>>>>>>> C-style cast. (Additional logic will be required here once more >>>>>>>> types of log outputs are added, but I don't think we need to worry >>>>>>>> about this right now.) >>>>>>>> >>>>>>>> * Don't print an error if an output is not rotatable, since it's >>>>>>>> not an error to have some log outputs rotated while others are >>>>>>>> not. If you really want some traceability here I suggest adding >>>>>>>> log messages on trace level, tagged with 'logging'. >>>>>>> >>>>>>> I added LogOutput::rotate(bool) as virtual function. >>>>>>> This function works nothing by default, but will rotate all logs at >>>>>>> LogFileOutput. >>>>>> >>>>>> This rotate function works I guess, but it will probably need some >>>>>> rework once there are other types of log outputs (rotate doesn't >>>>>> make sense on all types of log outputs). >>>>>> >>>>>>> >>>>>>> If true is passed to this function, all files will be rotated. >>>>>>> >>>>>>> >>>>>>>> logDiagnosticCommand.cpp/hpp: >>>>>>>> >>>>>>>> * I think the description could be improved to something like >>>>>>>> "Lists current log configuration, enables/disables/configures a >>>>>>>> log output, or disables/rotates all logs." >>>>>>>> >>>>>>>> * The rotate option description should clarify that all logs will >>>>>>>> be rotated ("current log" is too ambiguous). >>>>>>> >>>>>>> I've fixed. >>>>>> >>>>>> Great, thanks! >>>>>> >>>>>>> >>>>>>> >>>>>>>> logFileOutput.cpp/hpp: >>>>>>>> >>>>>>>> * Moving the MutexLocker like this introduces a race condition >>>>>>>> where a log might be rotated multiple times by different threads, >>>>>>>> instead of just once. >>>>>>>> Instead of making the rotate() function public and moving the >>>>>>>> mutexlocker, I suggest adding something like a public >>>>>>>> force_rotation() function that grabs the lock and calls the >>>>>>>> private rotate(). >>>>>>>> >>>>>>>> * Given the addition of is_rotated() in LogOutput, then >>>>>>>> get_archive_name() should be removed. >>>>>>> >>>>>>> I moved MutexLocker and should_rotate() to rotate(). >>>>>>> I think this change can avoid race condition. >>>>>>> >>>>>> >>>>>> There is a subtle problem with taking the lock to check if we should >>>>>> rotate. If we configure a rotated file output and then log something >>>>>> on that output before thread local storage initialization is >>>>>> complete, we will crash/hit an assert. The previous implementation >>>>>> avoided this as long as no rotation was needed before this >>>>>> initialization was complete. (This can be triggered using the >>>>>> following arguments >>>>>> "-Xlog:all=trace:file.txt::filesize=10,filecount=2 -Xlog:invalid", >>>>>> for example.) >>>>>> >>>>>> Thanks, >>>>>> Marcus >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2015/10/27 21:17, Marcus Larsson wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> On 2015-10-27 01:03, Yasumasa Suenaga wrote: >>>>>>>>> Hi Marcus, >>>>>>>>> >>>>>>>>> Thank you for replying. >>>>>>>>> >>>>>>>>> I filed this feature to JBS and change subject of this email. >>>>>>>>> Could you be a sponsor and review it? >>>>>>>> >>>>>>>> I'll sponsor it. >>>>>>>> >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.00/ >>>>>>>> >>>>>>>> logConfiguration.cpp/hpp: >>>>>>>> >>>>>>>> * I don't think we should rely on the archive_name or the output's >>>>>>>> name to decide whether or not an output should be rotated. It >>>>>>>> would be better to introduce an is_rotated() test function in >>>>>>>> LogOutput that could be used here. >>>>>>>> >>>>>>>> * rotate_all_logfile() should be renamed to rotate_all_outputs(). >>>>>>>> Currently there are only LogFileOutputs (other than >>>>>>>> stdout/stderr), but in the future there might be other types of >>>>>>>> outputs so I prefer having a more general name. >>>>>>>> Also, please use static_cast instead of the >>>>>>>> C-style cast. (Additional logic will be required here once more >>>>>>>> types of log outputs are added, but I don't think we need to worry >>>>>>>> about this right now.) >>>>>>>> >>>>>>>> * Don't print an error if an output is not rotatable, since it's >>>>>>>> not an error to have some log outputs rotated while others are >>>>>>>> not. If you really want some traceability here I suggest adding >>>>>>>> log messages on trace level, tagged with 'logging'. >>>>>>>> >>>>>>>> >>>>>>>> logDiagnosticCommand.cpp/hpp: >>>>>>>> >>>>>>>> * I think the description could be improved to something like >>>>>>>> "Lists current log configuration, enables/disables/configures a >>>>>>>> log output, or disables/rotates all logs." >>>>>>>> >>>>>>>> * The rotate option description should clarify that all logs will >>>>>>>> be rotated ("current log" is too ambiguous). >>>>>>>> >>>>>>>> >>>>>>>> logFileOutput.cpp/hpp: >>>>>>>> >>>>>>>> * Moving the MutexLocker like this introduces a race condition >>>>>>>> where a log might be rotated multiple times by different threads, >>>>>>>> instead of just once. >>>>>>>> Instead of making the rotate() function public and moving the >>>>>>>> mutexlocker, I suggest adding something like a public >>>>>>>> force_rotation() function that grabs the lock and calls the >>>>>>>> private rotate(). >>>>>>>> >>>>>>>> * Given the addition of is_rotated() in LogOutput, then >>>>>>>> get_archive_name() should be removed. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Marcus >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2015/10/26 18:56, Marcus Larsson wrote: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> Sorry for my late reply. >>>>>>>>>> >>>>>>>>>> I think being able to force rotation via jcmd seems like a good >>>>>>>>>> feature. Files are currently opened in append mode so it should >>>>>>>>>> already be possible to use external log rotation tools by >>>>>>>>>> copying and truncating the files. Still I think it would be nice >>>>>>>>>> to provide the jcmd for rotation as well. >>>>>>>>>> >>>>>>>>>> I can see some small issues with the implementation, but we can >>>>>>>>>> deal with that during the review. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Marcus >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 2015-10-26 00:26, Yasumasa Suenaga wrote: >>>>>>>>>>> Hi all, >>>>>>>>>>> >>>>>>>>>>> Have you ever seen my change? >>>>>>>>>>> I (and my customers) need log rotation function via external tool. >>>>>>>>>>> >>>>>>>>>>> I want to merge it by Feature Complete. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> 2015/10/16 22:55 "Yasumasa Suenaga" : >>>>>>>>>>> >>>>>>>>>>>> Hi all, >>>>>>>>>>>> >>>>>>>>>>>> I contributed JDK-7090324: gclog rotation via external tool to be >>>>>>>>>>>> synchronized with >>>>>>>>>>>> logrotated tool on Linux. >>>>>>>>>>>> >>>>>>>>>>>> I think JEP 158 is in progress. >>>>>>>>>>>> However, this JEP does not contain log rotation via external >>>>>>>>>>>> tool in the >>>>>>>>>>>> spec. >>>>>>>>>>>> I want to rotate logs via jcmd in this JEP. >>>>>>>>>>>> >>>>>>>>>>>> I've updated a patch for it: >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/jvmlogging-logrotate/ >>>>>>>>>>>> >>>>>>>>>>>> This patch provides new option "rotate" in VM.log command. >>>>>>>>>>>> If this change can be accepted, I will file it to JBS and send >>>>>>>>>>>> RFR. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Yasumasa >>>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>> >>>> From serguei.spitsyn at oracle.com Thu Nov 5 03:49:17 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 4 Nov 2015 19:49:17 -0800 Subject: RFR(M): 8140482: Various minor code improvements (runtime) In-Reply-To: <5639CFDE.1010302@oracle.com> References: <4295855A5C1DE049A61835A1887419CC41E9F0AA@DEWDFEMB12A.global.corp.sap> <563840B0.10401@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB1746@DEWDFEMB12A.global.corp.sap> <56388760.60800@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB1875@DEWDFEMB12A.global.corp.sap> <5639B058.9090609@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB1B57@DEWDFEMB12A.global.corp.sap> <5639CFDE.1010302@oracle.com> Message-ID: <563AD1BD.2020304@oracle.com> Hi Goetz, The fix looks good. Thanks for the improvements! The _NSIG related fix looks Ok to me but I do not feel myself qualified to make a final decision. A couple of minor comments: *src/share/vm/libadt/dict.cpp* 149 nb->_keyvals[nbcnt + nbcnt + 1] = b->_keyvals[j+j+1]; 152 b->_keyvals[j+j] = b->_keyvals[b->_cnt + b->_cnt]; 153 b->_keyvals[j+j+1] = b->_keyvals[b->_cnt + b->_cnt + 1]; Need spaces around the '+' sign for completeness. *src/os/linux/vm/attachListener_linux.cpp* 258 buf[max_len-1] = '\0'; Need spaces around the '-' sign. *src/share/vm/services/attachListener.hpp* 126 strncpy(_name, name, MIN2(strlen(name)+1, (size_t)name_length_max)); 143 strncpy(_arg[i], arg, MIN2(strlen(arg)+1, (size_t)arg_length_max)); Need spaces around the '+' sign. || *agent/src/os/linux/ps_core.c* 815 char interp_name[BUF_SIZE+1]; Need spaces around the '+' sign. Thanks, Serguei On 11/4/15 01:29, David Holmes wrote: > On 4/11/2015 6:01 PM, Lindenmaier, Goetz wrote: >> Hi David, >> >> attachListener.hpp: >> I agree that I can not find another possible issue >> with the strcpy. >> Still I think it's better to have the strncpy, as it would have >> protected against the bug in attachListener_windows.cpp. >> But if you insist I'll just remove it. > > I don't insist, but I do prefer to place all the guards at the > "boundary" of the VM rather than at every level when possible. > >> Should I remove the _NSIG issue from this change and >> open an issue of it's own discussed on the serviceability list? > > Let's give them a chance to respond. I'll ping them on the hotline ;-) > > Thanks, > David > >> Best regards, >> Goetz. >> >> >>> -----Original Message----- >>> From: David Holmes [mailto:david.holmes at oracle.com] >>> Sent: Mittwoch, 4. November 2015 08:15 >>> To: Lindenmaier, Goetz; hotspot-runtime-dev at openjdk.java.net; >>> serviceability-dev >>> Subject: Re: RFR(M): 8140482: Various minor code improvements (runtime) >>> >>> Hi Goetz, >>> >>> On 4/11/2015 12:10 AM, Lindenmaier, Goetz wrote: >>>> Hi David, >>>> >>>> the new scan is already through. I made a new webrev: >>>> http://cr.openjdk.java.net/~goetz/webrevs/8140482-covRt/webrev.01/ >>>> >>>> attachListener_linux.cpp: >>>> I had moved the string termination out of the loop, and read one >>>> less char from the file. The scan still claims "Passing unterminated >>>> string buf to strlen" in line 274. So I will undo it again for the >>>> webrev. >>>> >>>> codeBuffer.cpp >>>> Doing memset is fine. I'll use memset(). >>>> >>>>>>>> ps_core.c: >>>>>>>> Pread not necessarily terminates interp_name which is printed >>>>>>>> thereafter. Increase buffer size by 1 and add '\0'. >>>>>>> >>>>>>> Given: >>>>>>> #define BUF_SIZE (PATH_MAX + NAME_MAX + 1) >>>>>>> isn't it impossible to encounter that problem? >>>>>> As I understand, pread does not null-terminate what it read. So the >>>>>> null must come from the file. This protects against a corrupted >>>>>> file. >>>>> >>>>> So are you saying the nul is not present in the file? I'm not >>>>> familiar >>>>> with the ELF format. >>>> There should be a nul in the file, else the code would fail more >>>> obviously. The problem is if the file is corrupted. >>> >>> Thanks for clarifying. >>> >>> I'm still unclear why the attachListener.hpp changes are needed if we >>> have validated the entry points in the attachListener_.cpp files? >>> >>> Also we still need someone from serviceability to comment on the _NSIG >>> issue. >>> >>> Thanks, >>> David >>> >>> >>>> Best regards, >>>> Goetz. >>>> >>>> >>>>> -----Original Message----- >>>>> From: David Holmes [mailto:david.holmes at oracle.com] >>>>> Sent: Dienstag, 3. November 2015 11:07 >>>>> To: Lindenmaier, Goetz; hotspot-runtime-dev at openjdk.java.net; >>>>> serviceability-dev >>>>> Subject: Re: RFR(M): 8140482: Various minor code improvements >>> (runtime) >>>>> >>>>> Hi Goetz, >>>>> >>>>> Quick follow up on a couple of things ... >>>>> >>>>> On 3/11/2015 7:33 PM, Lindenmaier, Goetz wrote: >>>>>> Hi David, >>>>>> >>>>>>> Sorry, lots of folks very busy at the moment as we try to get >>>>>>> features >>>>>>> finalized before the deadline. >>>>>> thanks for looking at this! I know the Dec. 10 deadline, but I >>>>>> guess that >>> also >>>>>> holds for us ... at least J1 is over now. (Unfortunately we >>>>>> could not >>> attend >>>>>> this year.) >>>>> >>>>> Me neither :) >>>>> >>>>>>>> ps_core.c: >>>>>>>> Pread not necessarily terminates interp_name which is printed >>>>> thereafter. >>>>>>>> Increase buffer size by 1 and add '\0'. >>>>>>> >>>>>>> Given: >>>>>>> #define BUF_SIZE (PATH_MAX + NAME_MAX + 1) >>>>>>> isn't it impossible to encounter that problem? >>>>>> As I understand, pread does not null-terminate what it read. So the >>>>>> null must come from the file. This protects against a corrupted >>>>>> file. >>>>> >>>>> So are you saying the nul is not present in the file? I'm not >>>>> familiar >>>>> with the ELF format. >>>>> >>>>>>>> stubRoutines_x86.cpp: >>>>>>>> Cast to proper type. This way, left and right of '&' have the >>>>>>>> same type. >>>>>>> >>>>>>> I think you could just have changed the uint64_t to uint32_t as >>>>>>> applied >>>>>>> to the shift rather than casting the 1 to uint_32t. End result >>>>>>> is the >>>>>>> same though. >>>>>> What you propose did not work. It was my first fix, too. >>>>> >>>>> Hmm okay. The result of the shift must be an unsigned type and the >>>>> constant 1 is signed, so needs the cast (or use the unsigned constant >>>>> form - 1ud? ) >>>>> >>>>>>>> attachListener_linux.cpp: >>>>>>>> Read does not terminate buf. Size for '\0' is already considered. >>>>>>> >>>>>>> Looks a little odd being done on each iteration, but okay I guess. >>>>>> I'll try to move it out of the loop. Better: I'll check whether the >>>>>> scan groks it if I move it out of the loop :) >>>>>> >>>>>>>> os_linux.cpp: >>>>>>>> Array sigflags[] has size MAXSIGNUM==32. _NSIG is bigger than >>>>>>>> MAXSIGNUM (_NSIG == 65 on my machine). >>>>>>>> sig is checked to be smaller than _NSIG. Later, in >>>>>>>> set_our_sigflags(), >>>>>>>> sig is used to access sigflags[MAXSIGNUM] which can overflow the >>> array. >>>>>>>> Should we also increase MAXSIGNUM? >>>>>>> >>>>>>> Need to let the SR folk comment here as something definitely seems >>>>>>> wrong, but I'm not 100% sure the what the correct answer is. If >>>>>>> _JAVA_SR_SIGNUM is too big it should be validated somewhere and >>> an >>>>> error >>>>>>> or warning reported. >>>>>> I'm also not sure how to best handle this. Might even require a fix >>>>>> exceeding this change. But I think this is the best finding. >>>>>> >>>>>>>> codeBuffer.cpp: >>>>>>>> New_capacity is not initialized. Figure_expanded_capacities() >>>>>>>> handles >>>>> this >>>>>>>> correctly, but initializing this is cheap and safe. >>>>>>> >>>>>>> Hmmm ... I hate redundancy - this is pure wasted cycles. If we >>>>>>> had to >>> do >>>>>>> it would memset not be better? Or would the code-checker not >>>>>>> realize >>>>>>> what memset was doing? >>>>>> I guess it would work with memset, too. But I thought the 3-deep >>>>>> loop >>>>>> will be unrolled completely so that only three stores remain. >>>>> >>>>> I tend not to try and imagine what the compiler may or may not do. >>>>> Happy >>>>> to take other opinions. Though again I'd prefer if the checker >>>>> could be >>>>> shown that there is no missing initialization. >>>>> >>>>>>>> dict.cpp: >>>>>>>> If j-- is executed for j==0, the loop aborts because j is >>>>>>>> unsigned (0-- >= >>> b- >>>>>>>> _cnt). >>>>>>>> Instead, only do j++ if necessary. >>>>>>> >>>>>>> Not at all obvious to me that it is possible to do j-- when >>>>>>> j==0, but >>>>>>> the change seems reasonable. >>>>>> Yes, the scan does not understand there is j++ right after j-- >>>>>> because >>>>>> of the loop iteration. I saw it complaining about this pattern >>>>>> several >>> times. >>>>>> >>>>>>> Lots of spacing changes in that code make it hard to see the real >>> changes. >>>>>> Before, I was asked to fix indentation issues in a function I touch. >>>>>> Does that only hold for compiler files? >>>>> >>>>> Yes/no/maybe :) Fixing up bad formatting when you are touching an >>>>> area >>>>> can be convenient, however it can also obscure the real changes, >>>>> so it >>>>> depends on the ratio of functional changes to format changes. >>>>> >>>>>>> 145 // SAPJVM GL j-- can underflow, which will cause the >>>>>>> loop to >>> abort. >>>>>>> Seems unnecessary with the code change as noone will understand >>> what >>>>> j-- >>>>>>> you are referring to. >>>>>> Didn't mean to leave this in here. Removed. >>>>>> >>>>>>> 150 nb->_keyvals[nbcnt + nbcnt ] = key; >>>>>>> 151 nb->_keyvals[nbcnt + nbcnt + 1] = >>>>>>> b->_keyvals[j+j+1]; >>>>>>> hotspot-style doesn't align array index expressions like that. >>>>>>> Ditto >>>>>>> 154/155. >>>>>> Fixed. >>>>>> >>>>>>>> generateOopMap.cpp: >>>>>>>> Idx is read from String. This is only called with constant >>>>>>>> strings, so >>>>> compare >>>>>>>> should be folded away by optimizing compilers if inlined. >>>>>>> >>>>>>> Not a fan of adding conditions that should never be false (hence >>>>>>> the >>>>>>> assert) and relying on the compiler to elide them. >>>>>> OK, removed. >>>>>> >>>>>>>> deoptimization.cpp: >>>>>>>> If buflen == 0, buf[-1] is accessed. >>>>>>> >>>>>>> Okay - but an assert(buflen>0) would be better I think as we should >>>>>>> never be calling with a zero-length buffer. >>>>>> Ok, I added the assert. As this isn't critical code, I would >>>>>> like to leave the >>>>>> check in there, still. >>>>>> >>>>>>>> task.cpp: >>>>>>>> Fatal can return if -XX:SuppressErrorAt is used. Just don't >>>>>>>> access the >>>>>>>> array in this case. >>>>>>> >>>>>>> Okay. I would not be surprised if we have a lot of potential >>>>>>> errors if a >>>>>>> fatal/guarantee failure is suppressed. >>>>>>> >>>>>>>> attachListener.hpp: >>>>>>>> Do strncpy to not overflow buffer. Don't write more chars than >>> before. >>>>>>> >>>>>>> Again we have the assert to catch an error in the caller using an >>>>>>> invalid name. >>>>>> Hmm, the command comes from outside of the VM. It's not checked >>>>>> very thoroughly, see, e.g., attachListener_windows.cpp:194. Arg0 is >>>>>> checked twice, arg1 and arg2 are not checked at all. >>>>> >>>>> The libattach code is still part of our codebase so should be >>>>> doing the >>>>> right things. The linux and solaris code seems to be doing the >>>>> expected >>>>> name length check. On Windows the name is set using cmd, which is >>>>> also >>>>> subject to a length check: >>>>> >>>>> if (strlen(cmd) > AttachOperation::name_length_max) return >>>>> ATTACH_ERROR_ILLEGALARG; >>>>> >>>>>> I add fixes for attachListener_windows.cpp to this change. >>>>>> >>>>>>>> heapDumper.cpp: >>>>>>>> strncpy does not null terminate. >>>>>>> >>>>>>> 1973 if (total_length >= sizeof(base_path)) { >>>>>>> >>>>>>> total_length already adds +1 for the nul character so the == >>>>>>> case is >>>>>>> fine AFAICS. >>>>>>> >>>>>>> strncpy wont nul-terminate if the string exceeds the buffer >>>>>>> size. But >>> we >>>>>>> have already established that total_length <= sizeof(base_path), >>>>>>> and >>>>>>> total_path includes space for a bunch of stuff other than >>> HeapDumpPath, >>>>>>> so the strncpy of HeapDumpPath has to copy the nul character. >>>>>> Ok, removed. >>>>>> >>>>>>> > src/share/vm/services/memoryService.cpp >>>>>>> >>>>>>> Ok. >>>>>>> >>>>>>> > src/share/vm/utilities/xmlstream.cpp >>>>>>> >>>>>>> Ok - I'm more concerned about the "magic" 10 in that piece of code. >>>>>> I assume the 10 is the space needed for the "_done" plus some >>>>>> waste ... >>>>>> >>>>>> I'll do another run of the scan. That takes a day. I'll post a >>>>>> new webrev >>>>> after >>>>>> that. >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>>> Thank again for this thorough review, >>>>>> Goetz >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>>> Some of these, as the issue in codeBuffer.cpp, are actually >>>>>>>> handled >>>>> correctly. >>>>>>>> Nevertheless this is not that obvious so that somebody changing >>>>>>>> the >>>>> code >>>>>>>> Could oversee he has to add the initialization. >>>>>>> >>>>>>> Not an argument I buy en-masse as it leads to a lot of redundancy >>>>>>> through the code paths. Plus these tools that are being run should >>> show >>>>>>> if a code changes requires initialization that is not present. >>>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> >>>>>>>> Some of these fixes are part of SAP JVM for a long time. This >>>>>>>> change >>> has >>>>>>>> been tested with our nightly build of openJDK. >>>>>>>> >>>>>>>> Best regards, >>>>>>>> Goetz,. >>>>>>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Thu Nov 5 03:50:10 2015 From: david.holmes at oracle.com (David Holmes) Date: Thu, 5 Nov 2015 13:50:10 +1000 Subject: RFR: JDK-8140556: Add force rotation option to VM.log jcmd In-Reply-To: <563ACD6F.1050600@gmail.com> References: <562101D8.5080909@gmail.com> <562DF8D4.40501@oracle.com> <562EBF4B.4020607@gmail.com> <562F6B46.5050500@oracle.com> <563226AB.2030206@gmail.com> <5632341F.4060400@oracle.com> <56338125.9030802@gmail.com> <56338D58.9090709@oracle.com> <5634D9FC.6080502@gmail.com> <5636C66A.6020304@oracle.com> <563765FC.2090804@gmail.com> <563ACD6F.1050600@gmail.com> Message-ID: <563AD1F2.6060109@oracle.com> On 5/11/2015 1:30 PM, Yasumasa Suenaga wrote: > I've uploaded new webrev: > http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.03/ > > I added _initialized to LogConfiguration. > LogConfiguration::post_initialize() will be called after main thread > initialization. > So _initialized set to true in this function, and we can check this flag > at rotation. > > Could you review it? _post_initialized would be a more accurate name. This of course works, but I don't know whether the additional delay in allowing rotation will impact anything. Thanks, David > > Thanks, > > Yasumasa > > > On 2015/11/02 22:32, Yasumasa Suenaga wrote: >> Marcus, David, >> >> I've uploaded new webrev: >> http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.02/ >> >> Could you review it? >> >> >>>>>>> Still missing the renaming of rotate_all_logfile to >>>>>>> rotate_all_outputs. >> >> I've fixed. >> >> >>>>> Small typo, please make that is_rotatable(). >> >> I've added, and have used at LogConfiguration::rotate_all_outputs(). >> >> >>> You can't safely lock a mutex until the current thread has attached >>> to the VM and Thread::current() returns non-null (that is what >>> Thread::initialize_thread_local_storage() currently does). You may >>> get away with it, particularly in product builds, during VM >>> initialization, as the fast-lock path doesn't require access to >>> Thread::current(). >> >> I will check whether Thread::current() is NULL or not at >> LogFileOutput::is_rotatable() >> is_rotatable() is called at LogFileOutput::rotate() before locking >> _rotation_lock. >> So we can avoid crash/assert which is related to TLS. >> >> >> Thanks, >> >> Yasumasa >> >> >> On 2015/11/02 11:11, David Holmes wrote: >>> On 1/11/2015 1:10 AM, Yasumasa Suenaga wrote: >>>>> This in turn means that we are logging before the thread local storage >>>>> has been initialized, and the JVM will crash/hit the assert because we >>>>> are trying to take the rotation lock. >>>> >>>> I guess that you are saying about class member (not TLS - e.g. >>>> pthread_setspecific()). >>> >>> You can't safely lock a mutex until the current thread has attached >>> to the VM and Thread::current() returns non-null (that is what >>> Thread::initialize_thread_local_storage() currently does). You may >>> get away with it, particularly in product builds, during VM >>> initialization, as the fast-lock path doesn't require access to >>> Thread::current(). >>> >>> David >>> >>>> Most of fields (include _rotation_lock) in LogFileOutput are >>>> initialized >>>> at c'tor. >>>> Other field (_stream) is initialized at LogFileOutput::initialize(). >>>> >>>> We can avoid problem if we can move initialize() to c'tor because >>>> LogFileOutput >>>> will be instantiated at LogConfiguration::new_output(). >>>> >>>> Currently, LogFileOutput will be added to LogConfiguration::_outputs >>>> and >>>> to LogTagSet after calling LogFileOutput::initialize(). >>>> I wonder why we cannot avoid crash/assert with current >>>> implementation... >>>> >>>> >>>> I will keep current implementation about _rotation_lock if the above is >>>> incorrect. >>>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> On 2015/10/31 0:31, Marcus Larsson wrote: >>>>> Hey, >>>>> >>>>> On 2015-10-30 15:39, Yasumasa Suenaga wrote: >>>>>> Hi Marcus, >>>>>> >>>>>> Thank you for your comment. >>>>>> >>>>>> >>>>>>> Still missing the renaming of rotate_all_logfile to >>>>>>> rotate_all_outputs. >>>>>> >>>>>> Sorry, I will fix. >>>>>> >>>>> >>>>> Thanks! >>>>> >>>>>> >>>>>>> This rotate function works I guess, but it will probably need some >>>>>>> rework once there are other types of log outputs (rotate doesn't >>>>>>> make sense on all types of log outputs). >>>>>> >>>>>> I will add is_rotetable() to LogOutput as virtual function. >>>>>> This function return false (can not rotate) by default, return true >>>>>> if _file_count is >>>>>> greater than 0 in LogFileOutput. >>>>>> >>>>>> LogConfiguration::rotate_all_outputs() will rotate if this function >>>>>> returns true. >>>>>> >>>>> >>>>> Small typo, please make that is_rotatable(). >>>>> >>>>>> >>>>>>> If we configure a rotated file output and then log something on that >>>>>>> output before thread local storage initialization is complete, we >>>>>>> will crash/hit an assert. The previous implementation avoided this >>>>>>> as long as no rotation was needed before this initialization was >>>>>>> complete. (This can be triggered using the following arguments >>>>>>> "-Xlog:all=trace:file.txt::filesize=10,filecount=2 -Xlog:invalid", >>>>>>> for example.) >>>>>> >>>>>> I do not think so. >>>>>> Each -Xlog option creates unique instance of LogOutput (and >>>>>> subclass). >>>>>> So they are isolated. >>>>> >>>>> My example includes a second (and intentionally incorrect) -Xlog >>>>> option only to trigger logging on the newly configured rotating >>>>> LogFileOutput. This will cause some logging about the invalid -Xlog >>>>> argument, and this logging happens during the argument parsing. This >>>>> in turn means that we are logging before the thread local storage has >>>>> been initialized, and the JVM will crash/hit the assert because we are >>>>> trying to take the rotation lock. This has nothing to do with >>>>> concurrent fprintfs or freopens, even if that is a problem. >>>>> >>>>>> >>>>>> However, we might have to rotate at safepoint. >>>>> >>>>> Doing stuff at a safepoint will not give us much in this case. There >>>>> are threads that continue execution through a safepoint, which means >>>>> we will see log calls being made even during this time. >>>>> >>>>>> Currently, LogOutput might be used by multiple threads concurrently. >>>>>> If we rotate log when another thread is writing string to output, we >>>>>> encounter unexpected behavior. >>>>>> >>>>>> LogFileOutput::rotate() uses freopen(). >>>>>> LogFileStreamOutput::write() uses vfprintf() through jio_fprintf(). >>>>>> freopen() and vfprintf() are not atomic. >>>>> >>>>> You're right. There seems to be an overlooked concurrency issue with >>>>> vfprintfs during freopen. I don't think we should resolve this as part >>>>> of this particular fix, so instead I'll create a separate issue for >>>>> it. >>>>> >>>>>> >>>>>> I think that cause of crash/assertion which you say is it. >>>>>> (GC log will be rotated at safepoint.) >>>>>> >>>>> >>>>> See my comment above for what crash/assert I'm talking about. >>>>> >>>>> Thanks, >>>>> Marcus >>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2015/10/29 23:58, Marcus Larsson wrote: >>>>>>> Hi, >>>>>>> >>>>>>> On 2015-10-29 15:01, Yasumasa Suenaga wrote: >>>>>>>> Hi Marcus, >>>>>>>> >>>>>>>> Thank you for your comment. >>>>>>>> >>>>>>>>> I'll sponsor it. >>>>>>>> >>>>>>>> Thank you so much! >>>>>>>> I've uploaded new webrev: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.01/ >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> Still missing the renaming of rotate_all_logfile to >>>>>>> rotate_all_outputs. >>>>>>> >>>>>>>>> logConfiguration.cpp/hpp: >>>>>>>>> >>>>>>>>> * I don't think we should rely on the archive_name or the output's >>>>>>>>> name to decide whether or not an output should be rotated. It >>>>>>>>> would be better to introduce an is_rotated() test function in >>>>>>>>> LogOutput that could be used here. >>>>>>>>> >>>>>>>>> * rotate_all_logfile() should be renamed to rotate_all_outputs(). >>>>>>>>> Currently there are only LogFileOutputs (other than >>>>>>>>> stdout/stderr), but in the future there might be other types of >>>>>>>>> outputs so I prefer having a more general name. >>>>>>>>> Also, please use static_cast instead of the >>>>>>>>> C-style cast. (Additional logic will be required here once more >>>>>>>>> types of log outputs are added, but I don't think we need to worry >>>>>>>>> about this right now.) >>>>>>>>> >>>>>>>>> * Don't print an error if an output is not rotatable, since it's >>>>>>>>> not an error to have some log outputs rotated while others are >>>>>>>>> not. If you really want some traceability here I suggest adding >>>>>>>>> log messages on trace level, tagged with 'logging'. >>>>>>>> >>>>>>>> I added LogOutput::rotate(bool) as virtual function. >>>>>>>> This function works nothing by default, but will rotate all logs at >>>>>>>> LogFileOutput. >>>>>>> >>>>>>> This rotate function works I guess, but it will probably need some >>>>>>> rework once there are other types of log outputs (rotate doesn't >>>>>>> make sense on all types of log outputs). >>>>>>> >>>>>>>> >>>>>>>> If true is passed to this function, all files will be rotated. >>>>>>>> >>>>>>>> >>>>>>>>> logDiagnosticCommand.cpp/hpp: >>>>>>>>> >>>>>>>>> * I think the description could be improved to something like >>>>>>>>> "Lists current log configuration, enables/disables/configures a >>>>>>>>> log output, or disables/rotates all logs." >>>>>>>>> >>>>>>>>> * The rotate option description should clarify that all logs will >>>>>>>>> be rotated ("current log" is too ambiguous). >>>>>>>> >>>>>>>> I've fixed. >>>>>>> >>>>>>> Great, thanks! >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> logFileOutput.cpp/hpp: >>>>>>>>> >>>>>>>>> * Moving the MutexLocker like this introduces a race condition >>>>>>>>> where a log might be rotated multiple times by different threads, >>>>>>>>> instead of just once. >>>>>>>>> Instead of making the rotate() function public and moving the >>>>>>>>> mutexlocker, I suggest adding something like a public >>>>>>>>> force_rotation() function that grabs the lock and calls the >>>>>>>>> private rotate(). >>>>>>>>> >>>>>>>>> * Given the addition of is_rotated() in LogOutput, then >>>>>>>>> get_archive_name() should be removed. >>>>>>>> >>>>>>>> I moved MutexLocker and should_rotate() to rotate(). >>>>>>>> I think this change can avoid race condition. >>>>>>>> >>>>>>> >>>>>>> There is a subtle problem with taking the lock to check if we should >>>>>>> rotate. If we configure a rotated file output and then log something >>>>>>> on that output before thread local storage initialization is >>>>>>> complete, we will crash/hit an assert. The previous implementation >>>>>>> avoided this as long as no rotation was needed before this >>>>>>> initialization was complete. (This can be triggered using the >>>>>>> following arguments >>>>>>> "-Xlog:all=trace:file.txt::filesize=10,filecount=2 -Xlog:invalid", >>>>>>> for example.) >>>>>>> >>>>>>> Thanks, >>>>>>> Marcus >>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> On 2015/10/27 21:17, Marcus Larsson wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> On 2015-10-27 01:03, Yasumasa Suenaga wrote: >>>>>>>>>> Hi Marcus, >>>>>>>>>> >>>>>>>>>> Thank you for replying. >>>>>>>>>> >>>>>>>>>> I filed this feature to JBS and change subject of this email. >>>>>>>>>> Could you be a sponsor and review it? >>>>>>>>> >>>>>>>>> I'll sponsor it. >>>>>>>>> >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.00/ >>>>>>>>> >>>>>>>>> logConfiguration.cpp/hpp: >>>>>>>>> >>>>>>>>> * I don't think we should rely on the archive_name or the output's >>>>>>>>> name to decide whether or not an output should be rotated. It >>>>>>>>> would be better to introduce an is_rotated() test function in >>>>>>>>> LogOutput that could be used here. >>>>>>>>> >>>>>>>>> * rotate_all_logfile() should be renamed to rotate_all_outputs(). >>>>>>>>> Currently there are only LogFileOutputs (other than >>>>>>>>> stdout/stderr), but in the future there might be other types of >>>>>>>>> outputs so I prefer having a more general name. >>>>>>>>> Also, please use static_cast instead of the >>>>>>>>> C-style cast. (Additional logic will be required here once more >>>>>>>>> types of log outputs are added, but I don't think we need to worry >>>>>>>>> about this right now.) >>>>>>>>> >>>>>>>>> * Don't print an error if an output is not rotatable, since it's >>>>>>>>> not an error to have some log outputs rotated while others are >>>>>>>>> not. If you really want some traceability here I suggest adding >>>>>>>>> log messages on trace level, tagged with 'logging'. >>>>>>>>> >>>>>>>>> >>>>>>>>> logDiagnosticCommand.cpp/hpp: >>>>>>>>> >>>>>>>>> * I think the description could be improved to something like >>>>>>>>> "Lists current log configuration, enables/disables/configures a >>>>>>>>> log output, or disables/rotates all logs." >>>>>>>>> >>>>>>>>> * The rotate option description should clarify that all logs will >>>>>>>>> be rotated ("current log" is too ambiguous). >>>>>>>>> >>>>>>>>> >>>>>>>>> logFileOutput.cpp/hpp: >>>>>>>>> >>>>>>>>> * Moving the MutexLocker like this introduces a race condition >>>>>>>>> where a log might be rotated multiple times by different threads, >>>>>>>>> instead of just once. >>>>>>>>> Instead of making the rotate() function public and moving the >>>>>>>>> mutexlocker, I suggest adding something like a public >>>>>>>>> force_rotation() function that grabs the lock and calls the >>>>>>>>> private rotate(). >>>>>>>>> >>>>>>>>> * Given the addition of is_rotated() in LogOutput, then >>>>>>>>> get_archive_name() should be removed. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Marcus >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 2015/10/26 18:56, Marcus Larsson wrote: >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> Sorry for my late reply. >>>>>>>>>>> >>>>>>>>>>> I think being able to force rotation via jcmd seems like a good >>>>>>>>>>> feature. Files are currently opened in append mode so it should >>>>>>>>>>> already be possible to use external log rotation tools by >>>>>>>>>>> copying and truncating the files. Still I think it would be nice >>>>>>>>>>> to provide the jcmd for rotation as well. >>>>>>>>>>> >>>>>>>>>>> I can see some small issues with the implementation, but we can >>>>>>>>>>> deal with that during the review. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Marcus >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 2015-10-26 00:26, Yasumasa Suenaga wrote: >>>>>>>>>>>> Hi all, >>>>>>>>>>>> >>>>>>>>>>>> Have you ever seen my change? >>>>>>>>>>>> I (and my customers) need log rotation function via external >>>>>>>>>>>> tool. >>>>>>>>>>>> >>>>>>>>>>>> I want to merge it by Feature Complete. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Yasumasa >>>>>>>>>>>> 2015/10/16 22:55 "Yasumasa Suenaga" : >>>>>>>>>>>> >>>>>>>>>>>>> Hi all, >>>>>>>>>>>>> >>>>>>>>>>>>> I contributed JDK-7090324: gclog rotation via external tool >>>>>>>>>>>>> to be >>>>>>>>>>>>> synchronized with >>>>>>>>>>>>> logrotated tool on Linux. >>>>>>>>>>>>> >>>>>>>>>>>>> I think JEP 158 is in progress. >>>>>>>>>>>>> However, this JEP does not contain log rotation via external >>>>>>>>>>>>> tool in the >>>>>>>>>>>>> spec. >>>>>>>>>>>>> I want to rotate logs via jcmd in this JEP. >>>>>>>>>>>>> >>>>>>>>>>>>> I've updated a patch for it: >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/jvmlogging-logrotate/ >>>>>>>>>>>>> >>>>>>>>>>>>> This patch provides new option "rotate" in VM.log command. >>>>>>>>>>>>> If this change can be accepted, I will file it to JBS and send >>>>>>>>>>>>> RFR. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> >>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>> >>>>> From markus.gronlund at oracle.com Thu Nov 5 10:15:10 2015 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Thu, 5 Nov 2015 02:15:10 -0800 (PST) Subject: RFR(M): 8140482: Various minor code improvements (runtime) In-Reply-To: <5639CFDE.1010302@oracle.com> References: <4295855A5C1DE049A61835A1887419CC41E9F0AA@DEWDFEMB12A.global.corp.sap> <563840B0.10401@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB1746@DEWDFEMB12A.global.corp.sap> <56388760.60800@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB1875@DEWDFEMB12A.global.corp.sap> <5639B058.9090609@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB1B57@DEWDFEMB12A.global.corp.sap> <5639CFDE.1010302@oracle.com> Message-ID: <78c21008-fcd0-4745-9291-1fc5d6c98a15@default> Hi Goetz, Thanks for suggesting these fixes. Also thanks for pointing to the issue with _NSIG and MAXSIGNUM - looks like MAXSIGNUM is defined all over the place (platform specific + platform specific jsig's)... I also think it makes perfect sense to bound the signals to the dimensions of the arrays where they will be stored (sigact[MAXSIGNUM] and sigflags[MAXSIGNUM]). SR_initialize() will (after optionally fetching the env variable for SR_signum) eventually call into: void os::Linux::set_our_sigflags() { assert(sig > 0 && sig < MAXSIGNUM, "vm signal out of expected range"); sigflags[sig] = flags; } This assert makes me question the expression in SR_initialize(): if (sig > 0 || sig < _NSIG) { <<---- SR_signum = sig; } Due to shortcutting there is no upper bound range check on the signal here. If _NSIG is larger than MAXSIGNUM this could be a significant problem. Should most likely be changed to: (sig > 0 && sig < MAXSIGNUM) Thanks Markus -----Original Message----- From: David Holmes Sent: den 4 november 2015 10:29 To: Lindenmaier, Goetz; hotspot-runtime-dev at openjdk.java.net; serviceability-dev Subject: Re: RFR(M): 8140482: Various minor code improvements (runtime) On 4/11/2015 6:01 PM, Lindenmaier, Goetz wrote: > Hi David, > > attachListener.hpp: > I agree that I can not find another possible issue with the strcpy. > Still I think it's better to have the strncpy, as it would have > protected against the bug in attachListener_windows.cpp. > But if you insist I'll just remove it. I don't insist, but I do prefer to place all the guards at the "boundary" of the VM rather than at every level when possible. > Should I remove the _NSIG issue from this change and open an issue of > it's own discussed on the serviceability list? Let's give them a chance to respond. I'll ping them on the hotline ;-) Thanks, David > Best regards, > Goetz. > > >> -----Original Message----- >> From: David Holmes [mailto:david.holmes at oracle.com] >> Sent: Mittwoch, 4. November 2015 08:15 >> To: Lindenmaier, Goetz; hotspot-runtime-dev at openjdk.java.net; >> serviceability-dev >> Subject: Re: RFR(M): 8140482: Various minor code improvements >> (runtime) >> >> Hi Goetz, >> >> On 4/11/2015 12:10 AM, Lindenmaier, Goetz wrote: >>> Hi David, >>> >>> the new scan is already through. I made a new webrev: >>> http://cr.openjdk.java.net/~goetz/webrevs/8140482-covRt/webrev.01/ >>> >>> attachListener_linux.cpp: >>> I had moved the string termination out of the loop, and read one >>> less char from the file. The scan still claims "Passing unterminated >>> string buf to strlen" in line 274. So I will undo it again for the >>> webrev. >>> >>> codeBuffer.cpp >>> Doing memset is fine. I'll use memset(). >>> >>>>>>> ps_core.c: >>>>>>> Pread not necessarily terminates interp_name which is printed >>>>>>> thereafter. Increase buffer size by 1 and add '\0'. >>>>>> >>>>>> Given: >>>>>> #define BUF_SIZE (PATH_MAX + NAME_MAX + 1) >>>>>> isn't it impossible to encounter that problem? >>>>> As I understand, pread does not null-terminate what it read. So >>>>> the null must come from the file. This protects against a corrupted file. >>>> >>>> So are you saying the nul is not present in the file? I'm not >>>> familiar with the ELF format. >>> There should be a nul in the file, else the code would fail more >>> obviously. The problem is if the file is corrupted. >> >> Thanks for clarifying. >> >> I'm still unclear why the attachListener.hpp changes are needed if we >> have validated the entry points in the attachListener_.cpp files? >> >> Also we still need someone from serviceability to comment on the >> _NSIG issue. >> >> Thanks, >> David >> >> >>> Best regards, >>> Goetz. >>> >>> >>>> -----Original Message----- >>>> From: David Holmes [mailto:david.holmes at oracle.com] >>>> Sent: Dienstag, 3. November 2015 11:07 >>>> To: Lindenmaier, Goetz; hotspot-runtime-dev at openjdk.java.net; >>>> serviceability-dev >>>> Subject: Re: RFR(M): 8140482: Various minor code improvements >> (runtime) >>>> >>>> Hi Goetz, >>>> >>>> Quick follow up on a couple of things ... >>>> >>>> On 3/11/2015 7:33 PM, Lindenmaier, Goetz wrote: >>>>> Hi David, >>>>> >>>>>> Sorry, lots of folks very busy at the moment as we try to get >>>>>> features finalized before the deadline. >>>>> thanks for looking at this! I know the Dec. 10 deadline, but I >>>>> guess that >> also >>>>> holds for us ... at least J1 is over now. (Unfortunately we could >>>>> not >> attend >>>>> this year.) >>>> >>>> Me neither :) >>>> >>>>>>> ps_core.c: >>>>>>> Pread not necessarily terminates interp_name which is printed >>>> thereafter. >>>>>>> Increase buffer size by 1 and add '\0'. >>>>>> >>>>>> Given: >>>>>> #define BUF_SIZE (PATH_MAX + NAME_MAX + 1) >>>>>> isn't it impossible to encounter that problem? >>>>> As I understand, pread does not null-terminate what it read. So >>>>> the null must come from the file. This protects against a corrupted file. >>>> >>>> So are you saying the nul is not present in the file? I'm not >>>> familiar with the ELF format. >>>> >>>>>>> stubRoutines_x86.cpp: >>>>>>> Cast to proper type. This way, left and right of '&' have the same type. >>>>>> >>>>>> I think you could just have changed the uint64_t to uint32_t as >>>>>> applied to the shift rather than casting the 1 to uint_32t. End >>>>>> result is the same though. >>>>> What you propose did not work. It was my first fix, too. >>>> >>>> Hmm okay. The result of the shift must be an unsigned type and the >>>> constant 1 is signed, so needs the cast (or use the unsigned >>>> constant form - 1ud? ) >>>> >>>>>>> attachListener_linux.cpp: >>>>>>> Read does not terminate buf. Size for '\0' is already considered. >>>>>> >>>>>> Looks a little odd being done on each iteration, but okay I guess. >>>>> I'll try to move it out of the loop. Better: I'll check whether >>>>> the scan groks it if I move it out of the loop :) >>>>> >>>>>>> os_linux.cpp: >>>>>>> Array sigflags[] has size MAXSIGNUM==32. _NSIG is bigger than >>>>>>> MAXSIGNUM (_NSIG == 65 on my machine). >>>>>>> sig is checked to be smaller than _NSIG. Later, in >>>>>>> set_our_sigflags(), sig is used to access sigflags[MAXSIGNUM] >>>>>>> which can overflow the >> array. >>>>>>> Should we also increase MAXSIGNUM? >>>>>> >>>>>> Need to let the SR folk comment here as something definitely >>>>>> seems wrong, but I'm not 100% sure the what the correct answer >>>>>> is. If _JAVA_SR_SIGNUM is too big it should be validated >>>>>> somewhere and >> an >>>> error >>>>>> or warning reported. >>>>> I'm also not sure how to best handle this. Might even require a >>>>> fix exceeding this change. But I think this is the best finding. >>>>> >>>>>>> codeBuffer.cpp: >>>>>>> New_capacity is not initialized. Figure_expanded_capacities() >>>>>>> handles >>>> this >>>>>>> correctly, but initializing this is cheap and safe. >>>>>> >>>>>> Hmmm ... I hate redundancy - this is pure wasted cycles. If we >>>>>> had to >> do >>>>>> it would memset not be better? Or would the code-checker not >>>>>> realize what memset was doing? >>>>> I guess it would work with memset, too. But I thought the 3-deep >>>>> loop will be unrolled completely so that only three stores remain. >>>> >>>> I tend not to try and imagine what the compiler may or may not do. >>>> Happy to take other opinions. Though again I'd prefer if the >>>> checker could be shown that there is no missing initialization. >>>> >>>>>>> dict.cpp: >>>>>>> If j-- is executed for j==0, the loop aborts because j is >>>>>>> unsigned (0-- >= >> b- >>>>>>> _cnt). >>>>>>> Instead, only do j++ if necessary. >>>>>> >>>>>> Not at all obvious to me that it is possible to do j-- when j==0, >>>>>> but the change seems reasonable. >>>>> Yes, the scan does not understand there is j++ right after j-- >>>>> because of the loop iteration. I saw it complaining about this >>>>> pattern several >> times. >>>>> >>>>>> Lots of spacing changes in that code make it hard to see the real >> changes. >>>>> Before, I was asked to fix indentation issues in a function I touch. >>>>> Does that only hold for compiler files? >>>> >>>> Yes/no/maybe :) Fixing up bad formatting when you are touching an >>>> area can be convenient, however it can also obscure the real >>>> changes, so it depends on the ratio of functional changes to format changes. >>>> >>>>>> 145 // SAPJVM GL j-- can underflow, which will cause the loop to >> abort. >>>>>> Seems unnecessary with the code change as noone will understand >> what >>>> j-- >>>>>> you are referring to. >>>>> Didn't mean to leave this in here. Removed. >>>>> >>>>>> 150 nb->_keyvals[nbcnt + nbcnt ] = key; >>>>>> 151 nb->_keyvals[nbcnt + nbcnt + 1] = b->_keyvals[j+j+1]; >>>>>> hotspot-style doesn't align array index expressions like that. >>>>>> Ditto 154/155. >>>>> Fixed. >>>>> >>>>>>> generateOopMap.cpp: >>>>>>> Idx is read from String. This is only called with constant >>>>>>> strings, so >>>> compare >>>>>>> should be folded away by optimizing compilers if inlined. >>>>>> >>>>>> Not a fan of adding conditions that should never be false (hence >>>>>> the >>>>>> assert) and relying on the compiler to elide them. >>>>> OK, removed. >>>>> >>>>>>> deoptimization.cpp: >>>>>>> If buflen == 0, buf[-1] is accessed. >>>>>> >>>>>> Okay - but an assert(buflen>0) would be better I think as we >>>>>> should never be calling with a zero-length buffer. >>>>> Ok, I added the assert. As this isn't critical code, I would like >>>>> to leave the check in there, still. >>>>> >>>>>>> task.cpp: >>>>>>> Fatal can return if -XX:SuppressErrorAt is used. Just don't >>>>>>> access the array in this case. >>>>>> >>>>>> Okay. I would not be surprised if we have a lot of potential >>>>>> errors if a fatal/guarantee failure is suppressed. >>>>>> >>>>>>> attachListener.hpp: >>>>>>> Do strncpy to not overflow buffer. Don't write more chars than >> before. >>>>>> >>>>>> Again we have the assert to catch an error in the caller using an >>>>>> invalid name. >>>>> Hmm, the command comes from outside of the VM. It's not checked >>>>> very thoroughly, see, e.g., attachListener_windows.cpp:194. Arg0 >>>>> is checked twice, arg1 and arg2 are not checked at all. >>>> >>>> The libattach code is still part of our codebase so should be doing >>>> the right things. The linux and solaris code seems to be doing the >>>> expected name length check. On Windows the name is set using cmd, >>>> which is also subject to a length check: >>>> >>>> if (strlen(cmd) > AttachOperation::name_length_max) return >>>> ATTACH_ERROR_ILLEGALARG; >>>> >>>>> I add fixes for attachListener_windows.cpp to this change. >>>>> >>>>>>> heapDumper.cpp: >>>>>>> strncpy does not null terminate. >>>>>> >>>>>> 1973 if (total_length >= sizeof(base_path)) { >>>>>> >>>>>> total_length already adds +1 for the nul character so the == case >>>>>> is fine AFAICS. >>>>>> >>>>>> strncpy wont nul-terminate if the string exceeds the buffer size. >>>>>> But >> we >>>>>> have already established that total_length <= sizeof(base_path), >>>>>> and total_path includes space for a bunch of stuff other than >> HeapDumpPath, >>>>>> so the strncpy of HeapDumpPath has to copy the nul character. >>>>> Ok, removed. >>>>> >>>>>> > src/share/vm/services/memoryService.cpp >>>>>> >>>>>> Ok. >>>>>> >>>>>> > src/share/vm/utilities/xmlstream.cpp >>>>>> >>>>>> Ok - I'm more concerned about the "magic" 10 in that piece of code. >>>>> I assume the 10 is the space needed for the "_done" plus some waste ... >>>>> >>>>> I'll do another run of the scan. That takes a day. I'll post a >>>>> new webrev >>>> after >>>>> that. >>>> >>>> Thanks, >>>> David >>>> >>>>> Thank again for this thorough review, >>>>> Goetz >>>>> >>>>> >>>>> >>>>> >>>>>>> Some of these, as the issue in codeBuffer.cpp, are actually >>>>>>> handled >>>> correctly. >>>>>>> Nevertheless this is not that obvious so that somebody changing >>>>>>> the >>>> code >>>>>>> Could oversee he has to add the initialization. >>>>>> >>>>>> Not an argument I buy en-masse as it leads to a lot of redundancy >>>>>> through the code paths. Plus these tools that are being run >>>>>> should >> show >>>>>> if a code changes requires initialization that is not present. >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> >>>>>>> Some of these fixes are part of SAP JVM for a long time. This >>>>>>> change >> has >>>>>>> been tested with our nightly build of openJDK. >>>>>>> >>>>>>> Best regards, >>>>>>> Goetz,. >>>>>>> From dmitry.samersoff at oracle.com Thu Nov 5 10:25:34 2015 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Thu, 5 Nov 2015 13:25:34 +0300 Subject: RFR(M): 8140482: Various minor code improvements (runtime) In-Reply-To: <4295855A5C1DE049A61835A1887419CC41EB1B57@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC41E9F0AA@DEWDFEMB12A.global.corp.sap> <563840B0.10401@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB1746@DEWDFEMB12A.global.corp.sap> <56388760.60800@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB1875@DEWDFEMB12A.global.corp.sap> <5639B058.9090609@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB1B57@DEWDFEMB12A.global.corp.sap> Message-ID: <563B2E9E.8070704@oracle.com> Goetz, > Should I remove the _NSIG issue from this change and > open an issue of it's own discussed on the serviceability list? Yes please. -Dmitry On 2015-11-04 11:01, Lindenmaier, Goetz wrote: > Hi David, > > attachListener.hpp: > I agree that I can not find another possible issue > with the strcpy. > Still I think it's better to have the strncpy, as it would have > protected against the bug in attachListener_windows.cpp. > But if you insist I'll just remove it. > > Should I remove the _NSIG issue from this change and > open an issue of it's own discussed on the serviceability list? > > Best regards, > Goetz. > > >> -----Original Message----- >> From: David Holmes [mailto:david.holmes at oracle.com] >> Sent: Mittwoch, 4. November 2015 08:15 >> To: Lindenmaier, Goetz; hotspot-runtime-dev at openjdk.java.net; >> serviceability-dev >> Subject: Re: RFR(M): 8140482: Various minor code improvements (runtime) >> >> Hi Goetz, >> >> On 4/11/2015 12:10 AM, Lindenmaier, Goetz wrote: >>> Hi David, >>> >>> the new scan is already through. I made a new webrev: >>> http://cr.openjdk.java.net/~goetz/webrevs/8140482-covRt/webrev.01/ >>> >>> attachListener_linux.cpp: >>> I had moved the string termination out of the loop, and read one >>> less char from the file. The scan still claims "Passing unterminated >>> string buf to strlen" in line 274. So I will undo it again for the >>> webrev. >>> >>> codeBuffer.cpp >>> Doing memset is fine. I'll use memset(). >>> >>>>>>> ps_core.c: >>>>>>> Pread not necessarily terminates interp_name which is printed >>>>>>> thereafter. Increase buffer size by 1 and add '\0'. >>>>>> >>>>>> Given: >>>>>> #define BUF_SIZE (PATH_MAX + NAME_MAX + 1) >>>>>> isn't it impossible to encounter that problem? >>>>> As I understand, pread does not null-terminate what it read. So the >>>>> null must come from the file. This protects against a corrupted file. >>>> >>>> So are you saying the nul is not present in the file? I'm not familiar >>>> with the ELF format. >>> There should be a nul in the file, else the code would fail more >>> obviously. The problem is if the file is corrupted. >> >> Thanks for clarifying. >> >> I'm still unclear why the attachListener.hpp changes are needed if we >> have validated the entry points in the attachListener_.cpp files? >> >> Also we still need someone from serviceability to comment on the _NSIG >> issue. >> >> Thanks, >> David >> >> >>> Best regards, >>> Goetz. >>> >>> >>>> -----Original Message----- >>>> From: David Holmes [mailto:david.holmes at oracle.com] >>>> Sent: Dienstag, 3. November 2015 11:07 >>>> To: Lindenmaier, Goetz; hotspot-runtime-dev at openjdk.java.net; >>>> serviceability-dev >>>> Subject: Re: RFR(M): 8140482: Various minor code improvements >> (runtime) >>>> >>>> Hi Goetz, >>>> >>>> Quick follow up on a couple of things ... >>>> >>>> On 3/11/2015 7:33 PM, Lindenmaier, Goetz wrote: >>>>> Hi David, >>>>> >>>>>> Sorry, lots of folks very busy at the moment as we try to get features >>>>>> finalized before the deadline. >>>>> thanks for looking at this! I know the Dec. 10 deadline, but I guess that >> also >>>>> holds for us ... at least J1 is over now. (Unfortunately we could not >> attend >>>>> this year.) >>>> >>>> Me neither :) >>>> >>>>>>> ps_core.c: >>>>>>> Pread not necessarily terminates interp_name which is printed >>>> thereafter. >>>>>>> Increase buffer size by 1 and add '\0'. >>>>>> >>>>>> Given: >>>>>> #define BUF_SIZE (PATH_MAX + NAME_MAX + 1) >>>>>> isn't it impossible to encounter that problem? >>>>> As I understand, pread does not null-terminate what it read. So the >>>>> null must come from the file. This protects against a corrupted file. >>>> >>>> So are you saying the nul is not present in the file? I'm not familiar >>>> with the ELF format. >>>> >>>>>>> stubRoutines_x86.cpp: >>>>>>> Cast to proper type. This way, left and right of '&' have the same type. >>>>>> >>>>>> I think you could just have changed the uint64_t to uint32_t as applied >>>>>> to the shift rather than casting the 1 to uint_32t. End result is the >>>>>> same though. >>>>> What you propose did not work. It was my first fix, too. >>>> >>>> Hmm okay. The result of the shift must be an unsigned type and the >>>> constant 1 is signed, so needs the cast (or use the unsigned constant >>>> form - 1ud? ) >>>> >>>>>>> attachListener_linux.cpp: >>>>>>> Read does not terminate buf. Size for '\0' is already considered. >>>>>> >>>>>> Looks a little odd being done on each iteration, but okay I guess. >>>>> I'll try to move it out of the loop. Better: I'll check whether the >>>>> scan groks it if I move it out of the loop :) >>>>> >>>>>>> os_linux.cpp: >>>>>>> Array sigflags[] has size MAXSIGNUM==32. _NSIG is bigger than >>>>>>> MAXSIGNUM (_NSIG == 65 on my machine). >>>>>>> sig is checked to be smaller than _NSIG. Later, in set_our_sigflags(), >>>>>>> sig is used to access sigflags[MAXSIGNUM] which can overflow the >> array. >>>>>>> Should we also increase MAXSIGNUM? >>>>>> >>>>>> Need to let the SR folk comment here as something definitely seems >>>>>> wrong, but I'm not 100% sure the what the correct answer is. If >>>>>> _JAVA_SR_SIGNUM is too big it should be validated somewhere and >> an >>>> error >>>>>> or warning reported. >>>>> I'm also not sure how to best handle this. Might even require a fix >>>>> exceeding this change. But I think this is the best finding. >>>>> >>>>>>> codeBuffer.cpp: >>>>>>> New_capacity is not initialized. Figure_expanded_capacities() handles >>>> this >>>>>>> correctly, but initializing this is cheap and safe. >>>>>> >>>>>> Hmmm ... I hate redundancy - this is pure wasted cycles. If we had to >> do >>>>>> it would memset not be better? Or would the code-checker not realize >>>>>> what memset was doing? >>>>> I guess it would work with memset, too. But I thought the 3-deep loop >>>>> will be unrolled completely so that only three stores remain. >>>> >>>> I tend not to try and imagine what the compiler may or may not do. Happy >>>> to take other opinions. Though again I'd prefer if the checker could be >>>> shown that there is no missing initialization. >>>> >>>>>>> dict.cpp: >>>>>>> If j-- is executed for j==0, the loop aborts because j is unsigned (0-- >= >> b- >>>>>>> _cnt). >>>>>>> Instead, only do j++ if necessary. >>>>>> >>>>>> Not at all obvious to me that it is possible to do j-- when j==0, but >>>>>> the change seems reasonable. >>>>> Yes, the scan does not understand there is j++ right after j-- because >>>>> of the loop iteration. I saw it complaining about this pattern several >> times. >>>>> >>>>>> Lots of spacing changes in that code make it hard to see the real >> changes. >>>>> Before, I was asked to fix indentation issues in a function I touch. >>>>> Does that only hold for compiler files? >>>> >>>> Yes/no/maybe :) Fixing up bad formatting when you are touching an area >>>> can be convenient, however it can also obscure the real changes, so it >>>> depends on the ratio of functional changes to format changes. >>>> >>>>>> 145 // SAPJVM GL j-- can underflow, which will cause the loop to >> abort. >>>>>> Seems unnecessary with the code change as noone will understand >> what >>>> j-- >>>>>> you are referring to. >>>>> Didn't mean to leave this in here. Removed. >>>>> >>>>>> 150 nb->_keyvals[nbcnt + nbcnt ] = key; >>>>>> 151 nb->_keyvals[nbcnt + nbcnt + 1] = b->_keyvals[j+j+1]; >>>>>> hotspot-style doesn't align array index expressions like that. Ditto >>>>>> 154/155. >>>>> Fixed. >>>>> >>>>>>> generateOopMap.cpp: >>>>>>> Idx is read from String. This is only called with constant strings, so >>>> compare >>>>>>> should be folded away by optimizing compilers if inlined. >>>>>> >>>>>> Not a fan of adding conditions that should never be false (hence the >>>>>> assert) and relying on the compiler to elide them. >>>>> OK, removed. >>>>> >>>>>>> deoptimization.cpp: >>>>>>> If buflen == 0, buf[-1] is accessed. >>>>>> >>>>>> Okay - but an assert(buflen>0) would be better I think as we should >>>>>> never be calling with a zero-length buffer. >>>>> Ok, I added the assert. As this isn't critical code, I would like to leave the >>>>> check in there, still. >>>>> >>>>>>> task.cpp: >>>>>>> Fatal can return if -XX:SuppressErrorAt is used. Just don't access the >>>>>>> array in this case. >>>>>> >>>>>> Okay. I would not be surprised if we have a lot of potential errors if a >>>>>> fatal/guarantee failure is suppressed. >>>>>> >>>>>>> attachListener.hpp: >>>>>>> Do strncpy to not overflow buffer. Don't write more chars than >> before. >>>>>> >>>>>> Again we have the assert to catch an error in the caller using an >>>>>> invalid name. >>>>> Hmm, the command comes from outside of the VM. It's not checked >>>>> very thoroughly, see, e.g., attachListener_windows.cpp:194. Arg0 is >>>>> checked twice, arg1 and arg2 are not checked at all. >>>> >>>> The libattach code is still part of our codebase so should be doing the >>>> right things. The linux and solaris code seems to be doing the expected >>>> name length check. On Windows the name is set using cmd, which is also >>>> subject to a length check: >>>> >>>> if (strlen(cmd) > AttachOperation::name_length_max) return >>>> ATTACH_ERROR_ILLEGALARG; >>>> >>>>> I add fixes for attachListener_windows.cpp to this change. >>>>> >>>>>>> heapDumper.cpp: >>>>>>> strncpy does not null terminate. >>>>>> >>>>>> 1973 if (total_length >= sizeof(base_path)) { >>>>>> >>>>>> total_length already adds +1 for the nul character so the == case is >>>>>> fine AFAICS. >>>>>> >>>>>> strncpy wont nul-terminate if the string exceeds the buffer size. But >> we >>>>>> have already established that total_length <= sizeof(base_path), and >>>>>> total_path includes space for a bunch of stuff other than >> HeapDumpPath, >>>>>> so the strncpy of HeapDumpPath has to copy the nul character. >>>>> Ok, removed. >>>>> >>>>>> > src/share/vm/services/memoryService.cpp >>>>>> >>>>>> Ok. >>>>>> >>>>>> > src/share/vm/utilities/xmlstream.cpp >>>>>> >>>>>> Ok - I'm more concerned about the "magic" 10 in that piece of code. >>>>> I assume the 10 is the space needed for the "_done" plus some waste ... >>>>> >>>>> I'll do another run of the scan. That takes a day. I'll post a new webrev >>>> after >>>>> that. >>>> >>>> Thanks, >>>> David >>>> >>>>> Thank again for this thorough review, >>>>> Goetz >>>>> >>>>> >>>>> >>>>> >>>>>>> Some of these, as the issue in codeBuffer.cpp, are actually handled >>>> correctly. >>>>>>> Nevertheless this is not that obvious so that somebody changing the >>>> code >>>>>>> Could oversee he has to add the initialization. >>>>>> >>>>>> Not an argument I buy en-masse as it leads to a lot of redundancy >>>>>> through the code paths. Plus these tools that are being run should >> show >>>>>> if a code changes requires initialization that is not present. >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> >>>>>>> Some of these fixes are part of SAP JVM for a long time. This change >> has >>>>>>> been tested with our nightly build of openJDK. >>>>>>> >>>>>>> Best regards, >>>>>>> Goetz,. >>>>>>> -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From goetz.lindenmaier at sap.com Thu Nov 5 10:36:17 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 5 Nov 2015 10:36:17 +0000 Subject: RFR(M): 8140482: Various minor code improvements (runtime) In-Reply-To: <78c21008-fcd0-4745-9291-1fc5d6c98a15@default> References: <4295855A5C1DE049A61835A1887419CC41E9F0AA@DEWDFEMB12A.global.corp.sap> <563840B0.10401@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB1746@DEWDFEMB12A.global.corp.sap> <56388760.60800@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB1875@DEWDFEMB12A.global.corp.sap> <5639B058.9090609@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB1B57@DEWDFEMB12A.global.corp.sap> <5639CFDE.1010302@oracle.com> <78c21008-fcd0-4745-9291-1fc5d6c98a15@default> Message-ID: <4295855A5C1DE049A61835A1887419CC41EB2090@DEWDFEMB12A.global.corp.sap> Hi Markus, yes, what you say is exactly my understanding. _NSIG is larger than MAXSIGNUM! But see also my reply to Serguei. Thanks for looking at this! Best regards, Goetz > -----Original Message----- > From: Markus Gronlund [mailto:markus.gronlund at oracle.com] > Sent: Donnerstag, 5. November 2015 11:15 > To: Lindenmaier, Goetz > Cc: hotspot-runtime-dev at openjdk.java.net; serviceability-dev; David > Holmes > Subject: RE: RFR(M): 8140482: Various minor code improvements (runtime) > > Hi Goetz, > > Thanks for suggesting these fixes. > > Also thanks for pointing to the issue with _NSIG and MAXSIGNUM - looks like > MAXSIGNUM is defined all over the place (platform specific + platform > specific jsig's)... > > I also think it makes perfect sense to bound the signals to the dimensions of > the arrays where they will be stored (sigact[MAXSIGNUM] and > sigflags[MAXSIGNUM]). > > SR_initialize() will (after optionally fetching the env variable for SR_signum) > eventually call into: > > void os::Linux::set_our_sigflags() { > assert(sig > 0 && sig < MAXSIGNUM, "vm signal out of expected range"); > sigflags[sig] = flags; > } > > This assert makes me question the expression in SR_initialize(): > > if (sig > 0 || sig < _NSIG) { <<---- > SR_signum = sig; > } > > Due to shortcutting there is no upper bound range check on the signal here. > If _NSIG is larger than MAXSIGNUM this could be a significant problem. > > Should most likely be changed to: > > (sig > 0 && sig < MAXSIGNUM) > > > Thanks > Markus > > > > -----Original Message----- > From: David Holmes > Sent: den 4 november 2015 10:29 > To: Lindenmaier, Goetz; hotspot-runtime-dev at openjdk.java.net; > serviceability-dev > Subject: Re: RFR(M): 8140482: Various minor code improvements (runtime) > > On 4/11/2015 6:01 PM, Lindenmaier, Goetz wrote: > > Hi David, > > > > attachListener.hpp: > > I agree that I can not find another possible issue with the strcpy. > > Still I think it's better to have the strncpy, as it would have > > protected against the bug in attachListener_windows.cpp. > > But if you insist I'll just remove it. > > I don't insist, but I do prefer to place all the guards at the "boundary" of the > VM rather than at every level when possible. > > > Should I remove the _NSIG issue from this change and open an issue of > > it's own discussed on the serviceability list? > > Let's give them a chance to respond. I'll ping them on the hotline ;-) > > Thanks, > David > > > Best regards, > > Goetz. > > > > > >> -----Original Message----- > >> From: David Holmes [mailto:david.holmes at oracle.com] > >> Sent: Mittwoch, 4. November 2015 08:15 > >> To: Lindenmaier, Goetz; hotspot-runtime-dev at openjdk.java.net; > >> serviceability-dev > >> Subject: Re: RFR(M): 8140482: Various minor code improvements > >> (runtime) > >> > >> Hi Goetz, > >> > >> On 4/11/2015 12:10 AM, Lindenmaier, Goetz wrote: > >>> Hi David, > >>> > >>> the new scan is already through. I made a new webrev: > >>> http://cr.openjdk.java.net/~goetz/webrevs/8140482-covRt/webrev.01/ > >>> > >>> attachListener_linux.cpp: > >>> I had moved the string termination out of the loop, and read one > >>> less char from the file. The scan still claims "Passing unterminated > >>> string buf to strlen" in line 274. So I will undo it again for the > >>> webrev. > >>> > >>> codeBuffer.cpp > >>> Doing memset is fine. I'll use memset(). > >>> > >>>>>>> ps_core.c: > >>>>>>> Pread not necessarily terminates interp_name which is printed > >>>>>>> thereafter. Increase buffer size by 1 and add '\0'. > >>>>>> > >>>>>> Given: > >>>>>> #define BUF_SIZE (PATH_MAX + NAME_MAX + 1) > >>>>>> isn't it impossible to encounter that problem? > >>>>> As I understand, pread does not null-terminate what it read. So > >>>>> the null must come from the file. This protects against a corrupted > file. > >>>> > >>>> So are you saying the nul is not present in the file? I'm not > >>>> familiar with the ELF format. > >>> There should be a nul in the file, else the code would fail more > >>> obviously. The problem is if the file is corrupted. > >> > >> Thanks for clarifying. > >> > >> I'm still unclear why the attachListener.hpp changes are needed if we > >> have validated the entry points in the attachListener_.cpp files? > >> > >> Also we still need someone from serviceability to comment on the > >> _NSIG issue. > >> > >> Thanks, > >> David > >> > >> > >>> Best regards, > >>> Goetz. > >>> > >>> > >>>> -----Original Message----- > >>>> From: David Holmes [mailto:david.holmes at oracle.com] > >>>> Sent: Dienstag, 3. November 2015 11:07 > >>>> To: Lindenmaier, Goetz; hotspot-runtime-dev at openjdk.java.net; > >>>> serviceability-dev > >>>> Subject: Re: RFR(M): 8140482: Various minor code improvements > >> (runtime) > >>>> > >>>> Hi Goetz, > >>>> > >>>> Quick follow up on a couple of things ... > >>>> > >>>> On 3/11/2015 7:33 PM, Lindenmaier, Goetz wrote: > >>>>> Hi David, > >>>>> > >>>>>> Sorry, lots of folks very busy at the moment as we try to get > >>>>>> features finalized before the deadline. > >>>>> thanks for looking at this! I know the Dec. 10 deadline, but I > >>>>> guess that > >> also > >>>>> holds for us ... at least J1 is over now. (Unfortunately we could > >>>>> not > >> attend > >>>>> this year.) > >>>> > >>>> Me neither :) > >>>> > >>>>>>> ps_core.c: > >>>>>>> Pread not necessarily terminates interp_name which is printed > >>>> thereafter. > >>>>>>> Increase buffer size by 1 and add '\0'. > >>>>>> > >>>>>> Given: > >>>>>> #define BUF_SIZE (PATH_MAX + NAME_MAX + 1) > >>>>>> isn't it impossible to encounter that problem? > >>>>> As I understand, pread does not null-terminate what it read. So > >>>>> the null must come from the file. This protects against a corrupted > file. > >>>> > >>>> So are you saying the nul is not present in the file? I'm not > >>>> familiar with the ELF format. > >>>> > >>>>>>> stubRoutines_x86.cpp: > >>>>>>> Cast to proper type. This way, left and right of '&' have the same > type. > >>>>>> > >>>>>> I think you could just have changed the uint64_t to uint32_t as > >>>>>> applied to the shift rather than casting the 1 to uint_32t. End > >>>>>> result is the same though. > >>>>> What you propose did not work. It was my first fix, too. > >>>> > >>>> Hmm okay. The result of the shift must be an unsigned type and the > >>>> constant 1 is signed, so needs the cast (or use the unsigned > >>>> constant form - 1ud? ) > >>>> > >>>>>>> attachListener_linux.cpp: > >>>>>>> Read does not terminate buf. Size for '\0' is already considered. > >>>>>> > >>>>>> Looks a little odd being done on each iteration, but okay I guess. > >>>>> I'll try to move it out of the loop. Better: I'll check whether > >>>>> the scan groks it if I move it out of the loop :) > >>>>> > >>>>>>> os_linux.cpp: > >>>>>>> Array sigflags[] has size MAXSIGNUM==32. _NSIG is bigger than > >>>>>>> MAXSIGNUM (_NSIG == 65 on my machine). > >>>>>>> sig is checked to be smaller than _NSIG. Later, in > >>>>>>> set_our_sigflags(), sig is used to access sigflags[MAXSIGNUM] > >>>>>>> which can overflow the > >> array. > >>>>>>> Should we also increase MAXSIGNUM? > >>>>>> > >>>>>> Need to let the SR folk comment here as something definitely > >>>>>> seems wrong, but I'm not 100% sure the what the correct answer > >>>>>> is. If _JAVA_SR_SIGNUM is too big it should be validated > >>>>>> somewhere and > >> an > >>>> error > >>>>>> or warning reported. > >>>>> I'm also not sure how to best handle this. Might even require a > >>>>> fix exceeding this change. But I think this is the best finding. > >>>>> > >>>>>>> codeBuffer.cpp: > >>>>>>> New_capacity is not initialized. Figure_expanded_capacities() > >>>>>>> handles > >>>> this > >>>>>>> correctly, but initializing this is cheap and safe. > >>>>>> > >>>>>> Hmmm ... I hate redundancy - this is pure wasted cycles. If we > >>>>>> had to > >> do > >>>>>> it would memset not be better? Or would the code-checker not > >>>>>> realize what memset was doing? > >>>>> I guess it would work with memset, too. But I thought the 3-deep > >>>>> loop will be unrolled completely so that only three stores remain. > >>>> > >>>> I tend not to try and imagine what the compiler may or may not do. > >>>> Happy to take other opinions. Though again I'd prefer if the > >>>> checker could be shown that there is no missing initialization. > >>>> > >>>>>>> dict.cpp: > >>>>>>> If j-- is executed for j==0, the loop aborts because j is > >>>>>>> unsigned (0-- >= > >> b- > >>>>>>> _cnt). > >>>>>>> Instead, only do j++ if necessary. > >>>>>> > >>>>>> Not at all obvious to me that it is possible to do j-- when j==0, > >>>>>> but the change seems reasonable. > >>>>> Yes, the scan does not understand there is j++ right after j-- > >>>>> because of the loop iteration. I saw it complaining about this > >>>>> pattern several > >> times. > >>>>> > >>>>>> Lots of spacing changes in that code make it hard to see the real > >> changes. > >>>>> Before, I was asked to fix indentation issues in a function I touch. > >>>>> Does that only hold for compiler files? > >>>> > >>>> Yes/no/maybe :) Fixing up bad formatting when you are touching an > >>>> area can be convenient, however it can also obscure the real > >>>> changes, so it depends on the ratio of functional changes to format > changes. > >>>> > >>>>>> 145 // SAPJVM GL j-- can underflow, which will cause the loop to > >> abort. > >>>>>> Seems unnecessary with the code change as noone will understand > >> what > >>>> j-- > >>>>>> you are referring to. > >>>>> Didn't mean to leave this in here. Removed. > >>>>> > >>>>>> 150 nb->_keyvals[nbcnt + nbcnt ] = key; > >>>>>> 151 nb->_keyvals[nbcnt + nbcnt + 1] = b->_keyvals[j+j+1]; > >>>>>> hotspot-style doesn't align array index expressions like that. > >>>>>> Ditto 154/155. > >>>>> Fixed. > >>>>> > >>>>>>> generateOopMap.cpp: > >>>>>>> Idx is read from String. This is only called with constant > >>>>>>> strings, so > >>>> compare > >>>>>>> should be folded away by optimizing compilers if inlined. > >>>>>> > >>>>>> Not a fan of adding conditions that should never be false (hence > >>>>>> the > >>>>>> assert) and relying on the compiler to elide them. > >>>>> OK, removed. > >>>>> > >>>>>>> deoptimization.cpp: > >>>>>>> If buflen == 0, buf[-1] is accessed. > >>>>>> > >>>>>> Okay - but an assert(buflen>0) would be better I think as we > >>>>>> should never be calling with a zero-length buffer. > >>>>> Ok, I added the assert. As this isn't critical code, I would like > >>>>> to leave the check in there, still. > >>>>> > >>>>>>> task.cpp: > >>>>>>> Fatal can return if -XX:SuppressErrorAt is used. Just don't > >>>>>>> access the array in this case. > >>>>>> > >>>>>> Okay. I would not be surprised if we have a lot of potential > >>>>>> errors if a fatal/guarantee failure is suppressed. > >>>>>> > >>>>>>> attachListener.hpp: > >>>>>>> Do strncpy to not overflow buffer. Don't write more chars than > >> before. > >>>>>> > >>>>>> Again we have the assert to catch an error in the caller using an > >>>>>> invalid name. > >>>>> Hmm, the command comes from outside of the VM. It's not checked > >>>>> very thoroughly, see, e.g., attachListener_windows.cpp:194. Arg0 > >>>>> is checked twice, arg1 and arg2 are not checked at all. > >>>> > >>>> The libattach code is still part of our codebase so should be doing > >>>> the right things. The linux and solaris code seems to be doing the > >>>> expected name length check. On Windows the name is set using cmd, > >>>> which is also subject to a length check: > >>>> > >>>> if (strlen(cmd) > AttachOperation::name_length_max) return > >>>> ATTACH_ERROR_ILLEGALARG; > >>>> > >>>>> I add fixes for attachListener_windows.cpp to this change. > >>>>> > >>>>>>> heapDumper.cpp: > >>>>>>> strncpy does not null terminate. > >>>>>> > >>>>>> 1973 if (total_length >= sizeof(base_path)) { > >>>>>> > >>>>>> total_length already adds +1 for the nul character so the == case > >>>>>> is fine AFAICS. > >>>>>> > >>>>>> strncpy wont nul-terminate if the string exceeds the buffer size. > >>>>>> But > >> we > >>>>>> have already established that total_length <= sizeof(base_path), > >>>>>> and total_path includes space for a bunch of stuff other than > >> HeapDumpPath, > >>>>>> so the strncpy of HeapDumpPath has to copy the nul character. > >>>>> Ok, removed. > >>>>> > >>>>>> > src/share/vm/services/memoryService.cpp > >>>>>> > >>>>>> Ok. > >>>>>> > >>>>>> > src/share/vm/utilities/xmlstream.cpp > >>>>>> > >>>>>> Ok - I'm more concerned about the "magic" 10 in that piece of code. > >>>>> I assume the 10 is the space needed for the "_done" plus some waste > ... > >>>>> > >>>>> I'll do another run of the scan. That takes a day. I'll post a > >>>>> new webrev > >>>> after > >>>>> that. > >>>> > >>>> Thanks, > >>>> David > >>>> > >>>>> Thank again for this thorough review, > >>>>> Goetz > >>>>> > >>>>> > >>>>> > >>>>> > >>>>>>> Some of these, as the issue in codeBuffer.cpp, are actually > >>>>>>> handled > >>>> correctly. > >>>>>>> Nevertheless this is not that obvious so that somebody changing > >>>>>>> the > >>>> code > >>>>>>> Could oversee he has to add the initialization. > >>>>>> > >>>>>> Not an argument I buy en-masse as it leads to a lot of redundancy > >>>>>> through the code paths. Plus these tools that are being run > >>>>>> should > >> show > >>>>>> if a code changes requires initialization that is not present. > >>>>>> > >>>>>> Thanks, > >>>>>> David > >>>>>> > >>>>>>> Some of these fixes are part of SAP JVM for a long time. This > >>>>>>> change > >> has > >>>>>>> been tested with our nightly build of openJDK. > >>>>>>> > >>>>>>> Best regards, > >>>>>>> Goetz,. > >>>>>>> From goetz.lindenmaier at sap.com Thu Nov 5 10:49:13 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 5 Nov 2015 10:49:13 +0000 Subject: RFR(M): 8140482: Various minor code improvements (runtime) In-Reply-To: <563AD1BD.2020304@oracle.com> References: <4295855A5C1DE049A61835A1887419CC41E9F0AA@DEWDFEMB12A.global.corp.sap> <563840B0.10401@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB1746@DEWDFEMB12A.global.corp.sap> <56388760.60800@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB1875@DEWDFEMB12A.global.corp.sap> <5639B058.9090609@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB1B57@DEWDFEMB12A.global.corp.sap> <5639CFDE.1010302@oracle.com> <563AD1BD.2020304@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC41EB20B7@DEWDFEMB12A.global.corp.sap> Hi, Serguei, thanks for looking at the issue! I read a bit about the _NSIG issue. _JAVA_SR_SIGNUM is documented here: http://www.oracle.com/technetwork/java/javase/signals-139944.html. Linux has 32 predefined signals, and another 32 'real time signals' to be used by the user. http://www.linuxprogrammingblog.com/all-about-linux-signals?page=9 MAXSIGNUM is defined by hotspot to 32. _NSIG is defined on linux to 65. I think the current handling of this in SR_initilize() is quite bad. To avoid overwriting the sigflags array my change to set_our_sigflags() is sufficient. SR_initialize() should be improved in several means. - The code should check that the signal read is > MAX2(SIGSEGV, SIGBUS) (assert is not sufficient) - The code should check that the signal read is < MIN2(_NSIG, MAXSIGNUM) - It should warn if it is not. Currently the env var is silently ignored My current fix makes it impossible to use real time signals for _JAVA_SR_SIGNUM. Therefore I think we should increase MAXSIGNUM on linux to 65. Also, we should check bsd, which might have a similar issue. I will remove my fix from SR_initialize() and post a webrev of its own for this issue. @Dmitry, I just saw your mail ... In this new webrev I removed the change to SR_initialize() and fixed the spaces around the '+'.: http://cr.openjdk.java.net/~goetz/webrevs/8140482-covRt/webrev.02/ Best regards, Goetz. From: serguei.spitsyn at oracle.com [mailto:serguei.spitsyn at oracle.com] Sent: Donnerstag, 5. November 2015 04:49 To: David Holmes; Lindenmaier, Goetz; hotspot-runtime-dev at openjdk.java.net; serviceability-dev Subject: Re: RFR(M): 8140482: Various minor code improvements (runtime) Hi Goetz, The fix looks good. Thanks for the improvements! The _NSIG related fix looks Ok to me but I do not feel myself qualified to make a final decision. A couple of minor comments: src/share/vm/libadt/dict.cpp 149 nb->_keyvals[nbcnt + nbcnt + 1] = b->_keyvals[j+j+1]; 152 b->_keyvals[j+j] = b->_keyvals[b->_cnt + b->_cnt]; 153 b->_keyvals[j+j+1] = b->_keyvals[b->_cnt + b->_cnt + 1]; Need spaces around the '+' sign for completeness. src/os/linux/vm/attachListener_linux.cpp 258 buf[max_len-1] = '\0'; Need spaces around the '-' sign. src/share/vm/services/attachListener.hpp 126 strncpy(_name, name, MIN2(strlen(name)+1, (size_t)name_length_max)); 143 strncpy(_arg[i], arg, MIN2(strlen(arg)+1, (size_t)arg_length_max)); Need spaces around the '+' sign. agent/src/os/linux/ps_core.c 815 char interp_name[BUF_SIZE+1]; Need spaces around the '+' sign. Thanks, Serguei On 11/4/15 01:29, David Holmes wrote: On 4/11/2015 6:01 PM, Lindenmaier, Goetz wrote: Hi David, attachListener.hpp: I agree that I can not find another possible issue with the strcpy. Still I think it's better to have the strncpy, as it would have protected against the bug in attachListener_windows.cpp. But if you insist I'll just remove it. I don't insist, but I do prefer to place all the guards at the "boundary" of the VM rather than at every level when possible. Should I remove the _NSIG issue from this change and open an issue of it's own discussed on the serviceability list? Let's give them a chance to respond. I'll ping them on the hotline ;-) Thanks, David Best regards, Goetz. -----Original Message----- From: David Holmes [mailto:david.holmes at oracle.com] Sent: Mittwoch, 4. November 2015 08:15 To: Lindenmaier, Goetz; hotspot-runtime-dev at openjdk.java.net; serviceability-dev Subject: Re: RFR(M): 8140482: Various minor code improvements (runtime) Hi Goetz, On 4/11/2015 12:10 AM, Lindenmaier, Goetz wrote: Hi David, the new scan is already through. I made a new webrev: http://cr.openjdk.java.net/~goetz/webrevs/8140482-covRt/webrev.01/ attachListener_linux.cpp: I had moved the string termination out of the loop, and read one less char from the file. The scan still claims "Passing unterminated string buf to strlen" in line 274. So I will undo it again for the webrev. codeBuffer.cpp Doing memset is fine. I'll use memset(). ps_core.c: Pread not necessarily terminates interp_name which is printed thereafter. Increase buffer size by 1 and add '\0'. Given: #define BUF_SIZE (PATH_MAX + NAME_MAX + 1) isn't it impossible to encounter that problem? As I understand, pread does not null-terminate what it read. So the null must come from the file. This protects against a corrupted file. So are you saying the nul is not present in the file? I'm not familiar with the ELF format. There should be a nul in the file, else the code would fail more obviously. The problem is if the file is corrupted. Thanks for clarifying. I'm still unclear why the attachListener.hpp changes are needed if we have validated the entry points in the attachListener_.cpp files? Also we still need someone from serviceability to comment on the _NSIG issue. Thanks, David Best regards, Goetz. -----Original Message----- From: David Holmes [mailto:david.holmes at oracle.com] Sent: Dienstag, 3. November 2015 11:07 To: Lindenmaier, Goetz; hotspot-runtime-dev at openjdk.java.net; serviceability-dev Subject: Re: RFR(M): 8140482: Various minor code improvements (runtime) Hi Goetz, Quick follow up on a couple of things ... On 3/11/2015 7:33 PM, Lindenmaier, Goetz wrote: Hi David, Sorry, lots of folks very busy at the moment as we try to get features finalized before the deadline. thanks for looking at this! I know the Dec. 10 deadline, but I guess that also holds for us ... at least J1 is over now. (Unfortunately we could not attend this year.) Me neither :) ps_core.c: Pread not necessarily terminates interp_name which is printed thereafter. Increase buffer size by 1 and add '\0'. Given: #define BUF_SIZE (PATH_MAX + NAME_MAX + 1) isn't it impossible to encounter that problem? As I understand, pread does not null-terminate what it read. So the null must come from the file. This protects against a corrupted file. So are you saying the nul is not present in the file? I'm not familiar with the ELF format. stubRoutines_x86.cpp: Cast to proper type. This way, left and right of '&' have the same type. I think you could just have changed the uint64_t to uint32_t as applied to the shift rather than casting the 1 to uint_32t. End result is the same though. What you propose did not work. It was my first fix, too. Hmm okay. The result of the shift must be an unsigned type and the constant 1 is signed, so needs the cast (or use the unsigned constant form - 1ud? ) attachListener_linux.cpp: Read does not terminate buf. Size for '\0' is already considered. Looks a little odd being done on each iteration, but okay I guess. I'll try to move it out of the loop. Better: I'll check whether the scan groks it if I move it out of the loop :) os_linux.cpp: Array sigflags[] has size MAXSIGNUM==32. _NSIG is bigger than MAXSIGNUM (_NSIG == 65 on my machine). sig is checked to be smaller than _NSIG. Later, in set_our_sigflags(), sig is used to access sigflags[MAXSIGNUM] which can overflow the array. Should we also increase MAXSIGNUM? Need to let the SR folk comment here as something definitely seems wrong, but I'm not 100% sure the what the correct answer is. If _JAVA_SR_SIGNUM is too big it should be validated somewhere and an error or warning reported. I'm also not sure how to best handle this. Might even require a fix exceeding this change. But I think this is the best finding. codeBuffer.cpp: New_capacity is not initialized. Figure_expanded_capacities() handles this correctly, but initializing this is cheap and safe. Hmmm ... I hate redundancy - this is pure wasted cycles. If we had to do it would memset not be better? Or would the code-checker not realize what memset was doing? I guess it would work with memset, too. But I thought the 3-deep loop will be unrolled completely so that only three stores remain. I tend not to try and imagine what the compiler may or may not do. Happy to take other opinions. Though again I'd prefer if the checker could be shown that there is no missing initialization. dict.cpp: If j-- is executed for j==0, the loop aborts because j is unsigned (0-- >= b- _cnt). Instead, only do j++ if necessary. Not at all obvious to me that it is possible to do j-- when j==0, but the change seems reasonable. Yes, the scan does not understand there is j++ right after j-- because of the loop iteration. I saw it complaining about this pattern several times. Lots of spacing changes in that code make it hard to see the real changes. Before, I was asked to fix indentation issues in a function I touch. Does that only hold for compiler files? Yes/no/maybe :) Fixing up bad formatting when you are touching an area can be convenient, however it can also obscure the real changes, so it depends on the ratio of functional changes to format changes. 145 // SAPJVM GL j-- can underflow, which will cause the loop to abort. Seems unnecessary with the code change as noone will understand what j-- you are referring to. Didn't mean to leave this in here. Removed. 150 nb->_keyvals[nbcnt + nbcnt ] = key; 151 nb->_keyvals[nbcnt + nbcnt + 1] = b->_keyvals[j+j+1]; hotspot-style doesn't align array index expressions like that. Ditto 154/155. Fixed. generateOopMap.cpp: Idx is read from String. This is only called with constant strings, so compare should be folded away by optimizing compilers if inlined. Not a fan of adding conditions that should never be false (hence the assert) and relying on the compiler to elide them. OK, removed. deoptimization.cpp: If buflen == 0, buf[-1] is accessed. Okay - but an assert(buflen>0) would be better I think as we should never be calling with a zero-length buffer. Ok, I added the assert. As this isn't critical code, I would like to leave the check in there, still. task.cpp: Fatal can return if -XX:SuppressErrorAt is used. Just don't access the array in this case. Okay. I would not be surprised if we have a lot of potential errors if a fatal/guarantee failure is suppressed. attachListener.hpp: Do strncpy to not overflow buffer. Don't write more chars than before. Again we have the assert to catch an error in the caller using an invalid name. Hmm, the command comes from outside of the VM. It's not checked very thoroughly, see, e.g., attachListener_windows.cpp:194. Arg0 is checked twice, arg1 and arg2 are not checked at all. The libattach code is still part of our codebase so should be doing the right things. The linux and solaris code seems to be doing the expected name length check. On Windows the name is set using cmd, which is also subject to a length check: if (strlen(cmd) > AttachOperation::name_length_max) return ATTACH_ERROR_ILLEGALARG; I add fixes for attachListener_windows.cpp to this change. heapDumper.cpp: strncpy does not null terminate. 1973 if (total_length >= sizeof(base_path)) { total_length already adds +1 for the nul character so the == case is fine AFAICS. strncpy wont nul-terminate if the string exceeds the buffer size. But we have already established that total_length <= sizeof(base_path), and total_path includes space for a bunch of stuff other than HeapDumpPath, so the strncpy of HeapDumpPath has to copy the nul character. Ok, removed. > src/share/vm/services/memoryService.cpp Ok. > src/share/vm/utilities/xmlstream.cpp Ok - I'm more concerned about the "magic" 10 in that piece of code. I assume the 10 is the space needed for the "_done" plus some waste ... I'll do another run of the scan. That takes a day. I'll post a new webrev after that. Thanks, David Thank again for this thorough review, Goetz Some of these, as the issue in codeBuffer.cpp, are actually handled correctly. Nevertheless this is not that obvious so that somebody changing the code Could oversee he has to add the initialization. Not an argument I buy en-masse as it leads to a lot of redundancy through the code paths. Plus these tools that are being run should show if a code changes requires initialization that is not present. Thanks, David Some of these fixes are part of SAP JVM for a long time. This change has been tested with our nightly build of openJDK. Best regards, Goetz,. -------------- next part -------------- An HTML attachment was scrubbed... URL: From serguei.spitsyn at oracle.com Thu Nov 5 11:46:03 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 5 Nov 2015 03:46:03 -0800 Subject: RFR(M): 8140482: Various minor code improvements (runtime) In-Reply-To: <4295855A5C1DE049A61835A1887419CC41EB20B7@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC41E9F0AA@DEWDFEMB12A.global.corp.sap> <563840B0.10401@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB1746@DEWDFEMB12A.global.corp.sap> <56388760.60800@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB1875@DEWDFEMB12A.global.corp.sap> <5639B058.9090609@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB1B57@DEWDFEMB12A.global.corp.sap> <5639CFDE.1010302@oracle.com> <563AD1BD.2020304@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB20B7@DEWDFEMB12A.global.corp.sap> Message-ID: <563B417B.7080602@oracle.com> Hi Goetz, The fix looks good. Thank you for the explanations and separation of the SR_initialize() issue! Thanks, Serguei On 11/5/15 02:49, Lindenmaier, Goetz wrote: > > Hi, > > Serguei, thanks for looking at the issue! > > I read a bit about the _NSIG issue. > > _/JAVA_SR/_SIGNUM is documented here: > http://www.oracle.com/technetwork/java/javase/signals-139944.html. > > Linux has 32 predefined signals, and another 32 ?real time signals? to > be used by the user. > > http://www.linuxprogrammingblog.com/all-about-linux-signals?page=9 > > MAXSIGNUM is defined by hotspot to 32. > > _NSIG is defined on linux to 65. > > I think the current handling of this in SR_initilize() is quite bad. > > To avoid overwriting the sigflags array my change to > set_our_sigflags() is sufficient. > > SR_initialize() should be improved in several means. > > -The code should check that the signal read is > MAX2(SIGSEGV, SIGBUS) > (assert is not sufficient) > > -The code should check that the signal read is < MIN2(_NSIG, MAXSIGNUM) > > -It should warn if it is not. Currently the env var is silently ignored > > My current fix makes it impossible to use real time signals for > _/JAVA_SR/_SIGNUM. > > Therefore I think we should increase MAXSIGNUM on linux to 65. > > Also, we should check bsd, which might have a similar issue. > > I will remove my fix from SR_initialize() and post a webrev of its own > for > > this issue. @Dmitry, I just saw your mail ? > > In this new webrev I removed the change to SR_initialize() and > > fixed the spaces around the ?+?.: > > http://cr.openjdk.java.net/~goetz/webrevs/8140482-covRt/webrev.02/ > > > Best regards, > > Goetz. > > *From:*serguei.spitsyn at oracle.com [mailto:serguei.spitsyn at oracle.com] > *Sent:* Donnerstag, 5. November 2015 04:49 > *To:* David Holmes; Lindenmaier, Goetz; > hotspot-runtime-dev at openjdk.java.net; serviceability-dev > *Subject:* Re: RFR(M): 8140482: Various minor code improvements (runtime) > > Hi Goetz, > > The fix looks good. > Thanks for the improvements! > The _NSIG related fix looks Ok to me but I do not feel myself > qualified to make a final decision. > > A couple of minor comments: > > *src/share/vm/libadt/dict.cpp* > > 149 nb->_keyvals[nbcnt + nbcnt + 1] = b->_keyvals[j+j+1]; > 152 b->_keyvals[j+j] = b->_keyvals[b->_cnt + b->_cnt]; > 153 b->_keyvals[j+j+1] = b->_keyvals[b->_cnt + b->_cnt + 1]; > > Need spaces around the '+' sign for completeness. > > > *src/os/linux/vm/attachListener_linux.cpp* > > 258 buf[max_len-1] = '\0'; > > Need spaces around the '-' sign. > > > *src/share/vm/services/attachListener.hpp* > > 126 strncpy(_name, name, MIN2(strlen(name)+1, > (size_t)name_length_max)); > 143 strncpy(_arg[i], arg, MIN2(strlen(arg)+1, > (size_t)arg_length_max)); > > > Need spaces around the '+' sign. > > > *agent/src/os/linux/ps_core.c* > > 815 char interp_name[BUF_SIZE+1]; > > Need spaces around the '+' sign. > > > Thanks, > Serguei > > > On 11/4/15 01:29, David Holmes wrote: > > On 4/11/2015 6:01 PM, Lindenmaier, Goetz wrote: > > Hi David, > > attachListener.hpp: > I agree that I can not find another possible issue > with the strcpy. > Still I think it's better to have the strncpy, as it would have > protected against the bug in attachListener_windows.cpp. > But if you insist I'll just remove it. > > > I don't insist, but I do prefer to place all the guards at the > "boundary" of the VM rather than at every level when possible. > > > Should I remove the _NSIG issue from this change and > open an issue of it's own discussed on the serviceability list? > > > Let's give them a chance to respond. I'll ping them on the hotline > ;-) > > Thanks, > David > > > Best regards, > Goetz. > > > > -----Original Message----- > From: David Holmes [mailto:david.holmes at oracle.com] > Sent: Mittwoch, 4. November 2015 08:15 > To: Lindenmaier, Goetz; > hotspot-runtime-dev at openjdk.java.net > ; > serviceability-dev > Subject: Re: RFR(M): 8140482: Various minor code > improvements (runtime) > > Hi Goetz, > > On 4/11/2015 12:10 AM, Lindenmaier, Goetz wrote: > > Hi David, > > the new scan is already through. I made a new webrev: > http://cr.openjdk.java.net/~goetz/webrevs/8140482-covRt/webrev.01/ > > > > attachListener_linux.cpp: > I had moved the string termination out of the loop, > and read one > less char from the file. The scan still claims > "Passing unterminated > string buf to strlen" in line 274. So I will undo it > again for the > webrev. > > codeBuffer.cpp > Doing memset is fine. I'll use memset(). > > > ps_core.c: > Pread not necessarily terminates > interp_name which is printed > thereafter. Increase buffer size by 1 > and add '\0'. > > > Given: > #define BUF_SIZE (PATH_MAX + NAME_MAX > + 1) > isn't it impossible to encounter that > problem? > > As I understand, pread does not null-terminate > what it read. So the > null must come from the file. This protects > against a corrupted file. > > > So are you saying the nul is not present in the > file? I'm not familiar > with the ELF format. > > There should be a nul in the file, else the code would > fail more > obviously. The problem is if the file is corrupted. > > > Thanks for clarifying. > > I'm still unclear why the attachListener.hpp changes are > needed if we > have validated the entry points in the > attachListener_.cpp files? > > Also we still need someone from serviceability to comment > on the _NSIG > issue. > > Thanks, > David > > > > Best regards, > Goetz. > > > > -----Original Message----- > From: David Holmes [mailto:david.holmes at oracle.com] > Sent: Dienstag, 3. November 2015 11:07 > To: Lindenmaier, Goetz; > hotspot-runtime-dev at openjdk.java.net > ; > serviceability-dev > Subject: Re: RFR(M): 8140482: Various minor code > improvements > > (runtime) > > > Hi Goetz, > > Quick follow up on a couple of things ... > > On 3/11/2015 7:33 PM, Lindenmaier, Goetz wrote: > > Hi David, > > > Sorry, lots of folks very busy at the > moment as we try to get features > finalized before the deadline. > > thanks for looking at this! I know the Dec. 10 > deadline, but I guess that > > also > > holds for us ... at least J1 is over now. > (Unfortunately we could not > > attend > > this year.) > > > Me neither :) > > > ps_core.c: > Pread not necessarily terminates > interp_name which is printed > > thereafter. > > Increase buffer size by 1 and add '\0'. > > > Given: > #define BUF_SIZE (PATH_MAX + NAME_MAX > + 1) > isn't it impossible to encounter that > problem? > > As I understand, pread does not null-terminate > what it read. So the > null must come from the file. This protects > against a corrupted file. > > > So are you saying the nul is not present in the > file? I'm not familiar > with the ELF format. > > > stubRoutines_x86.cpp: > Cast to proper type. This way, left > and right of '&' have the same type. > > > I think you could just have changed the > uint64_t to uint32_t as applied > to the shift rather than casting the 1 to > uint_32t. End result is the > same though. > > What you propose did not work. It was my > first fix, too. > > > Hmm okay. The result of the shift must be an > unsigned type and the > constant 1 is signed, so needs the cast (or use > the unsigned constant > form - 1ud? ) > > > attachListener_linux.cpp: > Read does not terminate buf. Size for > '\0' is already considered. > > > Looks a little odd being done on each > iteration, but okay I guess. > > I'll try to move it out of the loop. Better: > I'll check whether the > scan groks it if I move it out of the loop :) > > > os_linux.cpp: > Array sigflags[] has size > MAXSIGNUM==32. _NSIG is bigger than > MAXSIGNUM (_NSIG == 65 on my machine). > sig is checked to be smaller than > _NSIG. Later, in set_our_sigflags(), > sig is used to access > sigflags[MAXSIGNUM] which can overflow > the > > array. > > Should we also increase MAXSIGNUM? > > > Need to let the SR folk comment here as > something definitely seems > wrong, but I'm not 100% sure the what the > correct answer is. If > _JAVA_SR_SIGNUM is too big it should be > validated somewhere and > > an > > error > > or warning reported. > > I'm also not sure how to best handle this. > Might even require a fix > exceeding this change. But I think this is > the best finding. > > > codeBuffer.cpp: > New_capacity is not initialized. > Figure_expanded_capacities() handles > > this > > correctly, but initializing this is > cheap and safe. > > > Hmmm ... I hate redundancy - this is pure > wasted cycles. If we had to > > do > > it would memset not be better? Or would > the code-checker not realize > what memset was doing? > > I guess it would work with memset, too. But I > thought the 3-deep loop > will be unrolled completely so that only three > stores remain. > > > I tend not to try and imagine what the compiler > may or may not do. Happy > to take other opinions. Though again I'd prefer if > the checker could be > shown that there is no missing initialization. > > > dict.cpp: > If j-- is executed for j==0, the loop > aborts because j is unsigned (0-- >= > > b- > > _cnt). > Instead, only do j++ if necessary. > > > Not at all obvious to me that it is > possible to do j-- when j==0, but > the change seems reasonable. > > Yes, the scan does not understand there is j++ > right after j-- because > of the loop iteration. I saw it complaining > about this pattern several > > times. > > > > Lots of spacing changes in that code make > it hard to see the real > > changes. > > Before, I was asked to fix indentation issues > in a function I touch. > Does that only hold for compiler files? > > > Yes/no/maybe :) Fixing up bad formatting when you > are touching an area > can be convenient, however it can also obscure the > real changes, so it > depends on the ratio of functional changes to > format changes. > > > 145 // SAPJVM GL j-- can underflow, > which will cause the loop to > > abort. > > Seems unnecessary with the code change as > noone will understand > > what > > j-- > > you are referring to. > > Didn't mean to leave this in here. Removed. > > > 150 nb->_keyvals[nbcnt + nbcnt ] = > key; > 151 nb->_keyvals[nbcnt + > nbcnt + 1] = b->_keyvals[j+j+1]; > hotspot-style doesn't align array index > expressions like that. Ditto > 154/155. > > Fixed. > > > generateOopMap.cpp: > Idx is read from String. This is only > called with constant strings, so > > compare > > should be folded away by optimizing > compilers if inlined. > > > Not a fan of adding conditions that should > never be false (hence the > assert) and relying on the compiler to > elide them. > > OK, removed. > > > deoptimization.cpp: > If buflen == 0, buf[-1] is accessed. > > > Okay - but an assert(buflen>0) would be > better I think as we should > never be calling with a zero-length buffer. > > Ok, I added the assert. As this isn't > critical code, I would like to leave the > check in there, still. > > > task.cpp: > Fatal can return if > -XX:SuppressErrorAt is used. Just > don't access the > array in this case. > > > Okay. I would not be surprised if we have > a lot of potential errors if a > fatal/guarantee failure is suppressed. > > > attachListener.hpp: > Do strncpy to not overflow buffer. > Don't write more chars than > > before. > > > Again we have the assert to catch an error > in the caller using an > invalid name. > > Hmm, the command comes from outside of the > VM. It's not checked > very thoroughly, see, e.g., > attachListener_windows.cpp:194. Arg0 is > checked twice, arg1 and arg2 are not checked > at all. > > > The libattach code is still part of our codebase > so should be doing the > right things. The linux and solaris code seems to > be doing the expected > name length check. On Windows the name is set > using cmd, which is also > subject to a length check: > > if (strlen(cmd) > > AttachOperation::name_length_max) return > ATTACH_ERROR_ILLEGALARG; > > > I add fixes for attachListener_windows.cpp to > this change. > > > heapDumper.cpp: > strncpy does not null terminate. > > > 1973 if (total_length >= > sizeof(base_path)) { > > total_length already adds +1 for the nul > character so the == case is > fine AFAICS. > > strncpy wont nul-terminate if the string > exceeds the buffer size. But > > we > > have already established that total_length > <= sizeof(base_path), and > total_path includes space for a bunch of > stuff other than > > HeapDumpPath, > > so the strncpy of HeapDumpPath has to copy > the nul character. > > Ok, removed. > > > > src/share/vm/services/memoryService.cpp > > Ok. > > > src/share/vm/utilities/xmlstream.cpp > > Ok - I'm more concerned about the "magic" > 10 in that piece of code. > > I assume the 10 is the space needed for the > "_done" plus some waste ... > > I'll do another run of the scan. That takes a > day. I'll post a new webrev > > after > > that. > > > Thanks, > David > > > Thank again for this thorough review, > Goetz > > > > > > Some of these, as the issue in > codeBuffer.cpp, are actually handled > > correctly. > > Nevertheless this is not that obvious > so that somebody changing the > > code > > Could oversee he has to add the > initialization. > > > Not an argument I buy en-masse as it leads > to a lot of redundancy > through the code paths. Plus these tools > that are being run should > > show > > if a code changes requires initialization > that is not present. > > Thanks, > David > > > Some of these fixes are part of SAP > JVM for a long time. This change > > has > > been tested with our nightly build of > openJDK. > > Best regards, > Goetz,. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From marcus.larsson at oracle.com Thu Nov 5 14:38:49 2015 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Thu, 5 Nov 2015 15:38:49 +0100 Subject: RFR: JDK-8140556: Add force rotation option to VM.log jcmd In-Reply-To: <563AD1F2.6060109@oracle.com> References: <562101D8.5080909@gmail.com> <562DF8D4.40501@oracle.com> <562EBF4B.4020607@gmail.com> <562F6B46.5050500@oracle.com> <563226AB.2030206@gmail.com> <5632341F.4060400@oracle.com> <56338125.9030802@gmail.com> <56338D58.9090709@oracle.com> <5634D9FC.6080502@gmail.com> <5636C66A.6020304@oracle.com> <563765FC.2090804@gmail.com> <563ACD6F.1050600@gmail.com> <563AD1F2.6060109@oracle.com> Message-ID: <563B69F9.5090102@oracle.com> Hey, On 2015-11-05 04:50, David Holmes wrote: > On 5/11/2015 1:30 PM, Yasumasa Suenaga wrote: >> I've uploaded new webrev: >> http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.03/ >> >> I added _initialized to LogConfiguration. >> LogConfiguration::post_initialize() will be called after main thread >> initialization. >> So _initialized set to true in this function, and we can check this flag >> at rotation. >> >> Could you review it? Looks OK to me. This is much better than checking for Thread::current() != NULL in is_rotatable(), IMHO. > > _post_initialized would be a more accurate name. > > This of course works, but I don't know whether the additional delay in > allowing rotation will impact anything. It shouldn't. If anyone ever generates that much logging during startup they will have to accept that the first rotated log won't be properly sized, that's all. Thanks, Marcus > > Thanks, > David > >> >> Thanks, >> >> Yasumasa >> >> >> On 2015/11/02 22:32, Yasumasa Suenaga wrote: >>> Marcus, David, >>> >>> I've uploaded new webrev: >>> http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.02/ >>> >>> Could you review it? >>> >>> >>>>>>>> Still missing the renaming of rotate_all_logfile to >>>>>>>> rotate_all_outputs. >>> >>> I've fixed. >>> >>> >>>>>> Small typo, please make that is_rotatable(). >>> >>> I've added, and have used at LogConfiguration::rotate_all_outputs(). >>> >>> >>>> You can't safely lock a mutex until the current thread has attached >>>> to the VM and Thread::current() returns non-null (that is what >>>> Thread::initialize_thread_local_storage() currently does). You may >>>> get away with it, particularly in product builds, during VM >>>> initialization, as the fast-lock path doesn't require access to >>>> Thread::current(). >>> >>> I will check whether Thread::current() is NULL or not at >>> LogFileOutput::is_rotatable() >>> is_rotatable() is called at LogFileOutput::rotate() before locking >>> _rotation_lock. >>> So we can avoid crash/assert which is related to TLS. >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> On 2015/11/02 11:11, David Holmes wrote: >>>> On 1/11/2015 1:10 AM, Yasumasa Suenaga wrote: >>>>>> This in turn means that we are logging before the thread local >>>>>> storage >>>>>> has been initialized, and the JVM will crash/hit the assert >>>>>> because we >>>>>> are trying to take the rotation lock. >>>>> >>>>> I guess that you are saying about class member (not TLS - e.g. >>>>> pthread_setspecific()). >>>> >>>> You can't safely lock a mutex until the current thread has attached >>>> to the VM and Thread::current() returns non-null (that is what >>>> Thread::initialize_thread_local_storage() currently does). You may >>>> get away with it, particularly in product builds, during VM >>>> initialization, as the fast-lock path doesn't require access to >>>> Thread::current(). >>>> >>>> David >>>> >>>>> Most of fields (include _rotation_lock) in LogFileOutput are >>>>> initialized >>>>> at c'tor. >>>>> Other field (_stream) is initialized at LogFileOutput::initialize(). >>>>> >>>>> We can avoid problem if we can move initialize() to c'tor because >>>>> LogFileOutput >>>>> will be instantiated at LogConfiguration::new_output(). >>>>> >>>>> Currently, LogFileOutput will be added to LogConfiguration::_outputs >>>>> and >>>>> to LogTagSet after calling LogFileOutput::initialize(). >>>>> I wonder why we cannot avoid crash/assert with current >>>>> implementation... >>>>> >>>>> >>>>> I will keep current implementation about _rotation_lock if the >>>>> above is >>>>> incorrect. >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2015/10/31 0:31, Marcus Larsson wrote: >>>>>> Hey, >>>>>> >>>>>> On 2015-10-30 15:39, Yasumasa Suenaga wrote: >>>>>>> Hi Marcus, >>>>>>> >>>>>>> Thank you for your comment. >>>>>>> >>>>>>> >>>>>>>> Still missing the renaming of rotate_all_logfile to >>>>>>>> rotate_all_outputs. >>>>>>> >>>>>>> Sorry, I will fix. >>>>>>> >>>>>> >>>>>> Thanks! >>>>>> >>>>>>> >>>>>>>> This rotate function works I guess, but it will probably need some >>>>>>>> rework once there are other types of log outputs (rotate doesn't >>>>>>>> make sense on all types of log outputs). >>>>>>> >>>>>>> I will add is_rotetable() to LogOutput as virtual function. >>>>>>> This function return false (can not rotate) by default, return true >>>>>>> if _file_count is >>>>>>> greater than 0 in LogFileOutput. >>>>>>> >>>>>>> LogConfiguration::rotate_all_outputs() will rotate if this function >>>>>>> returns true. >>>>>>> >>>>>> >>>>>> Small typo, please make that is_rotatable(). >>>>>> >>>>>>> >>>>>>>> If we configure a rotated file output and then log something on >>>>>>>> that >>>>>>>> output before thread local storage initialization is complete, we >>>>>>>> will crash/hit an assert. The previous implementation avoided this >>>>>>>> as long as no rotation was needed before this initialization was >>>>>>>> complete. (This can be triggered using the following arguments >>>>>>>> "-Xlog:all=trace:file.txt::filesize=10,filecount=2 -Xlog:invalid", >>>>>>>> for example.) >>>>>>> >>>>>>> I do not think so. >>>>>>> Each -Xlog option creates unique instance of LogOutput (and >>>>>>> subclass). >>>>>>> So they are isolated. >>>>>> >>>>>> My example includes a second (and intentionally incorrect) -Xlog >>>>>> option only to trigger logging on the newly configured rotating >>>>>> LogFileOutput. This will cause some logging about the invalid -Xlog >>>>>> argument, and this logging happens during the argument parsing. This >>>>>> in turn means that we are logging before the thread local storage >>>>>> has >>>>>> been initialized, and the JVM will crash/hit the assert because >>>>>> we are >>>>>> trying to take the rotation lock. This has nothing to do with >>>>>> concurrent fprintfs or freopens, even if that is a problem. >>>>>> >>>>>>> >>>>>>> However, we might have to rotate at safepoint. >>>>>> >>>>>> Doing stuff at a safepoint will not give us much in this case. There >>>>>> are threads that continue execution through a safepoint, which means >>>>>> we will see log calls being made even during this time. >>>>>> >>>>>>> Currently, LogOutput might be used by multiple threads >>>>>>> concurrently. >>>>>>> If we rotate log when another thread is writing string to >>>>>>> output, we >>>>>>> encounter unexpected behavior. >>>>>>> >>>>>>> LogFileOutput::rotate() uses freopen(). >>>>>>> LogFileStreamOutput::write() uses vfprintf() through jio_fprintf(). >>>>>>> freopen() and vfprintf() are not atomic. >>>>>> >>>>>> You're right. There seems to be an overlooked concurrency issue with >>>>>> vfprintfs during freopen. I don't think we should resolve this as >>>>>> part >>>>>> of this particular fix, so instead I'll create a separate issue for >>>>>> it. >>>>>> >>>>>>> >>>>>>> I think that cause of crash/assertion which you say is it. >>>>>>> (GC log will be rotated at safepoint.) >>>>>>> >>>>>> >>>>>> See my comment above for what crash/assert I'm talking about. >>>>>> >>>>>> Thanks, >>>>>> Marcus >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2015/10/29 23:58, Marcus Larsson wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> On 2015-10-29 15:01, Yasumasa Suenaga wrote: >>>>>>>>> Hi Marcus, >>>>>>>>> >>>>>>>>> Thank you for your comment. >>>>>>>>> >>>>>>>>>> I'll sponsor it. >>>>>>>>> >>>>>>>>> Thank you so much! >>>>>>>>> I've uploaded new webrev: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.01/ >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> Still missing the renaming of rotate_all_logfile to >>>>>>>> rotate_all_outputs. >>>>>>>> >>>>>>>>>> logConfiguration.cpp/hpp: >>>>>>>>>> >>>>>>>>>> * I don't think we should rely on the archive_name or the >>>>>>>>>> output's >>>>>>>>>> name to decide whether or not an output should be rotated. It >>>>>>>>>> would be better to introduce an is_rotated() test function in >>>>>>>>>> LogOutput that could be used here. >>>>>>>>>> >>>>>>>>>> * rotate_all_logfile() should be renamed to >>>>>>>>>> rotate_all_outputs(). >>>>>>>>>> Currently there are only LogFileOutputs (other than >>>>>>>>>> stdout/stderr), but in the future there might be other types of >>>>>>>>>> outputs so I prefer having a more general name. >>>>>>>>>> Also, please use static_cast instead of the >>>>>>>>>> C-style cast. (Additional logic will be required here once more >>>>>>>>>> types of log outputs are added, but I don't think we need to >>>>>>>>>> worry >>>>>>>>>> about this right now.) >>>>>>>>>> >>>>>>>>>> * Don't print an error if an output is not rotatable, since it's >>>>>>>>>> not an error to have some log outputs rotated while others are >>>>>>>>>> not. If you really want some traceability here I suggest adding >>>>>>>>>> log messages on trace level, tagged with 'logging'. >>>>>>>>> >>>>>>>>> I added LogOutput::rotate(bool) as virtual function. >>>>>>>>> This function works nothing by default, but will rotate all >>>>>>>>> logs at >>>>>>>>> LogFileOutput. >>>>>>>> >>>>>>>> This rotate function works I guess, but it will probably need some >>>>>>>> rework once there are other types of log outputs (rotate doesn't >>>>>>>> make sense on all types of log outputs). >>>>>>>> >>>>>>>>> >>>>>>>>> If true is passed to this function, all files will be rotated. >>>>>>>>> >>>>>>>>> >>>>>>>>>> logDiagnosticCommand.cpp/hpp: >>>>>>>>>> >>>>>>>>>> * I think the description could be improved to something like >>>>>>>>>> "Lists current log configuration, enables/disables/configures a >>>>>>>>>> log output, or disables/rotates all logs." >>>>>>>>>> >>>>>>>>>> * The rotate option description should clarify that all logs >>>>>>>>>> will >>>>>>>>>> be rotated ("current log" is too ambiguous). >>>>>>>>> >>>>>>>>> I've fixed. >>>>>>>> >>>>>>>> Great, thanks! >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> logFileOutput.cpp/hpp: >>>>>>>>>> >>>>>>>>>> * Moving the MutexLocker like this introduces a race condition >>>>>>>>>> where a log might be rotated multiple times by different >>>>>>>>>> threads, >>>>>>>>>> instead of just once. >>>>>>>>>> Instead of making the rotate() function public and moving the >>>>>>>>>> mutexlocker, I suggest adding something like a public >>>>>>>>>> force_rotation() function that grabs the lock and calls the >>>>>>>>>> private rotate(). >>>>>>>>>> >>>>>>>>>> * Given the addition of is_rotated() in LogOutput, then >>>>>>>>>> get_archive_name() should be removed. >>>>>>>>> >>>>>>>>> I moved MutexLocker and should_rotate() to rotate(). >>>>>>>>> I think this change can avoid race condition. >>>>>>>>> >>>>>>>> >>>>>>>> There is a subtle problem with taking the lock to check if we >>>>>>>> should >>>>>>>> rotate. If we configure a rotated file output and then log >>>>>>>> something >>>>>>>> on that output before thread local storage initialization is >>>>>>>> complete, we will crash/hit an assert. The previous implementation >>>>>>>> avoided this as long as no rotation was needed before this >>>>>>>> initialization was complete. (This can be triggered using the >>>>>>>> following arguments >>>>>>>> "-Xlog:all=trace:file.txt::filesize=10,filecount=2 -Xlog:invalid", >>>>>>>> for example.) >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Marcus >>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2015/10/27 21:17, Marcus Larsson wrote: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> On 2015-10-27 01:03, Yasumasa Suenaga wrote: >>>>>>>>>>> Hi Marcus, >>>>>>>>>>> >>>>>>>>>>> Thank you for replying. >>>>>>>>>>> >>>>>>>>>>> I filed this feature to JBS and change subject of this email. >>>>>>>>>>> Could you be a sponsor and review it? >>>>>>>>>> >>>>>>>>>> I'll sponsor it. >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.00/ >>>>>>>>>> >>>>>>>>>> logConfiguration.cpp/hpp: >>>>>>>>>> >>>>>>>>>> * I don't think we should rely on the archive_name or the >>>>>>>>>> output's >>>>>>>>>> name to decide whether or not an output should be rotated. It >>>>>>>>>> would be better to introduce an is_rotated() test function in >>>>>>>>>> LogOutput that could be used here. >>>>>>>>>> >>>>>>>>>> * rotate_all_logfile() should be renamed to >>>>>>>>>> rotate_all_outputs(). >>>>>>>>>> Currently there are only LogFileOutputs (other than >>>>>>>>>> stdout/stderr), but in the future there might be other types of >>>>>>>>>> outputs so I prefer having a more general name. >>>>>>>>>> Also, please use static_cast instead of the >>>>>>>>>> C-style cast. (Additional logic will be required here once more >>>>>>>>>> types of log outputs are added, but I don't think we need to >>>>>>>>>> worry >>>>>>>>>> about this right now.) >>>>>>>>>> >>>>>>>>>> * Don't print an error if an output is not rotatable, since it's >>>>>>>>>> not an error to have some log outputs rotated while others are >>>>>>>>>> not. If you really want some traceability here I suggest adding >>>>>>>>>> log messages on trace level, tagged with 'logging'. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> logDiagnosticCommand.cpp/hpp: >>>>>>>>>> >>>>>>>>>> * I think the description could be improved to something like >>>>>>>>>> "Lists current log configuration, enables/disables/configures a >>>>>>>>>> log output, or disables/rotates all logs." >>>>>>>>>> >>>>>>>>>> * The rotate option description should clarify that all logs >>>>>>>>>> will >>>>>>>>>> be rotated ("current log" is too ambiguous). >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> logFileOutput.cpp/hpp: >>>>>>>>>> >>>>>>>>>> * Moving the MutexLocker like this introduces a race condition >>>>>>>>>> where a log might be rotated multiple times by different >>>>>>>>>> threads, >>>>>>>>>> instead of just once. >>>>>>>>>> Instead of making the rotate() function public and moving the >>>>>>>>>> mutexlocker, I suggest adding something like a public >>>>>>>>>> force_rotation() function that grabs the lock and calls the >>>>>>>>>> private rotate(). >>>>>>>>>> >>>>>>>>>> * Given the addition of is_rotated() in LogOutput, then >>>>>>>>>> get_archive_name() should be removed. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Marcus >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 2015/10/26 18:56, Marcus Larsson wrote: >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> Sorry for my late reply. >>>>>>>>>>>> >>>>>>>>>>>> I think being able to force rotation via jcmd seems like a >>>>>>>>>>>> good >>>>>>>>>>>> feature. Files are currently opened in append mode so it >>>>>>>>>>>> should >>>>>>>>>>>> already be possible to use external log rotation tools by >>>>>>>>>>>> copying and truncating the files. Still I think it would be >>>>>>>>>>>> nice >>>>>>>>>>>> to provide the jcmd for rotation as well. >>>>>>>>>>>> >>>>>>>>>>>> I can see some small issues with the implementation, but we >>>>>>>>>>>> can >>>>>>>>>>>> deal with that during the review. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Marcus >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 2015-10-26 00:26, Yasumasa Suenaga wrote: >>>>>>>>>>>>> Hi all, >>>>>>>>>>>>> >>>>>>>>>>>>> Have you ever seen my change? >>>>>>>>>>>>> I (and my customers) need log rotation function via external >>>>>>>>>>>>> tool. >>>>>>>>>>>>> >>>>>>>>>>>>> I want to merge it by Feature Complete. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> >>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>> 2015/10/16 22:55 "Yasumasa Suenaga" : >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>> >>>>>>>>>>>>>> I contributed JDK-7090324: gclog rotation via external tool >>>>>>>>>>>>>> to be >>>>>>>>>>>>>> synchronized with >>>>>>>>>>>>>> logrotated tool on Linux. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I think JEP 158 is in progress. >>>>>>>>>>>>>> However, this JEP does not contain log rotation via external >>>>>>>>>>>>>> tool in the >>>>>>>>>>>>>> spec. >>>>>>>>>>>>>> I want to rotate logs via jcmd in this JEP. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I've updated a patch for it: >>>>>>>>>>>>>> >>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/jvmlogging-logrotate/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> This patch provides new option "rotate" in VM.log command. >>>>>>>>>>>>>> If this change can be accepted, I will file it to JBS and >>>>>>>>>>>>>> send >>>>>>>>>>>>>> RFR. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>> From yasuenag at gmail.com Thu Nov 5 14:50:08 2015 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Thu, 5 Nov 2015 23:50:08 +0900 Subject: RFR: JDK-8140556: Add force rotation option to VM.log jcmd In-Reply-To: <563B69F9.5090102@oracle.com> References: <562101D8.5080909@gmail.com> <562DF8D4.40501@oracle.com> <562EBF4B.4020607@gmail.com> <562F6B46.5050500@oracle.com> <563226AB.2030206@gmail.com> <5632341F.4060400@oracle.com> <56338125.9030802@gmail.com> <56338D58.9090709@oracle.com> <5634D9FC.6080502@gmail.com> <5636C66A.6020304@oracle.com> <563765FC.2090804@gmail.com> <563ACD6F.1050600@gmail.com> <563AD1F2.6060109@oracle.com> <563B69F9.5090102@oracle.com> Message-ID: <563B6CA0.9040107@gmail.com> >> _post_initialized would be a more accurate name. I've fixed it in new webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.04/ I will send changeset to Marcus if this changeset is reviewed. Thanks. Yasumasa On 2015/11/05 23:38, Marcus Larsson wrote: > Hey, > > On 2015-11-05 04:50, David Holmes wrote: >> On 5/11/2015 1:30 PM, Yasumasa Suenaga wrote: >>> I've uploaded new webrev: >>> http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.03/ >>> >>> I added _initialized to LogConfiguration. >>> LogConfiguration::post_initialize() will be called after main thread >>> initialization. >>> So _initialized set to true in this function, and we can check this flag >>> at rotation. >>> >>> Could you review it? > > Looks OK to me. This is much better than checking for Thread::current() != NULL in is_rotatable(), IMHO. > >> >> _post_initialized would be a more accurate name. >> >> This of course works, but I don't know whether the additional delay in allowing rotation will impact anything. > > It shouldn't. If anyone ever generates that much logging during startup they will have to accept that the first rotated log won't be properly sized, that's all. > > Thanks, > Marcus > >> >> Thanks, >> David >> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> On 2015/11/02 22:32, Yasumasa Suenaga wrote: >>>> Marcus, David, >>>> >>>> I've uploaded new webrev: >>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.02/ >>>> >>>> Could you review it? >>>> >>>> >>>>>>>>> Still missing the renaming of rotate_all_logfile to >>>>>>>>> rotate_all_outputs. >>>> >>>> I've fixed. >>>> >>>> >>>>>>> Small typo, please make that is_rotatable(). >>>> >>>> I've added, and have used at LogConfiguration::rotate_all_outputs(). >>>> >>>> >>>>> You can't safely lock a mutex until the current thread has attached >>>>> to the VM and Thread::current() returns non-null (that is what >>>>> Thread::initialize_thread_local_storage() currently does). You may >>>>> get away with it, particularly in product builds, during VM >>>>> initialization, as the fast-lock path doesn't require access to >>>>> Thread::current(). >>>> >>>> I will check whether Thread::current() is NULL or not at >>>> LogFileOutput::is_rotatable() >>>> is_rotatable() is called at LogFileOutput::rotate() before locking >>>> _rotation_lock. >>>> So we can avoid crash/assert which is related to TLS. >>>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> On 2015/11/02 11:11, David Holmes wrote: >>>>> On 1/11/2015 1:10 AM, Yasumasa Suenaga wrote: >>>>>>> This in turn means that we are logging before the thread local storage >>>>>>> has been initialized, and the JVM will crash/hit the assert because we >>>>>>> are trying to take the rotation lock. >>>>>> >>>>>> I guess that you are saying about class member (not TLS - e.g. >>>>>> pthread_setspecific()). >>>>> >>>>> You can't safely lock a mutex until the current thread has attached >>>>> to the VM and Thread::current() returns non-null (that is what >>>>> Thread::initialize_thread_local_storage() currently does). You may >>>>> get away with it, particularly in product builds, during VM >>>>> initialization, as the fast-lock path doesn't require access to >>>>> Thread::current(). >>>>> >>>>> David >>>>> >>>>>> Most of fields (include _rotation_lock) in LogFileOutput are >>>>>> initialized >>>>>> at c'tor. >>>>>> Other field (_stream) is initialized at LogFileOutput::initialize(). >>>>>> >>>>>> We can avoid problem if we can move initialize() to c'tor because >>>>>> LogFileOutput >>>>>> will be instantiated at LogConfiguration::new_output(). >>>>>> >>>>>> Currently, LogFileOutput will be added to LogConfiguration::_outputs >>>>>> and >>>>>> to LogTagSet after calling LogFileOutput::initialize(). >>>>>> I wonder why we cannot avoid crash/assert with current >>>>>> implementation... >>>>>> >>>>>> >>>>>> I will keep current implementation about _rotation_lock if the above is >>>>>> incorrect. >>>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2015/10/31 0:31, Marcus Larsson wrote: >>>>>>> Hey, >>>>>>> >>>>>>> On 2015-10-30 15:39, Yasumasa Suenaga wrote: >>>>>>>> Hi Marcus, >>>>>>>> >>>>>>>> Thank you for your comment. >>>>>>>> >>>>>>>> >>>>>>>>> Still missing the renaming of rotate_all_logfile to >>>>>>>>> rotate_all_outputs. >>>>>>>> >>>>>>>> Sorry, I will fix. >>>>>>>> >>>>>>> >>>>>>> Thanks! >>>>>>> >>>>>>>> >>>>>>>>> This rotate function works I guess, but it will probably need some >>>>>>>>> rework once there are other types of log outputs (rotate doesn't >>>>>>>>> make sense on all types of log outputs). >>>>>>>> >>>>>>>> I will add is_rotetable() to LogOutput as virtual function. >>>>>>>> This function return false (can not rotate) by default, return true >>>>>>>> if _file_count is >>>>>>>> greater than 0 in LogFileOutput. >>>>>>>> >>>>>>>> LogConfiguration::rotate_all_outputs() will rotate if this function >>>>>>>> returns true. >>>>>>>> >>>>>>> >>>>>>> Small typo, please make that is_rotatable(). >>>>>>> >>>>>>>> >>>>>>>>> If we configure a rotated file output and then log something on that >>>>>>>>> output before thread local storage initialization is complete, we >>>>>>>>> will crash/hit an assert. The previous implementation avoided this >>>>>>>>> as long as no rotation was needed before this initialization was >>>>>>>>> complete. (This can be triggered using the following arguments >>>>>>>>> "-Xlog:all=trace:file.txt::filesize=10,filecount=2 -Xlog:invalid", >>>>>>>>> for example.) >>>>>>>> >>>>>>>> I do not think so. >>>>>>>> Each -Xlog option creates unique instance of LogOutput (and >>>>>>>> subclass). >>>>>>>> So they are isolated. >>>>>>> >>>>>>> My example includes a second (and intentionally incorrect) -Xlog >>>>>>> option only to trigger logging on the newly configured rotating >>>>>>> LogFileOutput. This will cause some logging about the invalid -Xlog >>>>>>> argument, and this logging happens during the argument parsing. This >>>>>>> in turn means that we are logging before the thread local storage has >>>>>>> been initialized, and the JVM will crash/hit the assert because we are >>>>>>> trying to take the rotation lock. This has nothing to do with >>>>>>> concurrent fprintfs or freopens, even if that is a problem. >>>>>>> >>>>>>>> >>>>>>>> However, we might have to rotate at safepoint. >>>>>>> >>>>>>> Doing stuff at a safepoint will not give us much in this case. There >>>>>>> are threads that continue execution through a safepoint, which means >>>>>>> we will see log calls being made even during this time. >>>>>>> >>>>>>>> Currently, LogOutput might be used by multiple threads concurrently. >>>>>>>> If we rotate log when another thread is writing string to output, we >>>>>>>> encounter unexpected behavior. >>>>>>>> >>>>>>>> LogFileOutput::rotate() uses freopen(). >>>>>>>> LogFileStreamOutput::write() uses vfprintf() through jio_fprintf(). >>>>>>>> freopen() and vfprintf() are not atomic. >>>>>>> >>>>>>> You're right. There seems to be an overlooked concurrency issue with >>>>>>> vfprintfs during freopen. I don't think we should resolve this as part >>>>>>> of this particular fix, so instead I'll create a separate issue for >>>>>>> it. >>>>>>> >>>>>>>> >>>>>>>> I think that cause of crash/assertion which you say is it. >>>>>>>> (GC log will be rotated at safepoint.) >>>>>>>> >>>>>>> >>>>>>> See my comment above for what crash/assert I'm talking about. >>>>>>> >>>>>>> Thanks, >>>>>>> Marcus >>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> On 2015/10/29 23:58, Marcus Larsson wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> On 2015-10-29 15:01, Yasumasa Suenaga wrote: >>>>>>>>>> Hi Marcus, >>>>>>>>>> >>>>>>>>>> Thank you for your comment. >>>>>>>>>> >>>>>>>>>>> I'll sponsor it. >>>>>>>>>> >>>>>>>>>> Thank you so much! >>>>>>>>>> I've uploaded new webrev: >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.01/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> Still missing the renaming of rotate_all_logfile to >>>>>>>>> rotate_all_outputs. >>>>>>>>> >>>>>>>>>>> logConfiguration.cpp/hpp: >>>>>>>>>>> >>>>>>>>>>> * I don't think we should rely on the archive_name or the output's >>>>>>>>>>> name to decide whether or not an output should be rotated. It >>>>>>>>>>> would be better to introduce an is_rotated() test function in >>>>>>>>>>> LogOutput that could be used here. >>>>>>>>>>> >>>>>>>>>>> * rotate_all_logfile() should be renamed to rotate_all_outputs(). >>>>>>>>>>> Currently there are only LogFileOutputs (other than >>>>>>>>>>> stdout/stderr), but in the future there might be other types of >>>>>>>>>>> outputs so I prefer having a more general name. >>>>>>>>>>> Also, please use static_cast instead of the >>>>>>>>>>> C-style cast. (Additional logic will be required here once more >>>>>>>>>>> types of log outputs are added, but I don't think we need to worry >>>>>>>>>>> about this right now.) >>>>>>>>>>> >>>>>>>>>>> * Don't print an error if an output is not rotatable, since it's >>>>>>>>>>> not an error to have some log outputs rotated while others are >>>>>>>>>>> not. If you really want some traceability here I suggest adding >>>>>>>>>>> log messages on trace level, tagged with 'logging'. >>>>>>>>>> >>>>>>>>>> I added LogOutput::rotate(bool) as virtual function. >>>>>>>>>> This function works nothing by default, but will rotate all logs at >>>>>>>>>> LogFileOutput. >>>>>>>>> >>>>>>>>> This rotate function works I guess, but it will probably need some >>>>>>>>> rework once there are other types of log outputs (rotate doesn't >>>>>>>>> make sense on all types of log outputs). >>>>>>>>> >>>>>>>>>> >>>>>>>>>> If true is passed to this function, all files will be rotated. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> logDiagnosticCommand.cpp/hpp: >>>>>>>>>>> >>>>>>>>>>> * I think the description could be improved to something like >>>>>>>>>>> "Lists current log configuration, enables/disables/configures a >>>>>>>>>>> log output, or disables/rotates all logs." >>>>>>>>>>> >>>>>>>>>>> * The rotate option description should clarify that all logs will >>>>>>>>>>> be rotated ("current log" is too ambiguous). >>>>>>>>>> >>>>>>>>>> I've fixed. >>>>>>>>> >>>>>>>>> Great, thanks! >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> logFileOutput.cpp/hpp: >>>>>>>>>>> >>>>>>>>>>> * Moving the MutexLocker like this introduces a race condition >>>>>>>>>>> where a log might be rotated multiple times by different threads, >>>>>>>>>>> instead of just once. >>>>>>>>>>> Instead of making the rotate() function public and moving the >>>>>>>>>>> mutexlocker, I suggest adding something like a public >>>>>>>>>>> force_rotation() function that grabs the lock and calls the >>>>>>>>>>> private rotate(). >>>>>>>>>>> >>>>>>>>>>> * Given the addition of is_rotated() in LogOutput, then >>>>>>>>>>> get_archive_name() should be removed. >>>>>>>>>> >>>>>>>>>> I moved MutexLocker and should_rotate() to rotate(). >>>>>>>>>> I think this change can avoid race condition. >>>>>>>>>> >>>>>>>>> >>>>>>>>> There is a subtle problem with taking the lock to check if we should >>>>>>>>> rotate. If we configure a rotated file output and then log something >>>>>>>>> on that output before thread local storage initialization is >>>>>>>>> complete, we will crash/hit an assert. The previous implementation >>>>>>>>> avoided this as long as no rotation was needed before this >>>>>>>>> initialization was complete. (This can be triggered using the >>>>>>>>> following arguments >>>>>>>>> "-Xlog:all=trace:file.txt::filesize=10,filecount=2 -Xlog:invalid", >>>>>>>>> for example.) >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Marcus >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 2015/10/27 21:17, Marcus Larsson wrote: >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> On 2015-10-27 01:03, Yasumasa Suenaga wrote: >>>>>>>>>>>> Hi Marcus, >>>>>>>>>>>> >>>>>>>>>>>> Thank you for replying. >>>>>>>>>>>> >>>>>>>>>>>> I filed this feature to JBS and change subject of this email. >>>>>>>>>>>> Could you be a sponsor and review it? >>>>>>>>>>> >>>>>>>>>>> I'll sponsor it. >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.00/ >>>>>>>>>>> >>>>>>>>>>> logConfiguration.cpp/hpp: >>>>>>>>>>> >>>>>>>>>>> * I don't think we should rely on the archive_name or the output's >>>>>>>>>>> name to decide whether or not an output should be rotated. It >>>>>>>>>>> would be better to introduce an is_rotated() test function in >>>>>>>>>>> LogOutput that could be used here. >>>>>>>>>>> >>>>>>>>>>> * rotate_all_logfile() should be renamed to rotate_all_outputs(). >>>>>>>>>>> Currently there are only LogFileOutputs (other than >>>>>>>>>>> stdout/stderr), but in the future there might be other types of >>>>>>>>>>> outputs so I prefer having a more general name. >>>>>>>>>>> Also, please use static_cast instead of the >>>>>>>>>>> C-style cast. (Additional logic will be required here once more >>>>>>>>>>> types of log outputs are added, but I don't think we need to worry >>>>>>>>>>> about this right now.) >>>>>>>>>>> >>>>>>>>>>> * Don't print an error if an output is not rotatable, since it's >>>>>>>>>>> not an error to have some log outputs rotated while others are >>>>>>>>>>> not. If you really want some traceability here I suggest adding >>>>>>>>>>> log messages on trace level, tagged with 'logging'. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> logDiagnosticCommand.cpp/hpp: >>>>>>>>>>> >>>>>>>>>>> * I think the description could be improved to something like >>>>>>>>>>> "Lists current log configuration, enables/disables/configures a >>>>>>>>>>> log output, or disables/rotates all logs." >>>>>>>>>>> >>>>>>>>>>> * The rotate option description should clarify that all logs will >>>>>>>>>>> be rotated ("current log" is too ambiguous). >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> logFileOutput.cpp/hpp: >>>>>>>>>>> >>>>>>>>>>> * Moving the MutexLocker like this introduces a race condition >>>>>>>>>>> where a log might be rotated multiple times by different threads, >>>>>>>>>>> instead of just once. >>>>>>>>>>> Instead of making the rotate() function public and moving the >>>>>>>>>>> mutexlocker, I suggest adding something like a public >>>>>>>>>>> force_rotation() function that grabs the lock and calls the >>>>>>>>>>> private rotate(). >>>>>>>>>>> >>>>>>>>>>> * Given the addition of is_rotated() in LogOutput, then >>>>>>>>>>> get_archive_name() should be removed. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Marcus >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Yasumasa >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 2015/10/26 18:56, Marcus Larsson wrote: >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> Sorry for my late reply. >>>>>>>>>>>>> >>>>>>>>>>>>> I think being able to force rotation via jcmd seems like a good >>>>>>>>>>>>> feature. Files are currently opened in append mode so it should >>>>>>>>>>>>> already be possible to use external log rotation tools by >>>>>>>>>>>>> copying and truncating the files. Still I think it would be nice >>>>>>>>>>>>> to provide the jcmd for rotation as well. >>>>>>>>>>>>> >>>>>>>>>>>>> I can see some small issues with the implementation, but we can >>>>>>>>>>>>> deal with that during the review. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Marcus >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 2015-10-26 00:26, Yasumasa Suenaga wrote: >>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Have you ever seen my change? >>>>>>>>>>>>>> I (and my customers) need log rotation function via external >>>>>>>>>>>>>> tool. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I want to merge it by Feature Complete. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>> 2015/10/16 22:55 "Yasumasa Suenaga" : >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I contributed JDK-7090324: gclog rotation via external tool >>>>>>>>>>>>>>> to be >>>>>>>>>>>>>>> synchronized with >>>>>>>>>>>>>>> logrotated tool on Linux. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I think JEP 158 is in progress. >>>>>>>>>>>>>>> However, this JEP does not contain log rotation via external >>>>>>>>>>>>>>> tool in the >>>>>>>>>>>>>>> spec. >>>>>>>>>>>>>>> I want to rotate logs via jcmd in this JEP. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I've updated a patch for it: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/jvmlogging-logrotate/ >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> This patch provides new option "rotate" in VM.log command. >>>>>>>>>>>>>>> If this change can be accepted, I will file it to JBS and send >>>>>>>>>>>>>>> RFR. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>> > From david.holmes at oracle.com Fri Nov 6 07:32:27 2015 From: david.holmes at oracle.com (David Holmes) Date: Fri, 6 Nov 2015 17:32:27 +1000 Subject: RFR: JDK-8140556: Add force rotation option to VM.log jcmd In-Reply-To: <563B6CA0.9040107@gmail.com> References: <562101D8.5080909@gmail.com> <562DF8D4.40501@oracle.com> <562EBF4B.4020607@gmail.com> <562F6B46.5050500@oracle.com> <563226AB.2030206@gmail.com> <5632341F.4060400@oracle.com> <56338125.9030802@gmail.com> <56338D58.9090709@oracle.com> <5634D9FC.6080502@gmail.com> <5636C66A.6020304@oracle.com> <563765FC.2090804@gmail.com> <563ACD6F.1050600@gmail.com> <563AD1F2.6060109@oracle.com> <563B69F9.5090102@oracle.com> <563B6CA0.9040107@gmail.com> Message-ID: <563C578B.1040904@oracle.com> On 6/11/2015 12:50 AM, Yasumasa Suenaga wrote: >>> _post_initialized would be a more accurate name. > > I've fixed it in new webrev: > http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.04/ > > I will send changeset to Marcus if this changeset is reviewed. That change is fine by me. But again note this is not a full review as I am not familiar enough with the actual logging code. Thanks, David > > Thanks. > > Yasumasa > > > On 2015/11/05 23:38, Marcus Larsson wrote: >> Hey, >> >> On 2015-11-05 04:50, David Holmes wrote: >>> On 5/11/2015 1:30 PM, Yasumasa Suenaga wrote: >>>> I've uploaded new webrev: >>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.03/ >>>> >>>> I added _initialized to LogConfiguration. >>>> LogConfiguration::post_initialize() will be called after main thread >>>> initialization. >>>> So _initialized set to true in this function, and we can check this >>>> flag >>>> at rotation. >>>> >>>> Could you review it? >> >> Looks OK to me. This is much better than checking for >> Thread::current() != NULL in is_rotatable(), IMHO. >> >>> >>> _post_initialized would be a more accurate name. >>> >>> This of course works, but I don't know whether the additional delay >>> in allowing rotation will impact anything. >> >> It shouldn't. If anyone ever generates that much logging during >> startup they will have to accept that the first rotated log won't be >> properly sized, that's all. >> >> Thanks, >> Marcus >> >>> >>> Thanks, >>> David >>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> On 2015/11/02 22:32, Yasumasa Suenaga wrote: >>>>> Marcus, David, >>>>> >>>>> I've uploaded new webrev: >>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.02/ >>>>> >>>>> Could you review it? >>>>> >>>>> >>>>>>>>>> Still missing the renaming of rotate_all_logfile to >>>>>>>>>> rotate_all_outputs. >>>>> >>>>> I've fixed. >>>>> >>>>> >>>>>>>> Small typo, please make that is_rotatable(). >>>>> >>>>> I've added, and have used at LogConfiguration::rotate_all_outputs(). >>>>> >>>>> >>>>>> You can't safely lock a mutex until the current thread has attached >>>>>> to the VM and Thread::current() returns non-null (that is what >>>>>> Thread::initialize_thread_local_storage() currently does). You may >>>>>> get away with it, particularly in product builds, during VM >>>>>> initialization, as the fast-lock path doesn't require access to >>>>>> Thread::current(). >>>>> >>>>> I will check whether Thread::current() is NULL or not at >>>>> LogFileOutput::is_rotatable() >>>>> is_rotatable() is called at LogFileOutput::rotate() before locking >>>>> _rotation_lock. >>>>> So we can avoid crash/assert which is related to TLS. >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2015/11/02 11:11, David Holmes wrote: >>>>>> On 1/11/2015 1:10 AM, Yasumasa Suenaga wrote: >>>>>>>> This in turn means that we are logging before the thread local >>>>>>>> storage >>>>>>>> has been initialized, and the JVM will crash/hit the assert >>>>>>>> because we >>>>>>>> are trying to take the rotation lock. >>>>>>> >>>>>>> I guess that you are saying about class member (not TLS - e.g. >>>>>>> pthread_setspecific()). >>>>>> >>>>>> You can't safely lock a mutex until the current thread has attached >>>>>> to the VM and Thread::current() returns non-null (that is what >>>>>> Thread::initialize_thread_local_storage() currently does). You may >>>>>> get away with it, particularly in product builds, during VM >>>>>> initialization, as the fast-lock path doesn't require access to >>>>>> Thread::current(). >>>>>> >>>>>> David >>>>>> >>>>>>> Most of fields (include _rotation_lock) in LogFileOutput are >>>>>>> initialized >>>>>>> at c'tor. >>>>>>> Other field (_stream) is initialized at LogFileOutput::initialize(). >>>>>>> >>>>>>> We can avoid problem if we can move initialize() to c'tor because >>>>>>> LogFileOutput >>>>>>> will be instantiated at LogConfiguration::new_output(). >>>>>>> >>>>>>> Currently, LogFileOutput will be added to LogConfiguration::_outputs >>>>>>> and >>>>>>> to LogTagSet after calling LogFileOutput::initialize(). >>>>>>> I wonder why we cannot avoid crash/assert with current >>>>>>> implementation... >>>>>>> >>>>>>> >>>>>>> I will keep current implementation about _rotation_lock if the >>>>>>> above is >>>>>>> incorrect. >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2015/10/31 0:31, Marcus Larsson wrote: >>>>>>>> Hey, >>>>>>>> >>>>>>>> On 2015-10-30 15:39, Yasumasa Suenaga wrote: >>>>>>>>> Hi Marcus, >>>>>>>>> >>>>>>>>> Thank you for your comment. >>>>>>>>> >>>>>>>>> >>>>>>>>>> Still missing the renaming of rotate_all_logfile to >>>>>>>>>> rotate_all_outputs. >>>>>>>>> >>>>>>>>> Sorry, I will fix. >>>>>>>>> >>>>>>>> >>>>>>>> Thanks! >>>>>>>> >>>>>>>>> >>>>>>>>>> This rotate function works I guess, but it will probably need >>>>>>>>>> some >>>>>>>>>> rework once there are other types of log outputs (rotate doesn't >>>>>>>>>> make sense on all types of log outputs). >>>>>>>>> >>>>>>>>> I will add is_rotetable() to LogOutput as virtual function. >>>>>>>>> This function return false (can not rotate) by default, return >>>>>>>>> true >>>>>>>>> if _file_count is >>>>>>>>> greater than 0 in LogFileOutput. >>>>>>>>> >>>>>>>>> LogConfiguration::rotate_all_outputs() will rotate if this >>>>>>>>> function >>>>>>>>> returns true. >>>>>>>>> >>>>>>>> >>>>>>>> Small typo, please make that is_rotatable(). >>>>>>>> >>>>>>>>> >>>>>>>>>> If we configure a rotated file output and then log something >>>>>>>>>> on that >>>>>>>>>> output before thread local storage initialization is complete, we >>>>>>>>>> will crash/hit an assert. The previous implementation avoided >>>>>>>>>> this >>>>>>>>>> as long as no rotation was needed before this initialization was >>>>>>>>>> complete. (This can be triggered using the following arguments >>>>>>>>>> "-Xlog:all=trace:file.txt::filesize=10,filecount=2 >>>>>>>>>> -Xlog:invalid", >>>>>>>>>> for example.) >>>>>>>>> >>>>>>>>> I do not think so. >>>>>>>>> Each -Xlog option creates unique instance of LogOutput (and >>>>>>>>> subclass). >>>>>>>>> So they are isolated. >>>>>>>> >>>>>>>> My example includes a second (and intentionally incorrect) -Xlog >>>>>>>> option only to trigger logging on the newly configured rotating >>>>>>>> LogFileOutput. This will cause some logging about the invalid -Xlog >>>>>>>> argument, and this logging happens during the argument parsing. >>>>>>>> This >>>>>>>> in turn means that we are logging before the thread local >>>>>>>> storage has >>>>>>>> been initialized, and the JVM will crash/hit the assert because >>>>>>>> we are >>>>>>>> trying to take the rotation lock. This has nothing to do with >>>>>>>> concurrent fprintfs or freopens, even if that is a problem. >>>>>>>> >>>>>>>>> >>>>>>>>> However, we might have to rotate at safepoint. >>>>>>>> >>>>>>>> Doing stuff at a safepoint will not give us much in this case. >>>>>>>> There >>>>>>>> are threads that continue execution through a safepoint, which >>>>>>>> means >>>>>>>> we will see log calls being made even during this time. >>>>>>>> >>>>>>>>> Currently, LogOutput might be used by multiple threads >>>>>>>>> concurrently. >>>>>>>>> If we rotate log when another thread is writing string to >>>>>>>>> output, we >>>>>>>>> encounter unexpected behavior. >>>>>>>>> >>>>>>>>> LogFileOutput::rotate() uses freopen(). >>>>>>>>> LogFileStreamOutput::write() uses vfprintf() through >>>>>>>>> jio_fprintf(). >>>>>>>>> freopen() and vfprintf() are not atomic. >>>>>>>> >>>>>>>> You're right. There seems to be an overlooked concurrency issue >>>>>>>> with >>>>>>>> vfprintfs during freopen. I don't think we should resolve this >>>>>>>> as part >>>>>>>> of this particular fix, so instead I'll create a separate issue for >>>>>>>> it. >>>>>>>> >>>>>>>>> >>>>>>>>> I think that cause of crash/assertion which you say is it. >>>>>>>>> (GC log will be rotated at safepoint.) >>>>>>>>> >>>>>>>> >>>>>>>> See my comment above for what crash/assert I'm talking about. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Marcus >>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2015/10/29 23:58, Marcus Larsson wrote: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> On 2015-10-29 15:01, Yasumasa Suenaga wrote: >>>>>>>>>>> Hi Marcus, >>>>>>>>>>> >>>>>>>>>>> Thank you for your comment. >>>>>>>>>>> >>>>>>>>>>>> I'll sponsor it. >>>>>>>>>>> >>>>>>>>>>> Thank you so much! >>>>>>>>>>> I've uploaded new webrev: >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.01/ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Still missing the renaming of rotate_all_logfile to >>>>>>>>>> rotate_all_outputs. >>>>>>>>>> >>>>>>>>>>>> logConfiguration.cpp/hpp: >>>>>>>>>>>> >>>>>>>>>>>> * I don't think we should rely on the archive_name or the >>>>>>>>>>>> output's >>>>>>>>>>>> name to decide whether or not an output should be rotated. It >>>>>>>>>>>> would be better to introduce an is_rotated() test function in >>>>>>>>>>>> LogOutput that could be used here. >>>>>>>>>>>> >>>>>>>>>>>> * rotate_all_logfile() should be renamed to >>>>>>>>>>>> rotate_all_outputs(). >>>>>>>>>>>> Currently there are only LogFileOutputs (other than >>>>>>>>>>>> stdout/stderr), but in the future there might be other types of >>>>>>>>>>>> outputs so I prefer having a more general name. >>>>>>>>>>>> Also, please use static_cast instead of the >>>>>>>>>>>> C-style cast. (Additional logic will be required here once more >>>>>>>>>>>> types of log outputs are added, but I don't think we need to >>>>>>>>>>>> worry >>>>>>>>>>>> about this right now.) >>>>>>>>>>>> >>>>>>>>>>>> * Don't print an error if an output is not rotatable, since >>>>>>>>>>>> it's >>>>>>>>>>>> not an error to have some log outputs rotated while others are >>>>>>>>>>>> not. If you really want some traceability here I suggest adding >>>>>>>>>>>> log messages on trace level, tagged with 'logging'. >>>>>>>>>>> >>>>>>>>>>> I added LogOutput::rotate(bool) as virtual function. >>>>>>>>>>> This function works nothing by default, but will rotate all >>>>>>>>>>> logs at >>>>>>>>>>> LogFileOutput. >>>>>>>>>> >>>>>>>>>> This rotate function works I guess, but it will probably need >>>>>>>>>> some >>>>>>>>>> rework once there are other types of log outputs (rotate doesn't >>>>>>>>>> make sense on all types of log outputs). >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> If true is passed to this function, all files will be rotated. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> logDiagnosticCommand.cpp/hpp: >>>>>>>>>>>> >>>>>>>>>>>> * I think the description could be improved to something like >>>>>>>>>>>> "Lists current log configuration, enables/disables/configures a >>>>>>>>>>>> log output, or disables/rotates all logs." >>>>>>>>>>>> >>>>>>>>>>>> * The rotate option description should clarify that all logs >>>>>>>>>>>> will >>>>>>>>>>>> be rotated ("current log" is too ambiguous). >>>>>>>>>>> >>>>>>>>>>> I've fixed. >>>>>>>>>> >>>>>>>>>> Great, thanks! >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> logFileOutput.cpp/hpp: >>>>>>>>>>>> >>>>>>>>>>>> * Moving the MutexLocker like this introduces a race condition >>>>>>>>>>>> where a log might be rotated multiple times by different >>>>>>>>>>>> threads, >>>>>>>>>>>> instead of just once. >>>>>>>>>>>> Instead of making the rotate() function public and moving the >>>>>>>>>>>> mutexlocker, I suggest adding something like a public >>>>>>>>>>>> force_rotation() function that grabs the lock and calls the >>>>>>>>>>>> private rotate(). >>>>>>>>>>>> >>>>>>>>>>>> * Given the addition of is_rotated() in LogOutput, then >>>>>>>>>>>> get_archive_name() should be removed. >>>>>>>>>>> >>>>>>>>>>> I moved MutexLocker and should_rotate() to rotate(). >>>>>>>>>>> I think this change can avoid race condition. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> There is a subtle problem with taking the lock to check if we >>>>>>>>>> should >>>>>>>>>> rotate. If we configure a rotated file output and then log >>>>>>>>>> something >>>>>>>>>> on that output before thread local storage initialization is >>>>>>>>>> complete, we will crash/hit an assert. The previous >>>>>>>>>> implementation >>>>>>>>>> avoided this as long as no rotation was needed before this >>>>>>>>>> initialization was complete. (This can be triggered using the >>>>>>>>>> following arguments >>>>>>>>>> "-Xlog:all=trace:file.txt::filesize=10,filecount=2 >>>>>>>>>> -Xlog:invalid", >>>>>>>>>> for example.) >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Marcus >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 2015/10/27 21:17, Marcus Larsson wrote: >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> On 2015-10-27 01:03, Yasumasa Suenaga wrote: >>>>>>>>>>>>> Hi Marcus, >>>>>>>>>>>>> >>>>>>>>>>>>> Thank you for replying. >>>>>>>>>>>>> >>>>>>>>>>>>> I filed this feature to JBS and change subject of this email. >>>>>>>>>>>>> Could you be a sponsor and review it? >>>>>>>>>>>> >>>>>>>>>>>> I'll sponsor it. >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.00/ >>>>>>>>>>>> >>>>>>>>>>>> logConfiguration.cpp/hpp: >>>>>>>>>>>> >>>>>>>>>>>> * I don't think we should rely on the archive_name or the >>>>>>>>>>>> output's >>>>>>>>>>>> name to decide whether or not an output should be rotated. It >>>>>>>>>>>> would be better to introduce an is_rotated() test function in >>>>>>>>>>>> LogOutput that could be used here. >>>>>>>>>>>> >>>>>>>>>>>> * rotate_all_logfile() should be renamed to >>>>>>>>>>>> rotate_all_outputs(). >>>>>>>>>>>> Currently there are only LogFileOutputs (other than >>>>>>>>>>>> stdout/stderr), but in the future there might be other types of >>>>>>>>>>>> outputs so I prefer having a more general name. >>>>>>>>>>>> Also, please use static_cast instead of the >>>>>>>>>>>> C-style cast. (Additional logic will be required here once more >>>>>>>>>>>> types of log outputs are added, but I don't think we need to >>>>>>>>>>>> worry >>>>>>>>>>>> about this right now.) >>>>>>>>>>>> >>>>>>>>>>>> * Don't print an error if an output is not rotatable, since >>>>>>>>>>>> it's >>>>>>>>>>>> not an error to have some log outputs rotated while others are >>>>>>>>>>>> not. If you really want some traceability here I suggest adding >>>>>>>>>>>> log messages on trace level, tagged with 'logging'. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> logDiagnosticCommand.cpp/hpp: >>>>>>>>>>>> >>>>>>>>>>>> * I think the description could be improved to something like >>>>>>>>>>>> "Lists current log configuration, enables/disables/configures a >>>>>>>>>>>> log output, or disables/rotates all logs." >>>>>>>>>>>> >>>>>>>>>>>> * The rotate option description should clarify that all logs >>>>>>>>>>>> will >>>>>>>>>>>> be rotated ("current log" is too ambiguous). >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> logFileOutput.cpp/hpp: >>>>>>>>>>>> >>>>>>>>>>>> * Moving the MutexLocker like this introduces a race condition >>>>>>>>>>>> where a log might be rotated multiple times by different >>>>>>>>>>>> threads, >>>>>>>>>>>> instead of just once. >>>>>>>>>>>> Instead of making the rotate() function public and moving the >>>>>>>>>>>> mutexlocker, I suggest adding something like a public >>>>>>>>>>>> force_rotation() function that grabs the lock and calls the >>>>>>>>>>>> private rotate(). >>>>>>>>>>>> >>>>>>>>>>>> * Given the addition of is_rotated() in LogOutput, then >>>>>>>>>>>> get_archive_name() should be removed. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Marcus >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> >>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 2015/10/26 18:56, Marcus Larsson wrote: >>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Sorry for my late reply. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I think being able to force rotation via jcmd seems like a >>>>>>>>>>>>>> good >>>>>>>>>>>>>> feature. Files are currently opened in append mode so it >>>>>>>>>>>>>> should >>>>>>>>>>>>>> already be possible to use external log rotation tools by >>>>>>>>>>>>>> copying and truncating the files. Still I think it would >>>>>>>>>>>>>> be nice >>>>>>>>>>>>>> to provide the jcmd for rotation as well. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I can see some small issues with the implementation, but >>>>>>>>>>>>>> we can >>>>>>>>>>>>>> deal with that during the review. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Marcus >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 2015-10-26 00:26, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Have you ever seen my change? >>>>>>>>>>>>>>> I (and my customers) need log rotation function via external >>>>>>>>>>>>>>> tool. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I want to merge it by Feature Complete. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>> 2015/10/16 22:55 "Yasumasa Suenaga" : >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I contributed JDK-7090324: gclog rotation via external tool >>>>>>>>>>>>>>>> to be >>>>>>>>>>>>>>>> synchronized with >>>>>>>>>>>>>>>> logrotated tool on Linux. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I think JEP 158 is in progress. >>>>>>>>>>>>>>>> However, this JEP does not contain log rotation via >>>>>>>>>>>>>>>> external >>>>>>>>>>>>>>>> tool in the >>>>>>>>>>>>>>>> spec. >>>>>>>>>>>>>>>> I want to rotate logs via jcmd in this JEP. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I've updated a patch for it: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/jvmlogging-logrotate/ >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> This patch provides new option "rotate" in VM.log command. >>>>>>>>>>>>>>>> If this change can be accepted, I will file it to JBS >>>>>>>>>>>>>>>> and send >>>>>>>>>>>>>>>> RFR. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> >>>>>>>> >> From richard at lucentsky.com Fri Nov 6 11:11:19 2015 From: richard at lucentsky.com (Richard Callahan) Date: Fri, 6 Nov 2015 11:11:19 +0000 Subject: Technical question about the Java Dependency Analysis Tool Message-ID: Hi, I have a specific question about the output from the Java Dependency Analysis Tool (JDeps), released with Open JDK 8. Thank you very much for such a marvelous tool! I am parsing the output from JDeps with the "-v" flag set, such that a call to JDeps produces all class-level dependencies among and within a set of JAR files in a directory. Consider a graph G constructed from the JDeps output, with the nodes represented by classes and the edges represented by vertices. My question is, if a specific JAR file in the directory (call it A.jar) can compile using OpenJDK with the other JAR files in the directory designated as dependencies, then does there always exist at least one subgraph G' of G that corresponds to the dependency relationships that would be actually used by a Java compiler to compile that JAR file? If the answer is "yes," then I believe I should be able to recover that graph by starting with the set of classes in A.jar and constructing the subgraph G' in a manner similar to that used by a depth-first search. If this post is to the wrong mailing list, I beg your pardon, and thank you for your time! Best, Richard -------------- next part -------------- An HTML attachment was scrubbed... URL: From jaroslav.bachorik at oracle.com Fri Nov 6 13:36:09 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Fri, 6 Nov 2015 14:36:09 +0100 Subject: RFR 8141526: Allow to collect stdout/stderr from the FinalizationRunner even before the process returns Message-ID: <563CACC9.2070401@oracle.com> Please, review the following test change Issue : https://bugs.openjdk.java.net/browse/JDK-8141526 Webrev: top> http://cr.openjdk.java.net/~jbachorik/8141526/webrev.00 hotspot> http://cr.openjdk.java.net/~jbachorik/8141526/webrev.00/hotspot jdk> http://cr.openjdk.java.net/~jbachorik/8141526/webrev.00/jdk After the fix for https://bugs.openjdk.java.net/browse/JDK-8135188 we are not able to get the debug info about the run of the launcher FinalizationRunner application in case it gets stuck and harness times out. This is because the stdout/err of the application started via ProcessTools.executeProcess() is collected only after the application has exited. The solution is to use ProcessTools.startProcess() and consume the application stdout/err in a streaming mode. Because this method has only been available in the 'jdk' version of ProcessTools and not in the 'hotspot' one I decided to merge both of those versions and place the merged version into the shared location 'test/lib/share/classes/'. During this I decided to change the package for the shared ProcessTools class to be 'jdk.test.lib.process' to be more in line with the way this shared library is structured. I had to move few other lib classes required by ProcessTools to the shared lib as well. All the moved lib classes have been marked as deprecated in their original location. Thanks, -JB- From jaroslav.bachorik at oracle.com Fri Nov 6 16:11:39 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Fri, 6 Nov 2015 17:11:39 +0100 Subject: RFR 8141526: Allow to collect stdout/stderr from the FinalizationRunner even before the process returns Message-ID: <563CD13B.8040800@oracle.com> [wider audience included] Please, review the following test change Issue : https://bugs.openjdk.java.net/browse/JDK-8141526 Webrev: top> http://cr.openjdk.java.net/~jbachorik/8141526/webrev.00 hotspot> http://cr.openjdk.java.net/~jbachorik/8141526/webrev.00/hotspot jdk> http://cr.openjdk.java.net/~jbachorik/8141526/webrev.00/jdk After the fix for https://bugs.openjdk.java.net/browse/JDK-8135188 we are not able to get the debug info about the run of the launcher FinalizationRunner application in case it gets stuck and harness times out. This is because the stdout/err of the application started via ProcessTools.executeProcess() is collected only after the application has exited. The solution is to use ProcessTools.startProcess() and consume the application stdout/err in a streaming mode. Because this method has only been available in the 'jdk' version of ProcessTools and not in the 'hotspot' one I decided to merge both of those versions and place the merged version into the shared location 'test/lib/share/classes/'. During this I decided to change the package for the shared ProcessTools class to be 'jdk.test.lib.process' to be more in line with the way this shared library is structured. I had to move few other lib classes required by ProcessTools to the shared lib as well. All the moved lib classes have been marked as deprecated in their original location. Thanks, -JB- From staffan.larsen at oracle.com Mon Nov 9 07:25:41 2015 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Mon, 9 Nov 2015 08:25:41 +0100 Subject: Technical question about the Java Dependency Analysis Tool In-Reply-To: References: Message-ID: <521904EE-ADB2-4CFD-8C2F-23ECA2793D6C@oracle.com> Hi Richard, I think a better list for this question is either core-libs-dev@ or jigsaw-dev at . Thanks /Staffan > On 6 nov. 2015, at 12:11, Richard Callahan wrote: > > Hi, > > I have a specific question about the output from the Java Dependency Analysis Tool (JDeps), released with Open JDK 8. Thank you very much for such a marvelous tool! > > I am parsing the output from JDeps with the ?-v? flag set, such that a call to JDeps produces all class-level dependencies among and within a set of JAR files in a directory. Consider a graph G constructed from the JDeps output, with the nodes represented by classes and the edges represented by vertices. My question is, if a specific JAR file in the directory (call it A.jar) can compile using OpenJDK with the other JAR files in the directory designated as dependencies, then does there always exist at least one subgraph G? of G that corresponds to the dependency relationships that would be actually used by a Java compiler to compile that JAR file? If the answer is ?yes,? then I believe I should be able to recover that graph by starting with the set of classes in A.jar and constructing the subgraph G? in a manner similar to that used by a depth-first search. > > If this post is to the wrong mailing list, I beg your pardon, and thank you for your time! > > Best, > > Richard -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alan.Bateman at oracle.com Mon Nov 9 07:53:07 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 9 Nov 2015 07:53:07 +0000 Subject: Technical question about the Java Dependency Analysis Tool In-Reply-To: <521904EE-ADB2-4CFD-8C2F-23ECA2793D6C@oracle.com> References: <521904EE-ADB2-4CFD-8C2F-23ECA2793D6C@oracle.com> Message-ID: <564050E3.7020605@oracle.com> On 09/11/2015 07:25, Staffan Larsen wrote: > Hi Richard, > > I think a better list for this question is either core-libs-dev@ or > jigsaw-dev at . > Yes, jigsaw-dev is the mailing list for this. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From sgehwolf at redhat.com Mon Nov 9 09:32:05 2015 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Mon, 09 Nov 2015 10:32:05 +0100 Subject: jmx-dev RFR 6425769: jmx remote bind address In-Reply-To: <1446634476.3554.8.camel@redhat.com> References: <1446460682.10865.20.camel@redhat.com> <56377F17.4050006@oracle.com> <5637871B.30705@oracle.com> <1446487615.10865.57.camel@redhat.com> <5638C89F.2090406@oracle.com> <1446634476.3554.8.camel@redhat.com> Message-ID: <1447061525.3551.3.camel@redhat.com> On Wed, 2015-11-04 at 11:54 +0100, Severin Gehwolf wrote: > Hi, > > Updated webrev with jtreg test in Java: > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-6425769/02/ > bug:?https://bugs.openjdk.java.net/browse/JDK-6425769 > > I believe this updated webrev addresses all concerns and incorporates > suggestions brought up by Jaroslav and Daniel. > > I'm still looking for a sponsor and a hotspot/servicability-dev > reviewer. Could somebody maintaining javax.rmi.ssl have a look at > this > as well? Thank you! Ping? Friendly reminder that I still need reviewers and a sponsor for this. Thanks, Severin > Cheers, > Severin > > On Tue, 2015-11-03 at 15:45 +0100, Jaroslav Bachorik wrote: > > On 2.11.2015 19:06, Severin Gehwolf wrote: > > > Hi, > > > > > > Thanks Jaroslav and Daniel for the reviews! Comments inline. > > > > > > On Mon, 2015-11-02 at 16:54 +0100, Jaroslav Bachorik wrote: > > > > Hi, > > > > > > > > On 2.11.2015 16:19, Daniel Fuchs wrote: > > > > > Hi Severin, > > > > > > > > > > Adding serviceability-dev at openjdk.java.net into the loop - > > > > > that's > > > > > a better alias than hotspot-dev for this kind of changes - > > > > > maybe > > > > > someone from serviceability-dev will offer to sponsor :-) > > > > > > > > > > I will let serviceability team members comment on the hotspot > > > > > changes. > > > > > > > > > > ConnectorBootstrap.java > > > > > > > > > > I have one suggestion and one concern: > > > > > > > > > > Suggestion: I would suggest to reuse 'csf' (Client Socket > > > > > Factory) > > > > > and > > > > > ssf??(Server Socket Factory) variables rather than introduce > > > > > the > > > > > two > > > > > new variables rmiServerSocketFactory and > > > > > rmiClientSocketFactory. > > > > > You might want to create a new boolean 'useSocketFactory' > > > > > variable, > > > > > if that makes the code more readable. > > > > > > > > > > Concern: If I understand correctly how RMI socket factories > > > > > work, > > > > > the client socket factory will be serialized and sent to the > > > > > client > > > > > side. This is problematic for interoperability, as the class > > > > > may > > > > > not > > > > > present in the remote client - if the remote client is e.g. > > > > > jdk > > > > > 8. > > > > > > > > > > As far as I can see, your new DefaultClientSocketFactory > > > > > doesn't do > > > > > anything useful - so I would suggest to simply get rid of it, > > > > > and > > > > > only > > > > > set the Server Socket Factory when SSL is not involved. > > > > > > Thanks. Fixed in updated webrev. > > > > > > > > Tests: > > > > > > > > > > Concerning the tests - we're trying to get rid of shell > > > > > scripts > > > > > rather than introducing new ones :-) > > > > > Could the test be rewritten in pure java using the Process > > > > > API? > > > > > > > > > > I believe there's even a test library that will let you do > > > > > that > > > > > easily jdk/test/lib/testlibrary/jdk/testlibrary/ > > > > > (see ProcessTools.java) > > > > > > It'll take me a bit to rewrite the test in pure Java, but should > > > be > > > fine. This is not yet fixed in the updated webrev. > > > > > > > > Other: > > > > > > > > > > Also - I believe the new option should be documented in: > > > > > src/java.management/share/conf/management.properties > > > > > > Docs have been updated > > > in src/java.management/share/conf/management.properties. > > > > > > > I share Daniel's concerns. Also, the part of the changeset is > > > > related > > > > to javax.rmi.ssl - someone maintaining this library should also > > > > comment here. > > > > > > > > Also, the change is introducing new API (system property) and > > > > changing the existing one (adding SslRmiServerSocketFactory > > > > public > > > > constructors) so compatibility review process will have to be > > > > involved. > > > > > > OK. What exactly is there for me to do? I'm not familiar with > > > this > > > process. Note that the intent would be to get this backported to > > > JDK 8. > > Not much for you. But for the potential Oracle sponsor this means > > she > > will have to remember to go through some extra hoops before > > integrating your patch. > > I see. Thanks for clarifying it. > > > -JB- > > > > > > > > New webrev at: > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-6425769/01/ > > > > > > Thanks, > > > Severin > > > > > > > -JB- > > > > > > > > > > best regards, > > > > > > > > > > -- daniel > > > > > > > > > > On 02/11/15 11:38, Severin Gehwolf wrote: > > > > > > Hi, > > > > > > > > > > > > Here is a patch addressing JDK-6425769. The issue is that > > > > > > the > > > > > > JMX > > > > > > agent > > > > > > binds to the wildcard address by default, preventing users > > > > > > to > > > > > > use > > > > > > system properties for JMX agents on multi-homed hosts. > > > > > > Given > > > > > > a > > > > > > host > > > > > > with local network interfaces, 192.168.0.1 and 192.168.0.2 > > > > > > say, > > > > > > it's > > > > > > impossible to start two JMX agents binding to fixed ports > > > > > > but > > > > > > to > > > > > > different network interfaces, 192.168.0.1:{9111,9112} and > > > > > > 192.168.0.2:{9111,9112} say. > > > > > > > > > > > > The JDK would bind to all local interfaces by default. In > > > > > > the > > > > > > above > > > > > > example to 192.168.0.1 *and* 192.168.0.2. The effect is > > > > > > that > > > > > > the > > > > > > second > > > > > > Java process would get a "Connection refused" error because > > > > > > another > > > > > > process has already been bound to the specified JMX/RMI > > > > > > port > > > > > > pairs. > > > > > > > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-6425769 > > > > > > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-64 > > > > > > 25 > > > > > > 769/ > > > > > > 00/ > > > > > > > > > > > > Testing done: > > > > > > jdk_jmx and jdk_management tests all pass after this change > > > > > > (no > > > > > > regressions). There is also a new JTREG test which fails > > > > > > prior > > > > > > this > > > > > > change and passes after. Updates to the diagnostic command > > > > > > have > > > > > > been > > > > > > tested manually. > > > > > > > > > > > > I understand that, if approved, the JDK and hotspot changes > > > > > > should get > > > > > > pushed together? Hotspot changes are fairly trivial since > > > > > > it's > > > > > > only a > > > > > > doc-update for the new JDK property in the relevant > > > > > > diagnostic > > > > > > command. > > > > > > > > > > > > Could someone please review and sponsor this change? Please > > > > > > let > > > > > > me know > > > > > > if there are questions. > > > > > > > > > > > > Thanks, > > > > > > Severin > > > > > > > > > > > > > > > > > > > > > From alexander.kulyakhtin at oracle.com Mon Nov 9 15:40:58 2015 From: alexander.kulyakhtin at oracle.com (Alexander Kulyakhtin) Date: Mon, 9 Nov 2015 07:40:58 -0800 (PST) Subject: RFR: 8141591 : javax/management/remote/mandatory/threads/ExecutorTest.java fails intermittently Message-ID: <6f11862b-2e5c-406e-98dd-51b7347e326e@default> Hi, Could you, please, review this test-only fix: CR: https://bugs.openjdk.java.net/browse/JDK-8141591 Webrev: http://cr.openjdk.java.net/~akulyakh/8141591_01/test/javax/management/remote/mandatory/threads/ExecutorTest.java.udiff.html Before the fix, it was possible that we shut down the executor before all the jobs have been submitted. This resulted in RejectedExecutionException. We are modifying the test to wait on CountDownLatch untill all the jobs have completed their execute() Best regards, Alexander From jaroslav.bachorik at oracle.com Mon Nov 9 16:03:19 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Mon, 9 Nov 2015 17:03:19 +0100 Subject: RFR: 8141591 : javax/management/remote/mandatory/threads/ExecutorTest.java fails intermittently In-Reply-To: <6f11862b-2e5c-406e-98dd-51b7347e326e@default> References: <6f11862b-2e5c-406e-98dd-51b7347e326e@default> Message-ID: <5640C3C7.1060707@oracle.com> Hi Alexander, On 9.11.2015 16:40, Alexander Kulyakhtin wrote: > Hi, > > Could you, please, review this test-only fix: > > CR: https://bugs.openjdk.java.net/browse/JDK-8141591 > Webrev: http://cr.openjdk.java.net/~akulyakh/8141591_01/test/javax/management/remote/mandatory/threads/ExecutorTest.java.udiff.html > > Before the fix, it was possible that we shut down the executor before all the jobs have been submitted. This resulted in RejectedExecutionException. > We are modifying the test to wait on CountDownLatch untill all the jobs have completed their execute() On L175 you should be using the unbound version of await(). It would be better to let the harness deal with the timeout. Otherwise it looks good. Cheers, -JB- > > Best regards, > Alexander > From alexander.kulyakhtin at oracle.com Mon Nov 9 16:08:25 2015 From: alexander.kulyakhtin at oracle.com (Alexander Kulyakhtin) Date: Mon, 9 Nov 2015 08:08:25 -0800 (PST) Subject: RFR: 8141591 : javax/management/remote/mandatory/threads/ExecutorTest.java fails intermittently Message-ID: <801e35ae-823d-48cc-8666-4667c55bf05d@default> Hi Jaroslav, Thank you very much for the review. I wanted to use the bound version of the await(timeout) to provide for a better output in case the test fails. Would you advise just to replace with no-args await() or is it possible to somehow distinguish for L175 and other platforms where possible? Best regards, Alexander ----- Original Message ----- From: jaroslav.bachorik at oracle.com To: alexander.kulyakhtin at oracle.com, serviceability-dev at openjdk.java.net Sent: Monday, November 9, 2015 7:03:22 PM GMT +03:00 Iraq Subject: Re: RFR: 8141591 : javax/management/remote/mandatory/threads/ExecutorTest.java fails intermittently Hi Alexander, On 9.11.2015 16:40, Alexander Kulyakhtin wrote: > Hi, > > Could you, please, review this test-only fix: > > CR: https://bugs.openjdk.java.net/browse/JDK-8141591 > Webrev: http://cr.openjdk.java.net/~akulyakh/8141591_01/test/javax/management/remote/mandatory/threads/ExecutorTest.java.udiff.html > > Before the fix, it was possible that we shut down the executor before all the jobs have been submitted. This resulted in RejectedExecutionException. > We are modifying the test to wait on CountDownLatch untill all the jobs have completed their execute() On L175 you should be using the unbound version of await(). It would be better to let the harness deal with the timeout. Otherwise it looks good. Cheers, -JB- > > Best regards, > Alexander > From martinrb at google.com Mon Nov 9 16:53:06 2015 From: martinrb at google.com (Martin Buchholz) Date: Mon, 9 Nov 2015 08:53:06 -0800 Subject: RFR: 8141591 : javax/management/remote/mandatory/threads/ExecutorTest.java fails intermittently In-Reply-To: <5640C3C7.1060707@oracle.com> References: <6f11862b-2e5c-406e-98dd-51b7347e326e@default> <5640C3C7.1060707@oracle.com> Message-ID: The traditional way is to have all the worker tasks count down a latch when they're started and have the master thread wait on that before proceeding. You can use test.timeout.factor to do a scaled timed wait. On Mon, Nov 9, 2015 at 8:03 AM, Jaroslav Bachorik < jaroslav.bachorik at oracle.com> wrote: > Hi Alexander, > > On 9.11.2015 16:40, Alexander Kulyakhtin wrote: > >> Hi, >> >> Could you, please, review this test-only fix: >> >> CR: https://bugs.openjdk.java.net/browse/JDK-8141591 >> Webrev: >> http://cr.openjdk.java.net/~akulyakh/8141591_01/test/javax/management/remote/mandatory/threads/ExecutorTest.java.udiff.html >> >> Before the fix, it was possible that we shut down the executor before all >> the jobs have been submitted. This resulted in RejectedExecutionException. >> We are modifying the test to wait on CountDownLatch untill all the jobs >> have completed their execute() >> > > On L175 you should be using the unbound version of await(). It would be > better to let the harness deal with the timeout. > > Otherwise it looks good. > > Cheers, > > -JB- > > >> Best regards, >> Alexander >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.kulyakhtin at oracle.com Mon Nov 9 18:07:00 2015 From: alexander.kulyakhtin at oracle.com (Alexander Kulyakhtin) Date: Mon, 9 Nov 2015 10:07:00 -0800 (PST) Subject: RFR: 8141591 : javax/management/remote/mandatory/threads/ExecutorTest.java fails intermittently Message-ID: Hi, Following the comments from Jaroslav and Martin I've changed the test to use the unbound CountDownLatch.await() Since await() will wait undefinitely and thus the test, if fails, will now fail by timeout only the code has been additionally simplified to reflect that. Please, find the updated webrev at: CR: https://bugs.openjdk.java.net/browse/JDK-8141591 RFR: http://cr.openjdk.java.net/~akulyakh/8141591_03/test/javax/management/remote/mandatory/threads/ExecutorTest.java.udiff.html Best regards, Alexander ----- Original Message ----- From: martinrb at google.com To: jaroslav.bachorik at oracle.com Cc: alexander.kulyakhtin at oracle.com, serviceability-dev at openjdk.java.net Sent: Monday, November 9, 2015 7:53:08 PM GMT +03:00 Iraq Subject: Re: RFR: 8141591 : javax/management/remote/mandatory/threads/ExecutorTest.java fails intermittently The traditional way is to have all the worker tasks count down a latch when they're started and have the master thread wait on that before proceeding. You can use test.timeout.factor to do a scaled timed wait. On Mon, Nov 9, 2015 at 8:03 AM, Jaroslav Bachorik < jaroslav.bachorik at oracle.com > wrote: Hi Alexander, On 9.11.2015 16:40, Alexander Kulyakhtin wrote: Hi, Could you, please, review this test-only fix: CR: https://bugs.openjdk.java.net/browse/JDK-8141591 Webrev: http://cr.openjdk.java.net/~akulyakh/8141591_01/test/javax/management/remote/mandatory/threads/ExecutorTest.java.udiff.html Before the fix, it was possible that we shut down the executor before all the jobs have been submitted. This resulted in RejectedExecutionException. We are modifying the test to wait on CountDownLatch untill all the jobs have completed their execute() On L175 you should be using the unbound version of await(). It would be better to let the harness deal with the timeout. Otherwise it looks good. Cheers, -JB- Best regards, Alexander -------------- next part -------------- An HTML attachment was scrubbed... URL: From erik.joelsson at oracle.com Mon Nov 9 18:33:06 2015 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Mon, 9 Nov 2015 19:33:06 +0100 Subject: RFR: JDK-8142336: Convert the SA agent build to modular build-infra makefiles Message-ID: <5640E6E2.6090906@oracle.com> Hello, As a stepping stone in the hotspot makefile conversion, I have broken out the serviceability agent separately and converted it into proper modular build-infra makefiles. Doing this conversion separately has some value on its own by reducing the special cases currently needed for building the jdk.hotspot.agent module. The current SA java build compiles with the boot jdk javac with -source/-target JDK N-1. The proposed change instead builds SA with the interim-langtools javac for JDK N, like all the rest of the JDK classes. There is already a bug filed for reorganizing the source of the SA agent to conform to the Jigsaw style modular source layout: JDK-8067194, so I have left the source in its current location. The native compilation and linking has been changed to base the flags used on what configure sets up for the other JDK libraries. This has caused some changes in flag usage. From what I can tell, nothing important is different however. I have run the relevant jtreg tests on all OSes to verify that it still works. Some of the differences include: * Linux: "-Xlinker z -Xlinker defs" was added to LDFLAGS, which causes link failure unless "-ldl" was also added to LIBS. * Solaris: More warnings activated through "+w" caused need for disabling some warnings. I fixed one warning instance which was trivial (int->size_t), but couldn't figure out the rest. I will file a followup bug for fixing those if this patch is accepted. I tried to mimic the current behavior of excluding SA on linux-ppc and zero that Volker added a while back. Now it's excluded on the module level instead so that jdk.hotspot.agent isn't even built in that case. It would be good if this could be tested. Bug: https://bugs.openjdk.java.net/browse/JDK-8142336 Webrev: http://cr.openjdk.java.net/~erikj/8142336/webrev.01/ /Erik From rachel.protacio at oracle.com Mon Nov 9 22:55:14 2015 From: rachel.protacio at oracle.com (Rachel Protacio) Date: Mon, 9 Nov 2015 17:55:14 -0500 Subject: RFR: 8142366: Add develop_debug and develop_trace levels to Unified Logging Message-ID: <56412452.4060806@oracle.com> Hello, Please see my small changeset to add two develop levels to UL. Summary: This adds develop (that is, non-product) logging levels to the Unified Logging framework in order to support performance, footprint, and usefulness-of-output considerations while maintaining the ability for the user to specify levels of verbosity, i.e. default, "debug," and "trace" levels. Open webrev: http://cr.openjdk.java.net/~rprotacio/8142366/ Bug: https://bugs.openjdk.java.net/browse/JDK-8142366 I tested the added levels locally with sample log messages to ensure proper functioning. When I convert future tags to logging with these levels, those tags will have their own tests and inherently exercise the added levels. Thank you, Rachel From ioi.lam at oracle.com Mon Nov 9 23:49:24 2015 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 09 Nov 2015 15:49:24 -0800 Subject: RFR (S) 8079408 - Improve class loading log (with unified logging) Message-ID: <56413104.6010402@oracle.com> Please review a small fix, related to unified logging: http://cr.openjdk.java.net/~iklam/8079408-improve-class-load-log/ Bug: 8079408 Improve class loading log (with unified logging) https://bugs.openjdk.java.net/browse/JDK-8079408 Summary of fix: This fix has 2 parts: [a] Reimplement TraceClassData and TraceClassLoading as -Xlog:classloaderdata and -Xlog:classload [b] Add more detailed info to -Xlog:classload so we can perform more analysis Example: java -Xlog:classloaderdata=debug,classload=trace:file=log.txt \ -cp ~/tests Hello I have completely replaced TraceClassData with -Xlog:classloaderdata. The TraceClassData flag has been removed from globals.hpp. Since many existing tools use TraceClassLoading, I think it's better to deprecate it, but otherwise don't change how it works. Instead, we add a new -Xlog:classload flag that's independent of the old TraceClassLoading flag. (note: the deprecation of TraceClassLoading will be done in a separate bug). I have also not tied "-verbose:class" with the classload log, since I think "-verbose:class" is obsolete and should be replaced by -Xlog. Tests: RBT: I ran these testlists using RBT (more than 20000 total classes on Linux/Windows/MacOS/solaris) and saw no new failures. I think this means no existing tests depend on the removed -XX:+TraceClassLoaderData flag. hotspot/test/:hotspot_all vm.parallel_class_loading.testlist vm.regression.testlist vm.runtime.testlist vm.tmtools.testlist nsk.sajdi.testlist nsk.stress.testlist nsk.stress.jck.testlist vm.gc.testlist vm.metaspace.testlist Thanks - Ioi -------------- next part -------------- An HTML attachment was scrubbed... URL: From magnus.ihse.bursie at oracle.com Tue Nov 10 10:39:25 2015 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Tue, 10 Nov 2015 11:39:25 +0100 Subject: RFR: JDK-8142336: Convert the SA agent build to modular build-infra makefiles In-Reply-To: <5640E6E2.6090906@oracle.com> References: <5640E6E2.6090906@oracle.com> Message-ID: <5641C95D.8040103@oracle.com> On 2015-11-09 19:33, Erik Joelsson wrote: > Hello, > > As a stepping stone in the hotspot makefile conversion, I have broken > out the serviceability agent separately and converted it into proper > modular build-infra makefiles. Doing this conversion separately has > some value on its own by reducing the special cases currently needed > for building the jdk.hotspot.agent module. > > The current SA java build compiles with the boot jdk javac with > -source/-target JDK N-1. The proposed change instead builds SA with > the interim-langtools javac for JDK N, like all the rest of the JDK > classes. > > There is already a bug filed for reorganizing the source of the SA > agent to conform to the Jigsaw style modular source layout: > JDK-8067194, so I have left the source in its current location. > > The native compilation and linking has been changed to base the flags > used on what configure sets up for the other JDK libraries. This has > caused some changes in flag usage. From what I can tell, nothing > important is different however. I have run the relevant jtreg tests on > all OSes to verify that it still works. Some of the differences include: > > * Linux: "-Xlinker z -Xlinker defs" was added to LDFLAGS, which causes > link failure unless "-ldl" was also added to LIBS. > * Solaris: More warnings activated through "+w" caused need for > disabling some warnings. I fixed one warning instance which was > trivial (int->size_t), but couldn't figure out the rest. I will file a > followup bug for fixing those if this patch is accepted. > > I tried to mimic the current behavior of excluding SA on linux-ppc and > zero that Volker added a while back. Now it's excluded on the module > level instead so that jdk.hotspot.agent isn't even built in that case. > It would be good if this could be tested. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8142336 > Webrev: http://cr.openjdk.java.net/~erikj/8142336/webrev.01/ A few remarks: * Could you please document the new DISABLED_WARNINGS_CXX and DISABLED_WARNINGS_C in the function header? * I believe the use of {} here was to signify a set. When only jsig remains, it just looks strange: -# SYMFLAG is used by {jsig,saproc}.make +# SYMFLAG is used by {jsig}.make * The logic of setting up "--hash-style=both" is already done in configure for LDFLAGS_JDKLIB, so you do not need to repeat it, if you include the LDFLAGS_JDKLIB variable. Apart from this, it looks good. /Magnus From jaroslav.bachorik at oracle.com Tue Nov 10 11:10:46 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Tue, 10 Nov 2015 12:10:46 +0100 Subject: RFR: 8141591 : javax/management/remote/mandatory/threads/ExecutorTest.java fails intermittently In-Reply-To: References: Message-ID: <5641D0B6.9040905@oracle.com> On 9.11.2015 19:07, Alexander Kulyakhtin wrote: > Hi, > > Following the comments from Jaroslav and Martin I've changed the test > to use the unbound CountDownLatch.await() > > Since await() will wait undefinitely and thus the test, if fails, will > now fail by timeout only the code has been additionally simplified to > reflect that. > > Please, find the updated webrev at: > > CR: https://bugs.openjdk.java.net/browse/JDK-8141591 > RFR: > http://cr.openjdk.java.net/~akulyakh/8141591_03/test/javax/management/remote/mandatory/threads/ExecutorTest.java.udiff.html Looks good! -JB- > > Best regards, > Alexander > > ----- Original Message ----- > From: martinrb at google.com > To: jaroslav.bachorik at oracle.com > Cc: alexander.kulyakhtin at oracle.com, serviceability-dev at openjdk.java.net > Sent: Monday, November 9, 2015 7:53:08 PM GMT +03:00 Iraq > Subject: Re: RFR: 8141591 : > javax/management/remote/mandatory/threads/ExecutorTest.java fails > intermittently > > The traditional way is to have all the worker tasks count down a latch > when they're started and have the master thread wait on that before > proceeding. > > You can use test.timeout.factor to do a scaled timed wait. > > On Mon, Nov 9, 2015 at 8:03 AM, Jaroslav Bachorik > > wrote: > > Hi Alexander, > > On 9.11.2015 16:40, Alexander Kulyakhtin wrote: > > Hi, > > Could you, please, review this test-only fix: > > CR: https://bugs.openjdk.java.net/browse/JDK-8141591 > Webrev: > http://cr.openjdk.java.net/~akulyakh/8141591_01/test/javax/management/remote/mandatory/threads/ExecutorTest.java.udiff.html > > Before the fix, it was possible that we shut down the executor > before all the jobs have been submitted. This resulted in > RejectedExecutionException. > We are modifying the test to wait on CountDownLatch untill all > the jobs have completed their execute() > > > On L175 you should be using the unbound version of await(). It would > be better to let the harness deal with the timeout. > > Otherwise it looks good. > > Cheers, > > -JB- > > > Best regards, > Alexander > > > From alexander.kulyakhtin at oracle.com Tue Nov 10 11:36:53 2015 From: alexander.kulyakhtin at oracle.com (Alexander Kulyakhtin) Date: Tue, 10 Nov 2015 03:36:53 -0800 (PST) Subject: RFR: 8141591 : javax/management/remote/mandatory/threads/ExecutorTest.java fails intermittently Message-ID: Hi Jaroslav, Shanliang Jiang Thank you very much for the review. Following the comments from Shanliang Jiang on https://bugs.openjdk.java.net/browse/JDK-8141591 could it be possible to make, instead, a fix in the jdk source and leave the test unchanged as per the following webrev : http://cr.openjdk.java.net/~akulyakh/814591jdk/src/java.management/share/classes/com/sun/jmx/remote/internal/ClientNotifForwarder.java.udiff.html If these changes are problematic I will go for the test changes which you have already accepted. Best regards, Alexander ----- Original Message ----- From: jaroslav.bachorik at oracle.com To: alexander.kulyakhtin at oracle.com, serviceability-dev at openjdk.java.net Cc: martinrb at google.com Sent: Tuesday, November 10, 2015 2:10:48 PM GMT +03:00 Iraq Subject: Re: RFR: 8141591 : javax/management/remote/mandatory/threads/ExecutorTest.java fails intermittently On 9.11.2015 19:07, Alexander Kulyakhtin wrote: > Hi, > > Following the comments from Jaroslav and Martin I've changed the test > to use the unbound CountDownLatch.await() > > Since await() will wait undefinitely and thus the test, if fails, will > now fail by timeout only the code has been additionally simplified to > reflect that. > > Please, find the updated webrev at: > > CR: https://bugs.openjdk.java.net/browse/JDK-8141591 > RFR: > http://cr.openjdk.java.net/~akulyakh/8141591_03/test/javax/management/remote/mandatory/threads/ExecutorTest.java.udiff.html Looks good! -JB- > > Best regards, > Alexander > > ----- Original Message ----- > From: martinrb at google.com > To: jaroslav.bachorik at oracle.com > Cc: alexander.kulyakhtin at oracle.com, serviceability-dev at openjdk.java.net > Sent: Monday, November 9, 2015 7:53:08 PM GMT +03:00 Iraq > Subject: Re: RFR: 8141591 : > javax/management/remote/mandatory/threads/ExecutorTest.java fails > intermittently > > The traditional way is to have all the worker tasks count down a latch > when they're started and have the master thread wait on that before > proceeding. > > You can use test.timeout.factor to do a scaled timed wait. > > On Mon, Nov 9, 2015 at 8:03 AM, Jaroslav Bachorik > > wrote: > > Hi Alexander, > > On 9.11.2015 16:40, Alexander Kulyakhtin wrote: > > Hi, > > Could you, please, review this test-only fix: > > CR: https://bugs.openjdk.java.net/browse/JDK-8141591 > Webrev: > http://cr.openjdk.java.net/~akulyakh/8141591_01/test/javax/management/remote/mandatory/threads/ExecutorTest.java.udiff.html > > Before the fix, it was possible that we shut down the executor > before all the jobs have been submitted. This resulted in > RejectedExecutionException. > We are modifying the test to wait on CountDownLatch untill all > the jobs have completed their execute() > > > On L175 you should be using the unbound version of await(). It would > be better to let the harness deal with the timeout. > > Otherwise it looks good. > > Cheers, > > -JB- > > > Best regards, > Alexander > > > From shanliang.jiang at oracle.com Tue Nov 10 12:35:03 2015 From: shanliang.jiang at oracle.com (Shanliang JIANG) Date: Tue, 10 Nov 2015 13:35:03 +0100 Subject: RFR: 8141591 : javax/management/remote/mandatory/threads/ExecutorTest.java fails intermittently In-Reply-To: References: Message-ID: <7674A3F3-A954-4CE8-8FBF-E318559CF37A@oracle.com> When the executor is shutdown, the client and server were closed, so the test should not receive any exception. The better solution is fix in com.sun.jmx.remote.internal.ClientNotifForwarder Line 829: synchronized (ClientNotifForwarder.this) { if (state == STARTED) { executor.execute(new NotifFetcher()); } } Thanks to work on this issue. Shanliang > On 10 Nov 2015, at 12:36, Alexander Kulyakhtin wrote: > > Hi Jaroslav, Shanliang Jiang > > Thank you very much for the review. > > Following the comments from Shanliang Jiang on https://bugs.openjdk.java.net/browse/JDK-8141591 could it be possible to make, instead, a fix in the jdk source and leave the test unchanged > > as per the following webrev : http://cr.openjdk.java.net/~akulyakh/814591jdk/src/java.management/share/classes/com/sun/jmx/remote/internal/ClientNotifForwarder.java.udiff.html > > If these changes are problematic I will go for the test changes which you have already accepted. > > Best regards, > Alexander > > ----- Original Message ----- > From: jaroslav.bachorik at oracle.com > To: alexander.kulyakhtin at oracle.com, serviceability-dev at openjdk.java.net > Cc: martinrb at google.com > Sent: Tuesday, November 10, 2015 2:10:48 PM GMT +03:00 Iraq > Subject: Re: RFR: 8141591 : javax/management/remote/mandatory/threads/ExecutorTest.java fails intermittently > > On 9.11.2015 19:07, Alexander Kulyakhtin wrote: >> Hi, >> >> Following the comments from Jaroslav and Martin I've changed the test >> to use the unbound CountDownLatch.await() >> >> Since await() will wait undefinitely and thus the test, if fails, will >> now fail by timeout only the code has been additionally simplified to >> reflect that. >> >> Please, find the updated webrev at: >> >> CR: https://bugs.openjdk.java.net/browse/JDK-8141591 >> RFR: >> http://cr.openjdk.java.net/~akulyakh/8141591_03/test/javax/management/remote/mandatory/threads/ExecutorTest.java.udiff.html > > Looks good! > > -JB- > >> >> Best regards, >> Alexander >> >> ----- Original Message ----- >> From: martinrb at google.com >> To: jaroslav.bachorik at oracle.com >> Cc: alexander.kulyakhtin at oracle.com, serviceability-dev at openjdk.java.net >> Sent: Monday, November 9, 2015 7:53:08 PM GMT +03:00 Iraq >> Subject: Re: RFR: 8141591 : >> javax/management/remote/mandatory/threads/ExecutorTest.java fails >> intermittently >> >> The traditional way is to have all the worker tasks count down a latch >> when they're started and have the master thread wait on that before >> proceeding. >> >> You can use test.timeout.factor to do a scaled timed wait. >> >> On Mon, Nov 9, 2015 at 8:03 AM, Jaroslav Bachorik >> > wrote: >> >> Hi Alexander, >> >> On 9.11.2015 16:40, Alexander Kulyakhtin wrote: >> >> Hi, >> >> Could you, please, review this test-only fix: >> >> CR: https://bugs.openjdk.java.net/browse/JDK-8141591 >> Webrev: >> http://cr.openjdk.java.net/~akulyakh/8141591_01/test/javax/management/remote/mandatory/threads/ExecutorTest.java.udiff.html >> >> Before the fix, it was possible that we shut down the executor >> before all the jobs have been submitted. This resulted in >> RejectedExecutionException. >> We are modifying the test to wait on CountDownLatch untill all >> the jobs have completed their execute() >> >> >> On L175 you should be using the unbound version of await(). It would >> be better to let the harness deal with the timeout. >> >> Otherwise it looks good. >> >> Cheers, >> >> -JB- >> >> >> Best regards, >> Alexander >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.kulyakhtin at oracle.com Tue Nov 10 12:53:11 2015 From: alexander.kulyakhtin at oracle.com (Alexander Kulyakhtin) Date: Tue, 10 Nov 2015 04:53:11 -0800 (PST) Subject: RFR: 8141591 : javax/management/remote/mandatory/threads/ExecutorTest.java fails intermittently Message-ID: Based on the discussion in the group and in the CR comments I'm inclined to think this is not a test issue, as was initially treated but a JDK issue. The proper solution is still being discussed. I would suggest treating the issue as a JDK bug, possibly reassigning to an engineer familiar with this part of the JDK code. Best regards, Alexander ----- Original Message ----- From: shanliang.jiang at oracle.com To: alexander.kulyakhtin at oracle.com Cc: jaroslav.bachorik at oracle.com, martinrb at google.com, serviceability-dev at openjdk.java.net Sent: Tuesday, November 10, 2015 3:35:06 PM GMT +03:00 Iraq Subject: Re: RFR: 8141591 : javax/management/remote/mandatory/threads/ExecutorTest.java fails intermittently When the executor is shutdown, the client and server were closed, so the test should not receive any exception. The better solution is fix in com.sun.jmx.remote.internal.ClientNotifForwarder Line 829: synchronized (ClientNotifForwarder.this) { if (state == STARTED) { executor.execute(new NotifFetcher()); } } Thanks to work on this issue. Shanliang On 10 Nov 2015, at 12:36, Alexander Kulyakhtin < alexander.kulyakhtin at oracle.com > wrote: Hi Jaroslav, Shanliang Jiang Thank you very much for the review. Following the comments from Shanliang Jiang on https://bugs.openjdk.java.net/browse/JDK-8141591 could it be possible to make, instead, a fix in the jdk source and leave the test unchanged as per the following webrev : http://cr.openjdk.java.net/~akulyakh/814591jdk/src/java.management/share/classes/com/sun/jmx/remote/internal/ClientNotifForwarder.java.udiff.html If these changes are problematic I will go for the test changes which you have already accepted. Best regards, Alexander ----- Original Message ----- From: jaroslav.bachorik at oracle.com To: alexander.kulyakhtin at oracle.com , serviceability-dev at openjdk.java.net Cc: martinrb at google.com Sent: Tuesday, November 10, 2015 2:10:48 PM GMT +03:00 Iraq Subject: Re: RFR: 8141591 : javax/management/remote/mandatory/threads/ExecutorTest.java fails intermittently On 9.11.2015 19:07, Alexander Kulyakhtin wrote: Hi, Following the comments from Jaroslav and Martin I've changed the test to use the unbound CountDownLatch.await() Since await() will wait undefinitely and thus the test, if fails, will now fail by timeout only the code has been additionally simplified to reflect that. Please, find the updated webrev at: CR: https://bugs.openjdk.java.net/browse/JDK-8141591 RFR: http://cr.openjdk.java.net/~akulyakh/8141591_03/test/javax/management/remote/mandatory/threads/ExecutorTest.java.udiff.html Looks good! -JB- Best regards, Alexander ----- Original Message ----- From: martinrb at google.com To: jaroslav.bachorik at oracle.com Cc: alexander.kulyakhtin at oracle.com , serviceability-dev at openjdk.java.net Sent: Monday, November 9, 2015 7:53:08 PM GMT +03:00 Iraq Subject: Re: RFR: 8141591 : javax/management/remote/mandatory/threads/ExecutorTest.java fails intermittently The traditional way is to have all the worker tasks count down a latch when they're started and have the master thread wait on that before proceeding. You can use test.timeout.factor to do a scaled timed wait. On Mon, Nov 9, 2015 at 8:03 AM, Jaroslav Bachorik < jaroslav.bachorik at oracle.com < mailto:jaroslav.bachorik at oracle.com >> wrote: Hi Alexander, On 9.11.2015 16:40, Alexander Kulyakhtin wrote: Hi, Could you, please, review this test-only fix: CR: https://bugs.openjdk.java.net/browse/JDK-8141591 Webrev: http://cr.openjdk.java.net/~akulyakh/8141591_01/test/javax/management/remote/mandatory/threads/ExecutorTest.java.udiff.html Before the fix, it was possible that we shut down the executor before all the jobs have been submitted. This resulted in RejectedExecutionException. We are modifying the test to wait on CountDownLatch untill all the jobs have completed their execute() On L175 you should be using the unbound version of await(). It would be better to let the harness deal with the timeout. Otherwise it looks good. Cheers, -JB- Best regards, Alexander -------------- next part -------------- An HTML attachment was scrubbed... URL: From magnus.ihse.bursie at oracle.com Tue Nov 10 13:49:07 2015 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Tue, 10 Nov 2015 14:49:07 +0100 Subject: RFR: JDK-8142336: Convert the SA agent build to modular build-infra makefiles In-Reply-To: <5641C95D.8040103@oracle.com> References: <5640E6E2.6090906@oracle.com> <5641C95D.8040103@oracle.com> Message-ID: <5641F5D3.1000105@oracle.com> On 2015-11-10 11:39, Magnus Ihse Bursie wrote: > On 2015-11-09 19:33, Erik Joelsson wrote: >> Hello, >> >> As a stepping stone in the hotspot makefile conversion, I have broken >> out the serviceability agent separately and converted it into proper >> modular build-infra makefiles. Doing this conversion separately has >> some value on its own by reducing the special cases currently needed >> for building the jdk.hotspot.agent module. >> >> The current SA java build compiles with the boot jdk javac with >> -source/-target JDK N-1. The proposed change instead builds SA with >> the interim-langtools javac for JDK N, like all the rest of the JDK >> classes. >> >> There is already a bug filed for reorganizing the source of the SA >> agent to conform to the Jigsaw style modular source layout: >> JDK-8067194, so I have left the source in its current location. >> >> The native compilation and linking has been changed to base the flags >> used on what configure sets up for the other JDK libraries. This has >> caused some changes in flag usage. From what I can tell, nothing >> important is different however. I have run the relevant jtreg tests >> on all OSes to verify that it still works. Some of the differences >> include: >> >> * Linux: "-Xlinker z -Xlinker defs" was added to LDFLAGS, which >> causes link failure unless "-ldl" was also added to LIBS. >> * Solaris: More warnings activated through "+w" caused need for >> disabling some warnings. I fixed one warning instance which was >> trivial (int->size_t), but couldn't figure out the rest. I will file >> a followup bug for fixing those if this patch is accepted. >> >> I tried to mimic the current behavior of excluding SA on linux-ppc >> and zero that Volker added a while back. Now it's excluded on the >> module level instead so that jdk.hotspot.agent isn't even built in >> that case. It would be good if this could be tested. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8142336 >> Webrev: http://cr.openjdk.java.net/~erikj/8142336/webrev.01/ > > A few remarks: > > * Could you please document the new DISABLED_WARNINGS_CXX and > DISABLED_WARNINGS_C in the function header? > > * I believe the use of {} here was to signify a set. When only jsig > remains, it just looks strange: > -# SYMFLAG is used by {jsig,saproc}.make > +# SYMFLAG is used by {jsig}.make > > * The logic of setting up "--hash-style=both" is already done in > configure for LDFLAGS_JDKLIB, so you do not need to repeat it, if you > include the LDFLAGS_JDKLIB variable. Also, SA_WINDOWS_LDFLAGS is read but never defined. /Magnus From coleen.phillimore at oracle.com Tue Nov 10 23:13:04 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 10 Nov 2015 18:13:04 -0500 Subject: RFR: 8142366: Add develop_debug and develop_trace levels to Unified Logging In-Reply-To: <5642430A.6070500@oracle.com> References: <56412452.4060806@oracle.com> <564126B1.2020900@oracle.com> <5642430A.6070500@oracle.com> Message-ID: <56427A00.3090904@oracle.com> I think this looks good. I added serviceability-dev to the mailing list. Coleen On 11/10/15 2:18 PM, Rachel Protacio wrote: > Good point. I have changed Develop to DevelopInfo. I think it's better > to maintain the same levels of levels on the develop side so that the > logging can just shift laterally to non-product, as it were. > > And yes, they are specified as -Xlog:=develop_debug > > Updated webrev: http://cr.openjdk.java.net/~rprotacio/8142366.01/ > I re-tested with logging jtreg tests and ExecuteInternalVMTests test, > and for anyone who was curious (namely Max who had the good idea for > me to check), the debug levels do correctly "nest" with more verbose > ones printing out the less verbose and product mode logging as well. > > Thanks, > Rachel > > On 11/9/2015 6:05 PM, Coleen Phillimore wrote: >> >> Hi Rachel, >> >> I sort of thought DevelopDebug would replace plain Develop, and >> DevelopTrace would be additional lower level. Maybe plain Develop >> should be DevelopInfo then? >> >> Another question - how do you specify these levels? Is it >> -Xlog:itables=develop_debug ? >> >> Thanks, >> Coleen >> >> On 11/9/15 5:55 PM, Rachel Protacio wrote: >>> Hello, >>> >>> Please see my small changeset to add two develop levels to UL. >>> >>> Summary: This adds develop (that is, non-product) logging levels to >>> the Unified Logging framework in order to support performance, >>> footprint, and usefulness-of-output considerations while maintaining >>> the ability for the user to specify levels of verbosity, i.e. >>> default, "debug," and "trace" levels. >>> Open webrev: http://cr.openjdk.java.net/~rprotacio/8142366/ >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8142366 >>> >>> I tested the added levels locally with sample log messages to ensure >>> proper functioning. When I convert future tags to logging with these >>> levels, those tags will have their own tests and inherently exercise >>> the added levels. >>> >>> Thank you, >>> Rachel >> > From erik.joelsson at oracle.com Wed Nov 11 09:31:29 2015 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Wed, 11 Nov 2015 10:31:29 +0100 Subject: RFR: JDK-8142336: Convert the SA agent build to modular build-infra makefiles In-Reply-To: <5641F5D3.1000105@oracle.com> References: <5640E6E2.6090906@oracle.com> <5641C95D.8040103@oracle.com> <5641F5D3.1000105@oracle.com> Message-ID: <56430AF1.2050109@oracle.com> New webrev: http://cr.openjdk.java.net/~erikj/8142336/webrev.02/ Fixed the issues listed below. Reverted the faulty attempt at fixing a warning. Did a more thorough attempt at clearing out all references to SA in the old makefiles. /Erik On 2015-11-10 14:49, Magnus Ihse Bursie wrote: > On 2015-11-10 11:39, Magnus Ihse Bursie wrote: >> On 2015-11-09 19:33, Erik Joelsson wrote: >>> Hello, >>> >>> As a stepping stone in the hotspot makefile conversion, I have >>> broken out the serviceability agent separately and converted it into >>> proper modular build-infra makefiles. Doing this conversion >>> separately has some value on its own by reducing the special cases >>> currently needed for building the jdk.hotspot.agent module. >>> >>> The current SA java build compiles with the boot jdk javac with >>> -source/-target JDK N-1. The proposed change instead builds SA with >>> the interim-langtools javac for JDK N, like all the rest of the JDK >>> classes. >>> >>> There is already a bug filed for reorganizing the source of the SA >>> agent to conform to the Jigsaw style modular source layout: >>> JDK-8067194, so I have left the source in its current location. >>> >>> The native compilation and linking has been changed to base the >>> flags used on what configure sets up for the other JDK libraries. >>> This has caused some changes in flag usage. From what I can tell, >>> nothing important is different however. I have run the relevant >>> jtreg tests on all OSes to verify that it still works. Some of the >>> differences include: >>> >>> * Linux: "-Xlinker z -Xlinker defs" was added to LDFLAGS, which >>> causes link failure unless "-ldl" was also added to LIBS. >>> * Solaris: More warnings activated through "+w" caused need for >>> disabling some warnings. I fixed one warning instance which was >>> trivial (int->size_t), but couldn't figure out the rest. I will file >>> a followup bug for fixing those if this patch is accepted. >>> >>> I tried to mimic the current behavior of excluding SA on linux-ppc >>> and zero that Volker added a while back. Now it's excluded on the >>> module level instead so that jdk.hotspot.agent isn't even built in >>> that case. It would be good if this could be tested. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8142336 >>> Webrev: http://cr.openjdk.java.net/~erikj/8142336/webrev.01/ >> >> A few remarks: >> >> * Could you please document the new DISABLED_WARNINGS_CXX and >> DISABLED_WARNINGS_C in the function header? >> >> * I believe the use of {} here was to signify a set. When only jsig >> remains, it just looks strange: >> -# SYMFLAG is used by {jsig,saproc}.make >> +# SYMFLAG is used by {jsig}.make >> >> * The logic of setting up "--hash-style=both" is already done in >> configure for LDFLAGS_JDKLIB, so you do not need to repeat it, if you >> include the LDFLAGS_JDKLIB variable. > > Also, SA_WINDOWS_LDFLAGS is read but never defined. > > /Magnus From marcus.larsson at oracle.com Wed Nov 11 14:01:10 2015 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Wed, 11 Nov 2015 15:01:10 +0100 Subject: RFR: 8142366: Add develop_debug and develop_trace levels to Unified Logging In-Reply-To: <56427A00.3090904@oracle.com> References: <56412452.4060806@oracle.com> <564126B1.2020900@oracle.com> <5642430A.6070500@oracle.com> <56427A00.3090904@oracle.com> Message-ID: <56434A26.9020309@oracle.com> Hi, I don't think this is the right approach for this problem. These new develop levels are introduced as even finer levels than the trace, but have names that somewhat say otherwise. For example, develop_info is finer than trace, but it isn't unreasonable to expect that develop_info actually shares verbosity with the regular info level. I see two ways to look at this: 1) develop is a level, which is so verbose/expensive that it is not possible to include it in the product (current implementation). The following levels are used: error, warning, info, debug, trace and develop. There is no concept of levels for the develop only logging (it IS the level). Selecting/distinguishing different logging on develop level is done by using tags. 2) There is no concept of a develop level, either the logging is included in product or it isn't (compare to using #ifdefs around the logging calls). Development-only logging is orthogonal to the regular log levels, i.e. some log_info calls should only be made in non-product builds (log_develop_info), but should still follow the regular log level system (as all other logging does) and use the info level. The following levels are used: error, warning, info, debug, trace, and convenience macros are used to strip develop-logging from non-product builds. The downside of this approach is that it becomes harder to distinguish what is logged only in non-product builds and what is always logged. Depending on if the build is non-product or not one can get different output for "-Xlog". The current proposal mixes these two approaches and adds develop "levels" that are all finer than trace, which is confusing. In my opinion, there should be no reason to require multiple levels for non-product logging. I would assume all develop level logging to be very specific, i.e. have a very specific tagset, and require no support for selection/separation using log levels. If it does, then I would argue for that the logging is missing some classifying log tag that could be used instead of a level. The logging has already been classified as verbose/expensive, why classify it further on a level scale? Thanks, Marcus On 2015-11-11 00:13, Coleen Phillimore wrote: > > I think this looks good. I added serviceability-dev to the mailing > list. > > Coleen > > On 11/10/15 2:18 PM, Rachel Protacio wrote: >> Good point. I have changed Develop to DevelopInfo. I think it's >> better to maintain the same levels of levels on the develop side so >> that the logging can just shift laterally to non-product, as it were. >> >> And yes, they are specified as -Xlog:=develop_debug >> >> Updated webrev: http://cr.openjdk.java.net/~rprotacio/8142366.01/ >> I re-tested with logging jtreg tests and ExecuteInternalVMTests test, >> and for anyone who was curious (namely Max who had the good idea for >> me to check), the debug levels do correctly "nest" with more verbose >> ones printing out the less verbose and product mode logging as well. >> >> Thanks, >> Rachel >> >> On 11/9/2015 6:05 PM, Coleen Phillimore wrote: >>> >>> Hi Rachel, >>> >>> I sort of thought DevelopDebug would replace plain Develop, and >>> DevelopTrace would be additional lower level. Maybe plain Develop >>> should be DevelopInfo then? >>> >>> Another question - how do you specify these levels? Is it >>> -Xlog:itables=develop_debug ? >>> >>> Thanks, >>> Coleen >>> >>> On 11/9/15 5:55 PM, Rachel Protacio wrote: >>>> Hello, >>>> >>>> Please see my small changeset to add two develop levels to UL. >>>> >>>> Summary: This adds develop (that is, non-product) logging levels to >>>> the Unified Logging framework in order to support performance, >>>> footprint, and usefulness-of-output considerations while >>>> maintaining the ability for the user to specify levels of >>>> verbosity, i.e. default, "debug," and "trace" levels. >>>> Open webrev: http://cr.openjdk.java.net/~rprotacio/8142366/ >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8142366 >>>> >>>> I tested the added levels locally with sample log messages to >>>> ensure proper functioning. When I convert future tags to logging >>>> with these levels, those tags will have their own tests and >>>> inherently exercise the added levels. >>>> >>>> Thank you, >>>> Rachel >>> >> > From marcus.larsson at oracle.com Wed Nov 11 14:19:07 2015 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Wed, 11 Nov 2015 15:19:07 +0100 Subject: RFR (S): 8142482: Improve the support for prefix functions in unified logging Message-ID: <56434E5B.8010703@oracle.com> Hi, Please review the following small patch to improve the support for prefixes in unified logging. Instead of the previous format and value/callback, a prefix is now defined as a function that receives the log buffer as argument, allowing it to fill in an arbitrary (or empty!) prefix. (This enables GCId-prefixing to be skipped when GCId is undefined, for example.) Webrev: http://cr.openjdk.java.net/~mlarsson/8142482/webrev.00/ Issue: https://bugs.openjdk.java.net/browse/JDK-8142482 Thanks, Marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: From jaroslav.bachorik at oracle.com Wed Nov 11 15:52:43 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Wed, 11 Nov 2015 16:52:43 +0100 Subject: RFR 8142398: ManagementAgent.status diagnostic command only outputs the specifically set properties Message-ID: <5643644B.2080102@oracle.com> Please, review the following change Issue : https://bugs.openjdk.java.net/browse/JDK-8142398 Webrev: http://cr.openjdk.java.net/~jbachorik/8142398/webrev.00 This patch adds the list of the default agent properties with the info whether the default value has been overridden to the ManagementAgent.status DCMD output. Thanks, -JB- From magnus.ihse.bursie at oracle.com Wed Nov 11 16:43:17 2015 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Wed, 11 Nov 2015 17:43:17 +0100 Subject: RFR: JDK-8142336: Convert the SA agent build to modular build-infra makefiles In-Reply-To: <56430AF1.2050109@oracle.com> References: <5640E6E2.6090906@oracle.com> <5641C95D.8040103@oracle.com> <5641F5D3.1000105@oracle.com> <56430AF1.2050109@oracle.com> Message-ID: <56437025.4070504@oracle.com> On 2015-11-11 10:31, Erik Joelsson wrote: > New webrev: http://cr.openjdk.java.net/~erikj/8142336/webrev.02/ > > Fixed the issues listed below. Reverted the faulty attempt at fixing a > warning. Did a more thorough attempt at clearing out all references to > SA in the old makefiles. Looks great. There was a few lines in hotspot/make/bsd/makefiles/defs.make where you seem to have accidentally broken the indentation. Apart from this it looks good. If you just restore the indentation I'm okay (you don't need to respin the webrev for that). /Magnus > > /Erik > > On 2015-11-10 14:49, Magnus Ihse Bursie wrote: >> On 2015-11-10 11:39, Magnus Ihse Bursie wrote: >>> On 2015-11-09 19:33, Erik Joelsson wrote: >>>> Hello, >>>> >>>> As a stepping stone in the hotspot makefile conversion, I have >>>> broken out the serviceability agent separately and converted it >>>> into proper modular build-infra makefiles. Doing this conversion >>>> separately has some value on its own by reducing the special cases >>>> currently needed for building the jdk.hotspot.agent module. >>>> >>>> The current SA java build compiles with the boot jdk javac with >>>> -source/-target JDK N-1. The proposed change instead builds SA with >>>> the interim-langtools javac for JDK N, like all the rest of the JDK >>>> classes. >>>> >>>> There is already a bug filed for reorganizing the source of the SA >>>> agent to conform to the Jigsaw style modular source layout: >>>> JDK-8067194, so I have left the source in its current location. >>>> >>>> The native compilation and linking has been changed to base the >>>> flags used on what configure sets up for the other JDK libraries. >>>> This has caused some changes in flag usage. From what I can tell, >>>> nothing important is different however. I have run the relevant >>>> jtreg tests on all OSes to verify that it still works. Some of the >>>> differences include: >>>> >>>> * Linux: "-Xlinker z -Xlinker defs" was added to LDFLAGS, which >>>> causes link failure unless "-ldl" was also added to LIBS. >>>> * Solaris: More warnings activated through "+w" caused need for >>>> disabling some warnings. I fixed one warning instance which was >>>> trivial (int->size_t), but couldn't figure out the rest. I will >>>> file a followup bug for fixing those if this patch is accepted. >>>> >>>> I tried to mimic the current behavior of excluding SA on linux-ppc >>>> and zero that Volker added a while back. Now it's excluded on the >>>> module level instead so that jdk.hotspot.agent isn't even built in >>>> that case. It would be good if this could be tested. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8142336 >>>> Webrev: http://cr.openjdk.java.net/~erikj/8142336/webrev.01/ >>> >>> A few remarks: >>> >>> * Could you please document the new DISABLED_WARNINGS_CXX and >>> DISABLED_WARNINGS_C in the function header? >>> >>> * I believe the use of {} here was to signify a set. When only jsig >>> remains, it just looks strange: >>> -# SYMFLAG is used by {jsig,saproc}.make >>> +# SYMFLAG is used by {jsig}.make >>> >>> * The logic of setting up "--hash-style=both" is already done in >>> configure for LDFLAGS_JDKLIB, so you do not need to repeat it, if >>> you include the LDFLAGS_JDKLIB variable. >> >> Also, SA_WINDOWS_LDFLAGS is read but never defined. >> >> /Magnus > From bengt.rutisson at oracle.com Wed Nov 11 16:34:49 2015 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Wed, 11 Nov 2015 17:34:49 +0100 Subject: RFR (S): 8142482: Improve the support for prefix functions in unified logging In-Reply-To: <56434E5B.8010703@oracle.com> References: <56434E5B.8010703@oracle.com> Message-ID: <56436E29.2070308@oracle.com> Hi Marcus. On 2015-11-11 15:19, Marcus Larsson wrote: > Hi, > > Please review the following small patch to improve the support for > prefixes in unified logging. > > Instead of the previous format and value/callback, a prefix is now > defined as a function that receives the log buffer as argument, > allowing it to fill in an arbitrary (or empty!) prefix. (This enables > GCId-prefixing to be skipped when GCId is undefined, for example.) > > Webrev: > http://cr.openjdk.java.net/~mlarsson/8142482/webrev.00/ > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8142482 Looks good. Thanks for fixing this! Bengt > > Thanks, > Marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmitry.samersoff at oracle.com Wed Nov 11 16:58:00 2015 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Wed, 11 Nov 2015 19:58:00 +0300 Subject: RFR: JDK-8142336: Convert the SA agent build to modular build-infra makefiles In-Reply-To: <56430AF1.2050109@oracle.com> References: <5640E6E2.6090906@oracle.com> <5641C95D.8040103@oracle.com> <5641F5D3.1000105@oracle.com> <56430AF1.2050109@oracle.com> Message-ID: <56437398.7030502@oracle.com> Erik, Could you please file a new CR for warnings and assign it to me? -Dmitry On 2015-11-11 12:31, Erik Joelsson wrote: > New webrev: http://cr.openjdk.java.net/~erikj/8142336/webrev.02/ > > Fixed the issues listed below. Reverted the faulty attempt at fixing a > warning. Did a more thorough attempt at clearing out all references to > SA in the old makefiles. > > /Erik > > On 2015-11-10 14:49, Magnus Ihse Bursie wrote: >> On 2015-11-10 11:39, Magnus Ihse Bursie wrote: >>> On 2015-11-09 19:33, Erik Joelsson wrote: >>>> Hello, >>>> >>>> As a stepping stone in the hotspot makefile conversion, I have >>>> broken out the serviceability agent separately and converted it into >>>> proper modular build-infra makefiles. Doing this conversion >>>> separately has some value on its own by reducing the special cases >>>> currently needed for building the jdk.hotspot.agent module. >>>> >>>> The current SA java build compiles with the boot jdk javac with >>>> -source/-target JDK N-1. The proposed change instead builds SA with >>>> the interim-langtools javac for JDK N, like all the rest of the JDK >>>> classes. >>>> >>>> There is already a bug filed for reorganizing the source of the SA >>>> agent to conform to the Jigsaw style modular source layout: >>>> JDK-8067194, so I have left the source in its current location. >>>> >>>> The native compilation and linking has been changed to base the >>>> flags used on what configure sets up for the other JDK libraries. >>>> This has caused some changes in flag usage. From what I can tell, >>>> nothing important is different however. I have run the relevant >>>> jtreg tests on all OSes to verify that it still works. Some of the >>>> differences include: >>>> >>>> * Linux: "-Xlinker z -Xlinker defs" was added to LDFLAGS, which >>>> causes link failure unless "-ldl" was also added to LIBS. >>>> * Solaris: More warnings activated through "+w" caused need for >>>> disabling some warnings. I fixed one warning instance which was >>>> trivial (int->size_t), but couldn't figure out the rest. I will file >>>> a followup bug for fixing those if this patch is accepted. >>>> >>>> I tried to mimic the current behavior of excluding SA on linux-ppc >>>> and zero that Volker added a while back. Now it's excluded on the >>>> module level instead so that jdk.hotspot.agent isn't even built in >>>> that case. It would be good if this could be tested. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8142336 >>>> Webrev: http://cr.openjdk.java.net/~erikj/8142336/webrev.01/ >>> >>> A few remarks: >>> >>> * Could you please document the new DISABLED_WARNINGS_CXX and >>> DISABLED_WARNINGS_C in the function header? >>> >>> * I believe the use of {} here was to signify a set. When only jsig >>> remains, it just looks strange: >>> -# SYMFLAG is used by {jsig,saproc}.make >>> +# SYMFLAG is used by {jsig}.make >>> >>> * The logic of setting up "--hash-style=both" is already done in >>> configure for LDFLAGS_JDKLIB, so you do not need to repeat it, if you >>> include the LDFLAGS_JDKLIB variable. >> >> Also, SA_WINDOWS_LDFLAGS is read but never defined. >> >> /Magnus > -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From magnus.ihse.bursie at oracle.com Wed Nov 11 18:57:30 2015 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Wed, 11 Nov 2015 19:57:30 +0100 Subject: Dtrace on macosx? Message-ID: <56438F9A.6050007@oracle.com> Hi, In the hotspot build conversion project, I've started to analyze the dtrace build on macosx. From what I can understand, the only thing that are done on macosx is the dtrace_gen_headers target in dtrace.make, which creates hotspot.h, hotspot_jni.h and hs_private.h. No libraries seems to be build, so most of dtrace.make in bsd is dead code. The comment # Bsd does not build libjvm_db, does not compile on macosx # disabled in build: rule in vm.make has been there since the initial integration of the macosx port. My question is: are the generated headers still useful on their own? Or are they just remnants of an old but ultimately failed effort to get dtrace working on macosx? /Magnus From bengt.rutisson at oracle.com Wed Nov 11 19:25:57 2015 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Wed, 11 Nov 2015 20:25:57 +0100 Subject: RFR: 8142366: Add develop_debug and develop_trace levels to Unified Logging In-Reply-To: <56434A26.9020309@oracle.com> References: <56412452.4060806@oracle.com> <564126B1.2020900@oracle.com> <5642430A.6070500@oracle.com> <56427A00.3090904@oracle.com> <56434A26.9020309@oracle.com> Message-ID: <56439645.2040706@oracle.com> Hi all, I agree with Marcus here. Adding more levels below the Trace level seems odd. I interpret Marcus' suggestion 2) as that there would be for example a log_develop_trace() macro that does nothing in product builds and that calls log_trace() in debug builds. Similarly for info and debug. To me that seems like a reasonable approach. But I do see the problem that Marcus mentions. That it will be hard to distinguish develop logging from normal logging in the output. So far it's been enough for me with the single Develop level. Can you give an example of a use case where the different develop levels are needed? Thanks, Bengt On 2015-11-11 15:01, Marcus Larsson wrote: > Hi, > > I don't think this is the right approach for this problem. > > These new develop levels are introduced as even finer levels than the > trace, but have names that somewhat say otherwise. > For example, develop_info is finer than trace, but it isn't > unreasonable to expect that develop_info actually shares verbosity > with the regular info level. > > > I see two ways to look at this: > > 1) develop is a level, which is so verbose/expensive that it is not > possible to include it in the product (current implementation). > The following levels are used: error, warning, info, debug, trace and > develop. There is no concept of levels for the develop only logging > (it IS the level). > Selecting/distinguishing different logging on develop level is done by > using tags. > > 2) There is no concept of a develop level, either the logging is > included in product or it isn't (compare to using #ifdefs around the > logging calls). > Development-only logging is orthogonal to the regular log levels, i.e. > some log_info calls should only be made in non-product builds > (log_develop_info), > but should still follow the regular log level system (as all other > logging does) and use the info level. > The following levels are used: error, warning, info, debug, trace, and > convenience macros are used to strip develop-logging from non-product > builds. > > The downside of this approach is that it becomes harder to distinguish > what is logged only in non-product builds and what is always logged. > Depending on if the build is non-product or not one can get different > output for "-Xlog". > > > The current proposal mixes these two approaches and adds develop > "levels" that are all finer than trace, which is confusing. > > In my opinion, there should be no reason to require multiple levels > for non-product logging. I would assume all develop level logging to > be very specific, i.e. have a very specific tagset, > and require no support for selection/separation using log levels. If > it does, then I would argue for that the logging is missing some > classifying log tag that could be used instead of a level. > The logging has already been classified as verbose/expensive, why > classify it further on a level scale? > > Thanks, > Marcus > > > On 2015-11-11 00:13, Coleen Phillimore wrote: >> >> I think this looks good. I added serviceability-dev to the mailing >> list. >> >> Coleen >> >> On 11/10/15 2:18 PM, Rachel Protacio wrote: >>> Good point. I have changed Develop to DevelopInfo. I think it's >>> better to maintain the same levels of levels on the develop side so >>> that the logging can just shift laterally to non-product, as it were. >>> >>> And yes, they are specified as -Xlog:=develop_debug >>> >>> Updated webrev: http://cr.openjdk.java.net/~rprotacio/8142366.01/ >>> I re-tested with logging jtreg tests and ExecuteInternalVMTests >>> test, and for anyone who was curious (namely Max who had the good >>> idea for me to check), the debug levels do correctly "nest" with >>> more verbose ones printing out the less verbose and product mode >>> logging as well. >>> >>> Thanks, >>> Rachel >>> >>> On 11/9/2015 6:05 PM, Coleen Phillimore wrote: >>>> >>>> Hi Rachel, >>>> >>>> I sort of thought DevelopDebug would replace plain Develop, and >>>> DevelopTrace would be additional lower level. Maybe plain Develop >>>> should be DevelopInfo then? >>>> >>>> Another question - how do you specify these levels? Is it >>>> -Xlog:itables=develop_debug ? >>>> >>>> Thanks, >>>> Coleen >>>> >>>> On 11/9/15 5:55 PM, Rachel Protacio wrote: >>>>> Hello, >>>>> >>>>> Please see my small changeset to add two develop levels to UL. >>>>> >>>>> Summary: This adds develop (that is, non-product) logging levels >>>>> to the Unified Logging framework in order to support performance, >>>>> footprint, and usefulness-of-output considerations while >>>>> maintaining the ability for the user to specify levels of >>>>> verbosity, i.e. default, "debug," and "trace" levels. >>>>> Open webrev: http://cr.openjdk.java.net/~rprotacio/8142366/ >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8142366 >>>>> >>>>> I tested the added levels locally with sample log messages to >>>>> ensure proper functioning. When I convert future tags to logging >>>>> with these levels, those tags will have their own tests and >>>>> inherently exercise the added levels. >>>>> >>>>> Thank you, >>>>> Rachel >>>> >>> >> > From serguei.spitsyn at oracle.com Thu Nov 12 00:21:27 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 11 Nov 2015 16:21:27 -0800 Subject: Dtrace on macosx? In-Reply-To: <56438F9A.6050007@oracle.com> References: <56438F9A.6050007@oracle.com> Message-ID: <5643DB87.9020905@oracle.com> Hi Magnus, I'll try to provide the details I know. The libjvm_db is to support Solaris pstack utility that prints mixed java+native stacks. It can be potentially used on Linux and Mac OS X for the same purpose but needs an implementation in the OS. The generation of the JVM offsets headers on Solaris is used for both jhelper.d and libjvm_db. The jhelper.d is the dtrace helper supporting the jstack action that prints mixed java+native stacks like the pstack utility does. The dtrace jstack action has not been implemented yet on both Linux and MAC OS X. I have no information about any plans to implement it in the future. We keep this code in a hope it will be implemented at some point. My guess is that the hotspot.h, hotspot_jni.h and hs_private.h are still useful as they support the hotspot probes that should work on Linux and Mac OS X. I hope, someone else can comment on this as well. David H. may have some knowledge. :) Thanks, Serguei On 11/11/15 10:57, Magnus Ihse Bursie wrote: > Hi, > > In the hotspot build conversion project, I've started to analyze the > dtrace build on macosx. From what I can understand, the only thing > that are done on macosx is the dtrace_gen_headers target in > dtrace.make, which creates hotspot.h, hotspot_jni.h and hs_private.h. > No libraries seems to be build, so most of dtrace.make in bsd is dead > code. The comment > # Bsd does not build libjvm_db, does not compile on macosx > # disabled in build: rule in vm.make > has been there since the initial integration of the macosx port. > > My question is: are the generated headers still useful on their own? > Or are they just remnants of an old but ultimately failed effort to > get dtrace working on macosx? > > /Magnus From david.holmes at oracle.com Thu Nov 12 05:51:47 2015 From: david.holmes at oracle.com (David Holmes) Date: Thu, 12 Nov 2015 15:51:47 +1000 Subject: Dtrace on macosx? In-Reply-To: <5643DB87.9020905@oracle.com> References: <56438F9A.6050007@oracle.com> <5643DB87.9020905@oracle.com> Message-ID: <564428F3.4040008@oracle.com> On 12/11/2015 10:21 AM, serguei.spitsyn at oracle.com wrote: > Hi Magnus, > > I'll try to provide the details I know. > The libjvm_db is to support Solaris pstack utility that prints mixed > java+native stacks. > It can be potentially used on Linux and Mac OS X for the same purpose > but needs an implementation in the OS. > > The generation of the JVM offsets headers on Solaris is used for both > jhelper.d and libjvm_db. > The jhelper.d is the dtrace helper supporting the jstack action that > prints mixed > java+native stacks like the pstack utility does. > The dtrace jstack action has not been implemented yet on both Linux and > MAC OS X. > I have no information about any plans to implement it in the future. > We keep this code in a hope it will be implemented at some point. > > My guess is that the hotspot.h, hotspot_jni.h and hs_private.h are still > useful > as they support the hotspot probes that should work on Linux and Mac OS X. > > I hope, someone else can comment on this as well. > David H. may have some knowledge. :) I have the same guess - the headers support this: https://wiki.openjdk.java.net/display/MacOSXPort/Mac+OS+X+Port+Using+HotSpot+DTrace+Probes but don't know for sure. David H. :) > Thanks, > Serguei > > > On 11/11/15 10:57, Magnus Ihse Bursie wrote: >> Hi, >> >> In the hotspot build conversion project, I've started to analyze the >> dtrace build on macosx. From what I can understand, the only thing >> that are done on macosx is the dtrace_gen_headers target in >> dtrace.make, which creates hotspot.h, hotspot_jni.h and hs_private.h. >> No libraries seems to be build, so most of dtrace.make in bsd is dead >> code. The comment >> # Bsd does not build libjvm_db, does not compile on macosx >> # disabled in build: rule in vm.make >> has been there since the initial integration of the macosx port. >> >> My question is: are the generated headers still useful on their own? >> Or are they just remnants of an old but ultimately failed effort to >> get dtrace working on macosx? >> >> /Magnus > From david.holmes at oracle.com Thu Nov 12 07:35:25 2015 From: david.holmes at oracle.com (David Holmes) Date: Thu, 12 Nov 2015 17:35:25 +1000 Subject: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM In-Reply-To: <4295855A5C1DE049A61835A1887419CC41EB5DD6@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC41EB5DD6@DEWDFEMB12A.global.corp.sap> Message-ID: <5644413D.1000004@oracle.com> Hi Goetz, Adding in serviceability-dev On 9/11/2015 6:22 PM, Lindenmaier, Goetz wrote: > Hi, > > The environment variable _JAVA_SR_SIGNUM can be set to a signal number > do be used by the JVM's suspend/resume mechanism. > > If set, a signal handler is installed and the current signal handler is saved to an array. > On linux, this array had size MAXSIGNUM=32, and _JAVA_SR_SIGNUM was allowed > to range up to _NSIG=65. This could cause memory corruption. > > Further, in jsig.c, an unsinged int is used to set a bit for signals. This also > is too small, as only 32 signals can be supported. Further, the signals are mapped > wrong to these bits. '0' is not a valid signal, but '32' was. 1<<32 happens to map to > zero, so the signal could be stored, but this probably was not intended that way. > > This change increases MAXSIGNUM to 65 on linux, and to 64 on aix. It introduces > proper checking of the signal read from the env var, and issues a warning if it > does not use the signal set. It adapts the data types in jisig.c properly. > > Please review this change. I please need a sponsor. > http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.00 This all sounds very good to me. (I must find out why Solaris is not involved here :) ). On Linux you didn't add the bounds check to os::Linux::set_our_sigflags. I'm also wondering about documenting where we are determining the maximum from? Is it simply _NSIG on some/all distributions? And I see _NSIG is supposed to be the biggest signal number + one. Also linux defines NSIG = _NSIG so which should we be using? Thanks, David > Best regards, > Goetz. > From dmitry.samersoff at oracle.com Thu Nov 12 08:22:01 2015 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Thu, 12 Nov 2015 11:22:01 +0300 Subject: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM In-Reply-To: <5644413D.1000004@oracle.com> References: <4295855A5C1DE049A61835A1887419CC41EB5DD6@DEWDFEMB12A.global.corp.sap> <5644413D.1000004@oracle.com> Message-ID: <56444C29.8060606@oracle.com> David, I think it's better to use NSIG (without underscore) defined in signal.h -Dmitry On 2015-11-12 10:35, David Holmes wrote: > Hi Goetz, > > Adding in serviceability-dev > > On 9/11/2015 6:22 PM, Lindenmaier, Goetz wrote: >> Hi, >> >> The environment variable _JAVA_SR_SIGNUM can be set to a signal number >> do be used by the JVM's suspend/resume mechanism. >> >> If set, a signal handler is installed and the current signal handler >> is saved to an array. >> On linux, this array had size MAXSIGNUM=32, and _JAVA_SR_SIGNUM was >> allowed >> to range up to _NSIG=65. This could cause memory corruption. >> >> Further, in jsig.c, an unsinged int is used to set a bit for signals. >> This also >> is too small, as only 32 signals can be supported. Further, the >> signals are mapped >> wrong to these bits. '0' is not a valid signal, but '32' was. 1<<32 >> happens to map to >> zero, so the signal could be stored, but this probably was not >> intended that way. >> >> This change increases MAXSIGNUM to 65 on linux, and to 64 on aix. It >> introduces >> proper checking of the signal read from the env var, and issues a >> warning if it >> does not use the signal set. It adapts the data types in jisig.c >> properly. >> >> Please review this change. I please need a sponsor. >> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.00 > > This all sounds very good to me. (I must find out why Solaris is not > involved here :) ). > > On Linux you didn't add the bounds check to os::Linux::set_our_sigflags. > > I'm also wondering about documenting where we are determining the > maximum from? Is it simply _NSIG on some/all distributions? And I see > _NSIG is supposed to be the biggest signal number + one. Also linux > defines NSIG = _NSIG so which should we be using? > > Thanks, > David > >> Best regards, >> Goetz. >> -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From goetz.lindenmaier at sap.com Thu Nov 12 08:22:34 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 12 Nov 2015 08:22:34 +0000 Subject: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM In-Reply-To: <5644413D.1000004@oracle.com> References: <4295855A5C1DE049A61835A1887419CC41EB5DD6@DEWDFEMB12A.global.corp.sap> <5644413D.1000004@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC41EB821B@DEWDFEMB12A.global.corp.sap> Hi David, thanks for looking at this! > -----Original Message----- > From: David Holmes [mailto:david.holmes at oracle.com] > Sent: Donnerstag, 12. November 2015 08:35 > To: Lindenmaier, Goetz; hotspot-runtime-dev at openjdk.java.net; > serviceability-dev > Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM > > Hi Goetz, > > Adding in serviceability-dev > > On 9/11/2015 6:22 PM, Lindenmaier, Goetz wrote: > > Hi, > > > > The environment variable _JAVA_SR_SIGNUM can be set to a signal > number > > do be used by the JVM's suspend/resume mechanism. > > > > If set, a signal handler is installed and the current signal handler is saved to > an array. > > On linux, this array had size MAXSIGNUM=32, and _JAVA_SR_SIGNUM was > allowed > > to range up to _NSIG=65. This could cause memory corruption. > > > > Further, in jsig.c, an unsinged int is used to set a bit for signals. This also > > is too small, as only 32 signals can be supported. Further, the signals are > mapped > > wrong to these bits. '0' is not a valid signal, but '32' was. 1<<32 happens to > map to > > zero, so the signal could be stored, but this probably was not intended that > way. > > > > This change increases MAXSIGNUM to 65 on linux, and to 64 on aix. It > introduces > > proper checking of the signal read from the env var, and issues a warning if > it > > does not use the signal set. It adapts the data types in jisig.c properly. > > > > Please review this change. I please need a sponsor. > > http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.00 > > This all sounds very good to me. (I must find out why Solaris is not > involved here :) ). The mechanism is not implemented there. Why, I don't know. > On Linux you didn't add the bounds check to os::Linux::set_our_sigflags. It came with 8140482 http://hg.openjdk.java.net/jdk9/hs-rt/hotspot/rev/cd86b5699825 > I'm also wondering about documenting where we are determining the > maximum from? Is it simply _NSIG on some/all distributions? And I see > _NSIG is supposed to be the biggest signal number + one. Also linux > defines NSIG = _NSIG so which should we be using? I checked a row of linux distributions and Versions and always found NSIG=65. Also, as we compile NSIG into the code, we can not react on a difference between systems. So I guess that's fine. As NSIG = _NSIG, I don't really care which we use. I chose _NSIG because it was used before. On the other platforms I only found NSIG in the header files. Best regards, Goetz > > Thanks, > David > > > Best regards, > > Goetz. > > From david.holmes at oracle.com Thu Nov 12 08:29:09 2015 From: david.holmes at oracle.com (David Holmes) Date: Thu, 12 Nov 2015 18:29:09 +1000 Subject: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM In-Reply-To: <4295855A5C1DE049A61835A1887419CC41EB821B@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC41EB5DD6@DEWDFEMB12A.global.corp.sap> <5644413D.1000004@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB821B@DEWDFEMB12A.global.corp.sap> Message-ID: <56444DD5.2030507@oracle.com> Hi Goetz, On 12/11/2015 6:22 PM, Lindenmaier, Goetz wrote: > Hi David, > > thanks for looking at this! > >> -----Original Message----- >> From: David Holmes [mailto:david.holmes at oracle.com] >> Sent: Donnerstag, 12. November 2015 08:35 >> To: Lindenmaier, Goetz; hotspot-runtime-dev at openjdk.java.net; >> serviceability-dev >> Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM >> >> Hi Goetz, >> >> Adding in serviceability-dev >> >> On 9/11/2015 6:22 PM, Lindenmaier, Goetz wrote: >>> Hi, >>> >>> The environment variable _JAVA_SR_SIGNUM can be set to a signal >> number >>> do be used by the JVM's suspend/resume mechanism. >>> >>> If set, a signal handler is installed and the current signal handler is saved to >> an array. >>> On linux, this array had size MAXSIGNUM=32, and _JAVA_SR_SIGNUM was >> allowed >>> to range up to _NSIG=65. This could cause memory corruption. >>> >>> Further, in jsig.c, an unsinged int is used to set a bit for signals. This also >>> is too small, as only 32 signals can be supported. Further, the signals are >> mapped >>> wrong to these bits. '0' is not a valid signal, but '32' was. 1<<32 happens to >> map to >>> zero, so the signal could be stored, but this probably was not intended that >> way. >>> >>> This change increases MAXSIGNUM to 65 on linux, and to 64 on aix. It >> introduces >>> proper checking of the signal read from the env var, and issues a warning if >> it >>> does not use the signal set. It adapts the data types in jisig.c properly. >>> >>> Please review this change. I please need a sponsor. >>> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.00 >> >> This all sounds very good to me. (I must find out why Solaris is not >> involved here :) ). > The mechanism is not implemented there. Why, I don't know. > >> On Linux you didn't add the bounds check to os::Linux::set_our_sigflags. > It came with 8140482 http://hg.openjdk.java.net/jdk9/hs-rt/hotspot/rev/cd86b5699825 That change isn't present in your new code: void os::Linux::set_our_sigflags(int sig, int flags) { assert(sig > 0 && sig < MAXSIGNUM, "vm signal out of expected range"); sigflags[sig] = flags; } does it need to be re-based? >> I'm also wondering about documenting where we are determining the >> maximum from? Is it simply _NSIG on some/all distributions? And I see >> _NSIG is supposed to be the biggest signal number + one. Also linux >> defines NSIG = _NSIG so which should we be using? > I checked a row of linux distributions and Versions and always found NSIG=65. > Also, as we compile NSIG into the code, we can not react on a difference > between systems. So I guess that's fine. So why do we need MAXSIGNUM instead of just using NSIG ? > As NSIG = _NSIG, I don't really care which we use. I chose _NSIG because > it was used before. On the other platforms I only found NSIG in the header > files. I'd switch to NSIG as Dmitry suggested as it is the public value in signal.h. Thanks, David > Best regards, > Goetz > > >> >> Thanks, >> David >> >>> Best regards, >>> Goetz. >>> From goetz.lindenmaier at sap.com Thu Nov 12 08:36:20 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 12 Nov 2015 08:36:20 +0000 Subject: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM In-Reply-To: <56444C29.8060606@oracle.com> References: <4295855A5C1DE049A61835A1887419CC41EB5DD6@DEWDFEMB12A.global.corp.sap> <5644413D.1000004@oracle.com> <56444C29.8060606@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC41EB8252@DEWDFEMB12A.global.corp.sap> OK I'll change it to NSIG. That's used in other places in os_linux, too. So it's really more consistent. Best regards, Goetz > -----Original Message----- > From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com] > Sent: Donnerstag, 12. November 2015 09:22 > To: David Holmes; Lindenmaier, Goetz; hotspot-runtime- > dev at openjdk.java.net; serviceability-dev > Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM > > David, > > I think it's better to use NSIG (without underscore) defined in signal.h > > -Dmitry > > > On 2015-11-12 10:35, David Holmes wrote: > > Hi Goetz, > > > > Adding in serviceability-dev > > > > On 9/11/2015 6:22 PM, Lindenmaier, Goetz wrote: > >> Hi, > >> > >> The environment variable _JAVA_SR_SIGNUM can be set to a signal > number > >> do be used by the JVM's suspend/resume mechanism. > >> > >> If set, a signal handler is installed and the current signal handler > >> is saved to an array. > >> On linux, this array had size MAXSIGNUM=32, and _JAVA_SR_SIGNUM > was > >> allowed > >> to range up to _NSIG=65. This could cause memory corruption. > >> > >> Further, in jsig.c, an unsinged int is used to set a bit for signals. > >> This also > >> is too small, as only 32 signals can be supported. Further, the > >> signals are mapped > >> wrong to these bits. '0' is not a valid signal, but '32' was. 1<<32 > >> happens to map to > >> zero, so the signal could be stored, but this probably was not > >> intended that way. > >> > >> This change increases MAXSIGNUM to 65 on linux, and to 64 on aix. It > >> introduces > >> proper checking of the signal read from the env var, and issues a > >> warning if it > >> does not use the signal set. It adapts the data types in jisig.c > >> properly. > >> > >> Please review this change. I please need a sponsor. > >> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.00 > > > > This all sounds very good to me. (I must find out why Solaris is not > > involved here :) ). > > > > On Linux you didn't add the bounds check to os::Linux::set_our_sigflags. > > > > I'm also wondering about documenting where we are determining the > > maximum from? Is it simply _NSIG on some/all distributions? And I see > > _NSIG is supposed to be the biggest signal number + one. Also linux > > defines NSIG = _NSIG so which should we be using? > > > > Thanks, > > David > > > >> Best regards, > >> Goetz. > >> > > > -- > Dmitry Samersoff > Oracle Java development team, Saint Petersburg, Russia > * I would love to change the world, but they won't give me the sources. From dmitry.samersoff at oracle.com Thu Nov 12 09:05:10 2015 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Thu, 12 Nov 2015 12:05:10 +0300 Subject: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM In-Reply-To: <4295855A5C1DE049A61835A1887419CC41EB8252@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC41EB5DD6@DEWDFEMB12A.global.corp.sap> <5644413D.1000004@oracle.com> <56444C29.8060606@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB8252@DEWDFEMB12A.global.corp.sap> Message-ID: <56445646.4040707@oracle.com> Goetz, *BSD including OS X also defines NSIG (just checked) and if my memory is not bogus, AIX defines it too. So you may consider to use NSIG on all platform. -Dmitry On 2015-11-12 11:36, Lindenmaier, Goetz wrote: > OK I'll change it to NSIG. That's used in other places in os_linux, too. > So it's really more consistent. > > Best regards, > Goetz > >> -----Original Message----- >> From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com] >> Sent: Donnerstag, 12. November 2015 09:22 >> To: David Holmes; Lindenmaier, Goetz; hotspot-runtime- >> dev at openjdk.java.net; serviceability-dev >> Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM >> >> David, >> >> I think it's better to use NSIG (without underscore) defined in signal.h >> >> -Dmitry >> >> >> On 2015-11-12 10:35, David Holmes wrote: >>> Hi Goetz, >>> >>> Adding in serviceability-dev >>> >>> On 9/11/2015 6:22 PM, Lindenmaier, Goetz wrote: >>>> Hi, >>>> >>>> The environment variable _JAVA_SR_SIGNUM can be set to a signal >> number >>>> do be used by the JVM's suspend/resume mechanism. >>>> >>>> If set, a signal handler is installed and the current signal handler >>>> is saved to an array. >>>> On linux, this array had size MAXSIGNUM=32, and _JAVA_SR_SIGNUM >> was >>>> allowed >>>> to range up to _NSIG=65. This could cause memory corruption. >>>> >>>> Further, in jsig.c, an unsinged int is used to set a bit for signals. >>>> This also >>>> is too small, as only 32 signals can be supported. Further, the >>>> signals are mapped >>>> wrong to these bits. '0' is not a valid signal, but '32' was. 1<<32 >>>> happens to map to >>>> zero, so the signal could be stored, but this probably was not >>>> intended that way. >>>> >>>> This change increases MAXSIGNUM to 65 on linux, and to 64 on aix. It >>>> introduces >>>> proper checking of the signal read from the env var, and issues a >>>> warning if it >>>> does not use the signal set. It adapts the data types in jisig.c >>>> properly. >>>> >>>> Please review this change. I please need a sponsor. >>>> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.00 >>> >>> This all sounds very good to me. (I must find out why Solaris is not >>> involved here :) ). >>> >>> On Linux you didn't add the bounds check to os::Linux::set_our_sigflags. >>> >>> I'm also wondering about documenting where we are determining the >>> maximum from? Is it simply _NSIG on some/all distributions? And I see >>> _NSIG is supposed to be the biggest signal number + one. Also linux >>> defines NSIG = _NSIG so which should we be using? >>> >>> Thanks, >>> David >>> >>>> Best regards, >>>> Goetz. >>>> >> >> >> -- >> Dmitry Samersoff >> Oracle Java development team, Saint Petersburg, Russia >> * I would love to change the world, but they won't give me the sources. -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From goetz.lindenmaier at sap.com Thu Nov 12 10:15:00 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 12 Nov 2015 10:15:00 +0000 Subject: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM In-Reply-To: <56444DD5.2030507@oracle.com> References: <4295855A5C1DE049A61835A1887419CC41EB5DD6@DEWDFEMB12A.global.corp.sap> <5644413D.1000004@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB821B@DEWDFEMB12A.global.corp.sap> <56444DD5.2030507@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC41EB8354@DEWDFEMB12A.global.corp.sap> Hi, I'm not sure whether it's ideal to couple an internal data size to some constant defined on the system. But I'll change it. To make sure nothing breaks, I will add guard #if (64 < NSIG-1) #error "Not all signals can be encoded in jvmsigs. Adapt its type!" #endif to jsig.c. Best regards, Goetz. > -----Original Message----- > From: David Holmes [mailto:david.holmes at oracle.com] > Sent: Donnerstag, 12. November 2015 09:29 > To: Lindenmaier, Goetz; hotspot-runtime-dev at openjdk.java.net; > serviceability-dev > Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM > > Hi Goetz, > > On 12/11/2015 6:22 PM, Lindenmaier, Goetz wrote: > > Hi David, > > > > thanks for looking at this! > > > >> -----Original Message----- > >> From: David Holmes [mailto:david.holmes at oracle.com] > >> Sent: Donnerstag, 12. November 2015 08:35 > >> To: Lindenmaier, Goetz; hotspot-runtime-dev at openjdk.java.net; > >> serviceability-dev > >> Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM > >> > >> Hi Goetz, > >> > >> Adding in serviceability-dev > >> > >> On 9/11/2015 6:22 PM, Lindenmaier, Goetz wrote: > >>> Hi, > >>> > >>> The environment variable _JAVA_SR_SIGNUM can be set to a signal > >> number > >>> do be used by the JVM's suspend/resume mechanism. > >>> > >>> If set, a signal handler is installed and the current signal handler is saved > to > >> an array. > >>> On linux, this array had size MAXSIGNUM=32, and _JAVA_SR_SIGNUM > was > >> allowed > >>> to range up to _NSIG=65. This could cause memory corruption. > >>> > >>> Further, in jsig.c, an unsinged int is used to set a bit for signals. This also > >>> is too small, as only 32 signals can be supported. Further, the signals are > >> mapped > >>> wrong to these bits. '0' is not a valid signal, but '32' was. 1<<32 happens > to > >> map to > >>> zero, so the signal could be stored, but this probably was not intended > that > >> way. > >>> > >>> This change increases MAXSIGNUM to 65 on linux, and to 64 on aix. It > >> introduces > >>> proper checking of the signal read from the env var, and issues a warning > if > >> it > >>> does not use the signal set. It adapts the data types in jisig.c properly. > >>> > >>> Please review this change. I please need a sponsor. > >>> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.00 > >> > >> This all sounds very good to me. (I must find out why Solaris is not > >> involved here :) ). > > The mechanism is not implemented there. Why, I don't know. > > > >> On Linux you didn't add the bounds check to os::Linux::set_our_sigflags. > > It came with 8140482 http://hg.openjdk.java.net/jdk9/hs- > rt/hotspot/rev/cd86b5699825 > > That change isn't present in your new code: > > void os::Linux::set_our_sigflags(int sig, int flags) { > assert(sig > 0 && sig < MAXSIGNUM, "vm signal out of expected range"); > sigflags[sig] = flags; > } > > does it need to be re-based? > > >> I'm also wondering about documenting where we are determining the > >> maximum from? Is it simply _NSIG on some/all distributions? And I see > >> _NSIG is supposed to be the biggest signal number + one. Also linux > >> defines NSIG = _NSIG so which should we be using? > > I checked a row of linux distributions and Versions and always found > NSIG=65. > > Also, as we compile NSIG into the code, we can not react on a difference > > between systems. So I guess that's fine. > > So why do we need MAXSIGNUM instead of just using NSIG ? > > > As NSIG = _NSIG, I don't really care which we use. I chose _NSIG because > > it was used before. On the other platforms I only found NSIG in the header > > files. > > I'd switch to NSIG as Dmitry suggested as it is the public value in > signal.h. > > Thanks, > David > > > Best regards, > > Goetz > > > > > >> > >> Thanks, > >> David > >> > >>> Best regards, > >>> Goetz. > >>> From jaroslav.bachorik at oracle.com Thu Nov 12 12:13:07 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Thu, 12 Nov 2015 13:13:07 +0100 Subject: RFR 8141591: javax/management/remote/mandatory/threads/ExecutorTest.java fails intermittently Message-ID: <56448253.9020503@oracle.com> Please, review the following test change Issue : https://bugs.openjdk.java.net/browse/JDK-8141591 Webrev: http://cr.openjdk.java.net/~jbachorik/8141591/webrev.00 In rare circumstances, when an external executor is provided, the ClientNotifForwarder$NotifFetcher.doRun() method might fail because of RejectedExecutionException caused by the executor being externally shut down. The patch adds a guard against this possibility. If the executor has been shut down the fetcher will also properly stop. Thanks, -JB- From goetz.lindenmaier at sap.com Thu Nov 12 19:24:57 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 12 Nov 2015 19:24:57 +0000 Subject: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM In-Reply-To: <56445646.4040707@oracle.com> References: <4295855A5C1DE049A61835A1887419CC41EB5DD6@DEWDFEMB12A.global.corp.sap> <5644413D.1000004@oracle.com> <56444C29.8060606@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB8252@DEWDFEMB12A.global.corp.sap> <56445646.4040707@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC41EB85C3@DEWDFEMB12A.global.corp.sap> Hi David, Dmitry, I've come up with a new webrev: http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/ I hit on some more issues: - As proposed, I replaced MAXSIGNUM by NSIG - On AIX, NSIG=255. Therefore storing bits in a word does not work. I'm now using bitset functionality from signal.h as it's done in other places. sigset_t is >> NSIG on linux, so it's no good idea to use it there. - In the os files I found another bit vector that now is too small: sigs. I adapted that, too. Removed the dead declaration of this on solaris. Best regards, Goetz. > -----Original Message----- > From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com] > Sent: Donnerstag, 12. November 2015 10:05 > To: Lindenmaier, Goetz; David Holmes; hotspot-runtime- > dev at openjdk.java.net; serviceability-dev > Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM > > Goetz, > > *BSD including OS X also defines NSIG (just checked) and if my memory is > not bogus, AIX defines it too. > > So you may consider to use NSIG on all platform. > > -Dmitry > > On 2015-11-12 11:36, Lindenmaier, Goetz wrote: > > OK I'll change it to NSIG. That's used in other places in os_linux, too. > > So it's really more consistent. > > > > Best regards, > > Goetz > > > >> -----Original Message----- > >> From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com] > >> Sent: Donnerstag, 12. November 2015 09:22 > >> To: David Holmes; Lindenmaier, Goetz; hotspot-runtime- > >> dev at openjdk.java.net; serviceability-dev > >> Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM > >> > >> David, > >> > >> I think it's better to use NSIG (without underscore) defined in signal.h > >> > >> -Dmitry > >> > >> > >> On 2015-11-12 10:35, David Holmes wrote: > >>> Hi Goetz, > >>> > >>> Adding in serviceability-dev > >>> > >>> On 9/11/2015 6:22 PM, Lindenmaier, Goetz wrote: > >>>> Hi, > >>>> > >>>> The environment variable _JAVA_SR_SIGNUM can be set to a signal > >> number > >>>> do be used by the JVM's suspend/resume mechanism. > >>>> > >>>> If set, a signal handler is installed and the current signal handler > >>>> is saved to an array. > >>>> On linux, this array had size MAXSIGNUM=32, and _JAVA_SR_SIGNUM > >> was > >>>> allowed > >>>> to range up to _NSIG=65. This could cause memory corruption. > >>>> > >>>> Further, in jsig.c, an unsinged int is used to set a bit for signals. > >>>> This also > >>>> is too small, as only 32 signals can be supported. Further, the > >>>> signals are mapped > >>>> wrong to these bits. '0' is not a valid signal, but '32' was. 1<<32 > >>>> happens to map to > >>>> zero, so the signal could be stored, but this probably was not > >>>> intended that way. > >>>> > >>>> This change increases MAXSIGNUM to 65 on linux, and to 64 on aix. It > >>>> introduces > >>>> proper checking of the signal read from the env var, and issues a > >>>> warning if it > >>>> does not use the signal set. It adapts the data types in jisig.c > >>>> properly. > >>>> > >>>> Please review this change. I please need a sponsor. > >>>> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.00 > >>> > >>> This all sounds very good to me. (I must find out why Solaris is not > >>> involved here :) ). > >>> > >>> On Linux you didn't add the bounds check to os::Linux::set_our_sigflags. > >>> > >>> I'm also wondering about documenting where we are determining the > >>> maximum from? Is it simply _NSIG on some/all distributions? And I see > >>> _NSIG is supposed to be the biggest signal number + one. Also linux > >>> defines NSIG = _NSIG so which should we be using? > >>> > >>> Thanks, > >>> David > >>> > >>>> Best regards, > >>>> Goetz. > >>>> > >> > >> > >> -- > >> Dmitry Samersoff > >> Oracle Java development team, Saint Petersburg, Russia > >> * I would love to change the world, but they won't give me the sources. > > > -- > Dmitry Samersoff > Oracle Java development team, Saint Petersburg, Russia > * I would love to change the world, but they won't give me the sources. From david.holmes at oracle.com Fri Nov 13 04:16:20 2015 From: david.holmes at oracle.com (David Holmes) Date: Fri, 13 Nov 2015 14:16:20 +1000 Subject: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM In-Reply-To: <4295855A5C1DE049A61835A1887419CC41EB85C3@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC41EB5DD6@DEWDFEMB12A.global.corp.sap> <5644413D.1000004@oracle.com> <56444C29.8060606@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB8252@DEWDFEMB12A.global.corp.sap> <56445646.4040707@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB85C3@DEWDFEMB12A.global.corp.sap> Message-ID: <56456414.4020701@oracle.com> Hi Goetz, On 13/11/2015 5:24 AM, Lindenmaier, Goetz wrote: > Hi David, Dmitry, > > I've come up with a new webrev: > http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/ > > I hit on some more issues: That's what happens when you pull on a loose thread :) > - As proposed, I replaced MAXSIGNUM by NSIG > - On AIX, NSIG=255. Therefore storing bits in a word does not work. > I'm now using bitset functionality from signal.h as it's done in other places. Okay that's nice and neat. > sigset_t is >> NSIG on linux, so it's no good idea to use it there. That's unfortunate. I had a momentary vision of a single POSIX based shared implementation. The size may not be such a big deal - and we already use a number of sigset_t variables on linux. Future cleanup perhaps? > - In the os files I found another bit vector that now is too small: sigs. > I adapted that, too. Removed the dead declaration of this on solaris. Okay. This all seems okay to me. Thanks, David > Best regards, > Goetz. > >> -----Original Message----- >> From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com] >> Sent: Donnerstag, 12. November 2015 10:05 >> To: Lindenmaier, Goetz; David Holmes; hotspot-runtime- >> dev at openjdk.java.net; serviceability-dev >> Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM >> >> Goetz, >> >> *BSD including OS X also defines NSIG (just checked) and if my memory is >> not bogus, AIX defines it too. >> >> So you may consider to use NSIG on all platform. >> >> -Dmitry >> >> On 2015-11-12 11:36, Lindenmaier, Goetz wrote: >>> OK I'll change it to NSIG. That's used in other places in os_linux, too. >>> So it's really more consistent. >>> >>> Best regards, >>> Goetz >>> >>>> -----Original Message----- >>>> From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com] >>>> Sent: Donnerstag, 12. November 2015 09:22 >>>> To: David Holmes; Lindenmaier, Goetz; hotspot-runtime- >>>> dev at openjdk.java.net; serviceability-dev >>>> Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM >>>> >>>> David, >>>> >>>> I think it's better to use NSIG (without underscore) defined in signal.h >>>> >>>> -Dmitry >>>> >>>> >>>> On 2015-11-12 10:35, David Holmes wrote: >>>>> Hi Goetz, >>>>> >>>>> Adding in serviceability-dev >>>>> >>>>> On 9/11/2015 6:22 PM, Lindenmaier, Goetz wrote: >>>>>> Hi, >>>>>> >>>>>> The environment variable _JAVA_SR_SIGNUM can be set to a signal >>>> number >>>>>> do be used by the JVM's suspend/resume mechanism. >>>>>> >>>>>> If set, a signal handler is installed and the current signal handler >>>>>> is saved to an array. >>>>>> On linux, this array had size MAXSIGNUM=32, and _JAVA_SR_SIGNUM >>>> was >>>>>> allowed >>>>>> to range up to _NSIG=65. This could cause memory corruption. >>>>>> >>>>>> Further, in jsig.c, an unsinged int is used to set a bit for signals. >>>>>> This also >>>>>> is too small, as only 32 signals can be supported. Further, the >>>>>> signals are mapped >>>>>> wrong to these bits. '0' is not a valid signal, but '32' was. 1<<32 >>>>>> happens to map to >>>>>> zero, so the signal could be stored, but this probably was not >>>>>> intended that way. >>>>>> >>>>>> This change increases MAXSIGNUM to 65 on linux, and to 64 on aix. It >>>>>> introduces >>>>>> proper checking of the signal read from the env var, and issues a >>>>>> warning if it >>>>>> does not use the signal set. It adapts the data types in jisig.c >>>>>> properly. >>>>>> >>>>>> Please review this change. I please need a sponsor. >>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.00 >>>>> >>>>> This all sounds very good to me. (I must find out why Solaris is not >>>>> involved here :) ). >>>>> >>>>> On Linux you didn't add the bounds check to os::Linux::set_our_sigflags. >>>>> >>>>> I'm also wondering about documenting where we are determining the >>>>> maximum from? Is it simply _NSIG on some/all distributions? And I see >>>>> _NSIG is supposed to be the biggest signal number + one. Also linux >>>>> defines NSIG = _NSIG so which should we be using? >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>>> Best regards, >>>>>> Goetz. >>>>>> >>>> >>>> >>>> -- >>>> Dmitry Samersoff >>>> Oracle Java development team, Saint Petersburg, Russia >>>> * I would love to change the world, but they won't give me the sources. >> >> >> -- >> Dmitry Samersoff >> Oracle Java development team, Saint Petersburg, Russia >> * I would love to change the world, but they won't give me the sources. From shanliang.jiang at oracle.com Fri Nov 13 07:05:48 2015 From: shanliang.jiang at oracle.com (Shanliang JIANG) Date: Fri, 13 Nov 2015 08:05:48 +0100 Subject: jmx-dev RFR 8141591: javax/management/remote/mandatory/threads/ExecutorTest.java fails intermittently In-Reply-To: <56448253.9020503@oracle.com> References: <56448253.9020503@oracle.com> Message-ID: Hi Jaroslav, The issue is that after a JMX client is terminated, its ClientNotifForwarder continues deliver a job to a user specific Executor, I think a better fix to not allow this happen. I am not sure that it is a good solution to check RejectedExecutionException, here is the Java doc: "Exception thrown by an Executor when a task cannot be accepted for execution." it means that the exception is possibly thrown in other cases too, like too many tasks if it is shared. So ignore simply this exception in case of not ?shouldStop()? seems incorrect. And Executor is an interface and a user could provide any implementation class, so possible a user would throw any another RuntimeException or even an Error in this case. Shanliang > On 12 Nov 2015, at 13:13, Jaroslav Bachorik wrote: > > Please, review the following test change > > Issue : https://bugs.openjdk.java.net/browse/JDK-8141591 > Webrev: http://cr.openjdk.java.net/~jbachorik/8141591/webrev.00 > > In rare circumstances, when an external executor is provided, the ClientNotifForwarder$NotifFetcher.doRun() method might fail because of RejectedExecutionException caused by the executor being externally shut down. > > The patch adds a guard against this possibility. If the executor has been shut down the fetcher will also properly stop. > > Thanks, > > -JB- -------------- next part -------------- An HTML attachment was scrubbed... URL: From jaroslav.bachorik at oracle.com Fri Nov 13 08:04:54 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Fri, 13 Nov 2015 09:04:54 +0100 Subject: jmx-dev RFR 8141591: javax/management/remote/mandatory/threads/ExecutorTest.java fails intermittently In-Reply-To: References: <56448253.9020503@oracle.com> Message-ID: <564599A6.1030802@oracle.com> On 13.11.2015 08:05, Shanliang JIANG wrote: > Hi Jaroslav, > > The issue is that after a JMX client is terminated, its > ClientNotifForwarder continues deliver a job to a user specific > Executor, I think a better fix to not allow this happen. > > I am not sure that it is a good solution to check > RejectedExecutionException, here is the Java doc: > "Exception thrown by an |Executor| > when > a task cannot be accepted for execution." > > it means that the exception is possibly thrown in other cases too, like > too many tasks if it is shared. So ignore simply this exception in case > of not ?shouldStop()? seems incorrect. > > And Executor is an interface and a user could provide any implementation > class, so possible a user would throw any another RuntimeException or > even an Error in this case. Well, the main problem is the self-rescheduling part. Normally, a scheduled executor would be used to perform periodic tasks and it would know how to handle its own shutdown. But, unfortunately, the more generic Executor is required. If only it were at least ExecutorService where you can use 'isShutdown()' method ... The fact that the user provided executor can throw RejectedExecutionException at its discretion opens whole another can of worms. The code as it is now will simply bail out of the notification fetching loop with the thrown exception. Sadly, there is no cleanup performed so the ClientNotifForwarder will stay in inconsistent state (marked as STARTED when it is, in fact, TERMINATED, notification listeners not de-registered, etc.). To make things worse there seem to be no official documentation as what is the expected behaviour of the externally provided executor. The only documentation to the env property "jmx.remote.x.fetch.notifications.executor" is in ClientNotifForwarder.java (L125-128) and it is not exactly exhaustive. -JB- > > Shanliang > > On 12 Nov 2015, at 13:13, Jaroslav Bachorik > > > > > wrote: > > > > Please, review the following test change > > > > Issue : https://bugs.openjdk.java.net/browse/JDK-8141591 > > Webrev: http://cr.openjdk.java.net/~jbachorik/8141591/webrev.00 > > > > In rare circumstances, when an external executor is provided, the > > ClientNotifForwarder$NotifFetcher.doRun() method might fail because of > > RejectedExecutionException caused by the executor being externally > > shut down. > > > > The patch adds a guard against this possibility. If the executor has > > been shut down the fetcher will also properly stop. > > > > Thanks, > > > > -JB- > From per.liden at oracle.com Fri Nov 13 08:20:46 2015 From: per.liden at oracle.com (Per Liden) Date: Fri, 13 Nov 2015 09:20:46 +0100 Subject: RFR (S): 8142482: Improve the support for prefix functions in unified logging In-Reply-To: <56434E5B.8010703@oracle.com> References: <56434E5B.8010703@oracle.com> Message-ID: <56459D5E.60201@oracle.com> Hi Marcus, On 2015-11-11 15:19, Marcus Larsson wrote: > Hi, > > Please review the following small patch to improve the support for > prefixes in unified logging. > > Instead of the previous format and value/callback, a prefix is now > defined as a function that receives the log buffer as argument, > allowing it to fill in an arbitrary (or empty!) prefix. (This enables > GCId-prefixing to be skipped when GCId is undefined, for example.) > > Webrev: > http://cr.openjdk.java.net/~mlarsson/8142482/webrev.00/ Looks good. /Per > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8142482 > > Thanks, > Marcus From marcus.larsson at oracle.com Fri Nov 13 08:43:50 2015 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Fri, 13 Nov 2015 09:43:50 +0100 Subject: RFR (S): 8142482: Improve the support for prefix functions in unified logging In-Reply-To: <56436E29.2070308@oracle.com> References: <56434E5B.8010703@oracle.com> <56436E29.2070308@oracle.com> Message-ID: <5645A2C6.10207@oracle.com> Thanks for reviewing! Marcus On 2015-11-11 17:34, Bengt Rutisson wrote: > > Hi Marcus. > > On 2015-11-11 15:19, Marcus Larsson wrote: >> Hi, >> >> Please review the following small patch to improve the support for >> prefixes in unified logging. >> >> Instead of the previous format and value/callback, a prefix is now >> defined as a function that receives the log buffer as argument, >> allowing it to fill in an arbitrary (or empty!) prefix. (This enables >> GCId-prefixing to be skipped when GCId is undefined, for example.) >> >> Webrev: >> http://cr.openjdk.java.net/~mlarsson/8142482/webrev.00/ >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8142482 > > Looks good. Thanks for fixing this! > > Bengt > >> >> Thanks, >> Marcus > -------------- next part -------------- An HTML attachment was scrubbed... URL: From marcus.larsson at oracle.com Fri Nov 13 08:43:52 2015 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Fri, 13 Nov 2015 09:43:52 +0100 Subject: RFR (S): 8142482: Improve the support for prefix functions in unified logging In-Reply-To: <56459D5E.60201@oracle.com> References: <56434E5B.8010703@oracle.com> <56459D5E.60201@oracle.com> Message-ID: <5645A2C8.8010601@oracle.com> Thanks for reviewing! Marcus On 2015-11-13 09:20, Per Liden wrote: > Hi Marcus, > > On 2015-11-11 15:19, Marcus Larsson wrote: >> Hi, >> >> Please review the following small patch to improve the support for >> prefixes in unified logging. >> >> Instead of the previous format and value/callback, a prefix is now >> defined as a function that receives the log buffer as argument, >> allowing it to fill in an arbitrary (or empty!) prefix. (This enables >> GCId-prefixing to be skipped when GCId is undefined, for example.) >> >> Webrev: >> http://cr.openjdk.java.net/~mlarsson/8142482/webrev.00/ > > Looks good. > > /Per > >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8142482 >> >> Thanks, >> Marcus From marcus.larsson at oracle.com Fri Nov 13 09:20:15 2015 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Fri, 13 Nov 2015 10:20:15 +0100 Subject: RFR (XS): 8142483: Unified logging log instances cause warnings on windows when only static functions are used Message-ID: <5645AB4F.40907@oracle.com> Hi, Please review the following small patch to avoid 'unused variable' warnings on windows when only static functions are used on a log instance. Using log instances like this saves us from specifying the log tags for every function call, so it is something we should not warn about. The patch adds an empty constructor, which means that at least one non-static function is called for every log instance, and the warning is avoided. Webrev: http://cr.openjdk.java.net/~mlarsson/8142483/webrev.00/ Issue: https://bugs.openjdk.java.net/browse/JDK-8142483 Thanks, Marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.rutisson at oracle.com Fri Nov 13 09:39:56 2015 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Fri, 13 Nov 2015 10:39:56 +0100 Subject: RFR (XS): 8142483: Unified logging log instances cause warnings on windows when only static functions are used In-Reply-To: <5645AB4F.40907@oracle.com> References: <5645AB4F.40907@oracle.com> Message-ID: <5645AFEC.3010708@oracle.com> Hi Marcus, On 2015-11-13 10:20, Marcus Larsson wrote: > Hi, > > Please review the following small patch to avoid 'unused variable' > warnings on windows when only static functions are used on a log > instance. Using log instances like this saves us from specifying the > log tags for every function call, so it is something we should not > warn about. > > The patch adds an empty constructor, which means that at least one > non-static function is called for every log instance, and the warning > is avoided. > > Webrev: > http://cr.openjdk.java.net/~mlarsson/8142483/webrev.00/ > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8142483 Looks good. Bengt > > Thanks, > Marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.stuefe at gmail.com Fri Nov 13 09:53:44 2015 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 13 Nov 2015 10:53:44 +0100 Subject: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM In-Reply-To: <4295855A5C1DE049A61835A1887419CC41EB85C3@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC41EB5DD6@DEWDFEMB12A.global.corp.sap> <5644413D.1000004@oracle.com> <56444C29.8060606@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB8252@DEWDFEMB12A.global.corp.sap> <56445646.4040707@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB85C3@DEWDFEMB12A.global.corp.sap> Message-ID: Hi Goetz, sorry for not looking at this earlier. This is a nice cleanup. Some remarks: http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/aix/vm/os_aix.cpp.udiff.html + if (sig > MAX2(SIGSEGV, SIGBUS) && // See 4355769. + sig < NSIG) { // Must be legal signal and fit into sigflags[]. I do not like much the MAX2() construct. I would like it better to explicitly check whether the SR signal is one of the "forbidden" ones the VM uses. Maybe keep a mask defined centrally for each platform which contains signals the VM needs for itself ? +sigset_t os::Aix::sigs = { 0 }; I would not initialize the signal set this way. sigset_t is an opaque type; the only way to initialize it is with one of sigemptyset() or sigfillset(). http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/aix/vm/os_aix.hpp.udiff.html + static struct sigaction sigact[NSIG]; // saved preinstalled sigactions + static sigset_t sigs; // mask of signals that have + static int sigflags[NSIG]; I know this is not in the scope of your change, but I would like to see those removed from os::Aix and put into os_aix.cpp at static filescope. There is no need at all to export those, and you would get rid of the signal.h dependency you know have when including os_aix.hpp. http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/bsd/vm/jsig.c.udiff.html http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/bsd/vm/os_bsd.cpp.udiff.html On BSD, we have realtime signals. http://fxr.watson.org/fxr/source/sys/signal.h #define SIGRTMAX 126 and NSIG does not contain them: #define NSIG 32 The max. possible signal number would be 126, which unfortunately does not even fit into a 64bit mask. So the code in jsig.c is broken for the case that someone wants to register realtime signals, if the VM were to ever use realtime signals itself, which now is not the case. The same is true for os_bsd.cpp, where signal chaining will not work if the application did have handler for real time signals pre-installed before jvm is loaded. Solaris: The only platform where NSIG is missing? Here, we calculate the max. signal number dynamically in os_solaris.cpp, presumably because SIGRTMAX is not a constant and can be changed using system configuration. But then, on Linux we have the same situation (SIGRTMAX is dynamic) and there we do not go through the trouble of calculating the max. signal number dynamically. Instead we just use NSIG=64 and rely on the fact that NSIG is larger than the largest possible dynamic value for SIGRTMAX. Solaris does not seem to have NSIG defined, but I am sure there is also a max. possible value for SIGRTMAX (the default seems to be 48). So, one could probably safely define NSIG for Solaris too, so that we have NSIG defined on all Posix platforms. On Thu, Nov 12, 2015 at 8:24 PM, Lindenmaier, Goetz < goetz.lindenmaier at sap.com> wrote: > Hi David, Dmitry, > > I've come up with a new webrev: > http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/ > > I hit on some more issues: > - As proposed, I replaced MAXSIGNUM by NSIG > - On AIX, NSIG=255. Therefore storing bits in a word does not work. > I'm now using bitset functionality from signal.h as it's done in other > places. > sigset_t is >> NSIG on linux, so it's no good idea to use it there. > Why do we not do this on all platforms, provided sigset_t contains all signals (incl. realtime signals) ? > - In the os files I found another bit vector that now is too small: sigs. > I adapted that, too. Removed the dead declaration of this on solaris. > > Best regards, > Goetz. > > Kind Regards, Thomas > > -----Original Message----- > > From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com] > > Sent: Donnerstag, 12. November 2015 10:05 > > To: Lindenmaier, Goetz; David Holmes; hotspot-runtime- > > dev at openjdk.java.net; serviceability-dev > > Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM > > > > Goetz, > > > > *BSD including OS X also defines NSIG (just checked) and if my memory is > > not bogus, AIX defines it too. > > > > So you may consider to use NSIG on all platform. > > > > -Dmitry > > > > On 2015-11-12 11:36, Lindenmaier, Goetz wrote: > > > OK I'll change it to NSIG. That's used in other places in os_linux, > too. > > > So it's really more consistent. > > > > > > Best regards, > > > Goetz > > > > > >> -----Original Message----- > > >> From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com] > > >> Sent: Donnerstag, 12. November 2015 09:22 > > >> To: David Holmes; Lindenmaier, Goetz; hotspot-runtime- > > >> dev at openjdk.java.net; serviceability-dev > > >> Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM > > >> > > >> David, > > >> > > >> I think it's better to use NSIG (without underscore) defined in > signal.h > > >> > > >> -Dmitry > > >> > > >> > > >> On 2015-11-12 10:35, David Holmes wrote: > > >>> Hi Goetz, > > >>> > > >>> Adding in serviceability-dev > > >>> > > >>> On 9/11/2015 6:22 PM, Lindenmaier, Goetz wrote: > > >>>> Hi, > > >>>> > > >>>> The environment variable _JAVA_SR_SIGNUM can be set to a signal > > >> number > > >>>> do be used by the JVM's suspend/resume mechanism. > > >>>> > > >>>> If set, a signal handler is installed and the current signal handler > > >>>> is saved to an array. > > >>>> On linux, this array had size MAXSIGNUM=32, and _JAVA_SR_SIGNUM > > >> was > > >>>> allowed > > >>>> to range up to _NSIG=65. This could cause memory corruption. > > >>>> > > >>>> Further, in jsig.c, an unsinged int is used to set a bit for > signals. > > >>>> This also > > >>>> is too small, as only 32 signals can be supported. Further, the > > >>>> signals are mapped > > >>>> wrong to these bits. '0' is not a valid signal, but '32' was. > 1<<32 > > >>>> happens to map to > > >>>> zero, so the signal could be stored, but this probably was not > > >>>> intended that way. > > >>>> > > >>>> This change increases MAXSIGNUM to 65 on linux, and to 64 on aix. It > > >>>> introduces > > >>>> proper checking of the signal read from the env var, and issues a > > >>>> warning if it > > >>>> does not use the signal set. It adapts the data types in jisig.c > > >>>> properly. > > >>>> > > >>>> Please review this change. I please need a sponsor. > > >>>> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.00 > > >>> > > >>> This all sounds very good to me. (I must find out why Solaris is not > > >>> involved here :) ). > > >>> > > >>> On Linux you didn't add the bounds check to > os::Linux::set_our_sigflags. > > >>> > > >>> I'm also wondering about documenting where we are determining the > > >>> maximum from? Is it simply _NSIG on some/all distributions? And I see > > >>> _NSIG is supposed to be the biggest signal number + one. Also linux > > >>> defines NSIG = _NSIG so which should we be using? > > >>> > > >>> Thanks, > > >>> David > > >>> > > >>>> Best regards, > > >>>> Goetz. > > >>>> > > >> > > >> > > >> -- > > >> Dmitry Samersoff > > >> Oracle Java development team, Saint Petersburg, Russia > > >> * I would love to change the world, but they won't give me the > sources. > > > > > > -- > > Dmitry Samersoff > > Oracle Java development team, Saint Petersburg, Russia > > * I would love to change the world, but they won't give me the sources. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Fri Nov 13 13:38:24 2015 From: david.holmes at oracle.com (David Holmes) Date: Fri, 13 Nov 2015 23:38:24 +1000 Subject: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM In-Reply-To: References: <4295855A5C1DE049A61835A1887419CC41EB5DD6@DEWDFEMB12A.global.corp.sap> <5644413D.1000004@oracle.com> <56444C29.8060606@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB8252@DEWDFEMB12A.global.corp.sap> <56445646.4040707@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB85C3@DEWDFEMB12A.global.corp.sap> Message-ID: <5645E7D0.1000704@oracle.com> On 13/11/2015 7:53 PM, Thomas St?fe wrote: > Hi Goetz, > > sorry for not looking at this earlier. This is a nice cleanup. Some remarks: > > http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/aix/vm/os_aix.cpp.udiff.html > > + if (sig > MAX2(SIGSEGV, SIGBUS) && // See 4355769. > + sig < NSIG) { // Must be legal signal and fit > into sigflags[]. > > I do not like much the MAX2() construct. I would like it better to > explicitly check whether the SR signal is one of the "forbidden" ones > the VM uses. I must confess I had not looked into 4355769 but this check seems rather spurious. It is not at all clear to me what signals could be used here - other than SIGUSR1 or SIGUSR2 (if -Xrs is specified), or else a real-time signal (modulo discussion below). Hijacking anything else seems rather suspect. > Maybe keep a mask defined centrally for each platform which contains > signals the VM needs for itself ? Such masks already exist. > +sigset_t os::Aix::sigs = { 0 }; > > I would not initialize the signal set this way. sigset_t is an opaque > type; the only way to initialize it is with one of sigemptyset() or > sigfillset(). Good catch - I overlooked that. > http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/aix/vm/os_aix.hpp.udiff.html > > + static struct sigaction sigact[NSIG]; // saved preinstalled sigactions > + static sigset_t sigs; // mask of signals that have > > + static int sigflags[NSIG]; > > I know this is not in the scope of your change, but I would like to see > those removed from os::Aix and put into os_aix.cpp at static filescope. > There is no need at all to export those, and you would get rid of the > signal.h dependency you know have when including os_aix.hpp. > > > http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/bsd/vm/jsig.c.udiff.html > http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/bsd/vm/os_bsd.cpp.udiff.html > > On BSD, we have realtime signals. > > http://fxr.watson.org/fxr/source/sys/signal.h > #define SIGRTMAX 126 > and NSIG does not contain them: > #define NSIG 32 > > The max. possible signal number would be 126, which unfortunately does > not even fit into a 64bit mask. So this simply limits the signal choice to not be a real-time signal - same as today. > So the code in jsig.c is broken for the case that someone wants to > register realtime signals, if the VM were to ever use realtime signals > itself, which now is not the case. > > The same is true for os_bsd.cpp, where signal chaining will not work if > the application did have handler for real time signals pre-installed > before jvm is loaded. Chaining is only used when the JVM will catch signals. Aren't all the real-time signals going to be blocked by the VM by default and so chaining is not needed as no handler will exist in the VM ?? (Unless a real-time signal is supplied for SR_signum) I must admit I don't know if any of this code actually works for real-time signals. > Solaris: > > The only platform where NSIG is missing? > > Here, we calculate the max. signal number dynamically in os_solaris.cpp, > presumably because SIGRTMAX is not a constant and can be changed using > system configuration. But then, on Linux we have the same situation > (SIGRTMAX is dynamic) and there we do not go through the trouble of > calculating the max. signal number dynamically. Instead we just use > NSIG=64 and rely on the fact that NSIG is larger than the largest > possible dynamic value for SIGRTMAX. Linux ensures that _NSIG (and thus NSIG) includes all the real-time signals. But libc can expose a subset and steal some for its own use. > Solaris does not seem to have NSIG defined, but I am sure there is also > a max. possible value for SIGRTMAX (the default seems to be 48). So, one > could probably safely define NSIG for Solaris too, so that we have NSIG > defined on all Posix platforms. Solaris doesn't have any of this SR_signum related code. A more general cleanup of signal related code would potentially involve a lot of cleanup. David ----- > > On Thu, Nov 12, 2015 at 8:24 PM, Lindenmaier, Goetz > > wrote: > > Hi David, Dmitry, > > I've come up with a new webrev: > http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/ > > I hit on some more issues: > - As proposed, I replaced MAXSIGNUM by NSIG > - On AIX, NSIG=255. Therefore storing bits in a word does not work. > I'm now using bitset functionality from signal.h as it's done > in other places. > sigset_t is >> NSIG on linux, so it's no good idea to use it there. > > > Why do we not do this on all platforms, provided sigset_t contains all > signals (incl. realtime signals) ? > > - In the os files I found another bit vector that now is too > small: sigs. > I adapted that, too. Removed the dead declaration of this on > solaris. > > Best regards, > Goetz. > > > > Kind Regards, Thomas > > > -----Original Message----- > > From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com ] > > Sent: Donnerstag, 12. November 2015 10:05 > > To: Lindenmaier, Goetz; David Holmes; hotspot-runtime- > > dev at openjdk.java.net ; > serviceability-dev > > Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM > > > > Goetz, > > > > *BSD including OS X also defines NSIG (just checked) and if my > memory is > > not bogus, AIX defines it too. > > > > So you may consider to use NSIG on all platform. > > > > -Dmitry > > > > On 2015-11-12 11:36, Lindenmaier, Goetz wrote: > > > OK I'll change it to NSIG. That's used in other places in > os_linux, too. > > > So it's really more consistent. > > > > > > Best regards, > > > Goetz > > > > > >> -----Original Message----- > > >> From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com > ] > > >> Sent: Donnerstag, 12. November 2015 09:22 > > >> To: David Holmes; Lindenmaier, Goetz; hotspot-runtime- > > >> dev at openjdk.java.net ; > serviceability-dev > > >> Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM > > >> > > >> David, > > >> > > >> I think it's better to use NSIG (without underscore) defined > in signal.h > > >> > > >> -Dmitry > > >> > > >> > > >> On 2015-11-12 10:35, David Holmes wrote: > > >>> Hi Goetz, > > >>> > > >>> Adding in serviceability-dev > > >>> > > >>> On 9/11/2015 6:22 PM, Lindenmaier, Goetz wrote: > > >>>> Hi, > > >>>> > > >>>> The environment variable _JAVA_SR_SIGNUM can be set to a signal > > >> number > > >>>> do be used by the JVM's suspend/resume mechanism. > > >>>> > > >>>> If set, a signal handler is installed and the current signal > handler > > >>>> is saved to an array. > > >>>> On linux, this array had size MAXSIGNUM=32, and _JAVA_SR_SIGNUM > > >> was > > >>>> allowed > > >>>> to range up to _NSIG=65. This could cause memory corruption. > > >>>> > > >>>> Further, in jsig.c, an unsinged int is used to set a bit for > signals. > > >>>> This also > > >>>> is too small, as only 32 signals can be supported. Further, the > > >>>> signals are mapped > > >>>> wrong to these bits. '0' is not a valid signal, but '32' > was. 1<<32 > > >>>> happens to map to > > >>>> zero, so the signal could be stored, but this probably was not > > >>>> intended that way. > > >>>> > > >>>> This change increases MAXSIGNUM to 65 on linux, and to 64 on > aix. It > > >>>> introduces > > >>>> proper checking of the signal read from the env var, and > issues a > > >>>> warning if it > > >>>> does not use the signal set. It adapts the data types in > jisig.c > > >>>> properly. > > >>>> > > >>>> Please review this change. I please need a sponsor. > > >>>> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.00 > > >>> > > >>> This all sounds very good to me. (I must find out why Solaris > is not > > >>> involved here :) ). > > >>> > > >>> On Linux you didn't add the bounds check to > os::Linux::set_our_sigflags. > > >>> > > >>> I'm also wondering about documenting where we are determining the > > >>> maximum from? Is it simply _NSIG on some/all distributions? > And I see > > >>> _NSIG is supposed to be the biggest signal number + one. Also > linux > > >>> defines NSIG = _NSIG so which should we be using? > > >>> > > >>> Thanks, > > >>> David > > >>> > > >>>> Best regards, > > >>>> Goetz. > > >>>> > > >> > > >> > > >> -- > > >> Dmitry Samersoff > > >> Oracle Java development team, Saint Petersburg, Russia > > >> * I would love to change the world, but they won't give me the > sources. > > > > > > -- > > Dmitry Samersoff > > Oracle Java development team, Saint Petersburg, Russia > > * I would love to change the world, but they won't give me the > sources. > > From daniel.daugherty at oracle.com Fri Nov 13 14:45:43 2015 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 13 Nov 2015 07:45:43 -0700 Subject: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM In-Reply-To: <5645E7D0.1000704@oracle.com> References: <4295855A5C1DE049A61835A1887419CC41EB5DD6@DEWDFEMB12A.global.corp.sap> <5644413D.1000004@oracle.com> <56444C29.8060606@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB8252@DEWDFEMB12A.global.corp.sap> <56445646.4040707@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB85C3@DEWDFEMB12A.global.corp.sap> <5645E7D0.1000704@oracle.com> Message-ID: <5645F797.3010105@oracle.com> On 11/13/15 6:38 AM, David Holmes wrote: > On 13/11/2015 7:53 PM, Thomas St?fe wrote: >> Hi Goetz, >> >> sorry for not looking at this earlier. This is a nice cleanup. Some >> remarks: >> >> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/aix/vm/os_aix.cpp.udiff.html >> >> >> + if (sig > MAX2(SIGSEGV, SIGBUS) && // See 4355769. >> + sig < NSIG) { // Must be legal signal and fit >> into sigflags[]. >> >> I do not like much the MAX2() construct. I would like it better to >> explicitly check whether the SR signal is one of the "forbidden" ones >> the VM uses. > > I must confess I had not looked into 4355769 but this check seems > rather spurious. It is not at all clear to me what signals could be > used here - other than SIGUSR1 or SIGUSR2 (if -Xrs is specified), or > else a real-time signal (modulo discussion below). Hijacking anything > else seems rather suspect. > >> Maybe keep a mask defined centrally for each platform which contains >> signals the VM needs for itself ? > > Such masks already exist. > >> +sigset_t os::Aix::sigs = { 0 }; >> >> I would not initialize the signal set this way. sigset_t is an opaque >> type; the only way to initialize it is with one of sigemptyset() or >> sigfillset(). > > Good catch - I overlooked that. > >> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/aix/vm/os_aix.hpp.udiff.html >> >> >> + static struct sigaction sigact[NSIG]; // saved preinstalled >> sigactions >> + static sigset_t sigs; // mask of signals that have >> >> + static int sigflags[NSIG]; >> >> I know this is not in the scope of your change, but I would like to see >> those removed from os::Aix and put into os_aix.cpp at static filescope. >> There is no need at all to export those, and you would get rid of the >> signal.h dependency you know have when including os_aix.hpp. >> >> >> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/bsd/vm/jsig.c.udiff.html >> >> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/bsd/vm/os_bsd.cpp.udiff.html >> >> >> On BSD, we have realtime signals. >> >> http://fxr.watson.org/fxr/source/sys/signal.h >> #define SIGRTMAX 126 >> and NSIG does not contain them: >> #define NSIG 32 >> >> The max. possible signal number would be 126, which unfortunately does >> not even fit into a 64bit mask. > > So this simply limits the signal choice to not be a real-time signal - > same as today. > >> So the code in jsig.c is broken for the case that someone wants to >> register realtime signals, if the VM were to ever use realtime signals >> itself, which now is not the case. >> >> The same is true for os_bsd.cpp, where signal chaining will not work if >> the application did have handler for real time signals pre-installed >> before jvm is loaded. > > Chaining is only used when the JVM will catch signals. Aren't all the > real-time signals going to be blocked by the VM by default and so > chaining is not needed as no handler will exist in the VM ?? (Unless a > real-time signal is supplied for SR_signum) > > I must admit I don't know if any of this code actually works for > real-time signals. > >> Solaris: >> >> The only platform where NSIG is missing? >> >> Here, we calculate the max. signal number dynamically in os_solaris.cpp, >> presumably because SIGRTMAX is not a constant and can be changed using >> system configuration. But then, on Linux we have the same situation >> (SIGRTMAX is dynamic) and there we do not go through the trouble of >> calculating the max. signal number dynamically. Instead we just use >> NSIG=64 and rely on the fact that NSIG is larger than the largest >> possible dynamic value for SIGRTMAX. > > Linux ensures that _NSIG (and thus NSIG) includes all the real-time > signals. But libc can expose a subset and steal some for its own use. > >> Solaris does not seem to have NSIG defined, but I am sure there is also >> a max. possible value for SIGRTMAX (the default seems to be 48). So, one >> could probably safely define NSIG for Solaris too, so that we have NSIG >> defined on all Posix platforms. > > Solaris doesn't have any of this SR_signum related code. A more > general cleanup of signal related code would potentially involve a lot > of cleanup. I see this on my s10x_u11wos_24a X86 machine: $ rgrep NSIG /usr/include | grep -w NSIG /usr/include/sys/signal.h:#define NSIG 49 /* valid signals range from 1 to NSIG-1 */ /usr/include/sys/signal.h:#define MAXSIG 48 /* size of u_signal[], NSIG-1 <= MAXSIG */ But Solaris doesn't use NSIG because it does have the SR_signum code (as David said)... Dan > > David > ----- > >> >> On Thu, Nov 12, 2015 at 8:24 PM, Lindenmaier, Goetz >> > wrote: >> >> Hi David, Dmitry, >> >> I've come up with a new webrev: >> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/ >> >> I hit on some more issues: >> - As proposed, I replaced MAXSIGNUM by NSIG >> - On AIX, NSIG=255. Therefore storing bits in a word does not >> work. >> I'm now using bitset functionality from signal.h as it's done >> in other places. >> sigset_t is >> NSIG on linux, so it's no good idea to use it >> there. >> >> >> Why do we not do this on all platforms, provided sigset_t contains all >> signals (incl. realtime signals) ? >> >> - In the os files I found another bit vector that now is too >> small: sigs. >> I adapted that, too. Removed the dead declaration of this on >> solaris. >> >> Best regards, >> Goetz. >> >> >> >> Kind Regards, Thomas >> >> > -----Original Message----- >> > From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com >> ] >> > Sent: Donnerstag, 12. November 2015 10:05 >> > To: Lindenmaier, Goetz; David Holmes; hotspot-runtime- >> > dev at openjdk.java.net ; >> serviceability-dev >> > Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM >> > >> > Goetz, >> > >> > *BSD including OS X also defines NSIG (just checked) and if my >> memory is >> > not bogus, AIX defines it too. >> > >> > So you may consider to use NSIG on all platform. >> > >> > -Dmitry >> > >> > On 2015-11-12 11:36, Lindenmaier, Goetz wrote: >> > > OK I'll change it to NSIG. That's used in other places in >> os_linux, too. >> > > So it's really more consistent. >> > > >> > > Best regards, >> > > Goetz >> > > >> > >> -----Original Message----- >> > >> From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com >> ] >> > >> Sent: Donnerstag, 12. November 2015 09:22 >> > >> To: David Holmes; Lindenmaier, Goetz; hotspot-runtime- >> > >> dev at openjdk.java.net ; >> serviceability-dev >> > >> Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM >> > >> >> > >> David, >> > >> >> > >> I think it's better to use NSIG (without underscore) defined >> in signal.h >> > >> >> > >> -Dmitry >> > >> >> > >> >> > >> On 2015-11-12 10:35, David Holmes wrote: >> > >>> Hi Goetz, >> > >>> >> > >>> Adding in serviceability-dev >> > >>> >> > >>> On 9/11/2015 6:22 PM, Lindenmaier, Goetz wrote: >> > >>>> Hi, >> > >>>> >> > >>>> The environment variable _JAVA_SR_SIGNUM can be set to a >> signal >> > >> number >> > >>>> do be used by the JVM's suspend/resume mechanism. >> > >>>> >> > >>>> If set, a signal handler is installed and the current signal >> handler >> > >>>> is saved to an array. >> > >>>> On linux, this array had size MAXSIGNUM=32, and >> _JAVA_SR_SIGNUM >> > >> was >> > >>>> allowed >> > >>>> to range up to _NSIG=65. This could cause memory corruption. >> > >>>> >> > >>>> Further, in jsig.c, an unsinged int is used to set a bit for >> signals. >> > >>>> This also >> > >>>> is too small, as only 32 signals can be supported. >> Further, the >> > >>>> signals are mapped >> > >>>> wrong to these bits. '0' is not a valid signal, but '32' >> was. 1<<32 >> > >>>> happens to map to >> > >>>> zero, so the signal could be stored, but this probably >> was not >> > >>>> intended that way. >> > >>>> >> > >>>> This change increases MAXSIGNUM to 65 on linux, and to 64 on >> aix. It >> > >>>> introduces >> > >>>> proper checking of the signal read from the env var, and >> issues a >> > >>>> warning if it >> > >>>> does not use the signal set. It adapts the data types in >> jisig.c >> > >>>> properly. >> > >>>> >> > >>>> Please review this change. I please need a sponsor. >> > >>>> >> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.00 >> > >>> >> > >>> This all sounds very good to me. (I must find out why Solaris >> is not >> > >>> involved here :) ). >> > >>> >> > >>> On Linux you didn't add the bounds check to >> os::Linux::set_our_sigflags. >> > >>> >> > >>> I'm also wondering about documenting where we are >> determining the >> > >>> maximum from? Is it simply _NSIG on some/all distributions? >> And I see >> > >>> _NSIG is supposed to be the biggest signal number + one. Also >> linux >> > >>> defines NSIG = _NSIG so which should we be using? >> > >>> >> > >>> Thanks, >> > >>> David >> > >>> >> > >>>> Best regards, >> > >>>> Goetz. >> > >>>> >> > >> >> > >> >> > >> -- >> > >> Dmitry Samersoff >> > >> Oracle Java development team, Saint Petersburg, Russia >> > >> * I would love to change the world, but they won't give me the >> sources. >> > >> > >> > -- >> > Dmitry Samersoff >> > Oracle Java development team, Saint Petersburg, Russia >> > * I would love to change the world, but they won't give me the >> sources. >> >> From shanliang.jiang at oracle.com Fri Nov 13 17:07:42 2015 From: shanliang.jiang at oracle.com (Shanliang JIANG) Date: Fri, 13 Nov 2015 18:07:42 +0100 Subject: jmx-dev RFR 8141591: javax/management/remote/mandatory/threads/ExecutorTest.java fails intermittently In-Reply-To: <564599A6.1030802@oracle.com> References: <56448253.9020503@oracle.com> <564599A6.1030802@oracle.com> Message-ID: <0838E637-4F57-4BED-B973-3AAACABC719C@oracle.com> > On 13 Nov 2015, at 09:04, Jaroslav Bachorik wrote: > > On 13.11.2015 08:05, Shanliang JIANG wrote: >> Hi Jaroslav, >> >> The issue is that after a JMX client is terminated, its >> ClientNotifForwarder continues deliver a job to a user specific >> Executor, I think a better fix to not allow this happen. >> >> I am not sure that it is a good solution to check >> RejectedExecutionException, here is the Java doc: >> "Exception thrown by an |Executor| >> when >> a task cannot be accepted for execution." >> >> it means that the exception is possibly thrown in other cases too, like >> too many tasks if it is shared. So ignore simply this exception in case >> of not ?shouldStop()? seems incorrect. >> >> And Executor is an interface and a user could provide any implementation >> class, so possible a user would throw any another RuntimeException or >> even an Error in this case. > Well, the main problem is the self-rescheduling part. Normally, a scheduled executor would be used to perform periodic tasks and it would know how to handle its own shutdown. But, unfortunately, the more generic Executor is required. If only it were at least ExecutorService where you can use 'isShutdown()' method ... > > The fact that the user provided executor can throw RejectedExecutionException at its discretion opens whole another can of worms. The code as it is now will simply bail out of the notification fetching loop with the thrown exception. Sadly, there is no cleanup performed so the ClientNotifForwarder will stay in inconsistent state (marked as STARTED when it is, in fact, TERMINATED, notification listeners not de-registered, etc.). To make things worse there seem to be no official documentation as what is the expected behaviour of the externally provided executor. The only documentation to the env property "jmx.remote.x.fetch.notifications.executor" is in ClientNotifForwarder.java (L125-128) and it is not exactly exhaustive. Here is my suggestion (I create a webrev because it is easier to look at): http://cr.openjdk.java.net/~sjiang/JDK-8141591/00/ It does rescheduling within the synchronisation, which makes sure to not deliver a new task after a JMX client is terminated. This class is complicated because the fetching should be stopped if no more listener or during re-connection, but then could be re-started at anytime. Shanliang > > -JB- > > >> >> Shanliang >> > On 12 Nov 2015, at 13:13, Jaroslav Bachorik >> > > >> > wrote: >> > >> > Please, review the following test change >> > >> > Issue : https://bugs.openjdk.java.net/browse/JDK-8141591 >> > Webrev: http://cr.openjdk.java.net/~jbachorik/8141591/webrev.00 >> > >> > In rare circumstances, when an external executor is provided, the >> > ClientNotifForwarder$NotifFetcher.doRun() method might fail because of >> > RejectedExecutionException caused by the executor being externally >> > shut down. >> > >> > The patch adds a guard against this possibility. If the executor has >> > been shut down the fetcher will also properly stop. >> > >> > Thanks, >> > >> > -JB- >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From yasuenag at gmail.com Sun Nov 15 12:51:03 2015 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Sun, 15 Nov 2015 21:51:03 +0900 Subject: PING: RFR: JDK-8140556: Add force rotation option to VM.log jcmd In-Reply-To: <563B6CA0.9040107@gmail.com> References: <562101D8.5080909@gmail.com> <562DF8D4.40501@oracle.com> <562EBF4B.4020607@gmail.com> <562F6B46.5050500@oracle.com> <563226AB.2030206@gmail.com> <5632341F.4060400@oracle.com> <56338125.9030802@gmail.com> <56338D58.9090709@oracle.com> <5634D9FC.6080502@gmail.com> <5636C66A.6020304@oracle.com> <563765FC.2090804@gmail.com> <563ACD6F.1050600@gmail.com> <563AD1F2.6060109@oracle.com> <563B69F9.5090102@oracle.com> <563B6CA0.9040107@gmail.com> Message-ID: <56487FB7.9020908@gmail.com> PING: Could you review it? http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.04/ Yasumasa On 2015/11/05 23:50, Yasumasa Suenaga wrote: >>> _post_initialized would be a more accurate name. > > I've fixed it in new webrev: > http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.04/ > > I will send changeset to Marcus if this changeset is reviewed. > > > Thanks. > > Yasumasa > > > On 2015/11/05 23:38, Marcus Larsson wrote: >> Hey, >> >> On 2015-11-05 04:50, David Holmes wrote: >>> On 5/11/2015 1:30 PM, Yasumasa Suenaga wrote: >>>> I've uploaded new webrev: >>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.03/ >>>> >>>> I added _initialized to LogConfiguration. >>>> LogConfiguration::post_initialize() will be called after main thread >>>> initialization. >>>> So _initialized set to true in this function, and we can check this flag >>>> at rotation. >>>> >>>> Could you review it? >> >> Looks OK to me. This is much better than checking for Thread::current() != NULL in is_rotatable(), IMHO. >> >>> >>> _post_initialized would be a more accurate name. >>> >>> This of course works, but I don't know whether the additional delay in allowing rotation will impact anything. >> >> It shouldn't. If anyone ever generates that much logging during startup they will have to accept that the first rotated log won't be properly sized, that's all. >> >> Thanks, >> Marcus >> >>> >>> Thanks, >>> David >>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> On 2015/11/02 22:32, Yasumasa Suenaga wrote: >>>>> Marcus, David, >>>>> >>>>> I've uploaded new webrev: >>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.02/ >>>>> >>>>> Could you review it? >>>>> >>>>> >>>>>>>>>> Still missing the renaming of rotate_all_logfile to >>>>>>>>>> rotate_all_outputs. >>>>> >>>>> I've fixed. >>>>> >>>>> >>>>>>>> Small typo, please make that is_rotatable(). >>>>> >>>>> I've added, and have used at LogConfiguration::rotate_all_outputs(). >>>>> >>>>> >>>>>> You can't safely lock a mutex until the current thread has attached >>>>>> to the VM and Thread::current() returns non-null (that is what >>>>>> Thread::initialize_thread_local_storage() currently does). You may >>>>>> get away with it, particularly in product builds, during VM >>>>>> initialization, as the fast-lock path doesn't require access to >>>>>> Thread::current(). >>>>> >>>>> I will check whether Thread::current() is NULL or not at >>>>> LogFileOutput::is_rotatable() >>>>> is_rotatable() is called at LogFileOutput::rotate() before locking >>>>> _rotation_lock. >>>>> So we can avoid crash/assert which is related to TLS. >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2015/11/02 11:11, David Holmes wrote: >>>>>> On 1/11/2015 1:10 AM, Yasumasa Suenaga wrote: >>>>>>>> This in turn means that we are logging before the thread local storage >>>>>>>> has been initialized, and the JVM will crash/hit the assert because we >>>>>>>> are trying to take the rotation lock. >>>>>>> >>>>>>> I guess that you are saying about class member (not TLS - e.g. >>>>>>> pthread_setspecific()). >>>>>> >>>>>> You can't safely lock a mutex until the current thread has attached >>>>>> to the VM and Thread::current() returns non-null (that is what >>>>>> Thread::initialize_thread_local_storage() currently does). You may >>>>>> get away with it, particularly in product builds, during VM >>>>>> initialization, as the fast-lock path doesn't require access to >>>>>> Thread::current(). >>>>>> >>>>>> David >>>>>> >>>>>>> Most of fields (include _rotation_lock) in LogFileOutput are >>>>>>> initialized >>>>>>> at c'tor. >>>>>>> Other field (_stream) is initialized at LogFileOutput::initialize(). >>>>>>> >>>>>>> We can avoid problem if we can move initialize() to c'tor because >>>>>>> LogFileOutput >>>>>>> will be instantiated at LogConfiguration::new_output(). >>>>>>> >>>>>>> Currently, LogFileOutput will be added to LogConfiguration::_outputs >>>>>>> and >>>>>>> to LogTagSet after calling LogFileOutput::initialize(). >>>>>>> I wonder why we cannot avoid crash/assert with current >>>>>>> implementation... >>>>>>> >>>>>>> >>>>>>> I will keep current implementation about _rotation_lock if the above is >>>>>>> incorrect. >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2015/10/31 0:31, Marcus Larsson wrote: >>>>>>>> Hey, >>>>>>>> >>>>>>>> On 2015-10-30 15:39, Yasumasa Suenaga wrote: >>>>>>>>> Hi Marcus, >>>>>>>>> >>>>>>>>> Thank you for your comment. >>>>>>>>> >>>>>>>>> >>>>>>>>>> Still missing the renaming of rotate_all_logfile to >>>>>>>>>> rotate_all_outputs. >>>>>>>>> >>>>>>>>> Sorry, I will fix. >>>>>>>>> >>>>>>>> >>>>>>>> Thanks! >>>>>>>> >>>>>>>>> >>>>>>>>>> This rotate function works I guess, but it will probably need some >>>>>>>>>> rework once there are other types of log outputs (rotate doesn't >>>>>>>>>> make sense on all types of log outputs). >>>>>>>>> >>>>>>>>> I will add is_rotetable() to LogOutput as virtual function. >>>>>>>>> This function return false (can not rotate) by default, return true >>>>>>>>> if _file_count is >>>>>>>>> greater than 0 in LogFileOutput. >>>>>>>>> >>>>>>>>> LogConfiguration::rotate_all_outputs() will rotate if this function >>>>>>>>> returns true. >>>>>>>>> >>>>>>>> >>>>>>>> Small typo, please make that is_rotatable(). >>>>>>>> >>>>>>>>> >>>>>>>>>> If we configure a rotated file output and then log something on that >>>>>>>>>> output before thread local storage initialization is complete, we >>>>>>>>>> will crash/hit an assert. The previous implementation avoided this >>>>>>>>>> as long as no rotation was needed before this initialization was >>>>>>>>>> complete. (This can be triggered using the following arguments >>>>>>>>>> "-Xlog:all=trace:file.txt::filesize=10,filecount=2 -Xlog:invalid", >>>>>>>>>> for example.) >>>>>>>>> >>>>>>>>> I do not think so. >>>>>>>>> Each -Xlog option creates unique instance of LogOutput (and >>>>>>>>> subclass). >>>>>>>>> So they are isolated. >>>>>>>> >>>>>>>> My example includes a second (and intentionally incorrect) -Xlog >>>>>>>> option only to trigger logging on the newly configured rotating >>>>>>>> LogFileOutput. This will cause some logging about the invalid -Xlog >>>>>>>> argument, and this logging happens during the argument parsing. This >>>>>>>> in turn means that we are logging before the thread local storage has >>>>>>>> been initialized, and the JVM will crash/hit the assert because we are >>>>>>>> trying to take the rotation lock. This has nothing to do with >>>>>>>> concurrent fprintfs or freopens, even if that is a problem. >>>>>>>> >>>>>>>>> >>>>>>>>> However, we might have to rotate at safepoint. >>>>>>>> >>>>>>>> Doing stuff at a safepoint will not give us much in this case. There >>>>>>>> are threads that continue execution through a safepoint, which means >>>>>>>> we will see log calls being made even during this time. >>>>>>>> >>>>>>>>> Currently, LogOutput might be used by multiple threads concurrently. >>>>>>>>> If we rotate log when another thread is writing string to output, we >>>>>>>>> encounter unexpected behavior. >>>>>>>>> >>>>>>>>> LogFileOutput::rotate() uses freopen(). >>>>>>>>> LogFileStreamOutput::write() uses vfprintf() through jio_fprintf(). >>>>>>>>> freopen() and vfprintf() are not atomic. >>>>>>>> >>>>>>>> You're right. There seems to be an overlooked concurrency issue with >>>>>>>> vfprintfs during freopen. I don't think we should resolve this as part >>>>>>>> of this particular fix, so instead I'll create a separate issue for >>>>>>>> it. >>>>>>>> >>>>>>>>> >>>>>>>>> I think that cause of crash/assertion which you say is it. >>>>>>>>> (GC log will be rotated at safepoint.) >>>>>>>>> >>>>>>>> >>>>>>>> See my comment above for what crash/assert I'm talking about. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Marcus >>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2015/10/29 23:58, Marcus Larsson wrote: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> On 2015-10-29 15:01, Yasumasa Suenaga wrote: >>>>>>>>>>> Hi Marcus, >>>>>>>>>>> >>>>>>>>>>> Thank you for your comment. >>>>>>>>>>> >>>>>>>>>>>> I'll sponsor it. >>>>>>>>>>> >>>>>>>>>>> Thank you so much! >>>>>>>>>>> I've uploaded new webrev: >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.01/ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Still missing the renaming of rotate_all_logfile to >>>>>>>>>> rotate_all_outputs. >>>>>>>>>> >>>>>>>>>>>> logConfiguration.cpp/hpp: >>>>>>>>>>>> >>>>>>>>>>>> * I don't think we should rely on the archive_name or the output's >>>>>>>>>>>> name to decide whether or not an output should be rotated. It >>>>>>>>>>>> would be better to introduce an is_rotated() test function in >>>>>>>>>>>> LogOutput that could be used here. >>>>>>>>>>>> >>>>>>>>>>>> * rotate_all_logfile() should be renamed to rotate_all_outputs(). >>>>>>>>>>>> Currently there are only LogFileOutputs (other than >>>>>>>>>>>> stdout/stderr), but in the future there might be other types of >>>>>>>>>>>> outputs so I prefer having a more general name. >>>>>>>>>>>> Also, please use static_cast instead of the >>>>>>>>>>>> C-style cast. (Additional logic will be required here once more >>>>>>>>>>>> types of log outputs are added, but I don't think we need to worry >>>>>>>>>>>> about this right now.) >>>>>>>>>>>> >>>>>>>>>>>> * Don't print an error if an output is not rotatable, since it's >>>>>>>>>>>> not an error to have some log outputs rotated while others are >>>>>>>>>>>> not. If you really want some traceability here I suggest adding >>>>>>>>>>>> log messages on trace level, tagged with 'logging'. >>>>>>>>>>> >>>>>>>>>>> I added LogOutput::rotate(bool) as virtual function. >>>>>>>>>>> This function works nothing by default, but will rotate all logs at >>>>>>>>>>> LogFileOutput. >>>>>>>>>> >>>>>>>>>> This rotate function works I guess, but it will probably need some >>>>>>>>>> rework once there are other types of log outputs (rotate doesn't >>>>>>>>>> make sense on all types of log outputs). >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> If true is passed to this function, all files will be rotated. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> logDiagnosticCommand.cpp/hpp: >>>>>>>>>>>> >>>>>>>>>>>> * I think the description could be improved to something like >>>>>>>>>>>> "Lists current log configuration, enables/disables/configures a >>>>>>>>>>>> log output, or disables/rotates all logs." >>>>>>>>>>>> >>>>>>>>>>>> * The rotate option description should clarify that all logs will >>>>>>>>>>>> be rotated ("current log" is too ambiguous). >>>>>>>>>>> >>>>>>>>>>> I've fixed. >>>>>>>>>> >>>>>>>>>> Great, thanks! >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> logFileOutput.cpp/hpp: >>>>>>>>>>>> >>>>>>>>>>>> * Moving the MutexLocker like this introduces a race condition >>>>>>>>>>>> where a log might be rotated multiple times by different threads, >>>>>>>>>>>> instead of just once. >>>>>>>>>>>> Instead of making the rotate() function public and moving the >>>>>>>>>>>> mutexlocker, I suggest adding something like a public >>>>>>>>>>>> force_rotation() function that grabs the lock and calls the >>>>>>>>>>>> private rotate(). >>>>>>>>>>>> >>>>>>>>>>>> * Given the addition of is_rotated() in LogOutput, then >>>>>>>>>>>> get_archive_name() should be removed. >>>>>>>>>>> >>>>>>>>>>> I moved MutexLocker and should_rotate() to rotate(). >>>>>>>>>>> I think this change can avoid race condition. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> There is a subtle problem with taking the lock to check if we should >>>>>>>>>> rotate. If we configure a rotated file output and then log something >>>>>>>>>> on that output before thread local storage initialization is >>>>>>>>>> complete, we will crash/hit an assert. The previous implementation >>>>>>>>>> avoided this as long as no rotation was needed before this >>>>>>>>>> initialization was complete. (This can be triggered using the >>>>>>>>>> following arguments >>>>>>>>>> "-Xlog:all=trace:file.txt::filesize=10,filecount=2 -Xlog:invalid", >>>>>>>>>> for example.) >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Marcus >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 2015/10/27 21:17, Marcus Larsson wrote: >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> On 2015-10-27 01:03, Yasumasa Suenaga wrote: >>>>>>>>>>>>> Hi Marcus, >>>>>>>>>>>>> >>>>>>>>>>>>> Thank you for replying. >>>>>>>>>>>>> >>>>>>>>>>>>> I filed this feature to JBS and change subject of this email. >>>>>>>>>>>>> Could you be a sponsor and review it? >>>>>>>>>>>> >>>>>>>>>>>> I'll sponsor it. >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-8140556/webrev.00/ >>>>>>>>>>>> >>>>>>>>>>>> logConfiguration.cpp/hpp: >>>>>>>>>>>> >>>>>>>>>>>> * I don't think we should rely on the archive_name or the output's >>>>>>>>>>>> name to decide whether or not an output should be rotated. It >>>>>>>>>>>> would be better to introduce an is_rotated() test function in >>>>>>>>>>>> LogOutput that could be used here. >>>>>>>>>>>> >>>>>>>>>>>> * rotate_all_logfile() should be renamed to rotate_all_outputs(). >>>>>>>>>>>> Currently there are only LogFileOutputs (other than >>>>>>>>>>>> stdout/stderr), but in the future there might be other types of >>>>>>>>>>>> outputs so I prefer having a more general name. >>>>>>>>>>>> Also, please use static_cast instead of the >>>>>>>>>>>> C-style cast. (Additional logic will be required here once more >>>>>>>>>>>> types of log outputs are added, but I don't think we need to worry >>>>>>>>>>>> about this right now.) >>>>>>>>>>>> >>>>>>>>>>>> * Don't print an error if an output is not rotatable, since it's >>>>>>>>>>>> not an error to have some log outputs rotated while others are >>>>>>>>>>>> not. If you really want some traceability here I suggest adding >>>>>>>>>>>> log messages on trace level, tagged with 'logging'. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> logDiagnosticCommand.cpp/hpp: >>>>>>>>>>>> >>>>>>>>>>>> * I think the description could be improved to something like >>>>>>>>>>>> "Lists current log configuration, enables/disables/configures a >>>>>>>>>>>> log output, or disables/rotates all logs." >>>>>>>>>>>> >>>>>>>>>>>> * The rotate option description should clarify that all logs will >>>>>>>>>>>> be rotated ("current log" is too ambiguous). >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> logFileOutput.cpp/hpp: >>>>>>>>>>>> >>>>>>>>>>>> * Moving the MutexLocker like this introduces a race condition >>>>>>>>>>>> where a log might be rotated multiple times by different threads, >>>>>>>>>>>> instead of just once. >>>>>>>>>>>> Instead of making the rotate() function public and moving the >>>>>>>>>>>> mutexlocker, I suggest adding something like a public >>>>>>>>>>>> force_rotation() function that grabs the lock and calls the >>>>>>>>>>>> private rotate(). >>>>>>>>>>>> >>>>>>>>>>>> * Given the addition of is_rotated() in LogOutput, then >>>>>>>>>>>> get_archive_name() should be removed. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Marcus >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> >>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 2015/10/26 18:56, Marcus Larsson wrote: >>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Sorry for my late reply. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I think being able to force rotation via jcmd seems like a good >>>>>>>>>>>>>> feature. Files are currently opened in append mode so it should >>>>>>>>>>>>>> already be possible to use external log rotation tools by >>>>>>>>>>>>>> copying and truncating the files. Still I think it would be nice >>>>>>>>>>>>>> to provide the jcmd for rotation as well. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I can see some small issues with the implementation, but we can >>>>>>>>>>>>>> deal with that during the review. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Marcus >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 2015-10-26 00:26, Yasumasa Suenaga wrote: >>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Have you ever seen my change? >>>>>>>>>>>>>>> I (and my customers) need log rotation function via external >>>>>>>>>>>>>>> tool. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I want to merge it by Feature Complete. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>> 2015/10/16 22:55 "Yasumasa Suenaga" : >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I contributed JDK-7090324: gclog rotation via external tool >>>>>>>>>>>>>>>> to be >>>>>>>>>>>>>>>> synchronized with >>>>>>>>>>>>>>>> logrotated tool on Linux. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I think JEP 158 is in progress. >>>>>>>>>>>>>>>> However, this JEP does not contain log rotation via external >>>>>>>>>>>>>>>> tool in the >>>>>>>>>>>>>>>> spec. >>>>>>>>>>>>>>>> I want to rotate logs via jcmd in this JEP. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I've updated a patch for it: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/jvmlogging-logrotate/ >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> This patch provides new option "rotate" in VM.log command. >>>>>>>>>>>>>>>> If this change can be accepted, I will file it to JBS and send >>>>>>>>>>>>>>>> RFR. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> >>>>>>>> >> From david.holmes at oracle.com Mon Nov 16 04:03:54 2015 From: david.holmes at oracle.com (David Holmes) Date: Mon, 16 Nov 2015 14:03:54 +1000 Subject: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM In-Reply-To: <5645E7D0.1000704@oracle.com> References: <4295855A5C1DE049A61835A1887419CC41EB5DD6@DEWDFEMB12A.global.corp.sap> <5644413D.1000004@oracle.com> <56444C29.8060606@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB8252@DEWDFEMB12A.global.corp.sap> <56445646.4040707@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB85C3@DEWDFEMB12A.global.corp.sap> <5645E7D0.1000704@oracle.com> Message-ID: <564955AA.3020704@oracle.com> On 13/11/2015 11:38 PM, David Holmes wrote: > On 13/11/2015 7:53 PM, Thomas St?fe wrote: >> Hi Goetz, >> >> sorry for not looking at this earlier. This is a nice cleanup. Some >> remarks: >> >> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/aix/vm/os_aix.cpp.udiff.html >> >> >> + if (sig > MAX2(SIGSEGV, SIGBUS) && // See 4355769. >> + sig < NSIG) { // Must be legal signal and fit >> into sigflags[]. >> >> I do not like much the MAX2() construct. I would like it better to >> explicitly check whether the SR signal is one of the "forbidden" ones >> the VM uses. > > I must confess I had not looked into 4355769 but this check seems rather > spurious. It is not at all clear to me what signals could be used here - Okay should have looked into 4355769. The problem is how multiple pending signals are handled. It seems that in the past (no idea if still true) pending signals were handled in signal-number order (lowest first), not FIFO. The problem scenario is this: - thread accesses a null pointer in compiled Java code and the SEGV handler will cause NPE to be thrown - at the same time as the SEGV is being raised the thread is also hit with the SR signal to suspend it. - the SR signal will be delivered first and the SR handler starts to run - with signals unblocked. - the SEGV then gets delivered to the thread in the SR handler, and the regular signal handler is run - the regular signal handler tries to detect if we're running in Java code so it can post the NPE, but the presence of the SR handler causes that check to fail - so we abort thinking it is a real SEGV. I don't know how much of that is still true today. It seems strange to me that a kill based directed signal can usurp a synchronous signal. Anyway the fix, rather workaround, for that problem, was to ensure that the SR_signum is greater than any potential synchronous signal the VM cares about. Why SIGBUS was included there I don't know give that: a) it is already a lower signal number than SIGUSR1, SIGSEGV and SIGUSR2 b) we don't deliberately generate and use SIGBUS ... though perhaps unsafe-fetch needs to be considered. A better fix in my opinion, and as mentioned in the bug, would have been to disable delivery of SEGV whilst the SR handler is executing. But we start to touch on some grey areas of the POSIX spec there, and likely the implementation too. So I suggest that for this cleanup we simply leave this logic exactly as is. Thanks, David > other than SIGUSR1 or SIGUSR2 (if -Xrs is specified), or else a > real-time signal (modulo discussion below). Hijacking anything else > seems rather suspect. > >> Maybe keep a mask defined centrally for each platform which contains >> signals the VM needs for itself ? > > Such masks already exist. > >> +sigset_t os::Aix::sigs = { 0 }; >> >> I would not initialize the signal set this way. sigset_t is an opaque >> type; the only way to initialize it is with one of sigemptyset() or >> sigfillset(). > > Good catch - I overlooked that. > >> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/aix/vm/os_aix.hpp.udiff.html >> >> >> + static struct sigaction sigact[NSIG]; // saved preinstalled sigactions >> + static sigset_t sigs; // mask of signals that have >> >> + static int sigflags[NSIG]; >> >> I know this is not in the scope of your change, but I would like to see >> those removed from os::Aix and put into os_aix.cpp at static filescope. >> There is no need at all to export those, and you would get rid of the >> signal.h dependency you know have when including os_aix.hpp. >> >> >> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/bsd/vm/jsig.c.udiff.html >> >> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/bsd/vm/os_bsd.cpp.udiff.html >> >> >> On BSD, we have realtime signals. >> >> http://fxr.watson.org/fxr/source/sys/signal.h >> #define SIGRTMAX 126 >> and NSIG does not contain them: >> #define NSIG 32 >> >> The max. possible signal number would be 126, which unfortunately does >> not even fit into a 64bit mask. > > So this simply limits the signal choice to not be a real-time signal - > same as today. > >> So the code in jsig.c is broken for the case that someone wants to >> register realtime signals, if the VM were to ever use realtime signals >> itself, which now is not the case. >> >> The same is true for os_bsd.cpp, where signal chaining will not work if >> the application did have handler for real time signals pre-installed >> before jvm is loaded. > > Chaining is only used when the JVM will catch signals. Aren't all the > real-time signals going to be blocked by the VM by default and so > chaining is not needed as no handler will exist in the VM ?? (Unless a > real-time signal is supplied for SR_signum) > > I must admit I don't know if any of this code actually works for > real-time signals. > >> Solaris: >> >> The only platform where NSIG is missing? >> >> Here, we calculate the max. signal number dynamically in os_solaris.cpp, >> presumably because SIGRTMAX is not a constant and can be changed using >> system configuration. But then, on Linux we have the same situation >> (SIGRTMAX is dynamic) and there we do not go through the trouble of >> calculating the max. signal number dynamically. Instead we just use >> NSIG=64 and rely on the fact that NSIG is larger than the largest >> possible dynamic value for SIGRTMAX. > > Linux ensures that _NSIG (and thus NSIG) includes all the real-time > signals. But libc can expose a subset and steal some for its own use. > >> Solaris does not seem to have NSIG defined, but I am sure there is also >> a max. possible value for SIGRTMAX (the default seems to be 48). So, one >> could probably safely define NSIG for Solaris too, so that we have NSIG >> defined on all Posix platforms. > > Solaris doesn't have any of this SR_signum related code. A more general > cleanup of signal related code would potentially involve a lot of cleanup. > > David > ----- > >> >> On Thu, Nov 12, 2015 at 8:24 PM, Lindenmaier, Goetz >> > wrote: >> >> Hi David, Dmitry, >> >> I've come up with a new webrev: >> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/ >> >> I hit on some more issues: >> - As proposed, I replaced MAXSIGNUM by NSIG >> - On AIX, NSIG=255. Therefore storing bits in a word does not >> work. >> I'm now using bitset functionality from signal.h as it's done >> in other places. >> sigset_t is >> NSIG on linux, so it's no good idea to use it >> there. >> >> >> Why do we not do this on all platforms, provided sigset_t contains all >> signals (incl. realtime signals) ? >> >> - In the os files I found another bit vector that now is too >> small: sigs. >> I adapted that, too. Removed the dead declaration of this on >> solaris. >> >> Best regards, >> Goetz. >> >> >> >> Kind Regards, Thomas >> >> > -----Original Message----- >> > From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com >> ] >> > Sent: Donnerstag, 12. November 2015 10:05 >> > To: Lindenmaier, Goetz; David Holmes; hotspot-runtime- >> > dev at openjdk.java.net ; >> serviceability-dev >> > Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM >> > >> > Goetz, >> > >> > *BSD including OS X also defines NSIG (just checked) and if my >> memory is >> > not bogus, AIX defines it too. >> > >> > So you may consider to use NSIG on all platform. >> > >> > -Dmitry >> > >> > On 2015-11-12 11:36, Lindenmaier, Goetz wrote: >> > > OK I'll change it to NSIG. That's used in other places in >> os_linux, too. >> > > So it's really more consistent. >> > > >> > > Best regards, >> > > Goetz >> > > >> > >> -----Original Message----- >> > >> From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com >> ] >> > >> Sent: Donnerstag, 12. November 2015 09:22 >> > >> To: David Holmes; Lindenmaier, Goetz; hotspot-runtime- >> > >> dev at openjdk.java.net ; >> serviceability-dev >> > >> Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM >> > >> >> > >> David, >> > >> >> > >> I think it's better to use NSIG (without underscore) defined >> in signal.h >> > >> >> > >> -Dmitry >> > >> >> > >> >> > >> On 2015-11-12 10:35, David Holmes wrote: >> > >>> Hi Goetz, >> > >>> >> > >>> Adding in serviceability-dev >> > >>> >> > >>> On 9/11/2015 6:22 PM, Lindenmaier, Goetz wrote: >> > >>>> Hi, >> > >>>> >> > >>>> The environment variable _JAVA_SR_SIGNUM can be set to a >> signal >> > >> number >> > >>>> do be used by the JVM's suspend/resume mechanism. >> > >>>> >> > >>>> If set, a signal handler is installed and the current signal >> handler >> > >>>> is saved to an array. >> > >>>> On linux, this array had size MAXSIGNUM=32, and >> _JAVA_SR_SIGNUM >> > >> was >> > >>>> allowed >> > >>>> to range up to _NSIG=65. This could cause memory corruption. >> > >>>> >> > >>>> Further, in jsig.c, an unsinged int is used to set a bit for >> signals. >> > >>>> This also >> > >>>> is too small, as only 32 signals can be supported. >> Further, the >> > >>>> signals are mapped >> > >>>> wrong to these bits. '0' is not a valid signal, but '32' >> was. 1<<32 >> > >>>> happens to map to >> > >>>> zero, so the signal could be stored, but this probably was >> not >> > >>>> intended that way. >> > >>>> >> > >>>> This change increases MAXSIGNUM to 65 on linux, and to 64 on >> aix. It >> > >>>> introduces >> > >>>> proper checking of the signal read from the env var, and >> issues a >> > >>>> warning if it >> > >>>> does not use the signal set. It adapts the data types in >> jisig.c >> > >>>> properly. >> > >>>> >> > >>>> Please review this change. I please need a sponsor. >> > >>>> >> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.00 >> > >>> >> > >>> This all sounds very good to me. (I must find out why Solaris >> is not >> > >>> involved here :) ). >> > >>> >> > >>> On Linux you didn't add the bounds check to >> os::Linux::set_our_sigflags. >> > >>> >> > >>> I'm also wondering about documenting where we are >> determining the >> > >>> maximum from? Is it simply _NSIG on some/all distributions? >> And I see >> > >>> _NSIG is supposed to be the biggest signal number + one. Also >> linux >> > >>> defines NSIG = _NSIG so which should we be using? >> > >>> >> > >>> Thanks, >> > >>> David >> > >>> >> > >>>> Best regards, >> > >>>> Goetz. >> > >>>> >> > >> >> > >> >> > >> -- >> > >> Dmitry Samersoff >> > >> Oracle Java development team, Saint Petersburg, Russia >> > >> * I would love to change the world, but they won't give me the >> sources. >> > >> > >> > -- >> > Dmitry Samersoff >> > Oracle Java development team, Saint Petersburg, Russia >> > * I would love to change the world, but they won't give me the >> sources. >> >> From dmitry.samersoff at oracle.com Mon Nov 16 08:23:21 2015 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Mon, 16 Nov 2015 11:23:21 +0300 Subject: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM In-Reply-To: <564955AA.3020704@oracle.com> References: <4295855A5C1DE049A61835A1887419CC41EB5DD6@DEWDFEMB12A.global.corp.sap> <5644413D.1000004@oracle.com> <56444C29.8060606@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB8252@DEWDFEMB12A.global.corp.sap> <56445646.4040707@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB85C3@DEWDFEMB12A.global.corp.sap> <5645E7D0.1000704@oracle.com> <564955AA.3020704@oracle.com> Message-ID: <56499279.2050806@oracle.com> David, Thank you for detailed explanation. Could we put it as a comment right into the code? -Dmitry On 2015-11-16 07:03, David Holmes wrote: > On 13/11/2015 11:38 PM, David Holmes wrote: >> On 13/11/2015 7:53 PM, Thomas St?fe wrote: >>> Hi Goetz, >>> >>> sorry for not looking at this earlier. This is a nice cleanup. Some >>> remarks: >>> >>> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/aix/vm/os_aix.cpp.udiff.html >>> >>> >>> >>> + if (sig > MAX2(SIGSEGV, SIGBUS) && // See 4355769. >>> + sig < NSIG) { // Must be legal signal and fit >>> into sigflags[]. >>> >>> I do not like much the MAX2() construct. I would like it better to >>> explicitly check whether the SR signal is one of the "forbidden" ones >>> the VM uses. >> >> I must confess I had not looked into 4355769 but this check seems rather >> spurious. It is not at all clear to me what signals could be used here - > > Okay should have looked into 4355769. The problem is how multiple > pending signals are handled. It seems that in the past (no idea if still > true) pending signals were handled in signal-number order (lowest > first), not FIFO. The problem scenario is this: > - thread accesses a null pointer in compiled Java code and the SEGV > handler will cause NPE to be thrown > - at the same time as the SEGV is being raised the thread is also hit > with the SR signal to suspend it. > - the SR signal will be delivered first and the SR handler starts to run > - with signals unblocked. > - the SEGV then gets delivered to the thread in the SR handler, and the > regular signal handler is run > - the regular signal handler tries to detect if we're running in Java > code so it can post the NPE, but the presence of the SR handler causes > that check to fail - so we abort thinking it is a real SEGV. > > I don't know how much of that is still true today. It seems strange to > me that a kill based directed signal can usurp a synchronous signal. > > Anyway the fix, rather workaround, for that problem, was to ensure that > the SR_signum is greater than any potential synchronous signal the VM > cares about. Why SIGBUS was included there I don't know give that: > > a) it is already a lower signal number than SIGUSR1, SIGSEGV and SIGUSR2 > b) we don't deliberately generate and use SIGBUS ... though perhaps > unsafe-fetch needs to be considered. > > A better fix in my opinion, and as mentioned in the bug, would have been > to disable delivery of SEGV whilst the SR handler is executing. But we > start to touch on some grey areas of the POSIX spec there, and likely > the implementation too. > > So I suggest that for this cleanup we simply leave this logic exactly as > is. > > Thanks, > David > >> other than SIGUSR1 or SIGUSR2 (if -Xrs is specified), or else a >> real-time signal (modulo discussion below). Hijacking anything else >> seems rather suspect. >> >>> Maybe keep a mask defined centrally for each platform which contains >>> signals the VM needs for itself ? >> >> Such masks already exist. >> >>> +sigset_t os::Aix::sigs = { 0 }; >>> >>> I would not initialize the signal set this way. sigset_t is an opaque >>> type; the only way to initialize it is with one of sigemptyset() or >>> sigfillset(). >> >> Good catch - I overlooked that. >> >>> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/aix/vm/os_aix.hpp.udiff.html >>> >>> >>> >>> + static struct sigaction sigact[NSIG]; // saved preinstalled >>> sigactions >>> + static sigset_t sigs; // mask of signals that have >>> >>> + static int sigflags[NSIG]; >>> >>> I know this is not in the scope of your change, but I would like to see >>> those removed from os::Aix and put into os_aix.cpp at static filescope. >>> There is no need at all to export those, and you would get rid of the >>> signal.h dependency you know have when including os_aix.hpp. >>> >>> >>> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/bsd/vm/jsig.c.udiff.html >>> >>> >>> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/bsd/vm/os_bsd.cpp.udiff.html >>> >>> >>> >>> On BSD, we have realtime signals. >>> >>> http://fxr.watson.org/fxr/source/sys/signal.h >>> #define SIGRTMAX 126 >>> and NSIG does not contain them: >>> #define NSIG 32 >>> >>> The max. possible signal number would be 126, which unfortunately does >>> not even fit into a 64bit mask. >> >> So this simply limits the signal choice to not be a real-time signal - >> same as today. >> >>> So the code in jsig.c is broken for the case that someone wants to >>> register realtime signals, if the VM were to ever use realtime signals >>> itself, which now is not the case. >>> >>> The same is true for os_bsd.cpp, where signal chaining will not work if >>> the application did have handler for real time signals pre-installed >>> before jvm is loaded. >> >> Chaining is only used when the JVM will catch signals. Aren't all the >> real-time signals going to be blocked by the VM by default and so >> chaining is not needed as no handler will exist in the VM ?? (Unless a >> real-time signal is supplied for SR_signum) >> >> I must admit I don't know if any of this code actually works for >> real-time signals. >> >>> Solaris: >>> >>> The only platform where NSIG is missing? >>> >>> Here, we calculate the max. signal number dynamically in os_solaris.cpp, >>> presumably because SIGRTMAX is not a constant and can be changed using >>> system configuration. But then, on Linux we have the same situation >>> (SIGRTMAX is dynamic) and there we do not go through the trouble of >>> calculating the max. signal number dynamically. Instead we just use >>> NSIG=64 and rely on the fact that NSIG is larger than the largest >>> possible dynamic value for SIGRTMAX. >> >> Linux ensures that _NSIG (and thus NSIG) includes all the real-time >> signals. But libc can expose a subset and steal some for its own use. >> >>> Solaris does not seem to have NSIG defined, but I am sure there is also >>> a max. possible value for SIGRTMAX (the default seems to be 48). So, one >>> could probably safely define NSIG for Solaris too, so that we have NSIG >>> defined on all Posix platforms. >> >> Solaris doesn't have any of this SR_signum related code. A more general >> cleanup of signal related code would potentially involve a lot of >> cleanup. >> >> David >> ----- >> >>> >>> On Thu, Nov 12, 2015 at 8:24 PM, Lindenmaier, Goetz >>> > wrote: >>> >>> Hi David, Dmitry, >>> >>> I've come up with a new webrev: >>> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/ >>> >>> I hit on some more issues: >>> - As proposed, I replaced MAXSIGNUM by NSIG >>> - On AIX, NSIG=255. Therefore storing bits in a word does not >>> work. >>> I'm now using bitset functionality from signal.h as it's done >>> in other places. >>> sigset_t is >> NSIG on linux, so it's no good idea to use it >>> there. >>> >>> >>> Why do we not do this on all platforms, provided sigset_t contains all >>> signals (incl. realtime signals) ? >>> >>> - In the os files I found another bit vector that now is too >>> small: sigs. >>> I adapted that, too. Removed the dead declaration of this on >>> solaris. >>> >>> Best regards, >>> Goetz. >>> >>> >>> >>> Kind Regards, Thomas >>> >>> > -----Original Message----- >>> > From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com >>> ] >>> > Sent: Donnerstag, 12. November 2015 10:05 >>> > To: Lindenmaier, Goetz; David Holmes; hotspot-runtime- >>> > dev at openjdk.java.net ; >>> serviceability-dev >>> > Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM >>> > >>> > Goetz, >>> > >>> > *BSD including OS X also defines NSIG (just checked) and if my >>> memory is >>> > not bogus, AIX defines it too. >>> > >>> > So you may consider to use NSIG on all platform. >>> > >>> > -Dmitry >>> > >>> > On 2015-11-12 11:36, Lindenmaier, Goetz wrote: >>> > > OK I'll change it to NSIG. That's used in other places in >>> os_linux, too. >>> > > So it's really more consistent. >>> > > >>> > > Best regards, >>> > > Goetz >>> > > >>> > >> -----Original Message----- >>> > >> From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com >>> ] >>> > >> Sent: Donnerstag, 12. November 2015 09:22 >>> > >> To: David Holmes; Lindenmaier, Goetz; hotspot-runtime- >>> > >> dev at openjdk.java.net ; >>> serviceability-dev >>> > >> Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM >>> > >> >>> > >> David, >>> > >> >>> > >> I think it's better to use NSIG (without underscore) defined >>> in signal.h >>> > >> >>> > >> -Dmitry >>> > >> >>> > >> >>> > >> On 2015-11-12 10:35, David Holmes wrote: >>> > >>> Hi Goetz, >>> > >>> >>> > >>> Adding in serviceability-dev >>> > >>> >>> > >>> On 9/11/2015 6:22 PM, Lindenmaier, Goetz wrote: >>> > >>>> Hi, >>> > >>>> >>> > >>>> The environment variable _JAVA_SR_SIGNUM can be set to a >>> signal >>> > >> number >>> > >>>> do be used by the JVM's suspend/resume mechanism. >>> > >>>> >>> > >>>> If set, a signal handler is installed and the current signal >>> handler >>> > >>>> is saved to an array. >>> > >>>> On linux, this array had size MAXSIGNUM=32, and >>> _JAVA_SR_SIGNUM >>> > >> was >>> > >>>> allowed >>> > >>>> to range up to _NSIG=65. This could cause memory corruption. >>> > >>>> >>> > >>>> Further, in jsig.c, an unsinged int is used to set a bit for >>> signals. >>> > >>>> This also >>> > >>>> is too small, as only 32 signals can be supported. >>> Further, the >>> > >>>> signals are mapped >>> > >>>> wrong to these bits. '0' is not a valid signal, but '32' >>> was. 1<<32 >>> > >>>> happens to map to >>> > >>>> zero, so the signal could be stored, but this probably was >>> not >>> > >>>> intended that way. >>> > >>>> >>> > >>>> This change increases MAXSIGNUM to 65 on linux, and to 64 on >>> aix. It >>> > >>>> introduces >>> > >>>> proper checking of the signal read from the env var, and >>> issues a >>> > >>>> warning if it >>> > >>>> does not use the signal set. It adapts the data types in >>> jisig.c >>> > >>>> properly. >>> > >>>> >>> > >>>> Please review this change. I please need a sponsor. >>> > >>>> >>> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.00 >>> > >>> >>> > >>> This all sounds very good to me. (I must find out why Solaris >>> is not >>> > >>> involved here :) ). >>> > >>> >>> > >>> On Linux you didn't add the bounds check to >>> os::Linux::set_our_sigflags. >>> > >>> >>> > >>> I'm also wondering about documenting where we are >>> determining the >>> > >>> maximum from? Is it simply _NSIG on some/all distributions? >>> And I see >>> > >>> _NSIG is supposed to be the biggest signal number + one. Also >>> linux >>> > >>> defines NSIG = _NSIG so which should we be using? >>> > >>> >>> > >>> Thanks, >>> > >>> David >>> > >>> >>> > >>>> Best regards, >>> > >>>> Goetz. >>> > >>>> >>> > >> >>> > >> >>> > >> -- >>> > >> Dmitry Samersoff >>> > >> Oracle Java development team, Saint Petersburg, Russia >>> > >> * I would love to change the world, but they won't give me the >>> sources. >>> > >>> > >>> > -- >>> > Dmitry Samersoff >>> > Oracle Java development team, Saint Petersburg, Russia >>> > * I would love to change the world, but they won't give me the >>> sources. >>> >>> -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From thomas.stuefe at gmail.com Mon Nov 16 09:45:09 2015 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 16 Nov 2015 10:45:09 +0100 Subject: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM In-Reply-To: <564955AA.3020704@oracle.com> References: <4295855A5C1DE049A61835A1887419CC41EB5DD6@DEWDFEMB12A.global.corp.sap> <5644413D.1000004@oracle.com> <56444C29.8060606@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB8252@DEWDFEMB12A.global.corp.sap> <56445646.4040707@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB85C3@DEWDFEMB12A.global.corp.sap> <5645E7D0.1000704@oracle.com> <564955AA.3020704@oracle.com> Message-ID: Hi David, On Mon, Nov 16, 2015 at 5:03 AM, David Holmes wrote: > On 13/11/2015 11:38 PM, David Holmes wrote: > >> On 13/11/2015 7:53 PM, Thomas St?fe wrote: >> >>> Hi Goetz, >>> >>> sorry for not looking at this earlier. This is a nice cleanup. Some >>> remarks: >>> >>> >>> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/aix/vm/os_aix.cpp.udiff.html >>> >>> >>> + if (sig > MAX2(SIGSEGV, SIGBUS) && // See 4355769. >>> + sig < NSIG) { // Must be legal signal and fit >>> into sigflags[]. >>> >>> I do not like much the MAX2() construct. I would like it better to >>> explicitly check whether the SR signal is one of the "forbidden" ones >>> the VM uses. >>> >> >> I must confess I had not looked into 4355769 but this check seems rather >> spurious. It is not at all clear to me what signals could be used here - >> > > Okay should have looked into 4355769. The problem is how multiple pending > signals are handled. It seems that in the past (no idea if still true) > pending signals were handled in signal-number order (lowest first), not > FIFO. The problem scenario is this: > - thread accesses a null pointer in compiled Java code and the SEGV > handler will cause NPE to be thrown > - at the same time as the SEGV is being raised the thread is also hit with > the SR signal to suspend it. > - the SR signal will be delivered first and the SR handler starts to run - > with signals unblocked. > - the SEGV then gets delivered to the thread in the SR handler, and the > regular signal handler is run > - the regular signal handler tries to detect if we're running in Java code > so it can post the NPE, but the presence of the SR handler causes that > check to fail - so we abort thinking it is a real SEGV. > > I don't know how much of that is still true today. It seems strange to me > that a kill based directed signal can usurp a synchronous signal. > > Anyway the fix, rather workaround, for that problem, was to ensure that > the SR_signum is greater than any potential synchronous signal the VM cares > about. Why SIGBUS was included there I don't know give that: > > a) it is already a lower signal number than SIGUSR1, SIGSEGV and SIGUSR2 > b) we don't deliberately generate and use SIGBUS ... though perhaps > unsafe-fetch needs to be considered. > > A better fix in my opinion, and as mentioned in the bug, would have been > to disable delivery of SEGV whilst the SR handler is executing. But we > start to touch on some grey areas of the POSIX spec there, and likely the > implementation too. > How would this work? I think the process just dies immediately if an synchronous signal occurs while being blocked. At least that is why we needed to fix https://bugs.openjdk.java.net/browse/JDK-8065895. Kind Regards, Thomas > So I suggest that for this cleanup we simply leave this logic exactly as > is. > > Thanks, > David > > > other than SIGUSR1 or SIGUSR2 (if -Xrs is specified), or else a >> real-time signal (modulo discussion below). Hijacking anything else >> seems rather suspect. >> >> Maybe keep a mask defined centrally for each platform which contains >>> signals the VM needs for itself ? >>> >> >> Such masks already exist. >> >> +sigset_t os::Aix::sigs = { 0 }; >>> >>> I would not initialize the signal set this way. sigset_t is an opaque >>> type; the only way to initialize it is with one of sigemptyset() or >>> sigfillset(). >>> >> >> Good catch - I overlooked that. >> >> >>> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/aix/vm/os_aix.hpp.udiff.html >>> >>> >>> + static struct sigaction sigact[NSIG]; // saved preinstalled sigactions >>> + static sigset_t sigs; // mask of signals that have >>> >>> + static int sigflags[NSIG]; >>> >>> I know this is not in the scope of your change, but I would like to see >>> those removed from os::Aix and put into os_aix.cpp at static filescope. >>> There is no need at all to export those, and you would get rid of the >>> signal.h dependency you know have when including os_aix.hpp. >>> >>> >>> >>> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/bsd/vm/jsig.c.udiff.html >>> >>> >>> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/bsd/vm/os_bsd.cpp.udiff.html >>> >>> >>> On BSD, we have realtime signals. >>> >>> http://fxr.watson.org/fxr/source/sys/signal.h >>> #define SIGRTMAX 126 >>> and NSIG does not contain them: >>> #define NSIG 32 >>> >>> The max. possible signal number would be 126, which unfortunately does >>> not even fit into a 64bit mask. >>> >> >> So this simply limits the signal choice to not be a real-time signal - >> same as today. >> >> So the code in jsig.c is broken for the case that someone wants to >>> register realtime signals, if the VM were to ever use realtime signals >>> itself, which now is not the case. >>> >>> The same is true for os_bsd.cpp, where signal chaining will not work if >>> the application did have handler for real time signals pre-installed >>> before jvm is loaded. >>> >> >> Chaining is only used when the JVM will catch signals. Aren't all the >> real-time signals going to be blocked by the VM by default and so >> chaining is not needed as no handler will exist in the VM ?? (Unless a >> real-time signal is supplied for SR_signum) >> >> I must admit I don't know if any of this code actually works for >> real-time signals. >> >> Solaris: >>> >>> The only platform where NSIG is missing? >>> >>> Here, we calculate the max. signal number dynamically in os_solaris.cpp, >>> presumably because SIGRTMAX is not a constant and can be changed using >>> system configuration. But then, on Linux we have the same situation >>> (SIGRTMAX is dynamic) and there we do not go through the trouble of >>> calculating the max. signal number dynamically. Instead we just use >>> NSIG=64 and rely on the fact that NSIG is larger than the largest >>> possible dynamic value for SIGRTMAX. >>> >> >> Linux ensures that _NSIG (and thus NSIG) includes all the real-time >> signals. But libc can expose a subset and steal some for its own use. >> >> Solaris does not seem to have NSIG defined, but I am sure there is also >>> a max. possible value for SIGRTMAX (the default seems to be 48). So, one >>> could probably safely define NSIG for Solaris too, so that we have NSIG >>> defined on all Posix platforms. >>> >> >> Solaris doesn't have any of this SR_signum related code. A more general >> cleanup of signal related code would potentially involve a lot of cleanup. >> >> David >> ----- >> >> >>> On Thu, Nov 12, 2015 at 8:24 PM, Lindenmaier, Goetz >>> > wrote: >>> >>> Hi David, Dmitry, >>> >>> I've come up with a new webrev: >>> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/ >>> >>> I hit on some more issues: >>> - As proposed, I replaced MAXSIGNUM by NSIG >>> - On AIX, NSIG=255. Therefore storing bits in a word does not >>> work. >>> I'm now using bitset functionality from signal.h as it's done >>> in other places. >>> sigset_t is >> NSIG on linux, so it's no good idea to use it >>> there. >>> >>> >>> Why do we not do this on all platforms, provided sigset_t contains all >>> signals (incl. realtime signals) ? >>> >>> - In the os files I found another bit vector that now is too >>> small: sigs. >>> I adapted that, too. Removed the dead declaration of this on >>> solaris. >>> >>> Best regards, >>> Goetz. >>> >>> >>> >>> Kind Regards, Thomas >>> >>> > -----Original Message----- >>> > From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com >>> ] >>> > Sent: Donnerstag, 12. November 2015 10:05 >>> > To: Lindenmaier, Goetz; David Holmes; hotspot-runtime- >>> > dev at openjdk.java.net ; >>> serviceability-dev >>> > Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM >>> > >>> > Goetz, >>> > >>> > *BSD including OS X also defines NSIG (just checked) and if my >>> memory is >>> > not bogus, AIX defines it too. >>> > >>> > So you may consider to use NSIG on all platform. >>> > >>> > -Dmitry >>> > >>> > On 2015-11-12 11:36, Lindenmaier, Goetz wrote: >>> > > OK I'll change it to NSIG. That's used in other places in >>> os_linux, too. >>> > > So it's really more consistent. >>> > > >>> > > Best regards, >>> > > Goetz >>> > > >>> > >> -----Original Message----- >>> > >> From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com >>> ] >>> > >> Sent: Donnerstag, 12. November 2015 09:22 >>> > >> To: David Holmes; Lindenmaier, Goetz; hotspot-runtime- >>> > >> dev at openjdk.java.net ; >>> serviceability-dev >>> > >> Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM >>> > >> >>> > >> David, >>> > >> >>> > >> I think it's better to use NSIG (without underscore) defined >>> in signal.h >>> > >> >>> > >> -Dmitry >>> > >> >>> > >> >>> > >> On 2015-11-12 10:35, David Holmes wrote: >>> > >>> Hi Goetz, >>> > >>> >>> > >>> Adding in serviceability-dev >>> > >>> >>> > >>> On 9/11/2015 6:22 PM, Lindenmaier, Goetz wrote: >>> > >>>> Hi, >>> > >>>> >>> > >>>> The environment variable _JAVA_SR_SIGNUM can be set to a >>> signal >>> > >> number >>> > >>>> do be used by the JVM's suspend/resume mechanism. >>> > >>>> >>> > >>>> If set, a signal handler is installed and the current signal >>> handler >>> > >>>> is saved to an array. >>> > >>>> On linux, this array had size MAXSIGNUM=32, and >>> _JAVA_SR_SIGNUM >>> > >> was >>> > >>>> allowed >>> > >>>> to range up to _NSIG=65. This could cause memory corruption. >>> > >>>> >>> > >>>> Further, in jsig.c, an unsinged int is used to set a bit for >>> signals. >>> > >>>> This also >>> > >>>> is too small, as only 32 signals can be supported. >>> Further, the >>> > >>>> signals are mapped >>> > >>>> wrong to these bits. '0' is not a valid signal, but '32' >>> was. 1<<32 >>> > >>>> happens to map to >>> > >>>> zero, so the signal could be stored, but this probably was >>> not >>> > >>>> intended that way. >>> > >>>> >>> > >>>> This change increases MAXSIGNUM to 65 on linux, and to 64 on >>> aix. It >>> > >>>> introduces >>> > >>>> proper checking of the signal read from the env var, and >>> issues a >>> > >>>> warning if it >>> > >>>> does not use the signal set. It adapts the data types in >>> jisig.c >>> > >>>> properly. >>> > >>>> >>> > >>>> Please review this change. I please need a sponsor. >>> > >>>> >>> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.00 >>> > >>> >>> > >>> This all sounds very good to me. (I must find out why Solaris >>> is not >>> > >>> involved here :) ). >>> > >>> >>> > >>> On Linux you didn't add the bounds check to >>> os::Linux::set_our_sigflags. >>> > >>> >>> > >>> I'm also wondering about documenting where we are >>> determining the >>> > >>> maximum from? Is it simply _NSIG on some/all distributions? >>> And I see >>> > >>> _NSIG is supposed to be the biggest signal number + one. Also >>> linux >>> > >>> defines NSIG = _NSIG so which should we be using? >>> > >>> >>> > >>> Thanks, >>> > >>> David >>> > >>> >>> > >>>> Best regards, >>> > >>>> Goetz. >>> > >>>> >>> > >> >>> > >> >>> > >> -- >>> > >> Dmitry Samersoff >>> > >> Oracle Java development team, Saint Petersburg, Russia >>> > >> * I would love to change the world, but they won't give me the >>> sources. >>> > >>> > >>> > -- >>> > Dmitry Samersoff >>> > Oracle Java development team, Saint Petersburg, Russia >>> > * I would love to change the world, but they won't give me the >>> sources. >>> >>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From goetz.lindenmaier at sap.com Mon Nov 16 10:25:26 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 16 Nov 2015 10:25:26 +0000 Subject: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM In-Reply-To: References: <4295855A5C1DE049A61835A1887419CC41EB5DD6@DEWDFEMB12A.global.corp.sap> <5644413D.1000004@oracle.com> <56444C29.8060606@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB8252@DEWDFEMB12A.global.corp.sap> <56445646.4040707@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB85C3@DEWDFEMB12A.global.corp.sap> Message-ID: <4295855A5C1DE049A61835A1887419CC41EB8DB5@DEWDFEMB12A.global.corp.sap> Hi Thomas, thanks for looking at this. > MAX2(SIGSEGV, SIGBUS) I really would like to leave the code as-is. This would be a functional change, while I only intend to fix issues in this change. Also, as David explained, it might break for some os implementations. > the only way to initialize it is with one of sigemptyset() or sigfillset(). I added initialization with sigemptyset(). Unfortunately, there is no static initializer for this. > I would like to see those removed from os::Aix and put into os_aix.cpp at static filescope I moved these to static scope on the three oses. Here is the new webrev: http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.02/ Best regards, Goetz. From: Thomas St?fe [mailto:thomas.stuefe at gmail.com] Sent: Freitag, 13. November 2015 10:54 To: Lindenmaier, Goetz Cc: Dmitry Samersoff; David Holmes; hotspot-runtime-dev at openjdk.java.net; serviceability-dev Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM Hi Goetz, sorry for not looking at this earlier. This is a nice cleanup. Some remarks: http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/aix/vm/os_aix.cpp.udiff.html + if (sig > MAX2(SIGSEGV, SIGBUS) && // See 4355769. + sig < NSIG) { // Must be legal signal and fit into sigflags[]. I do not like much the MAX2() construct. I would like it better to explicitly check whether the SR signal is one of the "forbidden" ones the VM uses. Maybe keep a mask defined centrally for each platform which contains signals the VM needs for itself ? +sigset_t os::Aix::sigs = { 0 }; I would not initialize the signal set this way. sigset_t is an opaque type; the only way to initialize it is with one of sigemptyset() or sigfillset(). http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/aix/vm/os_aix.hpp.udiff.html + static struct sigaction sigact[NSIG]; // saved preinstalled sigactions + static sigset_t sigs; // mask of signals that have + static int sigflags[NSIG]; I know this is not in the scope of your change, but I would like to see those removed from os::Aix and put into os_aix.cpp at static filescope. There is no need at all to export those, and you would get rid of the signal.h dependency you know have when including os_aix.hpp. http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/bsd/vm/jsig.c.udiff.html http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/bsd/vm/os_bsd.cpp.udiff.html On BSD, we have realtime signals. http://fxr.watson.org/fxr/source/sys/signal.h #define SIGRTMAX 126 and NSIG does not contain them: #define NSIG 32 The max. possible signal number would be 126, which unfortunately does not even fit into a 64bit mask. So the code in jsig.c is broken for the case that someone wants to register realtime signals, if the VM were to ever use realtime signals itself, which now is not the case. The same is true for os_bsd.cpp, where signal chaining will not work if the application did have handler for real time signals pre-installed before jvm is loaded. Solaris: The only platform where NSIG is missing? Here, we calculate the max. signal number dynamically in os_solaris.cpp, presumably because SIGRTMAX is not a constant and can be changed using system configuration. But then, on Linux we have the same situation (SIGRTMAX is dynamic) and there we do not go through the trouble of calculating the max. signal number dynamically. Instead we just use NSIG=64 and rely on the fact that NSIG is larger than the largest possible dynamic value for SIGRTMAX. Solaris does not seem to have NSIG defined, but I am sure there is also a max. possible value for SIGRTMAX (the default seems to be 48). So, one could probably safely define NSIG for Solaris too, so that we have NSIG defined on all Posix platforms. On Thu, Nov 12, 2015 at 8:24 PM, Lindenmaier, Goetz > wrote: Hi David, Dmitry, I've come up with a new webrev: http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/ I hit on some more issues: - As proposed, I replaced MAXSIGNUM by NSIG - On AIX, NSIG=255. Therefore storing bits in a word does not work. I'm now using bitset functionality from signal.h as it's done in other places. sigset_t is >> NSIG on linux, so it's no good idea to use it there. Why do we not do this on all platforms, provided sigset_t contains all signals (incl. realtime signals) ? - In the os files I found another bit vector that now is too small: sigs. I adapted that, too. Removed the dead declaration of this on solaris. Best regards, Goetz. Kind Regards, Thomas > -----Original Message----- > From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com] > Sent: Donnerstag, 12. November 2015 10:05 > To: Lindenmaier, Goetz; David Holmes; hotspot-runtime- > dev at openjdk.java.net; serviceability-dev > Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM > > Goetz, > > *BSD including OS X also defines NSIG (just checked) and if my memory is > not bogus, AIX defines it too. > > So you may consider to use NSIG on all platform. > > -Dmitry > > On 2015-11-12 11:36, Lindenmaier, Goetz wrote: > > OK I'll change it to NSIG. That's used in other places in os_linux, too. > > So it's really more consistent. > > > > Best regards, > > Goetz > > > >> -----Original Message----- > >> From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com] > >> Sent: Donnerstag, 12. November 2015 09:22 > >> To: David Holmes; Lindenmaier, Goetz; hotspot-runtime- > >> dev at openjdk.java.net; serviceability-dev > >> Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM > >> > >> David, > >> > >> I think it's better to use NSIG (without underscore) defined in signal.h > >> > >> -Dmitry > >> > >> > >> On 2015-11-12 10:35, David Holmes wrote: > >>> Hi Goetz, > >>> > >>> Adding in serviceability-dev > >>> > >>> On 9/11/2015 6:22 PM, Lindenmaier, Goetz wrote: > >>>> Hi, > >>>> > >>>> The environment variable _JAVA_SR_SIGNUM can be set to a signal > >> number > >>>> do be used by the JVM's suspend/resume mechanism. > >>>> > >>>> If set, a signal handler is installed and the current signal handler > >>>> is saved to an array. > >>>> On linux, this array had size MAXSIGNUM=32, and _JAVA_SR_SIGNUM > >> was > >>>> allowed > >>>> to range up to _NSIG=65. This could cause memory corruption. > >>>> > >>>> Further, in jsig.c, an unsinged int is used to set a bit for signals. > >>>> This also > >>>> is too small, as only 32 signals can be supported. Further, the > >>>> signals are mapped > >>>> wrong to these bits. '0' is not a valid signal, but '32' was. 1<<32 > >>>> happens to map to > >>>> zero, so the signal could be stored, but this probably was not > >>>> intended that way. > >>>> > >>>> This change increases MAXSIGNUM to 65 on linux, and to 64 on aix. It > >>>> introduces > >>>> proper checking of the signal read from the env var, and issues a > >>>> warning if it > >>>> does not use the signal set. It adapts the data types in jisig.c > >>>> properly. > >>>> > >>>> Please review this change. I please need a sponsor. > >>>> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.00 > >>> > >>> This all sounds very good to me. (I must find out why Solaris is not > >>> involved here :) ). > >>> > >>> On Linux you didn't add the bounds check to os::Linux::set_our_sigflags. > >>> > >>> I'm also wondering about documenting where we are determining the > >>> maximum from? Is it simply _NSIG on some/all distributions? And I see > >>> _NSIG is supposed to be the biggest signal number + one. Also linux > >>> defines NSIG = _NSIG so which should we be using? > >>> > >>> Thanks, > >>> David > >>> > >>>> Best regards, > >>>> Goetz. > >>>> > >> > >> > >> -- > >> Dmitry Samersoff > >> Oracle Java development team, Saint Petersburg, Russia > >> * I would love to change the world, but they won't give me the sources. > > > -- > Dmitry Samersoff > Oracle Java development team, Saint Petersburg, Russia > * I would love to change the world, but they won't give me the sources. -------------- next part -------------- An HTML attachment was scrubbed... URL: From goetz.lindenmaier at sap.com Mon Nov 16 10:32:44 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 16 Nov 2015 10:32:44 +0000 Subject: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM In-Reply-To: <56499279.2050806@oracle.com> References: <4295855A5C1DE049A61835A1887419CC41EB5DD6@DEWDFEMB12A.global.corp.sap> <5644413D.1000004@oracle.com> <56444C29.8060606@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB8252@DEWDFEMB12A.global.corp.sap> <56445646.4040707@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB85C3@DEWDFEMB12A.global.corp.sap> <5645E7D0.1000704@oracle.com> <564955AA.3020704@oracle.com> <56499279.2050806@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC41EB8DD6@DEWDFEMB12A.global.corp.sap> Hi Dmitry, I think this is a rather complex issue which is better documented in the bug than in the code. The code already references the bugID. Also, I would have to copy the explanation into three files ... I you don't object I'll leave the comment as is. Best regards, Goetz. -----Original Message----- From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com] Sent: Montag, 16. November 2015 09:23 To: David Holmes; Thomas St?fe; Lindenmaier, Goetz Cc: serviceability-dev; hotspot-runtime-dev at openjdk.java.net Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM David, Thank you for detailed explanation. Could we put it as a comment right into the code? -Dmitry On 2015-11-16 07:03, David Holmes wrote: > On 13/11/2015 11:38 PM, David Holmes wrote: >> On 13/11/2015 7:53 PM, Thomas St?fe wrote: >>> Hi Goetz, >>> >>> sorry for not looking at this earlier. This is a nice cleanup. Some >>> remarks: >>> >>> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/aix/vm/os_aix.cpp.udiff.html >>> >>> >>> >>> + if (sig > MAX2(SIGSEGV, SIGBUS) && // See 4355769. >>> + sig < NSIG) { // Must be legal signal and fit >>> into sigflags[]. >>> >>> I do not like much the MAX2() construct. I would like it better to >>> explicitly check whether the SR signal is one of the "forbidden" ones >>> the VM uses. >> >> I must confess I had not looked into 4355769 but this check seems rather >> spurious. It is not at all clear to me what signals could be used here - > > Okay should have looked into 4355769. The problem is how multiple > pending signals are handled. It seems that in the past (no idea if still > true) pending signals were handled in signal-number order (lowest > first), not FIFO. The problem scenario is this: > - thread accesses a null pointer in compiled Java code and the SEGV > handler will cause NPE to be thrown > - at the same time as the SEGV is being raised the thread is also hit > with the SR signal to suspend it. > - the SR signal will be delivered first and the SR handler starts to run > - with signals unblocked. > - the SEGV then gets delivered to the thread in the SR handler, and the > regular signal handler is run > - the regular signal handler tries to detect if we're running in Java > code so it can post the NPE, but the presence of the SR handler causes > that check to fail - so we abort thinking it is a real SEGV. > > I don't know how much of that is still true today. It seems strange to > me that a kill based directed signal can usurp a synchronous signal. > > Anyway the fix, rather workaround, for that problem, was to ensure that > the SR_signum is greater than any potential synchronous signal the VM > cares about. Why SIGBUS was included there I don't know give that: > > a) it is already a lower signal number than SIGUSR1, SIGSEGV and SIGUSR2 > b) we don't deliberately generate and use SIGBUS ... though perhaps > unsafe-fetch needs to be considered. > > A better fix in my opinion, and as mentioned in the bug, would have been > to disable delivery of SEGV whilst the SR handler is executing. But we > start to touch on some grey areas of the POSIX spec there, and likely > the implementation too. > > So I suggest that for this cleanup we simply leave this logic exactly as > is. > > Thanks, > David > >> other than SIGUSR1 or SIGUSR2 (if -Xrs is specified), or else a >> real-time signal (modulo discussion below). Hijacking anything else >> seems rather suspect. >> >>> Maybe keep a mask defined centrally for each platform which contains >>> signals the VM needs for itself ? >> >> Such masks already exist. >> >>> +sigset_t os::Aix::sigs = { 0 }; >>> >>> I would not initialize the signal set this way. sigset_t is an opaque >>> type; the only way to initialize it is with one of sigemptyset() or >>> sigfillset(). >> >> Good catch - I overlooked that. >> >>> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/aix/vm/os_aix.hpp.udiff.html >>> >>> >>> >>> + static struct sigaction sigact[NSIG]; // saved preinstalled >>> sigactions >>> + static sigset_t sigs; // mask of signals that have >>> >>> + static int sigflags[NSIG]; >>> >>> I know this is not in the scope of your change, but I would like to see >>> those removed from os::Aix and put into os_aix.cpp at static filescope. >>> There is no need at all to export those, and you would get rid of the >>> signal.h dependency you know have when including os_aix.hpp. >>> >>> >>> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/bsd/vm/jsig.c.udiff.html >>> >>> >>> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/bsd/vm/os_bsd.cpp.udiff.html >>> >>> >>> >>> On BSD, we have realtime signals. >>> >>> http://fxr.watson.org/fxr/source/sys/signal.h >>> #define SIGRTMAX 126 >>> and NSIG does not contain them: >>> #define NSIG 32 >>> >>> The max. possible signal number would be 126, which unfortunately does >>> not even fit into a 64bit mask. >> >> So this simply limits the signal choice to not be a real-time signal - >> same as today. >> >>> So the code in jsig.c is broken for the case that someone wants to >>> register realtime signals, if the VM were to ever use realtime signals >>> itself, which now is not the case. >>> >>> The same is true for os_bsd.cpp, where signal chaining will not work if >>> the application did have handler for real time signals pre-installed >>> before jvm is loaded. >> >> Chaining is only used when the JVM will catch signals. Aren't all the >> real-time signals going to be blocked by the VM by default and so >> chaining is not needed as no handler will exist in the VM ?? (Unless a >> real-time signal is supplied for SR_signum) >> >> I must admit I don't know if any of this code actually works for >> real-time signals. >> >>> Solaris: >>> >>> The only platform where NSIG is missing? >>> >>> Here, we calculate the max. signal number dynamically in os_solaris.cpp, >>> presumably because SIGRTMAX is not a constant and can be changed using >>> system configuration. But then, on Linux we have the same situation >>> (SIGRTMAX is dynamic) and there we do not go through the trouble of >>> calculating the max. signal number dynamically. Instead we just use >>> NSIG=64 and rely on the fact that NSIG is larger than the largest >>> possible dynamic value for SIGRTMAX. >> >> Linux ensures that _NSIG (and thus NSIG) includes all the real-time >> signals. But libc can expose a subset and steal some for its own use. >> >>> Solaris does not seem to have NSIG defined, but I am sure there is also >>> a max. possible value for SIGRTMAX (the default seems to be 48). So, one >>> could probably safely define NSIG for Solaris too, so that we have NSIG >>> defined on all Posix platforms. >> >> Solaris doesn't have any of this SR_signum related code. A more general >> cleanup of signal related code would potentially involve a lot of >> cleanup. >> >> David >> ----- >> >>> >>> On Thu, Nov 12, 2015 at 8:24 PM, Lindenmaier, Goetz >>> > wrote: >>> >>> Hi David, Dmitry, >>> >>> I've come up with a new webrev: >>> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/ >>> >>> I hit on some more issues: >>> - As proposed, I replaced MAXSIGNUM by NSIG >>> - On AIX, NSIG=255. Therefore storing bits in a word does not >>> work. >>> I'm now using bitset functionality from signal.h as it's done >>> in other places. >>> sigset_t is >> NSIG on linux, so it's no good idea to use it >>> there. >>> >>> >>> Why do we not do this on all platforms, provided sigset_t contains all >>> signals (incl. realtime signals) ? >>> >>> - In the os files I found another bit vector that now is too >>> small: sigs. >>> I adapted that, too. Removed the dead declaration of this on >>> solaris. >>> >>> Best regards, >>> Goetz. >>> >>> >>> >>> Kind Regards, Thomas >>> >>> > -----Original Message----- >>> > From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com >>> ] >>> > Sent: Donnerstag, 12. November 2015 10:05 >>> > To: Lindenmaier, Goetz; David Holmes; hotspot-runtime- >>> > dev at openjdk.java.net ; >>> serviceability-dev >>> > Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM >>> > >>> > Goetz, >>> > >>> > *BSD including OS X also defines NSIG (just checked) and if my >>> memory is >>> > not bogus, AIX defines it too. >>> > >>> > So you may consider to use NSIG on all platform. >>> > >>> > -Dmitry >>> > >>> > On 2015-11-12 11:36, Lindenmaier, Goetz wrote: >>> > > OK I'll change it to NSIG. That's used in other places in >>> os_linux, too. >>> > > So it's really more consistent. >>> > > >>> > > Best regards, >>> > > Goetz >>> > > >>> > >> -----Original Message----- >>> > >> From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com >>> ] >>> > >> Sent: Donnerstag, 12. November 2015 09:22 >>> > >> To: David Holmes; Lindenmaier, Goetz; hotspot-runtime- >>> > >> dev at openjdk.java.net ; >>> serviceability-dev >>> > >> Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM >>> > >> >>> > >> David, >>> > >> >>> > >> I think it's better to use NSIG (without underscore) defined >>> in signal.h >>> > >> >>> > >> -Dmitry >>> > >> >>> > >> >>> > >> On 2015-11-12 10:35, David Holmes wrote: >>> > >>> Hi Goetz, >>> > >>> >>> > >>> Adding in serviceability-dev >>> > >>> >>> > >>> On 9/11/2015 6:22 PM, Lindenmaier, Goetz wrote: >>> > >>>> Hi, >>> > >>>> >>> > >>>> The environment variable _JAVA_SR_SIGNUM can be set to a >>> signal >>> > >> number >>> > >>>> do be used by the JVM's suspend/resume mechanism. >>> > >>>> >>> > >>>> If set, a signal handler is installed and the current signal >>> handler >>> > >>>> is saved to an array. >>> > >>>> On linux, this array had size MAXSIGNUM=32, and >>> _JAVA_SR_SIGNUM >>> > >> was >>> > >>>> allowed >>> > >>>> to range up to _NSIG=65. This could cause memory corruption. >>> > >>>> >>> > >>>> Further, in jsig.c, an unsinged int is used to set a bit for >>> signals. >>> > >>>> This also >>> > >>>> is too small, as only 32 signals can be supported. >>> Further, the >>> > >>>> signals are mapped >>> > >>>> wrong to these bits. '0' is not a valid signal, but '32' >>> was. 1<<32 >>> > >>>> happens to map to >>> > >>>> zero, so the signal could be stored, but this probably was >>> not >>> > >>>> intended that way. >>> > >>>> >>> > >>>> This change increases MAXSIGNUM to 65 on linux, and to 64 on >>> aix. It >>> > >>>> introduces >>> > >>>> proper checking of the signal read from the env var, and >>> issues a >>> > >>>> warning if it >>> > >>>> does not use the signal set. It adapts the data types in >>> jisig.c >>> > >>>> properly. >>> > >>>> >>> > >>>> Please review this change. I please need a sponsor. >>> > >>>> >>> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.00 >>> > >>> >>> > >>> This all sounds very good to me. (I must find out why Solaris >>> is not >>> > >>> involved here :) ). >>> > >>> >>> > >>> On Linux you didn't add the bounds check to >>> os::Linux::set_our_sigflags. >>> > >>> >>> > >>> I'm also wondering about documenting where we are >>> determining the >>> > >>> maximum from? Is it simply _NSIG on some/all distributions? >>> And I see >>> > >>> _NSIG is supposed to be the biggest signal number + one. Also >>> linux >>> > >>> defines NSIG = _NSIG so which should we be using? >>> > >>> >>> > >>> Thanks, >>> > >>> David >>> > >>> >>> > >>>> Best regards, >>> > >>>> Goetz. >>> > >>>> >>> > >> >>> > >> >>> > >> -- >>> > >> Dmitry Samersoff >>> > >> Oracle Java development team, Saint Petersburg, Russia >>> > >> * I would love to change the world, but they won't give me the >>> sources. >>> > >>> > >>> > -- >>> > Dmitry Samersoff >>> > Oracle Java development team, Saint Petersburg, Russia >>> > * I would love to change the world, but they won't give me the >>> sources. >>> >>> -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From goetz.lindenmaier at sap.com Mon Nov 16 10:40:17 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 16 Nov 2015 10:40:17 +0000 Subject: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM In-Reply-To: <564955AA.3020704@oracle.com> References: <4295855A5C1DE049A61835A1887419CC41EB5DD6@DEWDFEMB12A.global.corp.sap> <5644413D.1000004@oracle.com> <56444C29.8060606@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB8252@DEWDFEMB12A.global.corp.sap> <56445646.4040707@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB85C3@DEWDFEMB12A.global.corp.sap> <5645E7D0.1000704@oracle.com> <564955AA.3020704@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC41EB8DF0@DEWDFEMB12A.global.corp.sap> Hi David, thanks for looking into this issue. I'll leave the coding as is. Best regards, Goetz. -----Original Message----- From: David Holmes [mailto:david.holmes at oracle.com] Sent: Montag, 16. November 2015 05:04 To: Thomas St?fe; Lindenmaier, Goetz Cc: serviceability-dev; hotspot-runtime-dev at openjdk.java.net Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM On 13/11/2015 11:38 PM, David Holmes wrote: > On 13/11/2015 7:53 PM, Thomas St?fe wrote: >> Hi Goetz, >> >> sorry for not looking at this earlier. This is a nice cleanup. Some >> remarks: >> >> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/aix/vm/os_aix.cpp.udiff.html >> >> >> + if (sig > MAX2(SIGSEGV, SIGBUS) && // See 4355769. >> + sig < NSIG) { // Must be legal signal and fit >> into sigflags[]. >> >> I do not like much the MAX2() construct. I would like it better to >> explicitly check whether the SR signal is one of the "forbidden" ones >> the VM uses. > > I must confess I had not looked into 4355769 but this check seems rather > spurious. It is not at all clear to me what signals could be used here - Okay should have looked into 4355769. The problem is how multiple pending signals are handled. It seems that in the past (no idea if still true) pending signals were handled in signal-number order (lowest first), not FIFO. The problem scenario is this: - thread accesses a null pointer in compiled Java code and the SEGV handler will cause NPE to be thrown - at the same time as the SEGV is being raised the thread is also hit with the SR signal to suspend it. - the SR signal will be delivered first and the SR handler starts to run - with signals unblocked. - the SEGV then gets delivered to the thread in the SR handler, and the regular signal handler is run - the regular signal handler tries to detect if we're running in Java code so it can post the NPE, but the presence of the SR handler causes that check to fail - so we abort thinking it is a real SEGV. I don't know how much of that is still true today. It seems strange to me that a kill based directed signal can usurp a synchronous signal. Anyway the fix, rather workaround, for that problem, was to ensure that the SR_signum is greater than any potential synchronous signal the VM cares about. Why SIGBUS was included there I don't know give that: a) it is already a lower signal number than SIGUSR1, SIGSEGV and SIGUSR2 b) we don't deliberately generate and use SIGBUS ... though perhaps unsafe-fetch needs to be considered. A better fix in my opinion, and as mentioned in the bug, would have been to disable delivery of SEGV whilst the SR handler is executing. But we start to touch on some grey areas of the POSIX spec there, and likely the implementation too. So I suggest that for this cleanup we simply leave this logic exactly as is. Thanks, David > other than SIGUSR1 or SIGUSR2 (if -Xrs is specified), or else a > real-time signal (modulo discussion below). Hijacking anything else > seems rather suspect. > >> Maybe keep a mask defined centrally for each platform which contains >> signals the VM needs for itself ? > > Such masks already exist. > >> +sigset_t os::Aix::sigs = { 0 }; >> >> I would not initialize the signal set this way. sigset_t is an opaque >> type; the only way to initialize it is with one of sigemptyset() or >> sigfillset(). > > Good catch - I overlooked that. > >> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/aix/vm/os_aix.hpp.udiff.html >> >> >> + static struct sigaction sigact[NSIG]; // saved preinstalled sigactions >> + static sigset_t sigs; // mask of signals that have >> >> + static int sigflags[NSIG]; >> >> I know this is not in the scope of your change, but I would like to see >> those removed from os::Aix and put into os_aix.cpp at static filescope. >> There is no need at all to export those, and you would get rid of the >> signal.h dependency you know have when including os_aix.hpp. >> >> >> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/bsd/vm/jsig.c.udiff.html >> >> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/bsd/vm/os_bsd.cpp.udiff.html >> >> >> On BSD, we have realtime signals. >> >> http://fxr.watson.org/fxr/source/sys/signal.h >> #define SIGRTMAX 126 >> and NSIG does not contain them: >> #define NSIG 32 >> >> The max. possible signal number would be 126, which unfortunately does >> not even fit into a 64bit mask. > > So this simply limits the signal choice to not be a real-time signal - > same as today. > >> So the code in jsig.c is broken for the case that someone wants to >> register realtime signals, if the VM were to ever use realtime signals >> itself, which now is not the case. >> >> The same is true for os_bsd.cpp, where signal chaining will not work if >> the application did have handler for real time signals pre-installed >> before jvm is loaded. > > Chaining is only used when the JVM will catch signals. Aren't all the > real-time signals going to be blocked by the VM by default and so > chaining is not needed as no handler will exist in the VM ?? (Unless a > real-time signal is supplied for SR_signum) > > I must admit I don't know if any of this code actually works for > real-time signals. > >> Solaris: >> >> The only platform where NSIG is missing? >> >> Here, we calculate the max. signal number dynamically in os_solaris.cpp, >> presumably because SIGRTMAX is not a constant and can be changed using >> system configuration. But then, on Linux we have the same situation >> (SIGRTMAX is dynamic) and there we do not go through the trouble of >> calculating the max. signal number dynamically. Instead we just use >> NSIG=64 and rely on the fact that NSIG is larger than the largest >> possible dynamic value for SIGRTMAX. > > Linux ensures that _NSIG (and thus NSIG) includes all the real-time > signals. But libc can expose a subset and steal some for its own use. > >> Solaris does not seem to have NSIG defined, but I am sure there is also >> a max. possible value for SIGRTMAX (the default seems to be 48). So, one >> could probably safely define NSIG for Solaris too, so that we have NSIG >> defined on all Posix platforms. > > Solaris doesn't have any of this SR_signum related code. A more general > cleanup of signal related code would potentially involve a lot of cleanup. > > David > ----- > >> >> On Thu, Nov 12, 2015 at 8:24 PM, Lindenmaier, Goetz >> > wrote: >> >> Hi David, Dmitry, >> >> I've come up with a new webrev: >> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/ >> >> I hit on some more issues: >> - As proposed, I replaced MAXSIGNUM by NSIG >> - On AIX, NSIG=255. Therefore storing bits in a word does not >> work. >> I'm now using bitset functionality from signal.h as it's done >> in other places. >> sigset_t is >> NSIG on linux, so it's no good idea to use it >> there. >> >> >> Why do we not do this on all platforms, provided sigset_t contains all >> signals (incl. realtime signals) ? >> >> - In the os files I found another bit vector that now is too >> small: sigs. >> I adapted that, too. Removed the dead declaration of this on >> solaris. >> >> Best regards, >> Goetz. >> >> >> >> Kind Regards, Thomas >> >> > -----Original Message----- >> > From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com >> ] >> > Sent: Donnerstag, 12. November 2015 10:05 >> > To: Lindenmaier, Goetz; David Holmes; hotspot-runtime- >> > dev at openjdk.java.net ; >> serviceability-dev >> > Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM >> > >> > Goetz, >> > >> > *BSD including OS X also defines NSIG (just checked) and if my >> memory is >> > not bogus, AIX defines it too. >> > >> > So you may consider to use NSIG on all platform. >> > >> > -Dmitry >> > >> > On 2015-11-12 11:36, Lindenmaier, Goetz wrote: >> > > OK I'll change it to NSIG. That's used in other places in >> os_linux, too. >> > > So it's really more consistent. >> > > >> > > Best regards, >> > > Goetz >> > > >> > >> -----Original Message----- >> > >> From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com >> ] >> > >> Sent: Donnerstag, 12. November 2015 09:22 >> > >> To: David Holmes; Lindenmaier, Goetz; hotspot-runtime- >> > >> dev at openjdk.java.net ; >> serviceability-dev >> > >> Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM >> > >> >> > >> David, >> > >> >> > >> I think it's better to use NSIG (without underscore) defined >> in signal.h >> > >> >> > >> -Dmitry >> > >> >> > >> >> > >> On 2015-11-12 10:35, David Holmes wrote: >> > >>> Hi Goetz, >> > >>> >> > >>> Adding in serviceability-dev >> > >>> >> > >>> On 9/11/2015 6:22 PM, Lindenmaier, Goetz wrote: >> > >>>> Hi, >> > >>>> >> > >>>> The environment variable _JAVA_SR_SIGNUM can be set to a >> signal >> > >> number >> > >>>> do be used by the JVM's suspend/resume mechanism. >> > >>>> >> > >>>> If set, a signal handler is installed and the current signal >> handler >> > >>>> is saved to an array. >> > >>>> On linux, this array had size MAXSIGNUM=32, and >> _JAVA_SR_SIGNUM >> > >> was >> > >>>> allowed >> > >>>> to range up to _NSIG=65. This could cause memory corruption. >> > >>>> >> > >>>> Further, in jsig.c, an unsinged int is used to set a bit for >> signals. >> > >>>> This also >> > >>>> is too small, as only 32 signals can be supported. >> Further, the >> > >>>> signals are mapped >> > >>>> wrong to these bits. '0' is not a valid signal, but '32' >> was. 1<<32 >> > >>>> happens to map to >> > >>>> zero, so the signal could be stored, but this probably was >> not >> > >>>> intended that way. >> > >>>> >> > >>>> This change increases MAXSIGNUM to 65 on linux, and to 64 on >> aix. It >> > >>>> introduces >> > >>>> proper checking of the signal read from the env var, and >> issues a >> > >>>> warning if it >> > >>>> does not use the signal set. It adapts the data types in >> jisig.c >> > >>>> properly. >> > >>>> >> > >>>> Please review this change. I please need a sponsor. >> > >>>> >> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.00 >> > >>> >> > >>> This all sounds very good to me. (I must find out why Solaris >> is not >> > >>> involved here :) ). >> > >>> >> > >>> On Linux you didn't add the bounds check to >> os::Linux::set_our_sigflags. >> > >>> >> > >>> I'm also wondering about documenting where we are >> determining the >> > >>> maximum from? Is it simply _NSIG on some/all distributions? >> And I see >> > >>> _NSIG is supposed to be the biggest signal number + one. Also >> linux >> > >>> defines NSIG = _NSIG so which should we be using? >> > >>> >> > >>> Thanks, >> > >>> David >> > >>> >> > >>>> Best regards, >> > >>>> Goetz. >> > >>>> >> > >> >> > >> >> > >> -- >> > >> Dmitry Samersoff >> > >> Oracle Java development team, Saint Petersburg, Russia >> > >> * I would love to change the world, but they won't give me the >> sources. >> > >> > >> > -- >> > Dmitry Samersoff >> > Oracle Java development team, Saint Petersburg, Russia >> > * I would love to change the world, but they won't give me the >> sources. >> >> From dmitry.samersoff at oracle.com Mon Nov 16 10:53:13 2015 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Mon, 16 Nov 2015 13:53:13 +0300 Subject: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM In-Reply-To: <4295855A5C1DE049A61835A1887419CC41EB8DB5@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC41EB5DD6@DEWDFEMB12A.global.corp.sap> <5644413D.1000004@oracle.com> <56444C29.8060606@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB8252@DEWDFEMB12A.global.corp.sap> <56445646.4040707@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB85C3@DEWDFEMB12A.global.corp.sap> <4295855A5C1DE049A61835A1887419CC41EB8DB5@DEWDFEMB12A.global.corp.sap> Message-ID: <5649B599.60807@oracle.com> Goetz, Looks good for me. Thank you for a nice cleanup. -Dmitry On 2015-11-16 13:25, Lindenmaier, Goetz wrote: > Hi Thomas, > > > > thanks for looking at this. > > > >> MAX2(SIGSEGV, SIGBUS) > > I really would like to leave the code as-is. This would be a functional > > change, while I only intend to fix issues in this change. Also, as David > > explained, it might break for some os implementations. > > > >> the only way to initialize it is with one of sigemptyset() or sigfillset(). > > I added initialization with sigemptyset(). Unfortunately, there is no > static > > initializer for this. > > > >> I would like to see those removed from os::Aix and put into os_aix.cpp at static filescope > > I moved these to static scope on the three oses. > > > > Here is the new webrev: > > http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.02/ > > > > Best regards, > > Goetz. > > > > > > *From:*Thomas St?fe [mailto:thomas.stuefe at gmail.com] > *Sent:* Freitag, 13. November 2015 10:54 > *To:* Lindenmaier, Goetz > *Cc:* Dmitry Samersoff; David Holmes; > hotspot-runtime-dev at openjdk.java.net; serviceability-dev > *Subject:* Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM > > > > Hi Goetz, > > > > sorry for not looking at this earlier. This is a nice cleanup. Some remarks: > > > > http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/aix/vm/os_aix.cpp.udiff.html > > > > + if (sig > MAX2(SIGSEGV, SIGBUS) && // See 4355769. > > + sig < NSIG) { // Must be legal signal and fit > into sigflags[]. > > > > I do not like much the MAX2() construct. I would like it better to > explicitly check whether the SR signal is one of the "forbidden" ones > the VM uses. > > > > Maybe keep a mask defined centrally for each platform which contains > signals the VM needs for itself ? > > > > +sigset_t os::Aix::sigs = { 0 }; > > > > I would not initialize the signal set this way. sigset_t is an opaque > type; the only way to initialize it is with one of sigemptyset() or > sigfillset(). > > > > http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/aix/vm/os_aix.hpp.udiff.html > > > > + static struct sigaction sigact[NSIG]; // saved preinstalled sigactions > > + static sigset_t sigs; // mask of signals that have > > > > + static int sigflags[NSIG]; > > > > I know this is not in the scope of your change, but I would like to see > those removed from os::Aix and put into os_aix.cpp at static filescope. > There is no need at all to export those, and you would get rid of the > signal.h dependency you know have when including os_aix.hpp. > > > > > > http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/bsd/vm/jsig.c.udiff.html > > http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/bsd/vm/os_bsd.cpp.udiff.html > > > > On BSD, we have realtime signals. > > > > http://fxr.watson.org/fxr/source/sys/signal.h > > #define SIGRTMAX 126 > > and NSIG does not contain them: > > #define NSIG 32 > > > > The max. possible signal number would be 126, which unfortunately does > not even fit into a 64bit mask. > > > > So the code in jsig.c is broken for the case that someone wants to > register realtime signals, if the VM were to ever use realtime signals > itself, which now is not the case. > > > > The same is true for os_bsd.cpp, where signal chaining will not work if > the application did have handler for real time signals pre-installed > before jvm is loaded. > > > > Solaris: > > > > The only platform where NSIG is missing? > > > > Here, we calculate the max. signal number dynamically in os_solaris.cpp, > presumably because SIGRTMAX is not a constant and can be changed using > system configuration. But then, on Linux we have the same situation > (SIGRTMAX is dynamic) and there we do not go through the trouble of > calculating the max. signal number dynamically. Instead we just use > NSIG=64 and rely on the fact that NSIG is larger than the largest > possible dynamic value for SIGRTMAX. > > > > Solaris does not seem to have NSIG defined, but I am sure there is also > a max. possible value for SIGRTMAX (the default seems to be 48). So, one > could probably safely define NSIG for Solaris too, so that we have NSIG > defined on all Posix platforms. > > > > > > On Thu, Nov 12, 2015 at 8:24 PM, Lindenmaier, Goetz > > wrote: > > Hi David, Dmitry, > > I've come up with a new webrev: > http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/ > > I hit on some more issues: > - As proposed, I replaced MAXSIGNUM by NSIG > - On AIX, NSIG=255. Therefore storing bits in a word does not work. > I'm now using bitset functionality from signal.h as it's done in > other places. > sigset_t is >> NSIG on linux, so it's no good idea to use it there. > > > > Why do we not do this on all platforms, provided sigset_t contains all > signals (incl. realtime signals) ? > > > > - In the os files I found another bit vector that now is too small: > sigs. > I adapted that, too. Removed the dead declaration of this on > solaris. > > Best regards, > Goetz. > > > > > > Kind Regards, Thomas > > > > > > > -----Original Message----- > > From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com ] > > > Sent: Donnerstag, 12. November 2015 10:05 > > To: Lindenmaier, Goetz; David Holmes; hotspot-runtime- > > dev at openjdk.java.net ; serviceability-dev > > Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM > > > > Goetz, > > > > *BSD including OS X also defines NSIG (just checked) and if my > memory is > > not bogus, AIX defines it too. > > > > So you may consider to use NSIG on all platform. > > > > -Dmitry > > > > On 2015-11-12 11:36, Lindenmaier, Goetz wrote: > > > OK I'll change it to NSIG. That's used in other places in > os_linux, too. > > > So it's really more consistent. > > > > > > Best regards, > > > Goetz > > > > > >> -----Original Message----- > > >> From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com > ] > > >> Sent: Donnerstag, 12. November 2015 09:22 > > >> To: David Holmes; Lindenmaier, Goetz; hotspot-runtime- > > >> dev at openjdk.java.net ; > serviceability-dev > > >> Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM > > >> > > >> David, > > >> > > >> I think it's better to use NSIG (without underscore) defined in > signal.h > > >> > > >> -Dmitry > > >> > > >> > > >> On 2015-11-12 10:35, David Holmes wrote: > > >>> Hi Goetz, > > >>> > > >>> Adding in serviceability-dev > > >>> > > >>> On 9/11/2015 6:22 PM, Lindenmaier, Goetz wrote: > > >>>> Hi, > > >>>> > > >>>> The environment variable _JAVA_SR_SIGNUM can be set to a signal > > >> number > > >>>> do be used by the JVM's suspend/resume mechanism. > > >>>> > > >>>> If set, a signal handler is installed and the current signal > handler > > >>>> is saved to an array. > > >>>> On linux, this array had size MAXSIGNUM=32, and _JAVA_SR_SIGNUM > > >> was > > >>>> allowed > > >>>> to range up to _NSIG=65. This could cause memory corruption. > > >>>> > > >>>> Further, in jsig.c, an unsinged int is used to set a bit for > signals. > > >>>> This also > > >>>> is too small, as only 32 signals can be supported. Further, the > > >>>> signals are mapped > > >>>> wrong to these bits. '0' is not a valid signal, but '32' > was. 1<<32 > > >>>> happens to map to > > >>>> zero, so the signal could be stored, but this probably was not > > >>>> intended that way. > > >>>> > > >>>> This change increases MAXSIGNUM to 65 on linux, and to 64 on > aix. It > > >>>> introduces > > >>>> proper checking of the signal read from the env var, and issues a > > >>>> warning if it > > >>>> does not use the signal set. It adapts the data types in jisig.c > > >>>> properly. > > >>>> > > >>>> Please review this change. I please need a sponsor. > > >>>> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.00 > > >>> > > >>> This all sounds very good to me. (I must find out why Solaris > is not > > >>> involved here :) ). > > >>> > > >>> On Linux you didn't add the bounds check to > os::Linux::set_our_sigflags. > > >>> > > >>> I'm also wondering about documenting where we are determining the > > >>> maximum from? Is it simply _NSIG on some/all distributions? > And I see > > >>> _NSIG is supposed to be the biggest signal number + one. Also > linux > > >>> defines NSIG = _NSIG so which should we be using? > > >>> > > >>> Thanks, > > >>> David > > >>> > > >>>> Best regards, > > >>>> Goetz. > > >>>> > > >> > > >> > > >> -- > > >> Dmitry Samersoff > > >> Oracle Java development team, Saint Petersburg, Russia > > >> * I would love to change the world, but they won't give me the > sources. > > > > > > -- > > Dmitry Samersoff > > Oracle Java development team, Saint Petersburg, Russia > > * I would love to change the world, but they won't give me the > sources. > > > -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From dmitry.samersoff at oracle.com Mon Nov 16 10:54:19 2015 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Mon, 16 Nov 2015 13:54:19 +0300 Subject: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM In-Reply-To: <4295855A5C1DE049A61835A1887419CC41EB8DD6@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC41EB5DD6@DEWDFEMB12A.global.corp.sap> <5644413D.1000004@oracle.com> <56444C29.8060606@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB8252@DEWDFEMB12A.global.corp.sap> <56445646.4040707@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB85C3@DEWDFEMB12A.global.corp.sap> <5645E7D0.1000704@oracle.com> <564955AA.3020704@oracle.com> <56499279.2050806@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB8DD6@DEWDFEMB12A.global.corp.sap> Message-ID: <5649B5DB.3010105@oracle.com> Goetz, On 2015-11-16 13:32, Lindenmaier, Goetz wrote: > Hi Dmitry, > > I think this is a rather complex issue which is better documented in the > bug than in the code. The code already references the bugID. > Also, I would have to copy the explanation into three files ... > I you don't object I'll leave the comment as is. I'm OK with it. Leave it as is. -Dmitry > > Best regards, > Goetz. > > -----Original Message----- > From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com] > Sent: Montag, 16. November 2015 09:23 > To: David Holmes; Thomas St?fe; Lindenmaier, Goetz > Cc: serviceability-dev; hotspot-runtime-dev at openjdk.java.net > Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM > > David, > > Thank you for detailed explanation. Could we put it as a comment right > into the code? > > -Dmitry > > On 2015-11-16 07:03, David Holmes wrote: >> On 13/11/2015 11:38 PM, David Holmes wrote: >>> On 13/11/2015 7:53 PM, Thomas St?fe wrote: >>>> Hi Goetz, >>>> >>>> sorry for not looking at this earlier. This is a nice cleanup. Some >>>> remarks: >>>> >>>> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/aix/vm/os_aix.cpp.udiff.html >>>> >>>> >>>> >>>> + if (sig > MAX2(SIGSEGV, SIGBUS) && // See 4355769. >>>> + sig < NSIG) { // Must be legal signal and fit >>>> into sigflags[]. >>>> >>>> I do not like much the MAX2() construct. I would like it better to >>>> explicitly check whether the SR signal is one of the "forbidden" ones >>>> the VM uses. >>> >>> I must confess I had not looked into 4355769 but this check seems rather >>> spurious. It is not at all clear to me what signals could be used here - >> >> Okay should have looked into 4355769. The problem is how multiple >> pending signals are handled. It seems that in the past (no idea if still >> true) pending signals were handled in signal-number order (lowest >> first), not FIFO. The problem scenario is this: >> - thread accesses a null pointer in compiled Java code and the SEGV >> handler will cause NPE to be thrown >> - at the same time as the SEGV is being raised the thread is also hit >> with the SR signal to suspend it. >> - the SR signal will be delivered first and the SR handler starts to run >> - with signals unblocked. >> - the SEGV then gets delivered to the thread in the SR handler, and the >> regular signal handler is run >> - the regular signal handler tries to detect if we're running in Java >> code so it can post the NPE, but the presence of the SR handler causes >> that check to fail - so we abort thinking it is a real SEGV. >> >> I don't know how much of that is still true today. It seems strange to >> me that a kill based directed signal can usurp a synchronous signal. >> >> Anyway the fix, rather workaround, for that problem, was to ensure that >> the SR_signum is greater than any potential synchronous signal the VM >> cares about. Why SIGBUS was included there I don't know give that: >> >> a) it is already a lower signal number than SIGUSR1, SIGSEGV and SIGUSR2 >> b) we don't deliberately generate and use SIGBUS ... though perhaps >> unsafe-fetch needs to be considered. >> >> A better fix in my opinion, and as mentioned in the bug, would have been >> to disable delivery of SEGV whilst the SR handler is executing. But we >> start to touch on some grey areas of the POSIX spec there, and likely >> the implementation too. >> >> So I suggest that for this cleanup we simply leave this logic exactly as >> is. >> >> Thanks, >> David >> >>> other than SIGUSR1 or SIGUSR2 (if -Xrs is specified), or else a >>> real-time signal (modulo discussion below). Hijacking anything else >>> seems rather suspect. >>> >>>> Maybe keep a mask defined centrally for each platform which contains >>>> signals the VM needs for itself ? >>> >>> Such masks already exist. >>> >>>> +sigset_t os::Aix::sigs = { 0 }; >>>> >>>> I would not initialize the signal set this way. sigset_t is an opaque >>>> type; the only way to initialize it is with one of sigemptyset() or >>>> sigfillset(). >>> >>> Good catch - I overlooked that. >>> >>>> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/aix/vm/os_aix.hpp.udiff.html >>>> >>>> >>>> >>>> + static struct sigaction sigact[NSIG]; // saved preinstalled >>>> sigactions >>>> + static sigset_t sigs; // mask of signals that have >>>> >>>> + static int sigflags[NSIG]; >>>> >>>> I know this is not in the scope of your change, but I would like to see >>>> those removed from os::Aix and put into os_aix.cpp at static filescope. >>>> There is no need at all to export those, and you would get rid of the >>>> signal.h dependency you know have when including os_aix.hpp. >>>> >>>> >>>> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/bsd/vm/jsig.c.udiff.html >>>> >>>> >>>> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/bsd/vm/os_bsd.cpp.udiff.html >>>> >>>> >>>> >>>> On BSD, we have realtime signals. >>>> >>>> http://fxr.watson.org/fxr/source/sys/signal.h >>>> #define SIGRTMAX 126 >>>> and NSIG does not contain them: >>>> #define NSIG 32 >>>> >>>> The max. possible signal number would be 126, which unfortunately does >>>> not even fit into a 64bit mask. >>> >>> So this simply limits the signal choice to not be a real-time signal - >>> same as today. >>> >>>> So the code in jsig.c is broken for the case that someone wants to >>>> register realtime signals, if the VM were to ever use realtime signals >>>> itself, which now is not the case. >>>> >>>> The same is true for os_bsd.cpp, where signal chaining will not work if >>>> the application did have handler for real time signals pre-installed >>>> before jvm is loaded. >>> >>> Chaining is only used when the JVM will catch signals. Aren't all the >>> real-time signals going to be blocked by the VM by default and so >>> chaining is not needed as no handler will exist in the VM ?? (Unless a >>> real-time signal is supplied for SR_signum) >>> >>> I must admit I don't know if any of this code actually works for >>> real-time signals. >>> >>>> Solaris: >>>> >>>> The only platform where NSIG is missing? >>>> >>>> Here, we calculate the max. signal number dynamically in os_solaris.cpp, >>>> presumably because SIGRTMAX is not a constant and can be changed using >>>> system configuration. But then, on Linux we have the same situation >>>> (SIGRTMAX is dynamic) and there we do not go through the trouble of >>>> calculating the max. signal number dynamically. Instead we just use >>>> NSIG=64 and rely on the fact that NSIG is larger than the largest >>>> possible dynamic value for SIGRTMAX. >>> >>> Linux ensures that _NSIG (and thus NSIG) includes all the real-time >>> signals. But libc can expose a subset and steal some for its own use. >>> >>>> Solaris does not seem to have NSIG defined, but I am sure there is also >>>> a max. possible value for SIGRTMAX (the default seems to be 48). So, one >>>> could probably safely define NSIG for Solaris too, so that we have NSIG >>>> defined on all Posix platforms. >>> >>> Solaris doesn't have any of this SR_signum related code. A more general >>> cleanup of signal related code would potentially involve a lot of >>> cleanup. >>> >>> David >>> ----- >>> >>>> >>>> On Thu, Nov 12, 2015 at 8:24 PM, Lindenmaier, Goetz >>>> > wrote: >>>> >>>> Hi David, Dmitry, >>>> >>>> I've come up with a new webrev: >>>> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/ >>>> >>>> I hit on some more issues: >>>> - As proposed, I replaced MAXSIGNUM by NSIG >>>> - On AIX, NSIG=255. Therefore storing bits in a word does not >>>> work. >>>> I'm now using bitset functionality from signal.h as it's done >>>> in other places. >>>> sigset_t is >> NSIG on linux, so it's no good idea to use it >>>> there. >>>> >>>> >>>> Why do we not do this on all platforms, provided sigset_t contains all >>>> signals (incl. realtime signals) ? >>>> >>>> - In the os files I found another bit vector that now is too >>>> small: sigs. >>>> I adapted that, too. Removed the dead declaration of this on >>>> solaris. >>>> >>>> Best regards, >>>> Goetz. >>>> >>>> >>>> >>>> Kind Regards, Thomas >>>> >>>> > -----Original Message----- >>>> > From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com >>>> ] >>>> > Sent: Donnerstag, 12. November 2015 10:05 >>>> > To: Lindenmaier, Goetz; David Holmes; hotspot-runtime- >>>> > dev at openjdk.java.net ; >>>> serviceability-dev >>>> > Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM >>>> > >>>> > Goetz, >>>> > >>>> > *BSD including OS X also defines NSIG (just checked) and if my >>>> memory is >>>> > not bogus, AIX defines it too. >>>> > >>>> > So you may consider to use NSIG on all platform. >>>> > >>>> > -Dmitry >>>> > >>>> > On 2015-11-12 11:36, Lindenmaier, Goetz wrote: >>>> > > OK I'll change it to NSIG. That's used in other places in >>>> os_linux, too. >>>> > > So it's really more consistent. >>>> > > >>>> > > Best regards, >>>> > > Goetz >>>> > > >>>> > >> -----Original Message----- >>>> > >> From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com >>>> ] >>>> > >> Sent: Donnerstag, 12. November 2015 09:22 >>>> > >> To: David Holmes; Lindenmaier, Goetz; hotspot-runtime- >>>> > >> dev at openjdk.java.net ; >>>> serviceability-dev >>>> > >> Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM >>>> > >> >>>> > >> David, >>>> > >> >>>> > >> I think it's better to use NSIG (without underscore) defined >>>> in signal.h >>>> > >> >>>> > >> -Dmitry >>>> > >> >>>> > >> >>>> > >> On 2015-11-12 10:35, David Holmes wrote: >>>> > >>> Hi Goetz, >>>> > >>> >>>> > >>> Adding in serviceability-dev >>>> > >>> >>>> > >>> On 9/11/2015 6:22 PM, Lindenmaier, Goetz wrote: >>>> > >>>> Hi, >>>> > >>>> >>>> > >>>> The environment variable _JAVA_SR_SIGNUM can be set to a >>>> signal >>>> > >> number >>>> > >>>> do be used by the JVM's suspend/resume mechanism. >>>> > >>>> >>>> > >>>> If set, a signal handler is installed and the current signal >>>> handler >>>> > >>>> is saved to an array. >>>> > >>>> On linux, this array had size MAXSIGNUM=32, and >>>> _JAVA_SR_SIGNUM >>>> > >> was >>>> > >>>> allowed >>>> > >>>> to range up to _NSIG=65. This could cause memory corruption. >>>> > >>>> >>>> > >>>> Further, in jsig.c, an unsinged int is used to set a bit for >>>> signals. >>>> > >>>> This also >>>> > >>>> is too small, as only 32 signals can be supported. >>>> Further, the >>>> > >>>> signals are mapped >>>> > >>>> wrong to these bits. '0' is not a valid signal, but '32' >>>> was. 1<<32 >>>> > >>>> happens to map to >>>> > >>>> zero, so the signal could be stored, but this probably was >>>> not >>>> > >>>> intended that way. >>>> > >>>> >>>> > >>>> This change increases MAXSIGNUM to 65 on linux, and to 64 on >>>> aix. It >>>> > >>>> introduces >>>> > >>>> proper checking of the signal read from the env var, and >>>> issues a >>>> > >>>> warning if it >>>> > >>>> does not use the signal set. It adapts the data types in >>>> jisig.c >>>> > >>>> properly. >>>> > >>>> >>>> > >>>> Please review this change. I please need a sponsor. >>>> > >>>> >>>> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.00 >>>> > >>> >>>> > >>> This all sounds very good to me. (I must find out why Solaris >>>> is not >>>> > >>> involved here :) ). >>>> > >>> >>>> > >>> On Linux you didn't add the bounds check to >>>> os::Linux::set_our_sigflags. >>>> > >>> >>>> > >>> I'm also wondering about documenting where we are >>>> determining the >>>> > >>> maximum from? Is it simply _NSIG on some/all distributions? >>>> And I see >>>> > >>> _NSIG is supposed to be the biggest signal number + one. Also >>>> linux >>>> > >>> defines NSIG = _NSIG so which should we be using? >>>> > >>> >>>> > >>> Thanks, >>>> > >>> David >>>> > >>> >>>> > >>>> Best regards, >>>> > >>>> Goetz. >>>> > >>>> >>>> > >> >>>> > >> >>>> > >> -- >>>> > >> Dmitry Samersoff >>>> > >> Oracle Java development team, Saint Petersburg, Russia >>>> > >> * I would love to change the world, but they won't give me the >>>> sources. >>>> > >>>> > >>>> > -- >>>> > Dmitry Samersoff >>>> > Oracle Java development team, Saint Petersburg, Russia >>>> > * I would love to change the world, but they won't give me the >>>> sources. >>>> >>>> > > -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From david.holmes at oracle.com Mon Nov 16 11:18:34 2015 From: david.holmes at oracle.com (David Holmes) Date: Mon, 16 Nov 2015 21:18:34 +1000 Subject: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM In-Reply-To: References: <4295855A5C1DE049A61835A1887419CC41EB5DD6@DEWDFEMB12A.global.corp.sap> <5644413D.1000004@oracle.com> <56444C29.8060606@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB8252@DEWDFEMB12A.global.corp.sap> <56445646.4040707@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB85C3@DEWDFEMB12A.global.corp.sap> <5645E7D0.1000704@oracle.com> <564955AA.3020704@oracle.com> Message-ID: <5649BB8A.7070006@oracle.com> On 16/11/2015 7:45 PM, Thomas St?fe wrote: > Hi David, > > On Mon, Nov 16, 2015 at 5:03 AM, David Holmes > wrote: > > On 13/11/2015 11:38 PM, David Holmes wrote: > > On 13/11/2015 7:53 PM, Thomas St?fe wrote: > > Hi Goetz, > > sorry for not looking at this earlier. This is a nice > cleanup. Some > remarks: > > http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/aix/vm/os_aix.cpp.udiff.html > > > + if (sig > MAX2(SIGSEGV, SIGBUS) && // See 4355769. > + sig < NSIG) { // Must be legal > signal and fit > into sigflags[]. > > I do not like much the MAX2() construct. I would like it > better to > explicitly check whether the SR signal is one of the > "forbidden" ones > the VM uses. > > > I must confess I had not looked into 4355769 but this check > seems rather > spurious. It is not at all clear to me what signals could be > used here - > > > Okay should have looked into 4355769. The problem is how multiple > pending signals are handled. It seems that in the past (no idea if > still true) pending signals were handled in signal-number order > (lowest first), not FIFO. The problem scenario is this: > - thread accesses a null pointer in compiled Java code and the SEGV > handler will cause NPE to be thrown > - at the same time as the SEGV is being raised the thread is also > hit with the SR signal to suspend it. > - the SR signal will be delivered first and the SR handler starts to > run - with signals unblocked. > - the SEGV then gets delivered to the thread in the SR handler, and > the regular signal handler is run > - the regular signal handler tries to detect if we're running in > Java code so it can post the NPE, but the presence of the SR handler > causes that check to fail - so we abort thinking it is a real SEGV. > > I don't know how much of that is still true today. It seems strange > to me that a kill based directed signal can usurp a synchronous signal. > > Anyway the fix, rather workaround, for that problem, was to ensure > that the SR_signum is greater than any potential synchronous signal > the VM cares about. Why SIGBUS was included there I don't know give > that: > > a) it is already a lower signal number than SIGUSR1, SIGSEGV and SIGUSR2 > b) we don't deliberately generate and use SIGBUS ... though perhaps > unsafe-fetch needs to be considered. > > A better fix in my opinion, and as mentioned in the bug, would have > been to disable delivery of SEGV whilst the SR handler is executing. > But we start to touch on some grey areas of the POSIX spec there, > and likely the implementation too. > > > How would this work? I think the process just dies immediately if an > synchronous signal occurs while being blocked. At least that is why we > needed to fix https://bugs.openjdk.java.net/browse/JDK-8065895. POSIX specifies that it is undefined if such a signal is blocked - hence my reference to grey areas of the spec and implementation. It looks like once upon a time it didn't cause the process to die immediately, but perhaps now it does. If so we can't block it. But it may also be that the original delivery order problem is not possible as well. Hard to know - hence my suggestion to just leave it as is, at least for now. Cheers, David > Kind Regards, Thomas > > > So I suggest that for this cleanup we simply leave this logic > exactly as is. > > Thanks, > David > > > other than SIGUSR1 or SIGUSR2 (if -Xrs is specified), or else a > real-time signal (modulo discussion below). Hijacking anything else > seems rather suspect. > > Maybe keep a mask defined centrally for each platform which > contains > signals the VM needs for itself ? > > > Such masks already exist. > > +sigset_t os::Aix::sigs = { 0 }; > > I would not initialize the signal set this way. sigset_t is > an opaque > type; the only way to initialize it is with one of > sigemptyset() or > sigfillset(). > > > Good catch - I overlooked that. > > http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/aix/vm/os_aix.hpp.udiff.html > > > + static struct sigaction sigact[NSIG]; // saved > preinstalled sigactions > + static sigset_t sigs; // mask of signals > that have > > + static int sigflags[NSIG]; > > I know this is not in the scope of your change, but I would > like to see > those removed from os::Aix and put into os_aix.cpp at static > filescope. > There is no need at all to export those, and you would get > rid of the > signal.h dependency you know have when including os_aix.hpp. > > > http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/bsd/vm/jsig.c.udiff.html > > http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/bsd/vm/os_bsd.cpp.udiff.html > > > On BSD, we have realtime signals. > > http://fxr.watson.org/fxr/source/sys/signal.h > #define SIGRTMAX 126 > and NSIG does not contain them: > #define NSIG 32 > > The max. possible signal number would be 126, which > unfortunately does > not even fit into a 64bit mask. > > > So this simply limits the signal choice to not be a real-time > signal - > same as today. > > So the code in jsig.c is broken for the case that someone > wants to > register realtime signals, if the VM were to ever use > realtime signals > itself, which now is not the case. > > The same is true for os_bsd.cpp, where signal chaining will > not work if > the application did have handler for real time signals > pre-installed > before jvm is loaded. > > > Chaining is only used when the JVM will catch signals. Aren't > all the > real-time signals going to be blocked by the VM by default and so > chaining is not needed as no handler will exist in the VM ?? > (Unless a > real-time signal is supplied for SR_signum) > > I must admit I don't know if any of this code actually works for > real-time signals. > > Solaris: > > The only platform where NSIG is missing? > > Here, we calculate the max. signal number dynamically in > os_solaris.cpp, > presumably because SIGRTMAX is not a constant and can be > changed using > system configuration. But then, on Linux we have the same > situation > (SIGRTMAX is dynamic) and there we do not go through the > trouble of > calculating the max. signal number dynamically. Instead we > just use > NSIG=64 and rely on the fact that NSIG is larger than the > largest > possible dynamic value for SIGRTMAX. > > > Linux ensures that _NSIG (and thus NSIG) includes all the real-time > signals. But libc can expose a subset and steal some for its own > use. > > Solaris does not seem to have NSIG defined, but I am sure > there is also > a max. possible value for SIGRTMAX (the default seems to be > 48). So, one > could probably safely define NSIG for Solaris too, so that > we have NSIG > defined on all Posix platforms. > > > Solaris doesn't have any of this SR_signum related code. A more > general > cleanup of signal related code would potentially involve a lot > of cleanup. > > David > ----- > > > On Thu, Nov 12, 2015 at 8:24 PM, Lindenmaier, Goetz > > >> wrote: > > Hi David, Dmitry, > > I've come up with a new webrev: > http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/ > > I hit on some more issues: > - As proposed, I replaced MAXSIGNUM by NSIG > - On AIX, NSIG=255. Therefore storing bits in a word > does not > work. > I'm now using bitset functionality from signal.h > as it's done > in other places. > sigset_t is >> NSIG on linux, so it's no good idea > to use it > there. > > > Why do we not do this on all platforms, provided sigset_t > contains all > signals (incl. realtime signals) ? > > - In the os files I found another bit vector that now > is too > small: sigs. > I adapted that, too. Removed the dead declaration > of this on > solaris. > > Best regards, > Goetz. > > > > Kind Regards, Thomas > > > -----Original Message----- > > From: Dmitry Samersoff > [mailto:dmitry.samersoff at oracle.com > > >] > > Sent: Donnerstag, 12. November 2015 10:05 > > To: Lindenmaier, Goetz; David Holmes; hotspot-runtime- > > dev at openjdk.java.net > >; > serviceability-dev > > Subject: Re: RFR(M): 8141529: Fix handling of > _JAVA_SR_SIGNUM > > > > Goetz, > > > > *BSD including OS X also defines NSIG (just checked) > and if my > memory is > > not bogus, AIX defines it too. > > > > So you may consider to use NSIG on all platform. > > > > -Dmitry > > > > On 2015-11-12 11:36, Lindenmaier, Goetz wrote: > > > OK I'll change it to NSIG. That's used in other > places in > os_linux, too. > > > So it's really more consistent. > > > > > > Best regards, > > > Goetz > > > > > >> -----Original Message----- > > >> From: Dmitry Samersoff > [mailto:dmitry.samersoff at oracle.com > > >] > > >> Sent: Donnerstag, 12. November 2015 09:22 > > >> To: David Holmes; Lindenmaier, Goetz; > hotspot-runtime- > > >> dev at openjdk.java.net > >; > serviceability-dev > > >> Subject: Re: RFR(M): 8141529: Fix handling of > _JAVA_SR_SIGNUM > > >> > > >> David, > > >> > > >> I think it's better to use NSIG (without > underscore) defined > in signal.h > > >> > > >> -Dmitry > > >> > > >> > > >> On 2015-11-12 10:35, David Holmes wrote: > > >>> Hi Goetz, > > >>> > > >>> Adding in serviceability-dev > > >>> > > >>> On 9/11/2015 6:22 PM, Lindenmaier, Goetz wrote: > > >>>> Hi, > > >>>> > > >>>> The environment variable _JAVA_SR_SIGNUM can be > set to a > signal > > >> number > > >>>> do be used by the JVM's suspend/resume mechanism. > > >>>> > > >>>> If set, a signal handler is installed and the > current signal > handler > > >>>> is saved to an array. > > >>>> On linux, this array had size MAXSIGNUM=32, and > _JAVA_SR_SIGNUM > > >> was > > >>>> allowed > > >>>> to range up to _NSIG=65. This could cause > memory corruption. > > >>>> > > >>>> Further, in jsig.c, an unsinged int is used to > set a bit for > signals. > > >>>> This also > > >>>> is too small, as only 32 signals can be supported. > Further, the > > >>>> signals are mapped > > >>>> wrong to these bits. '0' is not a valid > signal, but '32' > was. 1<<32 > > >>>> happens to map to > > >>>> zero, so the signal could be stored, but this > probably was > not > > >>>> intended that way. > > >>>> > > >>>> This change increases MAXSIGNUM to 65 on linux, > and to 64 on > aix. It > > >>>> introduces > > >>>> proper checking of the signal read from the env > var, and > issues a > > >>>> warning if it > > >>>> does not use the signal set. It adapts the > data types in > jisig.c > > >>>> properly. > > >>>> > > >>>> Please review this change. I please need a > sponsor. > > >>>> > http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.00 > > >>> > > >>> This all sounds very good to me. (I must find > out why Solaris > is not > > >>> involved here :) ). > > >>> > > >>> On Linux you didn't add the bounds check to > os::Linux::set_our_sigflags. > > >>> > > >>> I'm also wondering about documenting where we are > determining the > > >>> maximum from? Is it simply _NSIG on some/all > distributions? > And I see > > >>> _NSIG is supposed to be the biggest signal > number + one. Also > linux > > >>> defines NSIG = _NSIG so which should we be using? > > >>> > > >>> Thanks, > > >>> David > > >>> > > >>>> Best regards, > > >>>> Goetz. > > >>>> > > >> > > >> > > >> -- > > >> Dmitry Samersoff > > >> Oracle Java development team, Saint Petersburg, > Russia > > >> * I would love to change the world, but they > won't give me the > sources. > > > > > > -- > > Dmitry Samersoff > > Oracle Java development team, Saint Petersburg, Russia > > * I would love to change the world, but they won't > give me the > sources. > > > From thomas.stuefe at gmail.com Tue Nov 17 06:57:31 2015 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 17 Nov 2015 07:57:31 +0100 Subject: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM In-Reply-To: <4295855A5C1DE049A61835A1887419CC41EB8DB5@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC41EB5DD6@DEWDFEMB12A.global.corp.sap> <5644413D.1000004@oracle.com> <56444C29.8060606@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB8252@DEWDFEMB12A.global.corp.sap> <56445646.4040707@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB85C3@DEWDFEMB12A.global.corp.sap> <4295855A5C1DE049A61835A1887419CC41EB8DB5@DEWDFEMB12A.global.corp.sap> Message-ID: Hi G?tz, looks good! ..Thomas On Mon, Nov 16, 2015 at 11:25 AM, Lindenmaier, Goetz < goetz.lindenmaier at sap.com> wrote: > Hi Thomas, > > > > thanks for looking at this. > > > > > MAX2(SIGSEGV, SIGBUS) > > I really would like to leave the code as-is. This would be a functional > > change, while I only intend to fix issues in this change. Also, as David > > explained, it might break for some os implementations. > > > > > the only way to initialize it is with one of sigemptyset() or > sigfillset(). > > I added initialization with sigemptyset(). Unfortunately, there is no > static > > initializer for this. > > > > > I would like to see those removed from os::Aix and put into os_aix.cpp > at static filescope > > I moved these to static scope on the three oses. > > > > Here is the new webrev: > > http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.02/ > > > > Best regards, > > Goetz. > > > > > > *From:* Thomas St?fe [mailto:thomas.stuefe at gmail.com] > *Sent:* Freitag, 13. November 2015 10:54 > *To:* Lindenmaier, Goetz > *Cc:* Dmitry Samersoff; David Holmes; hotspot-runtime-dev at openjdk.java.net; > serviceability-dev > > *Subject:* Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM > > > > Hi Goetz, > > > > sorry for not looking at this earlier. This is a nice cleanup. Some > remarks: > > > > > http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/aix/vm/os_aix.cpp.udiff.html > > > > + if (sig > MAX2(SIGSEGV, SIGBUS) && // See 4355769. > > + sig < NSIG) { // Must be legal signal and fit > into sigflags[]. > > > > I do not like much the MAX2() construct. I would like it better to > explicitly check whether the SR signal is one of the "forbidden" ones the > VM uses. > > > > Maybe keep a mask defined centrally for each platform which contains > signals the VM needs for itself ? > > > > +sigset_t os::Aix::sigs = { 0 }; > > > > I would not initialize the signal set this way. sigset_t is an opaque > type; the only way to initialize it is with one of sigemptyset() or > sigfillset(). > > > > > http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/aix/vm/os_aix.hpp.udiff.html > > > > + static struct sigaction sigact[NSIG]; // saved preinstalled sigactions > > + static sigset_t sigs; // mask of signals that have > > > > + static int sigflags[NSIG]; > > > > I know this is not in the scope of your change, but I would like to see > those removed from os::Aix and put into os_aix.cpp at static filescope. > There is no need at all to export those, and you would get rid of the > signal.h dependency you know have when including os_aix.hpp. > > > > > > > http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/bsd/vm/jsig.c.udiff.html > > > http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/src/os/bsd/vm/os_bsd.cpp.udiff.html > > > > On BSD, we have realtime signals. > > > > http://fxr.watson.org/fxr/source/sys/signal.h > > #define SIGRTMAX 126 > > and NSIG does not contain them: > > #define NSIG 32 > > > > The max. possible signal number would be 126, which unfortunately does not > even fit into a 64bit mask. > > > > So the code in jsig.c is broken for the case that someone wants to > register realtime signals, if the VM were to ever use realtime signals > itself, which now is not the case. > > > > The same is true for os_bsd.cpp, where signal chaining will not work if > the application did have handler for real time signals pre-installed before > jvm is loaded. > > > > Solaris: > > > > The only platform where NSIG is missing? > > > > Here, we calculate the max. signal number dynamically in os_solaris.cpp, > presumably because SIGRTMAX is not a constant and can be changed using > system configuration. But then, on Linux we have the same situation > (SIGRTMAX is dynamic) and there we do not go through the trouble of > calculating the max. signal number dynamically. Instead we just use NSIG=64 > and rely on the fact that NSIG is larger than the largest possible dynamic > value for SIGRTMAX. > > > > Solaris does not seem to have NSIG defined, but I am sure there is also a > max. possible value for SIGRTMAX (the default seems to be 48). So, one > could probably safely define NSIG for Solaris too, so that we have NSIG > defined on all Posix platforms. > > > > > > On Thu, Nov 12, 2015 at 8:24 PM, Lindenmaier, Goetz < > goetz.lindenmaier at sap.com> wrote: > > Hi David, Dmitry, > > I've come up with a new webrev: > http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/ > > I hit on some more issues: > - As proposed, I replaced MAXSIGNUM by NSIG > - On AIX, NSIG=255. Therefore storing bits in a word does not work. > I'm now using bitset functionality from signal.h as it's done in other > places. > sigset_t is >> NSIG on linux, so it's no good idea to use it there. > > > > Why do we not do this on all platforms, provided sigset_t contains all > signals (incl. realtime signals) ? > > > > - In the os files I found another bit vector that now is too small: sigs. > I adapted that, too. Removed the dead declaration of this on solaris. > > Best regards, > Goetz. > > > > > > Kind Regards, Thomas > > > > > > > -----Original Message----- > > From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com] > > > Sent: Donnerstag, 12. November 2015 10:05 > > To: Lindenmaier, Goetz; David Holmes; hotspot-runtime- > > dev at openjdk.java.net; serviceability-dev > > Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM > > > > Goetz, > > > > *BSD including OS X also defines NSIG (just checked) and if my memory is > > not bogus, AIX defines it too. > > > > So you may consider to use NSIG on all platform. > > > > -Dmitry > > > > On 2015-11-12 11:36, Lindenmaier, Goetz wrote: > > > OK I'll change it to NSIG. That's used in other places in os_linux, > too. > > > So it's really more consistent. > > > > > > Best regards, > > > Goetz > > > > > >> -----Original Message----- > > >> From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com] > > >> Sent: Donnerstag, 12. November 2015 09:22 > > >> To: David Holmes; Lindenmaier, Goetz; hotspot-runtime- > > >> dev at openjdk.java.net; serviceability-dev > > >> Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM > > >> > > >> David, > > >> > > >> I think it's better to use NSIG (without underscore) defined in > signal.h > > >> > > >> -Dmitry > > >> > > >> > > >> On 2015-11-12 10:35, David Holmes wrote: > > >>> Hi Goetz, > > >>> > > >>> Adding in serviceability-dev > > >>> > > >>> On 9/11/2015 6:22 PM, Lindenmaier, Goetz wrote: > > >>>> Hi, > > >>>> > > >>>> The environment variable _JAVA_SR_SIGNUM can be set to a signal > > >> number > > >>>> do be used by the JVM's suspend/resume mechanism. > > >>>> > > >>>> If set, a signal handler is installed and the current signal handler > > >>>> is saved to an array. > > >>>> On linux, this array had size MAXSIGNUM=32, and _JAVA_SR_SIGNUM > > >> was > > >>>> allowed > > >>>> to range up to _NSIG=65. This could cause memory corruption. > > >>>> > > >>>> Further, in jsig.c, an unsinged int is used to set a bit for > signals. > > >>>> This also > > >>>> is too small, as only 32 signals can be supported. Further, the > > >>>> signals are mapped > > >>>> wrong to these bits. '0' is not a valid signal, but '32' was. > 1<<32 > > >>>> happens to map to > > >>>> zero, so the signal could be stored, but this probably was not > > >>>> intended that way. > > >>>> > > >>>> This change increases MAXSIGNUM to 65 on linux, and to 64 on aix. It > > >>>> introduces > > >>>> proper checking of the signal read from the env var, and issues a > > >>>> warning if it > > >>>> does not use the signal set. It adapts the data types in jisig.c > > >>>> properly. > > >>>> > > >>>> Please review this change. I please need a sponsor. > > >>>> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.00 > > >>> > > >>> This all sounds very good to me. (I must find out why Solaris is not > > >>> involved here :) ). > > >>> > > >>> On Linux you didn't add the bounds check to > os::Linux::set_our_sigflags. > > >>> > > >>> I'm also wondering about documenting where we are determining the > > >>> maximum from? Is it simply _NSIG on some/all distributions? And I see > > >>> _NSIG is supposed to be the biggest signal number + one. Also linux > > >>> defines NSIG = _NSIG so which should we be using? > > >>> > > >>> Thanks, > > >>> David > > >>> > > >>>> Best regards, > > >>>> Goetz. > > >>>> > > >> > > >> > > >> -- > > >> Dmitry Samersoff > > >> Oracle Java development team, Saint Petersburg, Russia > > >> * I would love to change the world, but they won't give me the > sources. > > > > > > -- > > Dmitry Samersoff > > Oracle Java development team, Saint Petersburg, Russia > > * I would love to change the world, but they won't give me the sources. > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From goetz.lindenmaier at sap.com Tue Nov 17 07:52:37 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 17 Nov 2015 07:52:37 +0000 Subject: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM In-Reply-To: <5649B599.60807@oracle.com> References: <4295855A5C1DE049A61835A1887419CC41EB5DD6@DEWDFEMB12A.global.corp.sap> <5644413D.1000004@oracle.com> <56444C29.8060606@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB8252@DEWDFEMB12A.global.corp.sap> <56445646.4040707@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB85C3@DEWDFEMB12A.global.corp.sap> <4295855A5C1DE049A61835A1887419CC41EB8DB5@DEWDFEMB12A.global.corp.sap> <5649B599.60807@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC41EB93E1@DEWDFEMB12A.global.corp.sap> David, Dmitry, I think this is reviewed now. Could one of you please sponsor this? The final webrev, including a full changeset patch, is this: http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.02/ It ran successfully through our nightly tests, tonight. Best regards, Goetz. > -----Original Message----- > From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com] > Sent: Montag, 16. November 2015 11:53 > To: Lindenmaier, Goetz; Thomas St?fe > Cc: David Holmes; hotspot-runtime-dev at openjdk.java.net; serviceability- > dev > Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM > > Goetz, > > Looks good for me. > > Thank you for a nice cleanup. > > -Dmitry > > > On 2015-11-16 13:25, Lindenmaier, Goetz wrote: > > Hi Thomas, > > > > > > > > thanks for looking at this. > > > > > > > >> MAX2(SIGSEGV, SIGBUS) > > > > I really would like to leave the code as-is. This would be a functional > > > > change, while I only intend to fix issues in this change. Also, as David > > > > explained, it might break for some os implementations. > > > > > > > >> the only way to initialize it is with one of sigemptyset() or sigfillset(). > > > > I added initialization with sigemptyset(). Unfortunately, there is no > > static > > > > initializer for this. > > > > > > > >> I would like to see those removed from os::Aix and put into os_aix.cpp at > static filescope > > > > I moved these to static scope on the three oses. > > > > > > > > Here is the new webrev: > > > > http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.02/ > > > > > > > > Best regards, > > > > Goetz. > > > > > > > > > > > > *From:*Thomas St?fe [mailto:thomas.stuefe at gmail.com] > > *Sent:* Freitag, 13. November 2015 10:54 > > *To:* Lindenmaier, Goetz > > *Cc:* Dmitry Samersoff; David Holmes; > > hotspot-runtime-dev at openjdk.java.net; serviceability-dev > > *Subject:* Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM > > > > > > > > Hi Goetz, > > > > > > > > sorry for not looking at this earlier. This is a nice cleanup. Some remarks: > > > > > > > > http://cr.openjdk.java.net/~goetz/webrevs/8141529- > NSIG/webrev.01/src/os/aix/vm/os_aix.cpp.udiff.html > > > > > > > > + if (sig > MAX2(SIGSEGV, SIGBUS) && // See 4355769. > > > > + sig < NSIG) { // Must be legal signal and fit > > into sigflags[]. > > > > > > > > I do not like much the MAX2() construct. I would like it better to > > explicitly check whether the SR signal is one of the "forbidden" ones > > the VM uses. > > > > > > > > Maybe keep a mask defined centrally for each platform which contains > > signals the VM needs for itself ? > > > > > > > > +sigset_t os::Aix::sigs = { 0 }; > > > > > > > > I would not initialize the signal set this way. sigset_t is an opaque > > type; the only way to initialize it is with one of sigemptyset() or > > sigfillset(). > > > > > > > > http://cr.openjdk.java.net/~goetz/webrevs/8141529- > NSIG/webrev.01/src/os/aix/vm/os_aix.hpp.udiff.html > > > > > > > > + static struct sigaction sigact[NSIG]; // saved preinstalled sigactions > > > > + static sigset_t sigs; // mask of signals that have > > > > > > > > + static int sigflags[NSIG]; > > > > > > > > I know this is not in the scope of your change, but I would like to see > > those removed from os::Aix and put into os_aix.cpp at static filescope. > > There is no need at all to export those, and you would get rid of the > > signal.h dependency you know have when including os_aix.hpp. > > > > > > > > > > > > http://cr.openjdk.java.net/~goetz/webrevs/8141529- > NSIG/webrev.01/src/os/bsd/vm/jsig.c.udiff.html > > > > http://cr.openjdk.java.net/~goetz/webrevs/8141529- > NSIG/webrev.01/src/os/bsd/vm/os_bsd.cpp.udiff.html > > > > > > > > On BSD, we have realtime signals. > > > > > > > > http://fxr.watson.org/fxr/source/sys/signal.h > > > > #define SIGRTMAX 126 > > > > and NSIG does not contain them: > > > > #define NSIG 32 > > > > > > > > The max. possible signal number would be 126, which unfortunately does > > not even fit into a 64bit mask. > > > > > > > > So the code in jsig.c is broken for the case that someone wants to > > register realtime signals, if the VM were to ever use realtime signals > > itself, which now is not the case. > > > > > > > > The same is true for os_bsd.cpp, where signal chaining will not work if > > the application did have handler for real time signals pre-installed > > before jvm is loaded. > > > > > > > > Solaris: > > > > > > > > The only platform where NSIG is missing? > > > > > > > > Here, we calculate the max. signal number dynamically in os_solaris.cpp, > > presumably because SIGRTMAX is not a constant and can be changed using > > system configuration. But then, on Linux we have the same situation > > (SIGRTMAX is dynamic) and there we do not go through the trouble of > > calculating the max. signal number dynamically. Instead we just use > > NSIG=64 and rely on the fact that NSIG is larger than the largest > > possible dynamic value for SIGRTMAX. > > > > > > > > Solaris does not seem to have NSIG defined, but I am sure there is also > > a max. possible value for SIGRTMAX (the default seems to be 48). So, one > > could probably safely define NSIG for Solaris too, so that we have NSIG > > defined on all Posix platforms. > > > > > > > > > > > > On Thu, Nov 12, 2015 at 8:24 PM, Lindenmaier, Goetz > > > > wrote: > > > > Hi David, Dmitry, > > > > I've come up with a new webrev: > > http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/ > > > > I hit on some more issues: > > - As proposed, I replaced MAXSIGNUM by NSIG > > - On AIX, NSIG=255. Therefore storing bits in a word does not work. > > I'm now using bitset functionality from signal.h as it's done in > > other places. > > sigset_t is >> NSIG on linux, so it's no good idea to use it there. > > > > > > > > Why do we not do this on all platforms, provided sigset_t contains all > > signals (incl. realtime signals) ? > > > > > > > > - In the os files I found another bit vector that now is too small: > > sigs. > > I adapted that, too. Removed the dead declaration of this on > > solaris. > > > > Best regards, > > Goetz. > > > > > > > > > > > > Kind Regards, Thomas > > > > > > > > > > > > > -----Original Message----- > > > From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com > ] > > > > > Sent: Donnerstag, 12. November 2015 10:05 > > > To: Lindenmaier, Goetz; David Holmes; hotspot-runtime- > > > dev at openjdk.java.net ; serviceability- > dev > > > Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM > > > > > > Goetz, > > > > > > *BSD including OS X also defines NSIG (just checked) and if my > > memory is > > > not bogus, AIX defines it too. > > > > > > So you may consider to use NSIG on all platform. > > > > > > -Dmitry > > > > > > On 2015-11-12 11:36, Lindenmaier, Goetz wrote: > > > > OK I'll change it to NSIG. That's used in other places in > > os_linux, too. > > > > So it's really more consistent. > > > > > > > > Best regards, > > > > Goetz > > > > > > > >> -----Original Message----- > > > >> From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com > > ] > > > >> Sent: Donnerstag, 12. November 2015 09:22 > > > >> To: David Holmes; Lindenmaier, Goetz; hotspot-runtime- > > > >> dev at openjdk.java.net ; > > serviceability-dev > > > >> Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM > > > >> > > > >> David, > > > >> > > > >> I think it's better to use NSIG (without underscore) defined in > > signal.h > > > >> > > > >> -Dmitry > > > >> > > > >> > > > >> On 2015-11-12 10:35, David Holmes wrote: > > > >>> Hi Goetz, > > > >>> > > > >>> Adding in serviceability-dev > > > >>> > > > >>> On 9/11/2015 6:22 PM, Lindenmaier, Goetz wrote: > > > >>>> Hi, > > > >>>> > > > >>>> The environment variable _JAVA_SR_SIGNUM can be set to a > signal > > > >> number > > > >>>> do be used by the JVM's suspend/resume mechanism. > > > >>>> > > > >>>> If set, a signal handler is installed and the current signal > > handler > > > >>>> is saved to an array. > > > >>>> On linux, this array had size MAXSIGNUM=32, and > _JAVA_SR_SIGNUM > > > >> was > > > >>>> allowed > > > >>>> to range up to _NSIG=65. This could cause memory corruption. > > > >>>> > > > >>>> Further, in jsig.c, an unsinged int is used to set a bit for > > signals. > > > >>>> This also > > > >>>> is too small, as only 32 signals can be supported. Further, the > > > >>>> signals are mapped > > > >>>> wrong to these bits. '0' is not a valid signal, but '32' > > was. 1<<32 > > > >>>> happens to map to > > > >>>> zero, so the signal could be stored, but this probably was not > > > >>>> intended that way. > > > >>>> > > > >>>> This change increases MAXSIGNUM to 65 on linux, and to 64 on > > aix. It > > > >>>> introduces > > > >>>> proper checking of the signal read from the env var, and issues a > > > >>>> warning if it > > > >>>> does not use the signal set. It adapts the data types in jisig.c > > > >>>> properly. > > > >>>> > > > >>>> Please review this change. I please need a sponsor. > > > >>>> http://cr.openjdk.java.net/~goetz/webrevs/8141529- > NSIG/webrev.00 > > > >>> > > > >>> This all sounds very good to me. (I must find out why Solaris > > is not > > > >>> involved here :) ). > > > >>> > > > >>> On Linux you didn't add the bounds check to > > os::Linux::set_our_sigflags. > > > >>> > > > >>> I'm also wondering about documenting where we are determining > the > > > >>> maximum from? Is it simply _NSIG on some/all distributions? > > And I see > > > >>> _NSIG is supposed to be the biggest signal number + one. Also > > linux > > > >>> defines NSIG = _NSIG so which should we be using? > > > >>> > > > >>> Thanks, > > > >>> David > > > >>> > > > >>>> Best regards, > > > >>>> Goetz. > > > >>>> > > > >> > > > >> > > > >> -- > > > >> Dmitry Samersoff > > > >> Oracle Java development team, Saint Petersburg, Russia > > > >> * I would love to change the world, but they won't give me the > > sources. > > > > > > > > > -- > > > Dmitry Samersoff > > > Oracle Java development team, Saint Petersburg, Russia > > > * I would love to change the world, but they won't give me the > > sources. > > > > > > > > > -- > Dmitry Samersoff > Oracle Java development team, Saint Petersburg, Russia > * I would love to change the world, but they won't give me the sources. From dmitry.samersoff at oracle.com Tue Nov 17 08:33:40 2015 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Tue, 17 Nov 2015 11:33:40 +0300 Subject: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM In-Reply-To: <4295855A5C1DE049A61835A1887419CC41EB93E1@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC41EB5DD6@DEWDFEMB12A.global.corp.sap> <5644413D.1000004@oracle.com> <56444C29.8060606@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB8252@DEWDFEMB12A.global.corp.sap> <56445646.4040707@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB85C3@DEWDFEMB12A.global.corp.sap> <4295855A5C1DE049A61835A1887419CC41EB8DB5@DEWDFEMB12A.global.corp.sap> <5649B599.60807@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB93E1@DEWDFEMB12A.global.corp.sap> Message-ID: <564AE664.6000205@oracle.com> Goetz, I'll sponsor the push -Dmitry. On 2015-11-17 10:52, Lindenmaier, Goetz wrote: > David, Dmitry, > > I think this is reviewed now. Could one of you please sponsor this? > The final webrev, including a full changeset patch, is this: > http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.02/ > > It ran successfully through our nightly tests, tonight. > > Best regards, > Goetz. > >> -----Original Message----- >> From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com] >> Sent: Montag, 16. November 2015 11:53 >> To: Lindenmaier, Goetz; Thomas St?fe >> Cc: David Holmes; hotspot-runtime-dev at openjdk.java.net; serviceability- >> dev >> Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM >> >> Goetz, >> >> Looks good for me. >> >> Thank you for a nice cleanup. >> >> -Dmitry >> >> >> On 2015-11-16 13:25, Lindenmaier, Goetz wrote: >>> Hi Thomas, >>> >>> >>> >>> thanks for looking at this. >>> >>> >>> >>>> MAX2(SIGSEGV, SIGBUS) >>> >>> I really would like to leave the code as-is. This would be a functional >>> >>> change, while I only intend to fix issues in this change. Also, as David >>> >>> explained, it might break for some os implementations. >>> >>> >>> >>>> the only way to initialize it is with one of sigemptyset() or sigfillset(). >>> >>> I added initialization with sigemptyset(). Unfortunately, there is no >>> static >>> >>> initializer for this. >>> >>> >>> >>>> I would like to see those removed from os::Aix and put into os_aix.cpp at >> static filescope >>> >>> I moved these to static scope on the three oses. >>> >>> >>> >>> Here is the new webrev: >>> >>> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.02/ >>> >>> >>> >>> Best regards, >>> >>> Goetz. >>> >>> >>> >>> >>> >>> *From:*Thomas St?fe [mailto:thomas.stuefe at gmail.com] >>> *Sent:* Freitag, 13. November 2015 10:54 >>> *To:* Lindenmaier, Goetz >>> *Cc:* Dmitry Samersoff; David Holmes; >>> hotspot-runtime-dev at openjdk.java.net; serviceability-dev >>> *Subject:* Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM >>> >>> >>> >>> Hi Goetz, >>> >>> >>> >>> sorry for not looking at this earlier. This is a nice cleanup. Some remarks: >>> >>> >>> >>> http://cr.openjdk.java.net/~goetz/webrevs/8141529- >> NSIG/webrev.01/src/os/aix/vm/os_aix.cpp.udiff.html >>> >>> >>> >>> + if (sig > MAX2(SIGSEGV, SIGBUS) && // See 4355769. >>> >>> + sig < NSIG) { // Must be legal signal and fit >>> into sigflags[]. >>> >>> >>> >>> I do not like much the MAX2() construct. I would like it better to >>> explicitly check whether the SR signal is one of the "forbidden" ones >>> the VM uses. >>> >>> >>> >>> Maybe keep a mask defined centrally for each platform which contains >>> signals the VM needs for itself ? >>> >>> >>> >>> +sigset_t os::Aix::sigs = { 0 }; >>> >>> >>> >>> I would not initialize the signal set this way. sigset_t is an opaque >>> type; the only way to initialize it is with one of sigemptyset() or >>> sigfillset(). >>> >>> >>> >>> http://cr.openjdk.java.net/~goetz/webrevs/8141529- >> NSIG/webrev.01/src/os/aix/vm/os_aix.hpp.udiff.html >>> >>> >>> >>> + static struct sigaction sigact[NSIG]; // saved preinstalled sigactions >>> >>> + static sigset_t sigs; // mask of signals that have >>> >>> >>> >>> + static int sigflags[NSIG]; >>> >>> >>> >>> I know this is not in the scope of your change, but I would like to see >>> those removed from os::Aix and put into os_aix.cpp at static filescope. >>> There is no need at all to export those, and you would get rid of the >>> signal.h dependency you know have when including os_aix.hpp. >>> >>> >>> >>> >>> >>> http://cr.openjdk.java.net/~goetz/webrevs/8141529- >> NSIG/webrev.01/src/os/bsd/vm/jsig.c.udiff.html >>> >>> http://cr.openjdk.java.net/~goetz/webrevs/8141529- >> NSIG/webrev.01/src/os/bsd/vm/os_bsd.cpp.udiff.html >>> >>> >>> >>> On BSD, we have realtime signals. >>> >>> >>> >>> http://fxr.watson.org/fxr/source/sys/signal.h >>> >>> #define SIGRTMAX 126 >>> >>> and NSIG does not contain them: >>> >>> #define NSIG 32 >>> >>> >>> >>> The max. possible signal number would be 126, which unfortunately does >>> not even fit into a 64bit mask. >>> >>> >>> >>> So the code in jsig.c is broken for the case that someone wants to >>> register realtime signals, if the VM were to ever use realtime signals >>> itself, which now is not the case. >>> >>> >>> >>> The same is true for os_bsd.cpp, where signal chaining will not work if >>> the application did have handler for real time signals pre-installed >>> before jvm is loaded. >>> >>> >>> >>> Solaris: >>> >>> >>> >>> The only platform where NSIG is missing? >>> >>> >>> >>> Here, we calculate the max. signal number dynamically in os_solaris.cpp, >>> presumably because SIGRTMAX is not a constant and can be changed using >>> system configuration. But then, on Linux we have the same situation >>> (SIGRTMAX is dynamic) and there we do not go through the trouble of >>> calculating the max. signal number dynamically. Instead we just use >>> NSIG=64 and rely on the fact that NSIG is larger than the largest >>> possible dynamic value for SIGRTMAX. >>> >>> >>> >>> Solaris does not seem to have NSIG defined, but I am sure there is also >>> a max. possible value for SIGRTMAX (the default seems to be 48). So, one >>> could probably safely define NSIG for Solaris too, so that we have NSIG >>> defined on all Posix platforms. >>> >>> >>> >>> >>> >>> On Thu, Nov 12, 2015 at 8:24 PM, Lindenmaier, Goetz >>> > >> wrote: >>> >>> Hi David, Dmitry, >>> >>> I've come up with a new webrev: >>> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.01/ >>> >>> I hit on some more issues: >>> - As proposed, I replaced MAXSIGNUM by NSIG >>> - On AIX, NSIG=255. Therefore storing bits in a word does not work. >>> I'm now using bitset functionality from signal.h as it's done in >>> other places. >>> sigset_t is >> NSIG on linux, so it's no good idea to use it there. >>> >>> >>> >>> Why do we not do this on all platforms, provided sigset_t contains all >>> signals (incl. realtime signals) ? >>> >>> >>> >>> - In the os files I found another bit vector that now is too small: >>> sigs. >>> I adapted that, too. Removed the dead declaration of this on >>> solaris. >>> >>> Best regards, >>> Goetz. >>> >>> >>> >>> >>> >>> Kind Regards, Thomas >>> >>> >>> >>> >>> >>> > -----Original Message----- >>> > From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com >> ] >>> >>> > Sent: Donnerstag, 12. November 2015 10:05 >>> > To: Lindenmaier, Goetz; David Holmes; hotspot-runtime- >>> > dev at openjdk.java.net ; serviceability- >> dev >>> > Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM >>> > >>> > Goetz, >>> > >>> > *BSD including OS X also defines NSIG (just checked) and if my >>> memory is >>> > not bogus, AIX defines it too. >>> > >>> > So you may consider to use NSIG on all platform. >>> > >>> > -Dmitry >>> > >>> > On 2015-11-12 11:36, Lindenmaier, Goetz wrote: >>> > > OK I'll change it to NSIG. That's used in other places in >>> os_linux, too. >>> > > So it's really more consistent. >>> > > >>> > > Best regards, >>> > > Goetz >>> > > >>> > >> -----Original Message----- >>> > >> From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com >>> ] >>> > >> Sent: Donnerstag, 12. November 2015 09:22 >>> > >> To: David Holmes; Lindenmaier, Goetz; hotspot-runtime- >>> > >> dev at openjdk.java.net ; >>> serviceability-dev >>> > >> Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM >>> > >> >>> > >> David, >>> > >> >>> > >> I think it's better to use NSIG (without underscore) defined in >>> signal.h >>> > >> >>> > >> -Dmitry >>> > >> >>> > >> >>> > >> On 2015-11-12 10:35, David Holmes wrote: >>> > >>> Hi Goetz, >>> > >>> >>> > >>> Adding in serviceability-dev >>> > >>> >>> > >>> On 9/11/2015 6:22 PM, Lindenmaier, Goetz wrote: >>> > >>>> Hi, >>> > >>>> >>> > >>>> The environment variable _JAVA_SR_SIGNUM can be set to a >> signal >>> > >> number >>> > >>>> do be used by the JVM's suspend/resume mechanism. >>> > >>>> >>> > >>>> If set, a signal handler is installed and the current signal >>> handler >>> > >>>> is saved to an array. >>> > >>>> On linux, this array had size MAXSIGNUM=32, and >> _JAVA_SR_SIGNUM >>> > >> was >>> > >>>> allowed >>> > >>>> to range up to _NSIG=65. This could cause memory corruption. >>> > >>>> >>> > >>>> Further, in jsig.c, an unsinged int is used to set a bit for >>> signals. >>> > >>>> This also >>> > >>>> is too small, as only 32 signals can be supported. Further, the >>> > >>>> signals are mapped >>> > >>>> wrong to these bits. '0' is not a valid signal, but '32' >>> was. 1<<32 >>> > >>>> happens to map to >>> > >>>> zero, so the signal could be stored, but this probably was not >>> > >>>> intended that way. >>> > >>>> >>> > >>>> This change increases MAXSIGNUM to 65 on linux, and to 64 on >>> aix. It >>> > >>>> introduces >>> > >>>> proper checking of the signal read from the env var, and issues a >>> > >>>> warning if it >>> > >>>> does not use the signal set. It adapts the data types in jisig.c >>> > >>>> properly. >>> > >>>> >>> > >>>> Please review this change. I please need a sponsor. >>> > >>>> http://cr.openjdk.java.net/~goetz/webrevs/8141529- >> NSIG/webrev.00 >>> > >>> >>> > >>> This all sounds very good to me. (I must find out why Solaris >>> is not >>> > >>> involved here :) ). >>> > >>> >>> > >>> On Linux you didn't add the bounds check to >>> os::Linux::set_our_sigflags. >>> > >>> >>> > >>> I'm also wondering about documenting where we are determining >> the >>> > >>> maximum from? Is it simply _NSIG on some/all distributions? >>> And I see >>> > >>> _NSIG is supposed to be the biggest signal number + one. Also >>> linux >>> > >>> defines NSIG = _NSIG so which should we be using? >>> > >>> >>> > >>> Thanks, >>> > >>> David >>> > >>> >>> > >>>> Best regards, >>> > >>>> Goetz. >>> > >>>> >>> > >> >>> > >> >>> > >> -- >>> > >> Dmitry Samersoff >>> > >> Oracle Java development team, Saint Petersburg, Russia >>> > >> * I would love to change the world, but they won't give me the >>> sources. >>> > >>> > >>> > -- >>> > Dmitry Samersoff >>> > Oracle Java development team, Saint Petersburg, Russia >>> > * I would love to change the world, but they won't give me the >>> sources. >>> >>> >>> >> >> >> -- >> Dmitry Samersoff >> Oracle Java development team, Saint Petersburg, Russia >> * I would love to change the world, but they won't give me the sources. -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From goetz.lindenmaier at sap.com Tue Nov 17 08:35:21 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 17 Nov 2015 08:35:21 +0000 Subject: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM In-Reply-To: <564AE664.6000205@oracle.com> References: <4295855A5C1DE049A61835A1887419CC41EB5DD6@DEWDFEMB12A.global.corp.sap> <5644413D.1000004@oracle.com> <56444C29.8060606@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB8252@DEWDFEMB12A.global.corp.sap> <56445646.4040707@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB85C3@DEWDFEMB12A.global.corp.sap> <4295855A5C1DE049A61835A1887419CC41EB8DB5@DEWDFEMB12A.global.corp.sap> <5649B599.60807@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB93E1@DEWDFEMB12A.global.corp.sap> <564AE664.6000205@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC41EB9427@DEWDFEMB12A.global.corp.sap> That's great, thanks a lot! Best regards, Goetz. > -----Original Message----- > From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com] > Sent: Dienstag, 17. November 2015 09:34 > To: Lindenmaier, Goetz; Thomas St?fe > Cc: David Holmes; hotspot-runtime-dev at openjdk.java.net; serviceability- > dev > Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM > > Goetz, > > I'll sponsor the push > > -Dmitry. > > On 2015-11-17 10:52, Lindenmaier, Goetz wrote: > > David, Dmitry, > > > > I think this is reviewed now. Could one of you please sponsor this? > > The final webrev, including a full changeset patch, is this: > > http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.02/ > > > > It ran successfully through our nightly tests, tonight. > > > > Best regards, > > Goetz. > > > >> -----Original Message----- > >> From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com] > >> Sent: Montag, 16. November 2015 11:53 > >> To: Lindenmaier, Goetz; Thomas St?fe > >> Cc: David Holmes; hotspot-runtime-dev at openjdk.java.net; serviceability- > >> dev > >> Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM > >> > >> Goetz, > >> > >> Looks good for me. > >> > >> Thank you for a nice cleanup. > >> > >> -Dmitry > >> > >> > >> On 2015-11-16 13:25, Lindenmaier, Goetz wrote: > >>> Hi Thomas, > >>> > >>> > >>> > >>> thanks for looking at this. > >>> > >>> > >>> > >>>> MAX2(SIGSEGV, SIGBUS) > >>> > >>> I really would like to leave the code as-is. This would be a functional > >>> > >>> change, while I only intend to fix issues in this change. Also, as David > >>> > >>> explained, it might break for some os implementations. > >>> > >>> > >>> > >>>> the only way to initialize it is with one of sigemptyset() or sigfillset(). > >>> > >>> I added initialization with sigemptyset(). Unfortunately, there is no > >>> static > >>> > >>> initializer for this. > >>> > >>> > >>> > >>>> I would like to see those removed from os::Aix and put into os_aix.cpp > at > >> static filescope > >>> > >>> I moved these to static scope on the three oses. > >>> > >>> > >>> > >>> Here is the new webrev: > >>> > >>> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.02/ > >>> > >>> > >>> > >>> Best regards, > >>> > >>> Goetz. > >>> > >>> > >>> > >>> > >>> > >>> *From:*Thomas St?fe [mailto:thomas.stuefe at gmail.com] > >>> *Sent:* Freitag, 13. November 2015 10:54 > >>> *To:* Lindenmaier, Goetz > >>> *Cc:* Dmitry Samersoff; David Holmes; > >>> hotspot-runtime-dev at openjdk.java.net; serviceability-dev > >>> *Subject:* Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM > >>> > >>> > >>> > >>> Hi Goetz, > >>> > >>> > >>> > >>> sorry for not looking at this earlier. This is a nice cleanup. Some remarks: > >>> > >>> > >>> > >>> http://cr.openjdk.java.net/~goetz/webrevs/8141529- > >> NSIG/webrev.01/src/os/aix/vm/os_aix.cpp.udiff.html > >>> > >>> > >>> > >>> + if (sig > MAX2(SIGSEGV, SIGBUS) && // See 4355769. > >>> > >>> + sig < NSIG) { // Must be legal signal and fit > >>> into sigflags[]. > >>> > >>> > >>> > >>> I do not like much the MAX2() construct. I would like it better to > >>> explicitly check whether the SR signal is one of the "forbidden" ones > >>> the VM uses. > >>> > >>> > >>> > >>> Maybe keep a mask defined centrally for each platform which contains > >>> signals the VM needs for itself ? > >>> > >>> > >>> > >>> +sigset_t os::Aix::sigs = { 0 }; > >>> > >>> > >>> > >>> I would not initialize the signal set this way. sigset_t is an opaque > >>> type; the only way to initialize it is with one of sigemptyset() or > >>> sigfillset(). > >>> > >>> > >>> > >>> http://cr.openjdk.java.net/~goetz/webrevs/8141529- > >> NSIG/webrev.01/src/os/aix/vm/os_aix.hpp.udiff.html > >>> > >>> > >>> > >>> + static struct sigaction sigact[NSIG]; // saved preinstalled sigactions > >>> > >>> + static sigset_t sigs; // mask of signals that have > >>> > >>> > >>> > >>> + static int sigflags[NSIG]; > >>> > >>> > >>> > >>> I know this is not in the scope of your change, but I would like to see > >>> those removed from os::Aix and put into os_aix.cpp at static filescope. > >>> There is no need at all to export those, and you would get rid of the > >>> signal.h dependency you know have when including os_aix.hpp. > >>> > >>> > >>> > >>> > >>> > >>> http://cr.openjdk.java.net/~goetz/webrevs/8141529- > >> NSIG/webrev.01/src/os/bsd/vm/jsig.c.udiff.html > >>> > >>> http://cr.openjdk.java.net/~goetz/webrevs/8141529- > >> NSIG/webrev.01/src/os/bsd/vm/os_bsd.cpp.udiff.html > >>> > >>> > >>> > >>> On BSD, we have realtime signals. > >>> > >>> > >>> > >>> http://fxr.watson.org/fxr/source/sys/signal.h > >>> > >>> #define SIGRTMAX 126 > >>> > >>> and NSIG does not contain them: > >>> > >>> #define NSIG 32 > >>> > >>> > >>> > >>> The max. possible signal number would be 126, which unfortunately > does > >>> not even fit into a 64bit mask. > >>> > >>> > >>> > >>> So the code in jsig.c is broken for the case that someone wants to > >>> register realtime signals, if the VM were to ever use realtime signals > >>> itself, which now is not the case. > >>> > >>> > >>> > >>> The same is true for os_bsd.cpp, where signal chaining will not work if > >>> the application did have handler for real time signals pre-installed > >>> before jvm is loaded. > >>> > >>> > >>> > >>> Solaris: > >>> > >>> > >>> > >>> The only platform where NSIG is missing? > >>> > >>> > >>> > >>> Here, we calculate the max. signal number dynamically in os_solaris.cpp, > >>> presumably because SIGRTMAX is not a constant and can be changed > using > >>> system configuration. But then, on Linux we have the same situation > >>> (SIGRTMAX is dynamic) and there we do not go through the trouble of > >>> calculating the max. signal number dynamically. Instead we just use > >>> NSIG=64 and rely on the fact that NSIG is larger than the largest > >>> possible dynamic value for SIGRTMAX. > >>> > >>> > >>> > >>> Solaris does not seem to have NSIG defined, but I am sure there is also > >>> a max. possible value for SIGRTMAX (the default seems to be 48). So, > one > >>> could probably safely define NSIG for Solaris too, so that we have NSIG > >>> defined on all Posix platforms. > >>> > >>> > >>> > >>> > >>> > >>> On Thu, Nov 12, 2015 at 8:24 PM, Lindenmaier, Goetz > >>> > > >> wrote: > >>> > >>> Hi David, Dmitry, > >>> > >>> I've come up with a new webrev: > >>> http://cr.openjdk.java.net/~goetz/webrevs/8141529- > NSIG/webrev.01/ > >>> > >>> I hit on some more issues: > >>> - As proposed, I replaced MAXSIGNUM by NSIG > >>> - On AIX, NSIG=255. Therefore storing bits in a word does not work. > >>> I'm now using bitset functionality from signal.h as it's done in > >>> other places. > >>> sigset_t is >> NSIG on linux, so it's no good idea to use it there. > >>> > >>> > >>> > >>> Why do we not do this on all platforms, provided sigset_t contains all > >>> signals (incl. realtime signals) ? > >>> > >>> > >>> > >>> - In the os files I found another bit vector that now is too small: > >>> sigs. > >>> I adapted that, too. Removed the dead declaration of this on > >>> solaris. > >>> > >>> Best regards, > >>> Goetz. > >>> > >>> > >>> > >>> > >>> > >>> Kind Regards, Thomas > >>> > >>> > >>> > >>> > >>> > >>> > -----Original Message----- > >>> > From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com > >> ] > >>> > >>> > Sent: Donnerstag, 12. November 2015 10:05 > >>> > To: Lindenmaier, Goetz; David Holmes; hotspot-runtime- > >>> > dev at openjdk.java.net ; > serviceability- > >> dev > >>> > Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM > >>> > > >>> > Goetz, > >>> > > >>> > *BSD including OS X also defines NSIG (just checked) and if my > >>> memory is > >>> > not bogus, AIX defines it too. > >>> > > >>> > So you may consider to use NSIG on all platform. > >>> > > >>> > -Dmitry > >>> > > >>> > On 2015-11-12 11:36, Lindenmaier, Goetz wrote: > >>> > > OK I'll change it to NSIG. That's used in other places in > >>> os_linux, too. > >>> > > So it's really more consistent. > >>> > > > >>> > > Best regards, > >>> > > Goetz > >>> > > > >>> > >> -----Original Message----- > >>> > >> From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com > >>> ] > >>> > >> Sent: Donnerstag, 12. November 2015 09:22 > >>> > >> To: David Holmes; Lindenmaier, Goetz; hotspot-runtime- > >>> > >> dev at openjdk.java.net ; > >>> serviceability-dev > >>> > >> Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM > >>> > >> > >>> > >> David, > >>> > >> > >>> > >> I think it's better to use NSIG (without underscore) defined in > >>> signal.h > >>> > >> > >>> > >> -Dmitry > >>> > >> > >>> > >> > >>> > >> On 2015-11-12 10:35, David Holmes wrote: > >>> > >>> Hi Goetz, > >>> > >>> > >>> > >>> Adding in serviceability-dev > >>> > >>> > >>> > >>> On 9/11/2015 6:22 PM, Lindenmaier, Goetz wrote: > >>> > >>>> Hi, > >>> > >>>> > >>> > >>>> The environment variable _JAVA_SR_SIGNUM can be set to a > >> signal > >>> > >> number > >>> > >>>> do be used by the JVM's suspend/resume mechanism. > >>> > >>>> > >>> > >>>> If set, a signal handler is installed and the current signal > >>> handler > >>> > >>>> is saved to an array. > >>> > >>>> On linux, this array had size MAXSIGNUM=32, and > >> _JAVA_SR_SIGNUM > >>> > >> was > >>> > >>>> allowed > >>> > >>>> to range up to _NSIG=65. This could cause memory corruption. > >>> > >>>> > >>> > >>>> Further, in jsig.c, an unsinged int is used to set a bit for > >>> signals. > >>> > >>>> This also > >>> > >>>> is too small, as only 32 signals can be supported. Further, the > >>> > >>>> signals are mapped > >>> > >>>> wrong to these bits. '0' is not a valid signal, but '32' > >>> was. 1<<32 > >>> > >>>> happens to map to > >>> > >>>> zero, so the signal could be stored, but this probably was not > >>> > >>>> intended that way. > >>> > >>>> > >>> > >>>> This change increases MAXSIGNUM to 65 on linux, and to 64 on > >>> aix. It > >>> > >>>> introduces > >>> > >>>> proper checking of the signal read from the env var, and issues > a > >>> > >>>> warning if it > >>> > >>>> does not use the signal set. It adapts the data types in jisig.c > >>> > >>>> properly. > >>> > >>>> > >>> > >>>> Please review this change. I please need a sponsor. > >>> > >>>> http://cr.openjdk.java.net/~goetz/webrevs/8141529- > >> NSIG/webrev.00 > >>> > >>> > >>> > >>> This all sounds very good to me. (I must find out why Solaris > >>> is not > >>> > >>> involved here :) ). > >>> > >>> > >>> > >>> On Linux you didn't add the bounds check to > >>> os::Linux::set_our_sigflags. > >>> > >>> > >>> > >>> I'm also wondering about documenting where we are > determining > >> the > >>> > >>> maximum from? Is it simply _NSIG on some/all distributions? > >>> And I see > >>> > >>> _NSIG is supposed to be the biggest signal number + one. Also > >>> linux > >>> > >>> defines NSIG = _NSIG so which should we be using? > >>> > >>> > >>> > >>> Thanks, > >>> > >>> David > >>> > >>> > >>> > >>>> Best regards, > >>> > >>>> Goetz. > >>> > >>>> > >>> > >> > >>> > >> > >>> > >> -- > >>> > >> Dmitry Samersoff > >>> > >> Oracle Java development team, Saint Petersburg, Russia > >>> > >> * I would love to change the world, but they won't give me the > >>> sources. > >>> > > >>> > > >>> > -- > >>> > Dmitry Samersoff > >>> > Oracle Java development team, Saint Petersburg, Russia > >>> > * I would love to change the world, but they won't give me the > >>> sources. > >>> > >>> > >>> > >> > >> > >> -- > >> Dmitry Samersoff > >> Oracle Java development team, Saint Petersburg, Russia > >> * I would love to change the world, but they won't give me the sources. > > > -- > Dmitry Samersoff > Oracle Java development team, Saint Petersburg, Russia > * I would love to change the world, but they won't give me the sources. From dmitry.samersoff at oracle.com Tue Nov 17 13:41:54 2015 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Tue, 17 Nov 2015 16:41:54 +0300 Subject: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM In-Reply-To: <4295855A5C1DE049A61835A1887419CC41EB9427@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC41EB5DD6@DEWDFEMB12A.global.corp.sap> <5644413D.1000004@oracle.com> <56444C29.8060606@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB8252@DEWDFEMB12A.global.corp.sap> <56445646.4040707@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB85C3@DEWDFEMB12A.global.corp.sap> <4295855A5C1DE049A61835A1887419CC41EB8DB5@DEWDFEMB12A.global.corp.sap> <5649B599.60807@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB93E1@DEWDFEMB12A.global.corp.sap> <564AE664.6000205@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB9427@DEWDFEMB12A.global.corp.sap> Message-ID: <564B2EA2.2030302@oracle.com> Goetz, Push done. Please, take a look at the changeset: http://hg.openjdk.java.net/jdk9/hs-rt/hotspot/rev/149cc1f9f1aa -Dmitry On 2015-11-17 11:35, Lindenmaier, Goetz wrote: > That's great, thanks a lot! > > Best regards, > Goetz. > >> -----Original Message----- >> From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com] >> Sent: Dienstag, 17. November 2015 09:34 >> To: Lindenmaier, Goetz; Thomas St?fe >> Cc: David Holmes; hotspot-runtime-dev at openjdk.java.net; serviceability- >> dev >> Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM >> >> Goetz, >> >> I'll sponsor the push >> >> -Dmitry. >> >> On 2015-11-17 10:52, Lindenmaier, Goetz wrote: >>> David, Dmitry, >>> >>> I think this is reviewed now. Could one of you please sponsor this? >>> The final webrev, including a full changeset patch, is this: >>> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.02/ >>> >>> It ran successfully through our nightly tests, tonight. >>> >>> Best regards, >>> Goetz. >>> >>>> -----Original Message----- >>>> From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com] >>>> Sent: Montag, 16. November 2015 11:53 >>>> To: Lindenmaier, Goetz; Thomas St?fe >>>> Cc: David Holmes; hotspot-runtime-dev at openjdk.java.net; serviceability- >>>> dev >>>> Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM >>>> >>>> Goetz, >>>> >>>> Looks good for me. >>>> >>>> Thank you for a nice cleanup. >>>> >>>> -Dmitry >>>> >>>> >>>> On 2015-11-16 13:25, Lindenmaier, Goetz wrote: >>>>> Hi Thomas, >>>>> >>>>> >>>>> >>>>> thanks for looking at this. >>>>> >>>>> >>>>> >>>>>> MAX2(SIGSEGV, SIGBUS) >>>>> >>>>> I really would like to leave the code as-is. This would be a functional >>>>> >>>>> change, while I only intend to fix issues in this change. Also, as David >>>>> >>>>> explained, it might break for some os implementations. >>>>> >>>>> >>>>> >>>>>> the only way to initialize it is with one of sigemptyset() or sigfillset(). >>>>> >>>>> I added initialization with sigemptyset(). Unfortunately, there is no >>>>> static >>>>> >>>>> initializer for this. >>>>> >>>>> >>>>> >>>>>> I would like to see those removed from os::Aix and put into os_aix.cpp >> at >>>> static filescope >>>>> >>>>> I moved these to static scope on the three oses. >>>>> >>>>> >>>>> >>>>> Here is the new webrev: >>>>> >>>>> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.02/ >>>>> >>>>> >>>>> >>>>> Best regards, >>>>> >>>>> Goetz. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> *From:*Thomas St?fe [mailto:thomas.stuefe at gmail.com] >>>>> *Sent:* Freitag, 13. November 2015 10:54 >>>>> *To:* Lindenmaier, Goetz >>>>> *Cc:* Dmitry Samersoff; David Holmes; >>>>> hotspot-runtime-dev at openjdk.java.net; serviceability-dev >>>>> *Subject:* Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM >>>>> >>>>> >>>>> >>>>> Hi Goetz, >>>>> >>>>> >>>>> >>>>> sorry for not looking at this earlier. This is a nice cleanup. Some remarks: >>>>> >>>>> >>>>> >>>>> http://cr.openjdk.java.net/~goetz/webrevs/8141529- >>>> NSIG/webrev.01/src/os/aix/vm/os_aix.cpp.udiff.html >>>>> >>>>> >>>>> >>>>> + if (sig > MAX2(SIGSEGV, SIGBUS) && // See 4355769. >>>>> >>>>> + sig < NSIG) { // Must be legal signal and fit >>>>> into sigflags[]. >>>>> >>>>> >>>>> >>>>> I do not like much the MAX2() construct. I would like it better to >>>>> explicitly check whether the SR signal is one of the "forbidden" ones >>>>> the VM uses. >>>>> >>>>> >>>>> >>>>> Maybe keep a mask defined centrally for each platform which contains >>>>> signals the VM needs for itself ? >>>>> >>>>> >>>>> >>>>> +sigset_t os::Aix::sigs = { 0 }; >>>>> >>>>> >>>>> >>>>> I would not initialize the signal set this way. sigset_t is an opaque >>>>> type; the only way to initialize it is with one of sigemptyset() or >>>>> sigfillset(). >>>>> >>>>> >>>>> >>>>> http://cr.openjdk.java.net/~goetz/webrevs/8141529- >>>> NSIG/webrev.01/src/os/aix/vm/os_aix.hpp.udiff.html >>>>> >>>>> >>>>> >>>>> + static struct sigaction sigact[NSIG]; // saved preinstalled sigactions >>>>> >>>>> + static sigset_t sigs; // mask of signals that have >>>>> >>>>> >>>>> >>>>> + static int sigflags[NSIG]; >>>>> >>>>> >>>>> >>>>> I know this is not in the scope of your change, but I would like to see >>>>> those removed from os::Aix and put into os_aix.cpp at static filescope. >>>>> There is no need at all to export those, and you would get rid of the >>>>> signal.h dependency you know have when including os_aix.hpp. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> http://cr.openjdk.java.net/~goetz/webrevs/8141529- >>>> NSIG/webrev.01/src/os/bsd/vm/jsig.c.udiff.html >>>>> >>>>> http://cr.openjdk.java.net/~goetz/webrevs/8141529- >>>> NSIG/webrev.01/src/os/bsd/vm/os_bsd.cpp.udiff.html >>>>> >>>>> >>>>> >>>>> On BSD, we have realtime signals. >>>>> >>>>> >>>>> >>>>> http://fxr.watson.org/fxr/source/sys/signal.h >>>>> >>>>> #define SIGRTMAX 126 >>>>> >>>>> and NSIG does not contain them: >>>>> >>>>> #define NSIG 32 >>>>> >>>>> >>>>> >>>>> The max. possible signal number would be 126, which unfortunately >> does >>>>> not even fit into a 64bit mask. >>>>> >>>>> >>>>> >>>>> So the code in jsig.c is broken for the case that someone wants to >>>>> register realtime signals, if the VM were to ever use realtime signals >>>>> itself, which now is not the case. >>>>> >>>>> >>>>> >>>>> The same is true for os_bsd.cpp, where signal chaining will not work if >>>>> the application did have handler for real time signals pre-installed >>>>> before jvm is loaded. >>>>> >>>>> >>>>> >>>>> Solaris: >>>>> >>>>> >>>>> >>>>> The only platform where NSIG is missing? >>>>> >>>>> >>>>> >>>>> Here, we calculate the max. signal number dynamically in os_solaris.cpp, >>>>> presumably because SIGRTMAX is not a constant and can be changed >> using >>>>> system configuration. But then, on Linux we have the same situation >>>>> (SIGRTMAX is dynamic) and there we do not go through the trouble of >>>>> calculating the max. signal number dynamically. Instead we just use >>>>> NSIG=64 and rely on the fact that NSIG is larger than the largest >>>>> possible dynamic value for SIGRTMAX. >>>>> >>>>> >>>>> >>>>> Solaris does not seem to have NSIG defined, but I am sure there is also >>>>> a max. possible value for SIGRTMAX (the default seems to be 48). So, >> one >>>>> could probably safely define NSIG for Solaris too, so that we have NSIG >>>>> defined on all Posix platforms. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Thu, Nov 12, 2015 at 8:24 PM, Lindenmaier, Goetz >>>>> > >>>> wrote: >>>>> >>>>> Hi David, Dmitry, >>>>> >>>>> I've come up with a new webrev: >>>>> http://cr.openjdk.java.net/~goetz/webrevs/8141529- >> NSIG/webrev.01/ >>>>> >>>>> I hit on some more issues: >>>>> - As proposed, I replaced MAXSIGNUM by NSIG >>>>> - On AIX, NSIG=255. Therefore storing bits in a word does not work. >>>>> I'm now using bitset functionality from signal.h as it's done in >>>>> other places. >>>>> sigset_t is >> NSIG on linux, so it's no good idea to use it there. >>>>> >>>>> >>>>> >>>>> Why do we not do this on all platforms, provided sigset_t contains all >>>>> signals (incl. realtime signals) ? >>>>> >>>>> >>>>> >>>>> - In the os files I found another bit vector that now is too small: >>>>> sigs. >>>>> I adapted that, too. Removed the dead declaration of this on >>>>> solaris. >>>>> >>>>> Best regards, >>>>> Goetz. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> Kind Regards, Thomas >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> > -----Original Message----- >>>>> > From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com >>>> ] >>>>> >>>>> > Sent: Donnerstag, 12. November 2015 10:05 >>>>> > To: Lindenmaier, Goetz; David Holmes; hotspot-runtime- >>>>> > dev at openjdk.java.net ; >> serviceability- >>>> dev >>>>> > Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM >>>>> > >>>>> > Goetz, >>>>> > >>>>> > *BSD including OS X also defines NSIG (just checked) and if my >>>>> memory is >>>>> > not bogus, AIX defines it too. >>>>> > >>>>> > So you may consider to use NSIG on all platform. >>>>> > >>>>> > -Dmitry >>>>> > >>>>> > On 2015-11-12 11:36, Lindenmaier, Goetz wrote: >>>>> > > OK I'll change it to NSIG. That's used in other places in >>>>> os_linux, too. >>>>> > > So it's really more consistent. >>>>> > > >>>>> > > Best regards, >>>>> > > Goetz >>>>> > > >>>>> > >> -----Original Message----- >>>>> > >> From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com >>>>> ] >>>>> > >> Sent: Donnerstag, 12. November 2015 09:22 >>>>> > >> To: David Holmes; Lindenmaier, Goetz; hotspot-runtime- >>>>> > >> dev at openjdk.java.net ; >>>>> serviceability-dev >>>>> > >> Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM >>>>> > >> >>>>> > >> David, >>>>> > >> >>>>> > >> I think it's better to use NSIG (without underscore) defined in >>>>> signal.h >>>>> > >> >>>>> > >> -Dmitry >>>>> > >> >>>>> > >> >>>>> > >> On 2015-11-12 10:35, David Holmes wrote: >>>>> > >>> Hi Goetz, >>>>> > >>> >>>>> > >>> Adding in serviceability-dev >>>>> > >>> >>>>> > >>> On 9/11/2015 6:22 PM, Lindenmaier, Goetz wrote: >>>>> > >>>> Hi, >>>>> > >>>> >>>>> > >>>> The environment variable _JAVA_SR_SIGNUM can be set to a >>>> signal >>>>> > >> number >>>>> > >>>> do be used by the JVM's suspend/resume mechanism. >>>>> > >>>> >>>>> > >>>> If set, a signal handler is installed and the current signal >>>>> handler >>>>> > >>>> is saved to an array. >>>>> > >>>> On linux, this array had size MAXSIGNUM=32, and >>>> _JAVA_SR_SIGNUM >>>>> > >> was >>>>> > >>>> allowed >>>>> > >>>> to range up to _NSIG=65. This could cause memory corruption. >>>>> > >>>> >>>>> > >>>> Further, in jsig.c, an unsinged int is used to set a bit for >>>>> signals. >>>>> > >>>> This also >>>>> > >>>> is too small, as only 32 signals can be supported. Further, the >>>>> > >>>> signals are mapped >>>>> > >>>> wrong to these bits. '0' is not a valid signal, but '32' >>>>> was. 1<<32 >>>>> > >>>> happens to map to >>>>> > >>>> zero, so the signal could be stored, but this probably was not >>>>> > >>>> intended that way. >>>>> > >>>> >>>>> > >>>> This change increases MAXSIGNUM to 65 on linux, and to 64 on >>>>> aix. It >>>>> > >>>> introduces >>>>> > >>>> proper checking of the signal read from the env var, and issues >> a >>>>> > >>>> warning if it >>>>> > >>>> does not use the signal set. It adapts the data types in jisig.c >>>>> > >>>> properly. >>>>> > >>>> >>>>> > >>>> Please review this change. I please need a sponsor. >>>>> > >>>> http://cr.openjdk.java.net/~goetz/webrevs/8141529- >>>> NSIG/webrev.00 >>>>> > >>> >>>>> > >>> This all sounds very good to me. (I must find out why Solaris >>>>> is not >>>>> > >>> involved here :) ). >>>>> > >>> >>>>> > >>> On Linux you didn't add the bounds check to >>>>> os::Linux::set_our_sigflags. >>>>> > >>> >>>>> > >>> I'm also wondering about documenting where we are >> determining >>>> the >>>>> > >>> maximum from? Is it simply _NSIG on some/all distributions? >>>>> And I see >>>>> > >>> _NSIG is supposed to be the biggest signal number + one. Also >>>>> linux >>>>> > >>> defines NSIG = _NSIG so which should we be using? >>>>> > >>> >>>>> > >>> Thanks, >>>>> > >>> David >>>>> > >>> >>>>> > >>>> Best regards, >>>>> > >>>> Goetz. >>>>> > >>>> >>>>> > >> >>>>> > >> >>>>> > >> -- >>>>> > >> Dmitry Samersoff >>>>> > >> Oracle Java development team, Saint Petersburg, Russia >>>>> > >> * I would love to change the world, but they won't give me the >>>>> sources. >>>>> > >>>>> > >>>>> > -- >>>>> > Dmitry Samersoff >>>>> > Oracle Java development team, Saint Petersburg, Russia >>>>> > * I would love to change the world, but they won't give me the >>>>> sources. >>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> Dmitry Samersoff >>>> Oracle Java development team, Saint Petersburg, Russia >>>> * I would love to change the world, but they won't give me the sources. >> >> >> -- >> Dmitry Samersoff >> Oracle Java development team, Saint Petersburg, Russia >> * I would love to change the world, but they won't give me the sources. -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From goetz.lindenmaier at sap.com Tue Nov 17 15:30:56 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 17 Nov 2015 15:30:56 +0000 Subject: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM In-Reply-To: <564B2EA2.2030302@oracle.com> References: <4295855A5C1DE049A61835A1887419CC41EB5DD6@DEWDFEMB12A.global.corp.sap> <5644413D.1000004@oracle.com> <56444C29.8060606@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB8252@DEWDFEMB12A.global.corp.sap> <56445646.4040707@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB85C3@DEWDFEMB12A.global.corp.sap> <4295855A5C1DE049A61835A1887419CC41EB8DB5@DEWDFEMB12A.global.corp.sap> <5649B599.60807@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB93E1@DEWDFEMB12A.global.corp.sap> <564AE664.6000205@oracle.com> <4295855A5C1DE049A61835A1887419CC41EB9427@DEWDFEMB12A.global.corp.sap> <564B2EA2.2030302@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC41EB959C@DEWDFEMB12A.global.corp.sap> Great, looks good. Thanks a lot! Best regards, Goetz. > -----Original Message----- > From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com] > Sent: Dienstag, 17. November 2015 14:42 > To: Lindenmaier, Goetz; Thomas St?fe > Cc: David Holmes; hotspot-runtime-dev at openjdk.java.net; serviceability- > dev > Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM > > Goetz, > > Push done. > Please, take a look at the changeset: > > http://hg.openjdk.java.net/jdk9/hs-rt/hotspot/rev/149cc1f9f1aa > > -Dmitry > > On 2015-11-17 11:35, Lindenmaier, Goetz wrote: > > That's great, thanks a lot! > > > > Best regards, > > Goetz. > > > >> -----Original Message----- > >> From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com] > >> Sent: Dienstag, 17. November 2015 09:34 > >> To: Lindenmaier, Goetz; Thomas St?fe > >> Cc: David Holmes; hotspot-runtime-dev at openjdk.java.net; serviceability- > >> dev > >> Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM > >> > >> Goetz, > >> > >> I'll sponsor the push > >> > >> -Dmitry. > >> > >> On 2015-11-17 10:52, Lindenmaier, Goetz wrote: > >>> David, Dmitry, > >>> > >>> I think this is reviewed now. Could one of you please sponsor this? > >>> The final webrev, including a full changeset patch, is this: > >>> http://cr.openjdk.java.net/~goetz/webrevs/8141529-NSIG/webrev.02/ > >>> > >>> It ran successfully through our nightly tests, tonight. > >>> > >>> Best regards, > >>> Goetz. > >>> > >>>> -----Original Message----- > >>>> From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com] > >>>> Sent: Montag, 16. November 2015 11:53 > >>>> To: Lindenmaier, Goetz; Thomas St?fe > >>>> Cc: David Holmes; hotspot-runtime-dev at openjdk.java.net; > serviceability- > >>>> dev > >>>> Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM > >>>> > >>>> Goetz, > >>>> > >>>> Looks good for me. > >>>> > >>>> Thank you for a nice cleanup. > >>>> > >>>> -Dmitry > >>>> > >>>> > >>>> On 2015-11-16 13:25, Lindenmaier, Goetz wrote: > >>>>> Hi Thomas, > >>>>> > >>>>> > >>>>> > >>>>> thanks for looking at this. > >>>>> > >>>>> > >>>>> > >>>>>> MAX2(SIGSEGV, SIGBUS) > >>>>> > >>>>> I really would like to leave the code as-is. This would be a functional > >>>>> > >>>>> change, while I only intend to fix issues in this change. Also, as David > >>>>> > >>>>> explained, it might break for some os implementations. > >>>>> > >>>>> > >>>>> > >>>>>> the only way to initialize it is with one of sigemptyset() or sigfillset(). > >>>>> > >>>>> I added initialization with sigemptyset(). Unfortunately, there is no > >>>>> static > >>>>> > >>>>> initializer for this. > >>>>> > >>>>> > >>>>> > >>>>>> I would like to see those removed from os::Aix and put into > os_aix.cpp > >> at > >>>> static filescope > >>>>> > >>>>> I moved these to static scope on the three oses. > >>>>> > >>>>> > >>>>> > >>>>> Here is the new webrev: > >>>>> > >>>>> http://cr.openjdk.java.net/~goetz/webrevs/8141529- > NSIG/webrev.02/ > >>>>> > >>>>> > >>>>> > >>>>> Best regards, > >>>>> > >>>>> Goetz. > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> *From:*Thomas St?fe [mailto:thomas.stuefe at gmail.com] > >>>>> *Sent:* Freitag, 13. November 2015 10:54 > >>>>> *To:* Lindenmaier, Goetz > >>>>> *Cc:* Dmitry Samersoff; David Holmes; > >>>>> hotspot-runtime-dev at openjdk.java.net; serviceability-dev > >>>>> *Subject:* Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM > >>>>> > >>>>> > >>>>> > >>>>> Hi Goetz, > >>>>> > >>>>> > >>>>> > >>>>> sorry for not looking at this earlier. This is a nice cleanup. Some > remarks: > >>>>> > >>>>> > >>>>> > >>>>> http://cr.openjdk.java.net/~goetz/webrevs/8141529- > >>>> NSIG/webrev.01/src/os/aix/vm/os_aix.cpp.udiff.html > >>>>> > >>>>> > >>>>> > >>>>> + if (sig > MAX2(SIGSEGV, SIGBUS) && // See 4355769. > >>>>> > >>>>> + sig < NSIG) { // Must be legal signal and fit > >>>>> into sigflags[]. > >>>>> > >>>>> > >>>>> > >>>>> I do not like much the MAX2() construct. I would like it better to > >>>>> explicitly check whether the SR signal is one of the "forbidden" ones > >>>>> the VM uses. > >>>>> > >>>>> > >>>>> > >>>>> Maybe keep a mask defined centrally for each platform which > contains > >>>>> signals the VM needs for itself ? > >>>>> > >>>>> > >>>>> > >>>>> +sigset_t os::Aix::sigs = { 0 }; > >>>>> > >>>>> > >>>>> > >>>>> I would not initialize the signal set this way. sigset_t is an opaque > >>>>> type; the only way to initialize it is with one of sigemptyset() or > >>>>> sigfillset(). > >>>>> > >>>>> > >>>>> > >>>>> http://cr.openjdk.java.net/~goetz/webrevs/8141529- > >>>> NSIG/webrev.01/src/os/aix/vm/os_aix.hpp.udiff.html > >>>>> > >>>>> > >>>>> > >>>>> + static struct sigaction sigact[NSIG]; // saved preinstalled sigactions > >>>>> > >>>>> + static sigset_t sigs; // mask of signals that have > >>>>> > >>>>> > >>>>> > >>>>> + static int sigflags[NSIG]; > >>>>> > >>>>> > >>>>> > >>>>> I know this is not in the scope of your change, but I would like to see > >>>>> those removed from os::Aix and put into os_aix.cpp at static > filescope. > >>>>> There is no need at all to export those, and you would get rid of the > >>>>> signal.h dependency you know have when including os_aix.hpp. > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> http://cr.openjdk.java.net/~goetz/webrevs/8141529- > >>>> NSIG/webrev.01/src/os/bsd/vm/jsig.c.udiff.html > >>>>> > >>>>> http://cr.openjdk.java.net/~goetz/webrevs/8141529- > >>>> NSIG/webrev.01/src/os/bsd/vm/os_bsd.cpp.udiff.html > >>>>> > >>>>> > >>>>> > >>>>> On BSD, we have realtime signals. > >>>>> > >>>>> > >>>>> > >>>>> http://fxr.watson.org/fxr/source/sys/signal.h > >>>>> > >>>>> #define SIGRTMAX 126 > >>>>> > >>>>> and NSIG does not contain them: > >>>>> > >>>>> #define NSIG 32 > >>>>> > >>>>> > >>>>> > >>>>> The max. possible signal number would be 126, which unfortunately > >> does > >>>>> not even fit into a 64bit mask. > >>>>> > >>>>> > >>>>> > >>>>> So the code in jsig.c is broken for the case that someone wants to > >>>>> register realtime signals, if the VM were to ever use realtime signals > >>>>> itself, which now is not the case. > >>>>> > >>>>> > >>>>> > >>>>> The same is true for os_bsd.cpp, where signal chaining will not work if > >>>>> the application did have handler for real time signals pre-installed > >>>>> before jvm is loaded. > >>>>> > >>>>> > >>>>> > >>>>> Solaris: > >>>>> > >>>>> > >>>>> > >>>>> The only platform where NSIG is missing? > >>>>> > >>>>> > >>>>> > >>>>> Here, we calculate the max. signal number dynamically in > os_solaris.cpp, > >>>>> presumably because SIGRTMAX is not a constant and can be changed > >> using > >>>>> system configuration. But then, on Linux we have the same situation > >>>>> (SIGRTMAX is dynamic) and there we do not go through the trouble > of > >>>>> calculating the max. signal number dynamically. Instead we just use > >>>>> NSIG=64 and rely on the fact that NSIG is larger than the largest > >>>>> possible dynamic value for SIGRTMAX. > >>>>> > >>>>> > >>>>> > >>>>> Solaris does not seem to have NSIG defined, but I am sure there is > also > >>>>> a max. possible value for SIGRTMAX (the default seems to be 48). So, > >> one > >>>>> could probably safely define NSIG for Solaris too, so that we have > NSIG > >>>>> defined on all Posix platforms. > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> On Thu, Nov 12, 2015 at 8:24 PM, Lindenmaier, Goetz > >>>>> > > >>>> wrote: > >>>>> > >>>>> Hi David, Dmitry, > >>>>> > >>>>> I've come up with a new webrev: > >>>>> http://cr.openjdk.java.net/~goetz/webrevs/8141529- > >> NSIG/webrev.01/ > >>>>> > >>>>> I hit on some more issues: > >>>>> - As proposed, I replaced MAXSIGNUM by NSIG > >>>>> - On AIX, NSIG=255. Therefore storing bits in a word does not > work. > >>>>> I'm now using bitset functionality from signal.h as it's done in > >>>>> other places. > >>>>> sigset_t is >> NSIG on linux, so it's no good idea to use it there. > >>>>> > >>>>> > >>>>> > >>>>> Why do we not do this on all platforms, provided sigset_t contains all > >>>>> signals (incl. realtime signals) ? > >>>>> > >>>>> > >>>>> > >>>>> - In the os files I found another bit vector that now is too small: > >>>>> sigs. > >>>>> I adapted that, too. Removed the dead declaration of this on > >>>>> solaris. > >>>>> > >>>>> Best regards, > >>>>> Goetz. > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> Kind Regards, Thomas > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > -----Original Message----- > >>>>> > From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com > >>>> ] > >>>>> > >>>>> > Sent: Donnerstag, 12. November 2015 10:05 > >>>>> > To: Lindenmaier, Goetz; David Holmes; hotspot-runtime- > >>>>> > dev at openjdk.java.net ; > >> serviceability- > >>>> dev > >>>>> > Subject: Re: RFR(M): 8141529: Fix handling of _JAVA_SR_SIGNUM > >>>>> > > >>>>> > Goetz, > >>>>> > > >>>>> > *BSD including OS X also defines NSIG (just checked) and if my > >>>>> memory is > >>>>> > not bogus, AIX defines it too. > >>>>> > > >>>>> > So you may consider to use NSIG on all platform. > >>>>> > > >>>>> > -Dmitry > >>>>> > > >>>>> > On 2015-11-12 11:36, Lindenmaier, Goetz wrote: > >>>>> > > OK I'll change it to NSIG. That's used in other places in > >>>>> os_linux, too. > >>>>> > > So it's really more consistent. > >>>>> > > > >>>>> > > Best regards, > >>>>> > > Goetz > >>>>> > > > >>>>> > >> -----Original Message----- > >>>>> > >> From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com > >>>>> ] > >>>>> > >> Sent: Donnerstag, 12. November 2015 09:22 > >>>>> > >> To: David Holmes; Lindenmaier, Goetz; hotspot-runtime- > >>>>> > >> dev at openjdk.java.net ; > >>>>> serviceability-dev > >>>>> > >> Subject: Re: RFR(M): 8141529: Fix handling of > _JAVA_SR_SIGNUM > >>>>> > >> > >>>>> > >> David, > >>>>> > >> > >>>>> > >> I think it's better to use NSIG (without underscore) defined in > >>>>> signal.h > >>>>> > >> > >>>>> > >> -Dmitry > >>>>> > >> > >>>>> > >> > >>>>> > >> On 2015-11-12 10:35, David Holmes wrote: > >>>>> > >>> Hi Goetz, > >>>>> > >>> > >>>>> > >>> Adding in serviceability-dev > >>>>> > >>> > >>>>> > >>> On 9/11/2015 6:22 PM, Lindenmaier, Goetz wrote: > >>>>> > >>>> Hi, > >>>>> > >>>> > >>>>> > >>>> The environment variable _JAVA_SR_SIGNUM can be set to > a > >>>> signal > >>>>> > >> number > >>>>> > >>>> do be used by the JVM's suspend/resume mechanism. > >>>>> > >>>> > >>>>> > >>>> If set, a signal handler is installed and the current signal > >>>>> handler > >>>>> > >>>> is saved to an array. > >>>>> > >>>> On linux, this array had size MAXSIGNUM=32, and > >>>> _JAVA_SR_SIGNUM > >>>>> > >> was > >>>>> > >>>> allowed > >>>>> > >>>> to range up to _NSIG=65. This could cause memory > corruption. > >>>>> > >>>> > >>>>> > >>>> Further, in jsig.c, an unsinged int is used to set a bit for > >>>>> signals. > >>>>> > >>>> This also > >>>>> > >>>> is too small, as only 32 signals can be supported. Further, > the > >>>>> > >>>> signals are mapped > >>>>> > >>>> wrong to these bits. '0' is not a valid signal, but '32' > >>>>> was. 1<<32 > >>>>> > >>>> happens to map to > >>>>> > >>>> zero, so the signal could be stored, but this probably was not > >>>>> > >>>> intended that way. > >>>>> > >>>> > >>>>> > >>>> This change increases MAXSIGNUM to 65 on linux, and to 64 > on > >>>>> aix. It > >>>>> > >>>> introduces > >>>>> > >>>> proper checking of the signal read from the env var, and > issues > >> a > >>>>> > >>>> warning if it > >>>>> > >>>> does not use the signal set. It adapts the data types in jisig.c > >>>>> > >>>> properly. > >>>>> > >>>> > >>>>> > >>>> Please review this change. I please need a sponsor. > >>>>> > >>>> http://cr.openjdk.java.net/~goetz/webrevs/8141529- > >>>> NSIG/webrev.00 > >>>>> > >>> > >>>>> > >>> This all sounds very good to me. (I must find out why Solaris > >>>>> is not > >>>>> > >>> involved here :) ). > >>>>> > >>> > >>>>> > >>> On Linux you didn't add the bounds check to > >>>>> os::Linux::set_our_sigflags. > >>>>> > >>> > >>>>> > >>> I'm also wondering about documenting where we are > >> determining > >>>> the > >>>>> > >>> maximum from? Is it simply _NSIG on some/all distributions? > >>>>> And I see > >>>>> > >>> _NSIG is supposed to be the biggest signal number + one. > Also > >>>>> linux > >>>>> > >>> defines NSIG = _NSIG so which should we be using? > >>>>> > >>> > >>>>> > >>> Thanks, > >>>>> > >>> David > >>>>> > >>> > >>>>> > >>>> Best regards, > >>>>> > >>>> Goetz. > >>>>> > >>>> > >>>>> > >> > >>>>> > >> > >>>>> > >> -- > >>>>> > >> Dmitry Samersoff > >>>>> > >> Oracle Java development team, Saint Petersburg, Russia > >>>>> > >> * I would love to change the world, but they won't give me the > >>>>> sources. > >>>>> > > >>>>> > > >>>>> > -- > >>>>> > Dmitry Samersoff > >>>>> > Oracle Java development team, Saint Petersburg, Russia > >>>>> > * I would love to change the world, but they won't give me the > >>>>> sources. > >>>>> > >>>>> > >>>>> > >>>> > >>>> > >>>> -- > >>>> Dmitry Samersoff > >>>> Oracle Java development team, Saint Petersburg, Russia > >>>> * I would love to change the world, but they won't give me the > sources. > >> > >> > >> -- > >> Dmitry Samersoff > >> Oracle Java development team, Saint Petersburg, Russia > >> * I would love to change the world, but they won't give me the sources. > > > -- > Dmitry Samersoff > Oracle Java development team, Saint Petersburg, Russia > * I would love to change the world, but they won't give me the sources. From rachel.protacio at oracle.com Tue Nov 17 19:50:47 2015 From: rachel.protacio at oracle.com (Rachel Protacio) Date: Tue, 17 Nov 2015 14:50:47 -0500 Subject: RFR: 8143157: Convert TraceVMOperation to Unified Logging Message-ID: <564B8517.7080105@oracle.com> Hi, Please review the following small logging enhancement. Summary: The former -XX:+TraceVMOperation flag is updated to the unified logging framework and is now replaced with -Xlog:vmoperation in product mode. Open webrev: http://cr.openjdk.java.net/~rprotacio/8143157/ Bug: https://bugs.openjdk.java.net/browse/JDK-8143157 Testing: Passes jtreg, JPRT, RBT quick tests, and refworkload performance tests. A compatability request has been accepted with regard to this change. Thank you very much, Rachel From david.holmes at oracle.com Wed Nov 18 09:22:46 2015 From: david.holmes at oracle.com (David Holmes) Date: Wed, 18 Nov 2015 19:22:46 +1000 Subject: RFR: 8143157: Convert TraceVMOperation to Unified Logging In-Reply-To: <564B8517.7080105@oracle.com> References: <564B8517.7080105@oracle.com> Message-ID: <564C4366.4070803@oracle.com> Hi Rachel, On 18/11/2015 5:50 AM, Rachel Protacio wrote: > Hi, > > Please review the following small logging enhancement. > > Summary: The former -XX:+TraceVMOperation flag is updated to the unified > logging framework and is now replaced with -Xlog:vmoperation in product > mode. > > Open webrev: http://cr.openjdk.java.net/~rprotacio/8143157/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8143157 > Testing: Passes jtreg, JPRT, RBT quick tests, and refworkload > performance tests. Meta-question: the logging framework is safe to be called when at a safepoint isn't it? --- src/share/vm/runtime/vm_operations.cpp void VM_Operation::evaluate() { ResourceMark rm; ! outputStream* debugstream = LogHandle(vmoperation)::debug_stream(); ! if (log_is_enabled(Debug, vmoperation)) { ! debugstream->print("["); ! print_on_error(debugstream); ! debugstream->cr(); } doit(); ! if (log_is_enabled(Debug, vmoperation)) { ! print_on_error(debugstream); ! debugstream->print_cr("]"); } } Why are you calling print_on_error twice? Why is the only logging level for this tag the "debug" level? I think I may be missing part of the way UL works here - can you enable logging both by explicit tag (ie vmoperation) and the level (ie debug)? And I know I sound like a broken record but I'm concerned about the overhead of the logging actions when it is not enabled ie: outputStream* debugstream = LogHandle(vmoperation)::debug_stream(); always gets executed - and we evaluate is_enabled twice. --- test/runtime/logging/VMOperationTestMain.java Can you add a comment explaining what the logic is attempting to do please. I'm curious how many times the loop executes before you get a safepoint-based GC (and how it varies for different GC's). Style nit: while( -> while ( > A compatability request has been accepted with regard to this change. I'll record my objections again to the conversion of develop flags to product. Thanks, David > Thank you very much, > Rachel From markus.gronlund at oracle.com Wed Nov 18 13:30:57 2015 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Wed, 18 Nov 2015 05:30:57 -0800 (PST) Subject: 8143226 (M): Minor updates to Event Based tracing Message-ID: Greetings, Kindly asking for reviews for the following bug: https://bugs.openjdk.java.net/browse/JDK-8143226 Webrev: http://cr.openjdk.java.net/~mgronlun/8143226/webrev01/ Summary: - Remove obsoleted files from make system - Consolidate to a single Thread abstraction - Expand relational keys to u8's to introduce uniqueness property - Tracing hook to be notified on out of memory - Tracing hook in early vm startup - Event "starttime" to become the primary time stamp for the event (previously it was "endtime") Thanks in advance Markus -------------- next part -------------- An HTML attachment was scrubbed... URL: From markus.gronlund at oracle.com Wed Nov 18 14:51:31 2015 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Wed, 18 Nov 2015 06:51:31 -0800 (PST) Subject: FW: RFR(L): 8140485: Class load and creation clean up Message-ID: <0c675874-1559-4202-a52a-787288e59773@default> Ping!! Could I please ask (kindly) for reviews on this? Thanks in advance Markus -----Original Message----- From: Markus Gronlund Sent: den 12 november 2015 13:08 To: hotspot-runtime-dev at openjdk.java.net; serviceability-dev at openjdk.net Subject: RE: RFR(L): 8140485: Class load and creation clean up Hi again, I have reworked and simplified this clean up/refactoring suggestion: Updated high-level description: http://cr.openjdk.java.net/~mgronlun/8140485/unified/unified_review/8140485_updated.jpg Updated webrev: http://cr.openjdk.java.net/~mgronlun/8140485/unified/unified_review/webrev02/ Updated patch: http://cr.openjdk.java.net/~mgronlun/8140485/unified/unified_review/8140485_open_unified_updated.patch Thanks in advance Markus -----Original Message----- From: Markus Gronlund Sent: den 27 oktober 2015 13:21 To: hotspot-runtime-dev at openjdk.java.net; serviceability-dev at openjdk.net Subject: RFR(L): 8140485: Class load and creation clean up Greetings, I have spent some time refactoring and cleaning up parts of the class load and creation sequence and code. Bug: https://bugs.openjdk.java.net/browse/JDK-8140485 Overview: There are quite a lot of changes here - hopefully it will not be a deterrent of taking this on- first of all, a lot of changes are mechanical and applicable to "form" more than content- for example, variable, parameter and member function "constness". Same applies to other "domino changes", for example when moving an included header from a .hpp file (maybe replacing with a fwd decl instead). I have tried to stage the changes for easier overview / review (including a high level diagram). The patches/webrevs are split into a patch series of 7 patches (there is also a unified patch). Use case: My initial use case for starting all of this clean up / refactoring work is a need to reuse mechanics of the class parsing and creation sequence. Cleanup / refactoring objectives: - Allow for class parsing implementation reuse - Decouple class parsing and Klass creation - Decouple JVMTI class file load hook from ClassFileParser - Add compiler assisted read-only intent in order to assist future refactoring attempts - "componentify" ClassFileParser / apply to Klass/IK/AK creations - Take advantage of some optimizations opportunities (constructors and underlying "calloc" memory) - Maintain class load performance High level diagram explaining the updated suggested sequence (high level): http://cr.openjdk.java.net/~mgronlun/8140485/8140485_Class_load_and_creation_cleanup_high_level_sequence.jpg Split webrevs/patches for easier overview / review (do note that not all of these splits will compile individually - there is a unified patch link that includes everything, pls see below): Sequence: "first" (order of changes) Name: "8140485_1_unified entry point" Link: http://cr.openjdk.java.net/~mgronlun/8140485/split/first/webrev01/ Summary: Split out JVMTI agent CFLH from CFP - moved to SD::create_vm_representation_prologue(). Channel all class loading via a unified entry point - SD::create_vm_representation() Split out ClassPathEntry into own header. Creation of new class ClassPathFileStream (specialized ClassPathStream) to retain ClassLoaderExt::Context functionality in the refactored code. "Need verify" property is carried within the CFS itself. Comment: SystemDictionary::load_instance_class() will now have a call to "load_class_from_classpath" - the relevance of this name might need to be modified (modules etc?) Sequence: "second" Name: "8140485_2_read_only_class_file_stream" Link: http://cr.openjdk.java.net/~mgronlun/8140485/split/second/webrev01/ Summary: The underlying bytestream (buffer) in a CFS representation should not be altered (advertently or inadvertently) once created, made read-only. Comment: Modifications should work with copies. Sequence: "third" Name: "8140485_3_componentify_class_file_parser" Link: http://cr.openjdk.java.net/~mgronlun/8140485/split/third/webrev01/ Summary: In order to allow for code refactoring of CFP - stack allocated variables are modified into fields. The entire CFP acts as a generic RAII class for the allocated structures (note CHECK macros). I have not fulfilled refactoring of some of the longest methods since it's is hard to really understand the impact of the inlining I guess we are after here.. The performance work done so far indicated improvements in still passing parameters instead of reading the fields directly (so you might still see "fat" methods taking a lot of parameters even though the fields are set). "Consts everywhere.." One of the real hard issues trying to refactor this code is the lack of expressed read-only intent - it's hard to determine (both on reading but more importantly when compiling) what can/should change. Yes, I have entered the "const" rabbit hole related to this code (in some cases it might even be termed gratuitous (event 'consting' function parameter pointers in some cases for example)) - hopefully this will help in upcoming attempts to refactor/simplify this code. Accessors - CFP becomes a data container to be used in constructors (see change seven). In addition, you will now have a possibility to fetch data from a parser without creating a fullblown Klass Comment: I think the code could be improved and more refactoring could be accomplished by introducing a Metaspace allocation RAII class - this would allow the functions to decouple even more that is currently (might be next steps) Sequence: "fourth" Name: "8140485_4_hide_impl_details_class_file_parser" Link: http://cr.openjdk.java.net/~mgronlun/8140485/split/fourth/webrev01/ Summary: Reduce the visibility of CFP impl methods by internal linkage instead of external linkage. Comment: In my opinion, we should attempt to break the pattern of having private functions declared in header files when the private function does not need to reach inside "this". (that is, it can use an external interface alt. might not need access at all (helpers)) Sequence: "fifth" Name: "8140485_5_update_entry_point" Link: http://cr.openjdk.java.net/~mgronlun/8140485/split/fifth/webrev01/ Summary: Update signatures. Remove the parameter passing where not needed ("parsed_name" reference for example) Sequence: "sixth" Name: "8140485_6_types_const_includes_repercussions" Link: http://cr.openjdk.java.net/~mgronlun/8140485/split/sixth/webrev01/ Summary: Rippling effects of stronger constness. Forward includes where can be applied. This revealed other code modules not having includes -added. Downcasting of const qualified types (applied Scott Meyers idiom of dual const_cast additions (see Effective C++)) Sequence: "seventh" Name: "8140485_7_applied_use_in_constructors_and_some_optimizations" Link: http://cr.openjdk.java.net/~mgronlun/8140485/split/seventh/webrev01/ Summary: Take advantage of above modifications to gain direct upshots in other areas. Use CFP as the data container when constructing Klass/IK/AK. Optimizations: Reduce/optimize Klass/InstanceKlass/ArrayKlass/ConstantPool constructors by using underlying invariant on "calloc" equivalent MetaspaceObj allocations. Unified change/patch: Webrev: http://cr.openjdk.java.net/~mgronlun/8140485/unified/webrev01/ Patch: http://cr.openjdk.java.net/~mgronlun/8140485/unified/8140485_unified.patch Summary: Unified (folded) patch for easier application/imports, for compilation/try out Testing: I have done targeted testing during development especially using the "runtime/ParallelClassLoading" test suite in order to ensure functional equivalency. JPRT testing. Performance tests (see below) Performance: Unstructured: I have used Java Flight Recorder (JFR), which envelops the class loading sequence with high resolution timers - I have used this information to ensure performance has been stable regarding the changes. Structured: Startup/footprint-benchmarks (Linux32, Linux64, Windows32, Windows64, Solaris SPARC-T) It is hard to read anything decisive around these (the effects most likely hides in noise), but at least determine that the changes do not introduce anything statistically significant (+/-) (both time and space) Thanks in advance Markus From sgehwolf at redhat.com Wed Nov 18 17:06:26 2015 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Wed, 18 Nov 2015 18:06:26 +0100 Subject: jmx-dev RFR 6425769: jmx remote bind address In-Reply-To: <1447061525.3551.3.camel@redhat.com> References: <1446460682.10865.20.camel@redhat.com> <56377F17.4050006@oracle.com> <5637871B.30705@oracle.com> <1446487615.10865.57.camel@redhat.com> <5638C89F.2090406@oracle.com> <1446634476.3554.8.camel@redhat.com> <1447061525.3551.3.camel@redhat.com> Message-ID: <1447866386.3660.53.camel@redhat.com> Hi, Re-posting this for review since I've done another version which duplicates some of the code in?SslRMIServerSocketFactory.java but does not change API other than adding the new property. I personally don't like it, but maybe this version is preferred? Bug:?https://bugs.openjdk.java.net/browse/JDK-6425769 webrev:?http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-6425769/03.no-rmi-ssl-factory-changes/ Thanks, Severin On Mon, 2015-11-09 at 10:32 +0100, Severin Gehwolf wrote: > On Wed, 2015-11-04 at 11:54 +0100, Severin Gehwolf wrote: > > Hi, > > > > Updated webrev with jtreg test in Java: > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-6425769/02/ > > bug:?https://bugs.openjdk.java.net/browse/JDK-6425769 > > > > I believe this updated webrev addresses all concerns and > > incorporates > > suggestions brought up by Jaroslav and Daniel. > > > > I'm still looking for a sponsor and a hotspot/servicability-dev > > reviewer. Could somebody maintaining javax.rmi.ssl have a look at > > this > > as well? Thank you! > > Ping? Friendly reminder that I still need reviewers and a sponsor for > this. > > Thanks, > Severin > > > Cheers, > > Severin > > > > On Tue, 2015-11-03 at 15:45 +0100, Jaroslav Bachorik wrote: > > > On 2.11.2015 19:06, Severin Gehwolf wrote: > > > > Hi, > > > > > > > > Thanks Jaroslav and Daniel for the reviews! Comments inline. > > > > > > > > On Mon, 2015-11-02 at 16:54 +0100, Jaroslav Bachorik wrote: > > > > > Hi, > > > > > > > > > > On 2.11.2015 16:19, Daniel Fuchs wrote: > > > > > > Hi Severin, > > > > > > > > > > > > Adding serviceability-dev at openjdk.java.net into the loop - > > > > > > that's > > > > > > a better alias than hotspot-dev for this kind of changes - > > > > > > maybe > > > > > > someone from serviceability-dev will offer to sponsor :-) > > > > > > > > > > > > I will let serviceability team members comment on the > > > > > > hotspot > > > > > > changes. > > > > > > > > > > > > ConnectorBootstrap.java > > > > > > > > > > > > I have one suggestion and one concern: > > > > > > > > > > > > Suggestion: I would suggest to reuse 'csf' (Client Socket > > > > > > Factory) > > > > > > and > > > > > > ssf??(Server Socket Factory) variables rather than > > > > > > introduce > > > > > > the > > > > > > two > > > > > > new variables rmiServerSocketFactory and > > > > > > rmiClientSocketFactory. > > > > > > You might want to create a new boolean 'useSocketFactory' > > > > > > variable, > > > > > > if that makes the code more readable. > > > > > > > > > > > > Concern: If I understand correctly how RMI socket factories > > > > > > work, > > > > > > the client socket factory will be serialized and sent to > > > > > > the > > > > > > client > > > > > > side. This is problematic for interoperability, as the > > > > > > class > > > > > > may > > > > > > not > > > > > > present in the remote client - if the remote client is e.g. > > > > > > jdk > > > > > > 8. > > > > > > > > > > > > As far as I can see, your new DefaultClientSocketFactory > > > > > > doesn't do > > > > > > anything useful - so I would suggest to simply get rid of > > > > > > it, > > > > > > and > > > > > > only > > > > > > set the Server Socket Factory when SSL is not involved. > > > > > > > > Thanks. Fixed in updated webrev. > > > > > > > > > > Tests: > > > > > > > > > > > > Concerning the tests - we're trying to get rid of shell > > > > > > scripts > > > > > > rather than introducing new ones :-) > > > > > > Could the test be rewritten in pure java using the Process > > > > > > API? > > > > > > > > > > > > I believe there's even a test library that will let you do > > > > > > that > > > > > > easily jdk/test/lib/testlibrary/jdk/testlibrary/ > > > > > > (see ProcessTools.java) > > > > > > > > It'll take me a bit to rewrite the test in pure Java, but > > > > should > > > > be > > > > fine. This is not yet fixed in the updated webrev. > > > > > > > > > > Other: > > > > > > > > > > > > Also - I believe the new option should be documented in: > > > > > > src/java.management/share/conf/management.properties > > > > > > > > Docs have been updated > > > > in src/java.management/share/conf/management.properties. > > > > > > > > > I share Daniel's concerns. Also, the part of the changeset is > > > > > related > > > > > to javax.rmi.ssl - someone maintaining this library should > > > > > also > > > > > comment here. > > > > > > > > > > Also, the change is introducing new API (system property) and > > > > > changing the existing one (adding SslRmiServerSocketFactory > > > > > public > > > > > constructors) so compatibility review process will have to be > > > > > involved. > > > > > > > > OK. What exactly is there for me to do? I'm not familiar with > > > > this > > > > process. Note that the intent would be to get this backported > > > > to > > > > JDK 8. > > > Not much for you. But for the potential Oracle sponsor this means > > > she > > > will have to remember to go through some extra hoops before > > > integrating your patch. > > > > I see. Thanks for clarifying it. > > > > > -JB- > > > > > > > > > > > New webrev at: > > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-6425769/01/ > > > > > > > > Thanks, > > > > Severin > > > > > > > > > -JB- > > > > > > > > > > > > best regards, > > > > > > > > > > > > -- daniel > > > > > > > > > > > > On 02/11/15 11:38, Severin Gehwolf wrote: > > > > > > > Hi, > > > > > > > > > > > > > > Here is a patch addressing JDK-6425769. The issue is that > > > > > > > the > > > > > > > JMX > > > > > > > agent > > > > > > > binds to the wildcard address by default, preventing > > > > > > > users > > > > > > > to > > > > > > > use > > > > > > > system properties for JMX agents on multi-homed hosts. > > > > > > > Given > > > > > > > a > > > > > > > host > > > > > > > with local network interfaces, 192.168.0.1 and > > > > > > > 192.168.0.2 > > > > > > > say, > > > > > > > it's > > > > > > > impossible to start two JMX agents binding to fixed ports > > > > > > > but > > > > > > > to > > > > > > > different network interfaces, 192.168.0.1:{9111,9112} and > > > > > > > 192.168.0.2:{9111,9112} say. > > > > > > > > > > > > > > The JDK would bind to all local interfaces by default. In > > > > > > > the > > > > > > > above > > > > > > > example to 192.168.0.1 *and* 192.168.0.2. The effect is > > > > > > > that > > > > > > > the > > > > > > > second > > > > > > > Java process would get a "Connection refused" error > > > > > > > because > > > > > > > another > > > > > > > process has already been bound to the specified JMX/RMI > > > > > > > port > > > > > > > pairs. > > > > > > > > > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-6425769 > > > > > > > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK- > > > > > > > 64 > > > > > > > 25 > > > > > > > 769/ > > > > > > > 00/ > > > > > > > > > > > > > > Testing done: > > > > > > > jdk_jmx and jdk_management tests all pass after this > > > > > > > change > > > > > > > (no > > > > > > > regressions). There is also a new JTREG test which fails > > > > > > > prior > > > > > > > this > > > > > > > change and passes after. Updates to the diagnostic > > > > > > > command > > > > > > > have > > > > > > > been > > > > > > > tested manually. > > > > > > > > > > > > > > I understand that, if approved, the JDK and hotspot > > > > > > > changes > > > > > > > should get > > > > > > > pushed together? Hotspot changes are fairly trivial since > > > > > > > it's > > > > > > > only a > > > > > > > doc-update for the new JDK property in the relevant > > > > > > > diagnostic > > > > > > > command. > > > > > > > > > > > > > > Could someone please review and sponsor this change? > > > > > > > Please > > > > > > > let > > > > > > > me know > > > > > > > if there are questions. > > > > > > > > > > > > > > Thanks, > > > > > > > Severin > > > > > > > > > > > > > > > > > > > > > > > > > > > > From markus.gronlund at oracle.com Wed Nov 18 17:45:23 2015 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Wed, 18 Nov 2015 09:45:23 -0800 (PST) Subject: RFR(XS): 8143228: Update module exports for Java Flight Recorder Message-ID: Greetings, Looking for review of the following to update module exports for Java Flight Recorder: Bug: https://bugs.openjdk.java.net/browse/JDK-8143228 Change: Patch is attached and also unrolled here: # HG changeset patch # User mgronlun # Date 1447764614 -3600 # Tue Nov 17 13:50:14 2015 +0100 # Node ID e452cc5ea183f4db155514a5ae6201e83ef70c95 # Parent 93f2a16f683dd52cbe08ab3aa277535db13473ee [mq]: jdk_top_level diff --git a/common/bin/compare_exceptions.sh.incl b/common/bin/compare_exceptions.sh.incl --- a/common/bin/compare_exceptions.sh.incl +++ b/common/bin/compare_exceptions.sh.incl @@ -231,7 +231,6 @@ ./lib/amd64/libjava.so ./lib/amd64/libjawt.so ./lib/amd64/libjdwp.so -./lib/amd64/libjfr.so ./lib/amd64/libjpeg.so ./lib/amd64/libjsdt.so ./lib/amd64/libjsound.so @@ -364,7 +363,6 @@ ./lib/sparcv9/libjava.so ./lib/sparcv9/libjawt.so ./lib/sparcv9/libjdwp.so -./lib/sparcv9/libjfr.so ./lib/sparcv9/libjpeg.so ./lib/sparcv9/libjsdt.so ./lib/sparcv9/libjsound.so diff --git a/modules.xml b/modules.xml --- a/modules.xml +++ b/modules.xml @@ -236,6 +236,22 @@ jdk.scripting.nashorn + jdk.internal.org.xml.sax + jdk.jfr + + + jdk.internal.org.xml.sax.helpers + jdk.jfr + + + jdk.internal.util.xml + jdk.jfr + + + jdk.internal.util.xml.impl + jdk.jfr + + jdk.internal.org.objectweb.asm java.instrument jdk.jfr @@ -287,6 +303,7 @@ jdk.httpserver jdk.jartool jdk.jconsole + jdk.jfr jdk.jvmstat jdk.management.resource jdk.pack200 @@ -869,6 +886,7 @@ sun.management.spi jdk.management jdk.management.cmm + jdk.management.jfr Thanks in advance Markus -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: jdk_top_level_open_modules_exports.patch Type: application/octet-stream Size: 1889 bytes Desc: not available URL: From rachel.protacio at oracle.com Wed Nov 18 18:41:01 2015 From: rachel.protacio at oracle.com (Rachel Protacio) Date: Wed, 18 Nov 2015 13:41:01 -0500 Subject: RFR: 8143157: Convert TraceVMOperation to Unified Logging In-Reply-To: <564CB632.9060201@oracle.com> References: <564B8517.7080105@oracle.com> <564C4366.4070803@oracle.com> <564CB632.9060201@oracle.com> Message-ID: <564CC63D.2000509@oracle.com> Hi, On 11/18/2015 12:32 PM, Max Ockner wrote: > I think the issue is that doit() happens regardless of what is being > logged. I don't think we can reorder the logging statements relative > to doit(). In that case, we would be forced to split into 2 > conditionals for the logging. > > Still I don't think it is necessary to call log_is_enabled twice, and > I don't think it is necessary to always define the outputStream. If > this turns out to be a problem, we could flip things around a bit: > > void VM_Operation::evaluate() { > ResourceMark rm; > ! bool enabled = false; > ! if (log_is_enabled(Debug, vmoperation)) { > ! outputStream* debugstream = LogHandle(vmoperation)::debug_stream(); > ! enabled = true; > ! if (enabled) { > ! print_on_error(debugstream); > ! debugstream->cr(); > } > doit(); > ! if (enabled) { > ! print_on_error(debugstream); > ! debugstream->print_cr("]"); > } > } > I'm fine with moving the definition into a conditional (but I can't move the declaration for scope reasons). It will look like this outputStream* debugstream; if (log_is_enabled(Debug, vmoperation)) { debugstream = LogHandle(vmoperation)::debug_stream(); To be honest, I'm not sure whether evaluating log_is_enabled twice is worse than adding a boolean check and cluttering the code and making it less readable. The argument for changing log_is_enabled() to a boolean could be extrapolated to say that it should be a boolean for every check for every tag. And that might be a good idea. But I feel like the problem with it here is that it comes across as confusing to add "enabled", which only actually needs to be used once. If we want it changed, maybe we should change the macro? Or put it outside the function, even, to make it only evaluate once? > There really is nothing quite like typing code into an editor with > misaligned columns. > Anyway, the change looks pretty good to me. > -Max > > On 11/18/2015 4:22 AM, David Holmes wrote: >> Hi Rachel, >> >> On 18/11/2015 5:50 AM, Rachel Protacio wrote: >>> Hi, >>> >>> Please review the following small logging enhancement. >>> >>> Summary: The former -XX:+TraceVMOperation flag is updated to the >>> unified >>> logging framework and is now replaced with -Xlog:vmoperation in product >>> mode. >>> >>> Open webrev: http://cr.openjdk.java.net/~rprotacio/8143157/ >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8143157 >>> Testing: Passes jtreg, JPRT, RBT quick tests, and refworkload >>> performance tests. >> >> Meta-question: the logging framework is safe to be called when at a >> safepoint isn't it? I haven't seen any problems, but I also don't know what would cause it to be unsafe. I'd appreciate any guidance in figuring it out! >> >> --- >> >> src/share/vm/runtime/vm_operations.cpp >> >> void VM_Operation::evaluate() { >> ResourceMark rm; >> ! outputStream* debugstream = LogHandle(vmoperation)::debug_stream(); >> ! if (log_is_enabled(Debug, vmoperation)) { >> ! debugstream->print("["); >> ! print_on_error(debugstream); >> ! debugstream->cr(); >> } >> doit(); >> ! if (log_is_enabled(Debug, vmoperation)) { >> ! print_on_error(debugstream); >> ! debugstream->print_cr("]"); >> } >> } >> >> Why are you calling print_on_error twice? The problem with switching VMOperation to logging is that in the current system, the output goes straight to tty and so can be concatenated. However, since the output triggered by the flag needs to be printed in a log stream (which is not necessarily tty and which has bracketed decorations), the "[" portion comes out in that stream, which could then be broken be other output, making it confusing to have a closing "]" sitting around later. Coleen and I agreed that it would improve output readability to be able to match the closing bracket by restating the VM operation info. In other words: [VM_Operation (0x00007fa3b44ce700): G1CollectFull, mode: safepoint, requested by thread 0x00007fa3ac018000 ] becomes [0.257s][debug ][vmoperation] [VM_Operation (0x00007f74e589c700): G1CollectFull, mode: safepoint, requested by thread 0x00007f74dc018000 [0.272s][debug ][vmoperation] VM_Operation (0x00007f74e589c700): G1CollectFull, mode: safepoint, requested by thread 0x00007f74dc018000] rather than [0.257s][debug ][vmoperation] [VM_Operation (0x00007f74e589c700): G1CollectFull, mode: safepoint, requested by thread 0x00007f74dc018000 [0.272s][debug ][vmoperation] ] It should be easier for users to parse this. >> >> Why is the only logging level for this tag the "debug" level? I think >> I may be missing part of the way UL works here - can you enable >> logging both by explicit tag (ie vmoperation) and the level (ie debug)? >> You enable the logging with "-Xlog:vmoperation=debug". If you leave of the "=" portion, it is by default parsed as "=info". We don't believe this vmoperation logging needs to come out by default in the case where someone asks for "-Xlog:all", so we put it one level below. >> And I know I sound like a broken record but I'm concerned about the >> overhead of the logging actions when it is not enabled ie: >> >> outputStream* debugstream = LogHandle(vmoperation)::debug_stream(); >> >> always gets executed - and we evaluate is_enabled twice. >> See my reply to Max above. >> --- >> >> test/runtime/logging/VMOperationTestMain.java >> >> Can you add a comment explaining what the logic is attempting to do >> please. I'm curious how many times the loop executes before you get a >> safepoint-based GC (and how it varies for different GC's). >> >> Style nit: while( -> while ( Yes, will fix. >> >>> A compatability request has been accepted with regard to this change. >> >> I'll record my objections again to the conversion of develop flags to >> product. Thanks for voicing your concern. The justification for this tag being product is that it could be useful to helping diagnose user problems, it does not slow performance when off, and it minimally increases the size of the code. If any of these prove to be false in aggregated logging options, we can move it to develop level during code freeze. Thanks, Rachel >> >> >> Thanks, >> David >> >>> Thank you very much, >>> Rachel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexhenrie24 at gmail.com Wed Nov 18 19:11:13 2015 From: alexhenrie24 at gmail.com (Alex Henrie) Date: Wed, 18 Nov 2015 12:11:13 -0700 Subject: [PATCH] Skip the null pointer check before calling realloc Message-ID: # HG changeset patch # User Alex Henrie # Date 1447827030 25200 # Tue Nov 17 23:10:30 2015 -0700 # Node ID ae564686e3b2797ebc4d115ce9c49aa15da964ea # Parent 28f55851bf3890dcdaa81d64d135cf35b527b987 Skip the null pointer check before calling realloc. "If ptr is a null pointer, realloc() shall be equivalent to malloc() for the specified size." http://pubs.opengroup.org/onlinepubs/9699919799/functions/realloc.html Also, sizeof(char*) is more correct here than sizeof(char**), although in practice it doesn't make a difference. diff --git a/src/java.instrument/share/native/libinstrument/InvocationAdapter.c b/src/java.instrument/share/native/libinstrument/InvocationAdapter.c --- a/src/java.instrument/share/native/libinstrument/InvocationAdapter.c +++ b/src/java.instrument/share/native/libinstrument/InvocationAdapter.c @@ -519,21 +519,17 @@ splitPathList(const char* str, int* path int count = 0; char** segments = NULL; char* c = (char*) str; while (*c != '\0') { while (*c == ' ') c++; /* skip leading spaces */ if (*c == '\0') { break; } - if (segments == NULL) { - segments = (char**)malloc( sizeof(char**) ); - } else { - segments = (char**)realloc( segments, (count+1)*sizeof(char**) ); - } + segments = (char**)realloc( segments, (count+1)*sizeof(char*) ); jplis_assert(segments != (char**)NULL); segments[count++] = c; c = strchr(c, ' '); if (c == NULL) { break; } *c = '\0'; c++; From Alan.Bateman at oracle.com Wed Nov 18 19:25:05 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 18 Nov 2015 19:25:05 +0000 Subject: RFR(XS): 8143228: Update module exports for Java Flight Recorder In-Reply-To: References: Message-ID: <564CD091.4080509@oracle.com> On 18/11/2015 17:45, Markus Gronlund wrote: > Greetings, > > > > Looking for review of the following to update module exports for Java Flight Recorder: > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8143228 > > > > Change: > > The changes to modules.xml look okay although I think we should figure out a way to drop these dependences at some point. -Alan From markus.gronlund at oracle.com Wed Nov 18 19:58:47 2015 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Wed, 18 Nov 2015 11:58:47 -0800 (PST) Subject: RFR(XS): 8143228: Update module exports for Java Flight Recorder In-Reply-To: <564CD091.4080509@oracle.com> References: <564CD091.4080509@oracle.com> Message-ID: <12c349e5-d9c2-4849-899a-2be37935c386@default> Thank you Alan. I agree on coming up with a evolution path here. Thanks for now. /Markus -----Original Message----- From: Alan Bateman Sent: den 18 november 2015 20:25 To: Markus Gronlund; jigsaw-dev at openjdk.java.net; serviceability-dev Subject: Re: RFR(XS): 8143228: Update module exports for Java Flight Recorder On 18/11/2015 17:45, Markus Gronlund wrote: > Greetings, > > > > Looking for review of the following to update module exports for Java Flight Recorder: > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8143228 > > > > Change: > > The changes to modules.xml look okay although I think we should figure out a way to drop these dependences at some point. -Alan From markus.gronlund at oracle.com Wed Nov 18 20:09:33 2015 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Wed, 18 Nov 2015 12:09:33 -0800 (PST) Subject: RFR(XS): 8143235: Remove libjfr mapfile Message-ID: Greetings, Kindly asking for reviews for the following: Bug: https://bugs.openjdk.java.net/browse/JDK-8143235 Summary: Remove libjfr mapfile (cleanup) + adds "jdk.management.jfr" to boot modules Patch is attached and also unrolled here: # HG changeset patch # User mgronlun # Date 1447765226 -3600 # Tue Nov 17 14:00:26 2015 +0100 # Node ID 13e7746e92f9b9651c0f77ed04057f382cc9e477 # Parent 85c9ddeda1c3fe13ef64ce1005c09af8e0bf1f42 [mq]: jdk_src_open_remove_mapfile diff --git a/make/mapfiles/libjfr/mapfile-vers b/make/mapfiles/libjfr/mapfile-vers deleted file mode 100644 --- a/make/mapfiles/libjfr/mapfile-vers +++ /dev/null @@ -1,45 +0,0 @@ -# -# Copyright (c) 2012, 2013, Oracle and/or its affiliates. All rights reserved. -# ORACLE PROPRIETARY/CONFIDENTIAL. Use is subject to license terms. -# - -# Define library interface. - -SUNWprivate_1.1 { - global: - Java_oracle_jrockit_jfr_Process_getpid; - Java_oracle_jrockit_jfr_Timing_counterTime; - Java_oracle_jrockit_jfr_Timing_init; - Java_oracle_jrockit_jfr_Logger_output0; - Java_oracle_jrockit_jfr_JFR_isCommercialFeaturesUnlocked; - Java_oracle_jrockit_jfr_JFR_isStarted; - Java_oracle_jrockit_jfr_JFR_isSupportedInVM; - Java_oracle_jrockit_jfr_JFR_startFlightRecorder; - Java_oracle_jrockit_jfr_JFR_isDisabledOnCommandLine; - Java_oracle_jrockit_jfr_JFR_isEnabled; - Java_oracle_jrockit_jfr_VMJFR_options; - Java_oracle_jrockit_jfr_VMJFR_init; - Java_oracle_jrockit_jfr_VMJFR_addConstPool; - Java_oracle_jrockit_jfr_VMJFR_removeConstPool; - Java_oracle_jrockit_jfr_VMJFR_storeConstPool; - Java_oracle_jrockit_jfr_VMJFR_classID0; - Java_oracle_jrockit_jfr_VMJFR_stackTraceID; - Java_oracle_jrockit_jfr_VMJFR_threadID; - Java_oracle_jrockit_jfr_VMJFR_rotate; - Java_oracle_jrockit_jfr_VMJFR_shutdown; - Java_oracle_jrockit_jfr_VMJFR_start; - Java_oracle_jrockit_jfr_VMJFR_stop; - Java_oracle_jrockit_jfr_VMJFR_buffer; - Java_oracle_jrockit_jfr_VMJFR_flush; - Java_oracle_jrockit_jfr_VMJFR_write; - Java_oracle_jrockit_jfr_VMJFR_add; - Java_oracle_jrockit_jfr_VMJFR_remove; - Java_oracle_jrockit_jfr_VMJFR_setThreshold; - Java_oracle_jrockit_jfr_VMJFR_setPeriod; - Java_oracle_jrockit_jfr_VMJFR_getPeriod; - Java_oracle_jrockit_jfr_VMJFR_descriptors; - Java_oracle_jrockit_jfr_VMJFR_retransformClasses0; - JNI_OnLoad; - local: - *; -}; diff --git a/make/src/classes/build/tools/module/boot.modules b/make/src/classes/build/tools/module/boot.modules --- a/make/src/classes/build/tools/module/boot.modules +++ b/make/src/classes/build/tools/module/boot.modules @@ -23,6 +23,7 @@ jdk.jfr jdk.management jdk.management.cmm +jdk.management.jfr jdk.management.resource jdk.naming.rmi jdk.sctp -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: jdk_src_open_remove_mapfile.patch Type: application/octet-stream Size: 2592 bytes Desc: not available URL: From david.holmes at oracle.com Wed Nov 18 22:06:41 2015 From: david.holmes at oracle.com (David Holmes) Date: Thu, 19 Nov 2015 08:06:41 +1000 Subject: RFR: 8143157: Convert TraceVMOperation to Unified Logging In-Reply-To: <564CC63D.2000509@oracle.com> References: <564B8517.7080105@oracle.com> <564C4366.4070803@oracle.com> <564CB632.9060201@oracle.com> <564CC63D.2000509@oracle.com> Message-ID: <564CF671.9060400@oracle.com> Hi Rachel, On 19/11/2015 4:41 AM, Rachel Protacio wrote: > Hi, > > On 11/18/2015 12:32 PM, Max Ockner wrote: >> I think the issue is that doit() happens regardless of what is being >> logged. I don't think we can reorder the logging statements relative >> to doit(). In that case, we would be forced to split into 2 >> conditionals for the logging. >> >> Still I don't think it is necessary to call log_is_enabled twice, and >> I don't think it is necessary to always define the outputStream. If >> this turns out to be a problem, we could flip things around a bit: >> >> void VM_Operation::evaluate() { >> ResourceMark rm; >> ! bool enabled = false; >> ! if (log_is_enabled(Debug, vmoperation)) { >> ! outputStream* debugstream = LogHandle(vmoperation)::debug_stream(); >> ! enabled = true; >> ! if (enabled) { >> ! print_on_error(debugstream); >> ! debugstream->cr(); >> } >> doit(); >> ! if (enabled) { >> ! print_on_error(debugstream); >> ! debugstream->print_cr("]"); >> } >> } >> > I'm fine with moving the definition into a conditional (but I can't move > the declaration for scope reasons). It will look like this > > outputStream* debugstream; > if (log_is_enabled(Debug, vmoperation)) { > debugstream = LogHandle(vmoperation)::debug_stream(); You may run afoul of the static analyzer when you use debugstream in the logging clause after the doit(). It won't know that log_is_enabled has to return the same answer both times. The use of the enabled flag would address that. > To be honest, I'm not sure whether evaluating log_is_enabled twice is > worse than adding a boolean check and cluttering the code and making it > less readable. The argument for changing log_is_enabled() to a boolean > could be extrapolated to say that it should be a boolean for every check > for every tag. And that might be a good idea. But I feel like the > problem with it here is that it comes across as confusing to add > "enabled", which only actually needs to be used once. If we want it > changed, maybe we should change the macro? Or put it outside the > function, even, to make it only evaluate once? I would argue that within a function log_is_enabled should only be called once per tag/level combination. I don't see it impeding readability at all - quite the opposite. (The new logging code in the current example, looks worse to me, than the original, from a clutter/readability perspective.) >> There really is nothing quite like typing code into an editor with >> misaligned columns. >> Anyway, the change looks pretty good to me. >> -Max >> >> On 11/18/2015 4:22 AM, David Holmes wrote: >>> Hi Rachel, >>> >>> On 18/11/2015 5:50 AM, Rachel Protacio wrote: >>>> Hi, >>>> >>>> Please review the following small logging enhancement. >>>> >>>> Summary: The former -XX:+TraceVMOperation flag is updated to the >>>> unified >>>> logging framework and is now replaced with -Xlog:vmoperation in product >>>> mode. >>>> >>>> Open webrev: http://cr.openjdk.java.net/~rprotacio/8143157/ >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8143157 >>>> Testing: Passes jtreg, JPRT, RBT quick tests, and refworkload >>>> performance tests. >>> >>> Meta-question: the logging framework is safe to be called when at a >>> safepoint isn't it? > I haven't seen any problems, but I also don't know what would cause it > to be unsafe. I'd appreciate any guidance in figuring it out! The main thing would be locking of course. >>> >>> --- >>> >>> src/share/vm/runtime/vm_operations.cpp >>> >>> void VM_Operation::evaluate() { >>> ResourceMark rm; >>> ! outputStream* debugstream = LogHandle(vmoperation)::debug_stream(); >>> ! if (log_is_enabled(Debug, vmoperation)) { >>> ! debugstream->print("["); >>> ! print_on_error(debugstream); >>> ! debugstream->cr(); >>> } >>> doit(); >>> ! if (log_is_enabled(Debug, vmoperation)) { >>> ! print_on_error(debugstream); >>> ! debugstream->print_cr("]"); >>> } >>> } >>> >>> Why are you calling print_on_error twice? > The problem with switching VMOperation to logging is that in the current > system, the output goes straight to tty and so can be concatenated. > However, since the output triggered by the flag needs to be printed in a > log stream (which is not necessarily tty and which has bracketed > decorations), the "[" portion comes out in that > stream, which could then be broken be other output, making it confusing > to have a closing "]" sitting around later. Coleen and I agreed that it > would improve output readability to be able to match the closing bracket > by restating the VM operation info. In other words: > > [VM_Operation (0x00007fa3b44ce700): G1CollectFull, mode: safepoint, > requested by thread 0x00007fa3ac018000 > > ] > > becomes > > [0.257s][debug ][vmoperation] [VM_Operation (0x00007f74e589c700): > G1CollectFull, mode: safepoint, requested by thread 0x00007f74dc018000 > > [0.272s][debug ][vmoperation] VM_Operation (0x00007f74e589c700): > G1CollectFull, mode: safepoint, requested by thread 0x00007f74dc018000] > > rather than > > [0.257s][debug ][vmoperation] [VM_Operation (0x00007f74e589c700): > G1CollectFull, mode: safepoint, requested by thread 0x00007f74dc018000 > > [0.272s][debug ][vmoperation] ] > > It should be easier for users to parse this. Okay but in that case each part should have the trailing ] Or, perhaps better, given all the other decoration why not just drop the [ ] bracketing altogether? The intent to was capture all output that occurred while the VMOperation was active, but that is now much clearer in the output anyway. Though perhaps a "start" and "end" indicator would aid parsing - especially when nested VMOperations might occur. >>> >>> Why is the only logging level for this tag the "debug" level? I think >>> I may be missing part of the way UL works here - can you enable >>> logging both by explicit tag (ie vmoperation) and the level (ie debug)? >>> > You enable the logging with "-Xlog:vmoperation=debug". If you leave of > the "=" portion, it is by default parsed as "=info". We don't > believe this vmoperation logging needs to come out by default in the > case where someone asks for "-Xlog:all", so we put it one level below. So does -Xlog:all signify all tags at info level or ??? I think this is real conceptual problem with the UL framework. I should be able to define the info/trace/debug levels for a given tag, without having to think about how they interact with a "log all" request. >>> And I know I sound like a broken record but I'm concerned about the >>> overhead of the logging actions when it is not enabled ie: >>> >>> outputStream* debugstream = LogHandle(vmoperation)::debug_stream(); >>> >>> always gets executed - and we evaluate is_enabled twice. >>> > See my reply to Max above. >>> --- >>> >>> test/runtime/logging/VMOperationTestMain.java >>> >>> Can you add a comment explaining what the logic is attempting to do >>> please. I'm curious how many times the loop executes before you get a >>> safepoint-based GC (and how it varies for different GC's). >>> >>> Style nit: while( -> while ( > Yes, will fix. >>> >>>> A compatability request has been accepted with regard to this change. >>> >>> I'll record my objections again to the conversion of develop flags to >>> product. > Thanks for voicing your concern. The justification for this tag being > product is that it could be useful to helping diagnose user problems, it > does not slow performance when off, and it minimally increases the size > of the code. If any of these prove to be false in aggregated logging > options, we can move it to develop level during code freeze. You could make that argument for every piece of logging and end up with a very simple flat logging system. :) It is all very subjective of course. Thanks, David > Thanks, > Rachel >>> >>> >>> Thanks, >>> David >>> >>>> Thank you very much, >>>> Rachel >> > From david.holmes at oracle.com Wed Nov 18 22:20:00 2015 From: david.holmes at oracle.com (David Holmes) Date: Thu, 19 Nov 2015 08:20:00 +1000 Subject: RFR: 8143157: Convert TraceVMOperation to Unified Logging In-Reply-To: <564CF671.9060400@oracle.com> References: <564B8517.7080105@oracle.com> <564C4366.4070803@oracle.com> <564CB632.9060201@oracle.com> <564CC63D.2000509@oracle.com> <564CF671.9060400@oracle.com> Message-ID: <564CF990.1060004@oracle.com> Thinking more .... Given we have three logging levels that are supposed to increase in detail, I think our responsibility is to simply identify what constitutes general information, "debug" level detail, and "tracing" level detail. To me what we show for the VM operation is basic information. If someone uses log:all then they get what they get - they have no idea a-priori what that will be so it seems pointless for us to try and second guess what they might want by choosing to start our logging at a lower level than info. I think real users of logging will be enabling exactly what they want when they want it. Cheers, David On 19/11/2015 8:06 AM, David Holmes wrote: > Hi Rachel, > > On 19/11/2015 4:41 AM, Rachel Protacio wrote: >> Hi, >> >> On 11/18/2015 12:32 PM, Max Ockner wrote: >>> I think the issue is that doit() happens regardless of what is being >>> logged. I don't think we can reorder the logging statements relative >>> to doit(). In that case, we would be forced to split into 2 >>> conditionals for the logging. >>> >>> Still I don't think it is necessary to call log_is_enabled twice, and >>> I don't think it is necessary to always define the outputStream. If >>> this turns out to be a problem, we could flip things around a bit: >>> >>> void VM_Operation::evaluate() { >>> ResourceMark rm; >>> ! bool enabled = false; >>> ! if (log_is_enabled(Debug, vmoperation)) { >>> ! outputStream* debugstream = >>> LogHandle(vmoperation)::debug_stream(); >>> ! enabled = true; >>> ! if (enabled) { >>> ! print_on_error(debugstream); >>> ! debugstream->cr(); >>> } >>> doit(); >>> ! if (enabled) { >>> ! print_on_error(debugstream); >>> ! debugstream->print_cr("]"); >>> } >>> } >>> >> I'm fine with moving the definition into a conditional (but I can't move >> the declaration for scope reasons). It will look like this >> >> outputStream* debugstream; >> if (log_is_enabled(Debug, vmoperation)) { >> debugstream = LogHandle(vmoperation)::debug_stream(); > > You may run afoul of the static analyzer when you use debugstream in the > logging clause after the doit(). It won't know that log_is_enabled has > to return the same answer both times. The use of the enabled flag would > address that. > >> To be honest, I'm not sure whether evaluating log_is_enabled twice is >> worse than adding a boolean check and cluttering the code and making it >> less readable. The argument for changing log_is_enabled() to a boolean >> could be extrapolated to say that it should be a boolean for every check >> for every tag. And that might be a good idea. But I feel like the >> problem with it here is that it comes across as confusing to add >> "enabled", which only actually needs to be used once. If we want it >> changed, maybe we should change the macro? Or put it outside the >> function, even, to make it only evaluate once? > > I would argue that within a function log_is_enabled should only be > called once per tag/level combination. I don't see it impeding > readability at all - quite the opposite. (The new logging code in the > current example, looks worse to me, than the original, from a > clutter/readability perspective.) > >>> There really is nothing quite like typing code into an editor with >>> misaligned columns. >>> Anyway, the change looks pretty good to me. >>> -Max >>> >>> On 11/18/2015 4:22 AM, David Holmes wrote: >>>> Hi Rachel, >>>> >>>> On 18/11/2015 5:50 AM, Rachel Protacio wrote: >>>>> Hi, >>>>> >>>>> Please review the following small logging enhancement. >>>>> >>>>> Summary: The former -XX:+TraceVMOperation flag is updated to the >>>>> unified >>>>> logging framework and is now replaced with -Xlog:vmoperation in >>>>> product >>>>> mode. >>>>> >>>>> Open webrev: http://cr.openjdk.java.net/~rprotacio/8143157/ >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8143157 >>>>> Testing: Passes jtreg, JPRT, RBT quick tests, and refworkload >>>>> performance tests. >>>> >>>> Meta-question: the logging framework is safe to be called when at a >>>> safepoint isn't it? >> I haven't seen any problems, but I also don't know what would cause it >> to be unsafe. I'd appreciate any guidance in figuring it out! > > The main thing would be locking of course. > >>>> >>>> --- >>>> >>>> src/share/vm/runtime/vm_operations.cpp >>>> >>>> void VM_Operation::evaluate() { >>>> ResourceMark rm; >>>> ! outputStream* debugstream = LogHandle(vmoperation)::debug_stream(); >>>> ! if (log_is_enabled(Debug, vmoperation)) { >>>> ! debugstream->print("["); >>>> ! print_on_error(debugstream); >>>> ! debugstream->cr(); >>>> } >>>> doit(); >>>> ! if (log_is_enabled(Debug, vmoperation)) { >>>> ! print_on_error(debugstream); >>>> ! debugstream->print_cr("]"); >>>> } >>>> } >>>> >>>> Why are you calling print_on_error twice? >> The problem with switching VMOperation to logging is that in the current >> system, the output goes straight to tty and so can be concatenated. >> However, since the output triggered by the flag needs to be printed in a >> log stream (which is not necessarily tty and which has bracketed >> decorations), the "[" portion comes out in that >> stream, which could then be broken be other output, making it confusing >> to have a closing "]" sitting around later. Coleen and I agreed that it >> would improve output readability to be able to match the closing bracket >> by restating the VM operation info. In other words: >> >> [VM_Operation (0x00007fa3b44ce700): G1CollectFull, mode: safepoint, >> requested by thread 0x00007fa3ac018000 >> >> ] >> >> becomes >> >> [0.257s][debug ][vmoperation] [VM_Operation (0x00007f74e589c700): >> G1CollectFull, mode: safepoint, requested by thread >> 0x00007f74dc018000 >> >> [0.272s][debug ][vmoperation] VM_Operation (0x00007f74e589c700): >> G1CollectFull, mode: safepoint, requested by thread >> 0x00007f74dc018000] >> >> rather than >> >> [0.257s][debug ][vmoperation] [VM_Operation (0x00007f74e589c700): >> G1CollectFull, mode: safepoint, requested by thread >> 0x00007f74dc018000 >> >> [0.272s][debug ][vmoperation] ] >> >> It should be easier for users to parse this. > > Okay but in that case each part should have the trailing ] > > Or, perhaps better, given all the other decoration why not just drop the > [ ] bracketing altogether? The intent to was capture all output that > occurred while the VMOperation was active, but that is now much clearer > in the output anyway. Though perhaps a "start" and "end" indicator would > aid parsing - especially when nested VMOperations might occur. > >>>> >>>> Why is the only logging level for this tag the "debug" level? I think >>>> I may be missing part of the way UL works here - can you enable >>>> logging both by explicit tag (ie vmoperation) and the level (ie debug)? >>>> >> You enable the logging with "-Xlog:vmoperation=debug". If you leave of >> the "=" portion, it is by default parsed as "=info". We don't >> believe this vmoperation logging needs to come out by default in the >> case where someone asks for "-Xlog:all", so we put it one level below. > > So does -Xlog:all signify all tags at info level or ??? > > I think this is real conceptual problem with the UL framework. I should > be able to define the info/trace/debug levels for a given tag, without > having to think about how they interact with a "log all" request. > >>>> And I know I sound like a broken record but I'm concerned about the >>>> overhead of the logging actions when it is not enabled ie: >>>> >>>> outputStream* debugstream = LogHandle(vmoperation)::debug_stream(); >>>> >>>> always gets executed - and we evaluate is_enabled twice. >>>> >> See my reply to Max above. >>>> --- >>>> >>>> test/runtime/logging/VMOperationTestMain.java >>>> >>>> Can you add a comment explaining what the logic is attempting to do >>>> please. I'm curious how many times the loop executes before you get a >>>> safepoint-based GC (and how it varies for different GC's). >>>> >>>> Style nit: while( -> while ( >> Yes, will fix. >>>> >>>>> A compatability request has been accepted with regard to this change. >>>> >>>> I'll record my objections again to the conversion of develop flags to >>>> product. >> Thanks for voicing your concern. The justification for this tag being >> product is that it could be useful to helping diagnose user problems, it >> does not slow performance when off, and it minimally increases the size >> of the code. If any of these prove to be false in aggregated logging >> options, we can move it to develop level during code freeze. > > You could make that argument for every piece of logging and end up with > a very simple flat logging system. :) It is all very subjective of course. > > Thanks, > David > >> Thanks, >> Rachel >>>> >>>> >>>> Thanks, >>>> David >>>> >>>>> Thank you very much, >>>>> Rachel >>> >> From kirk.pepperdine at gmail.com Thu Nov 19 08:48:49 2015 From: kirk.pepperdine at gmail.com (kirk.pepperdine at gmail.com) Date: Thu, 19 Nov 2015 09:48:49 +0100 Subject: RFR: 8143157: Convert TraceVMOperation to Unified Logging In-Reply-To: <564CF671.9060400@oracle.com> References: <564B8517.7080105@oracle.com> <564C4366.4070803@oracle.com> <564CB632.9060201@oracle.com> <564CC63D.2000509@oracle.com> <564CF671.9060400@oracle.com> Message-ID: <2DE32236-0866-4BB5-87EB-3946039B05A0@gmail.com> Hi David, >>>> >> You enable the logging with "-Xlog:vmoperation=debug". If you leave of >> the "=" portion, it is by default parsed as "=info". We don't >> believe this vmoperation logging needs to come out by default in the >> case where someone asks for "-Xlog:all", so we put it one level below. > > So does -Xlog:all signify all tags at info level or ??? > > I think this is real conceptual problem with the UL framework. I should be able to define the info/trace/debug levels for a given tag, without having to think about how they interact with a "log all" request. This has been one of my concerns about UL since the initial JEP and why I was speaking towards UL being tag based instead of level based. Now, IMO, UL is backwards in that it is geared towards levels but that is confused with tags. Being geared towards tags with levels would have created less of a conceptual problem. And, it is more inline with how other messaging systems work (given that logging is a specialization of messaging). >> Thanks for voicing your concern. The justification for this tag being >> product is that it could be useful to helping diagnose user problems, it >> does not slow performance when off, and it minimally increases the size >> of the code. If any of these prove to be false in aggregated logging >> options, we can move it to develop level during code freeze. > > You could make that argument for every piece of logging and end up with a very simple flat logging system. :) It is all very subjective of course. Agreed, slippery slope but in this case I?m happy that we will now have access to this type of information in production environments. What we currently have are indications of a problem without any real means of seeing what the problem is. My rule of thumb for using log levels is debug - detailed information that a developer needs. It would have limited value to diagnosing problems in production systems warning - defensive action needed to be taken, or an action couldn?t be taken to avert something bad from happening info - an event occurred (such as jvm operation, GC, safe-point, CHA event?..) I would not want ?all" to include ?debug?. So, not exactly the hierarchical escalation that is being forced on us. Kind regards, Kirk From marcus.larsson at oracle.com Thu Nov 19 12:23:16 2015 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Thu, 19 Nov 2015 13:23:16 +0100 Subject: RFR: 8143157: Convert TraceVMOperation to Unified Logging In-Reply-To: <564CF671.9060400@oracle.com> References: <564B8517.7080105@oracle.com> <564C4366.4070803@oracle.com> <564CB632.9060201@oracle.com> <564CC63D.2000509@oracle.com> <564CF671.9060400@oracle.com> Message-ID: <564DBF34.7060003@oracle.com> Hi, On 2015-11-18 23:06, David Holmes wrote: > Hi Rachel, > > On 19/11/2015 4:41 AM, Rachel Protacio wrote: >> Hi, >> >> On 11/18/2015 12:32 PM, Max Ockner wrote: >>> I think the issue is that doit() happens regardless of what is being >>> logged. I don't think we can reorder the logging statements relative >>> to doit(). In that case, we would be forced to split into 2 >>> conditionals for the logging. >>> >>> Still I don't think it is necessary to call log_is_enabled twice, and >>> I don't think it is necessary to always define the outputStream. If >>> this turns out to be a problem, we could flip things around a bit: >>> >>> void VM_Operation::evaluate() { >>> ResourceMark rm; >>> ! bool enabled = false; >>> ! if (log_is_enabled(Debug, vmoperation)) { >>> ! outputStream* debugstream = >>> LogHandle(vmoperation)::debug_stream(); >>> ! enabled = true; >>> ! if (enabled) { >>> ! print_on_error(debugstream); >>> ! debugstream->cr(); >>> } >>> doit(); >>> ! if (enabled) { >>> ! print_on_error(debugstream); >>> ! debugstream->print_cr("]"); >>> } >>> } >>> >> I'm fine with moving the definition into a conditional (but I can't move >> the declaration for scope reasons). It will look like this >> >> outputStream* debugstream; >> if (log_is_enabled(Debug, vmoperation)) { >> debugstream = LogHandle(vmoperation)::debug_stream(); > > You may run afoul of the static analyzer when you use debugstream in > the logging clause after the doit(). It won't know that log_is_enabled > has to return the same answer both times. The use of the enabled flag > would address that. The log configuration can change at runtime, so in this case the two log_is_enabled() tests might in fact give different answers. > >> To be honest, I'm not sure whether evaluating log_is_enabled twice is >> worse than adding a boolean check and cluttering the code and making it >> less readable. The argument for changing log_is_enabled() to a boolean >> could be extrapolated to say that it should be a boolean for every check >> for every tag. And that might be a good idea. But I feel like the >> problem with it here is that it comes across as confusing to add >> "enabled", which only actually needs to be used once. If we want it >> changed, maybe we should change the macro? Or put it outside the >> function, even, to make it only evaluate once? > > I would argue that within a function log_is_enabled should only be > called once per tag/level combination. I don't see it impeding > readability at all - quite the opposite. (The new logging code in the > current example, looks worse to me, than the original, from a > clutter/readability perspective.) > >>> There really is nothing quite like typing code into an editor with >>> misaligned columns. >>> Anyway, the change looks pretty good to me. >>> -Max >>> >>> On 11/18/2015 4:22 AM, David Holmes wrote: >>>> Hi Rachel, >>>> >>>> On 18/11/2015 5:50 AM, Rachel Protacio wrote: >>>>> Hi, >>>>> >>>>> Please review the following small logging enhancement. >>>>> >>>>> Summary: The former -XX:+TraceVMOperation flag is updated to the >>>>> unified >>>>> logging framework and is now replaced with -Xlog:vmoperation in >>>>> product >>>>> mode. >>>>> >>>>> Open webrev: http://cr.openjdk.java.net/~rprotacio/8143157/ >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8143157 >>>>> Testing: Passes jtreg, JPRT, RBT quick tests, and refworkload >>>>> performance tests. >>>> >>>> Meta-question: the logging framework is safe to be called when at a >>>> safepoint isn't it? >> I haven't seen any problems, but I also don't know what would cause it >> to be unsafe. I'd appreciate any guidance in figuring it out! > > The main thing would be locking of course. Naturally, logging is and should be available for use at any time, including during safepoints. > >>>> >>>> --- >>>> >>>> src/share/vm/runtime/vm_operations.cpp >>>> >>>> void VM_Operation::evaluate() { >>>> ResourceMark rm; >>>> ! outputStream* debugstream = >>>> LogHandle(vmoperation)::debug_stream(); >>>> ! if (log_is_enabled(Debug, vmoperation)) { >>>> ! debugstream->print("["); >>>> ! print_on_error(debugstream); >>>> ! debugstream->cr(); >>>> } >>>> doit(); >>>> ! if (log_is_enabled(Debug, vmoperation)) { >>>> ! print_on_error(debugstream); >>>> ! debugstream->print_cr("]"); >>>> } >>>> } >>>> >>>> Why are you calling print_on_error twice? >> The problem with switching VMOperation to logging is that in the current >> system, the output goes straight to tty and so can be concatenated. >> However, since the output triggered by the flag needs to be printed in a >> log stream (which is not necessarily tty and which has bracketed >> decorations), the "[" portion comes out in that >> stream, which could then be broken be other output, making it confusing >> to have a closing "]" sitting around later. Coleen and I agreed that it >> would improve output readability to be able to match the closing bracket >> by restating the VM operation info. In other words: >> >> [VM_Operation (0x00007fa3b44ce700): G1CollectFull, mode: safepoint, >> requested by thread 0x00007fa3ac018000 >> >> ] >> >> becomes >> >> [0.257s][debug ][vmoperation] [VM_Operation (0x00007f74e589c700): >> G1CollectFull, mode: safepoint, requested by thread >> 0x00007f74dc018000 >> >> [0.272s][debug ][vmoperation] VM_Operation (0x00007f74e589c700): >> G1CollectFull, mode: safepoint, requested by thread >> 0x00007f74dc018000] >> >> rather than >> >> [0.257s][debug ][vmoperation] [VM_Operation (0x00007f74e589c700): >> G1CollectFull, mode: safepoint, requested by thread >> 0x00007f74dc018000 >> >> [0.272s][debug ][vmoperation] ] >> >> It should be easier for users to parse this. > > Okay but in that case each part should have the trailing ] > > Or, perhaps better, given all the other decoration why not just drop > the [ ] bracketing altogether? The intent to was capture all output > that occurred while the VMOperation was active, but that is now much > clearer in the output anyway. Though perhaps a "start" and "end" > indicator would aid parsing - especially when nested VMOperations > might occur. I agree. Adding some sort of end message and skipping the brackets altogether seems clearer to me. > >>>> >>>> Why is the only logging level for this tag the "debug" level? I think >>>> I may be missing part of the way UL works here - can you enable >>>> logging both by explicit tag (ie vmoperation) and the level (ie >>>> debug)? >>>> >> You enable the logging with "-Xlog:vmoperation=debug". If you leave of >> the "=" portion, it is by default parsed as "=info". We don't >> believe this vmoperation logging needs to come out by default in the >> case where someone asks for "-Xlog:all", so we put it one level below. > > So does -Xlog:all signify all tags at info level or ??? Yes. > > I think this is real conceptual problem with the UL framework. I > should be able to define the info/trace/debug levels for a given tag, > without having to think about how they interact with a "log all" request. I believe focus should be on classifying each log message according to its component & type (tags) and verbosity & cost (level), and less about which users will get it using what command line. The log configuration is very flexible to allow users full control over the logging they want, all we need to do on the development side is to keep the logging logically classified and ordered so that the users can easily select from it. Thanks, Marcus > >>>> And I know I sound like a broken record but I'm concerned about the >>>> overhead of the logging actions when it is not enabled ie: >>>> >>>> outputStream* debugstream = LogHandle(vmoperation)::debug_stream(); >>>> >>>> always gets executed - and we evaluate is_enabled twice. >>>> >> See my reply to Max above. >>>> --- >>>> >>>> test/runtime/logging/VMOperationTestMain.java >>>> >>>> Can you add a comment explaining what the logic is attempting to do >>>> please. I'm curious how many times the loop executes before you get a >>>> safepoint-based GC (and how it varies for different GC's). >>>> >>>> Style nit: while( -> while ( >> Yes, will fix. >>>> >>>>> A compatability request has been accepted with regard to this change. >>>> >>>> I'll record my objections again to the conversion of develop flags to >>>> product. >> Thanks for voicing your concern. The justification for this tag being >> product is that it could be useful to helping diagnose user problems, it >> does not slow performance when off, and it minimally increases the size >> of the code. If any of these prove to be false in aggregated logging >> options, we can move it to develop level during code freeze. > > You could make that argument for every piece of logging and end up > with a very simple flat logging system. :) It is all very subjective > of course. > > Thanks, > David > >> Thanks, >> Rachel >>>> >>>> >>>> Thanks, >>>> David >>>> >>>>> Thank you very much, >>>>> Rachel >>> >> From erik.gahlin at oracle.com Thu Nov 19 15:21:45 2015 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Thu, 19 Nov 2015 16:21:45 +0100 Subject: RFR(XS): 8066814: Reduce accessibility in TraceEvent In-Reply-To: <2c9d1e54-7ee2-444d-8ee2-36fefb866f78@default> References: <2c9d1e54-7ee2-444d-8ee2-36fefb866f78@default> Message-ID: <564DE909.3070303@oracle.com> Looks good, not a reviewer. Erik On 2014-12-17 15:45, Markus Gr?nlund wrote: > > Greetings, > > Kindly asking for reviews for the following changeset: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8066814 > > Webrev: http://cr.openjdk.java.net/~mgronlun/8066814/webrev01/ > > > Description: > > TraceEvent currently exposes internals unnecessarily. > > Therefore: > > Remove unnecessarily exposed methods. > Add assert for not committing a cancelled event. > Add method stubs for !INCLUDE_TRACE > > Thanks in advance > > Markus > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erik.gahlin at oracle.com Thu Nov 19 15:41:22 2015 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Thu, 19 Nov 2015 16:41:22 +0100 Subject: RFR(XS): 8143235: Remove libjfr mapfile In-Reply-To: References: Message-ID: <564DEDA2.2060009@oracle.com> Looks good, not a reviewer. Erik On 2015-11-18 21:09, Markus Gronlund wrote: > > Greetings, > > Kindly asking for reviews for the following: > > Bug: > > https://bugs.openjdk.java.net/browse/JDK-8143235 > > Summary: > > Remove libjfr mapfile (cleanup) + adds ?jdk.management.jfr? to boot > modules > > Patch is attached and also unrolled here: > > # HG changeset patch > > # User mgronlun > > # Date 1447765226 -3600 > > # Tue Nov 17 14:00:26 2015 +0100 > > # Node ID 13e7746e92f9b9651c0f77ed04057f382cc9e477 > > # Parent 85c9ddeda1c3fe13ef64ce1005c09af8e0bf1f42 > > [mq]: jdk_src_open_remove_mapfile > > diff --git a/make/mapfiles/libjfr/mapfile-vers > b/make/mapfiles/libjfr/mapfile-vers > > deleted file mode 100644 > > --- a/make/mapfiles/libjfr/mapfile-vers > > +++ /dev/null > > @@ -1,45 +0,0 @@ > > -# > > -# Copyright (c) 2012, 2013, Oracle and/or its affiliates. All rights > reserved. > > -# ORACLE PROPRIETARY/CONFIDENTIAL. Use is subject to license terms. > > -# > > - > > -# Define library interface. > > - > > -SUNWprivate_1.1 { > > - global: > > - Java_oracle_jrockit_jfr_Process_getpid; > > - Java_oracle_jrockit_jfr_Timing_counterTime; > > - Java_oracle_jrockit_jfr_Timing_init; > > - Java_oracle_jrockit_jfr_Logger_output0; > > - Java_oracle_jrockit_jfr_JFR_isCommercialFeaturesUnlocked; > > - Java_oracle_jrockit_jfr_JFR_isStarted; > > - Java_oracle_jrockit_jfr_JFR_isSupportedInVM; > > - Java_oracle_jrockit_jfr_JFR_startFlightRecorder; > > - Java_oracle_jrockit_jfr_JFR_isDisabledOnCommandLine; > > - Java_oracle_jrockit_jfr_JFR_isEnabled; > > - Java_oracle_jrockit_jfr_VMJFR_options; > > - Java_oracle_jrockit_jfr_VMJFR_init; > > - Java_oracle_jrockit_jfr_VMJFR_addConstPool; > > - Java_oracle_jrockit_jfr_VMJFR_removeConstPool; > > - Java_oracle_jrockit_jfr_VMJFR_storeConstPool; > > - Java_oracle_jrockit_jfr_VMJFR_classID0; > > - Java_oracle_jrockit_jfr_VMJFR_stackTraceID; > > - Java_oracle_jrockit_jfr_VMJFR_threadID; > > - Java_oracle_jrockit_jfr_VMJFR_rotate; > > - Java_oracle_jrockit_jfr_VMJFR_shutdown; > > - Java_oracle_jrockit_jfr_VMJFR_start; > > - Java_oracle_jrockit_jfr_VMJFR_stop; > > - Java_oracle_jrockit_jfr_VMJFR_buffer; > > - Java_oracle_jrockit_jfr_VMJFR_flush; > > - Java_oracle_jrockit_jfr_VMJFR_write; > > - Java_oracle_jrockit_jfr_VMJFR_add; > > - Java_oracle_jrockit_jfr_VMJFR_remove; > > - Java_oracle_jrockit_jfr_VMJFR_setThreshold; > > - Java_oracle_jrockit_jfr_VMJFR_setPeriod; > > - Java_oracle_jrockit_jfr_VMJFR_getPeriod; > > - Java_oracle_jrockit_jfr_VMJFR_descriptors; > > - Java_oracle_jrockit_jfr_VMJFR_retransformClasses0; > > - JNI_OnLoad; > > - local: > > - *; > > -}; > > diff --git a/make/src/classes/build/tools/module/boot.modules > b/make/src/classes/build/tools/module/boot.modules > > --- a/make/src/classes/build/tools/module/boot.modules > > +++ b/make/src/classes/build/tools/module/boot.modules > > @@ -23,6 +23,7 @@ > > jdk.jfr > > jdk.management > > jdk.management.cmm > > +jdk.management.jfr > > jdk.management.resource > > jdk.naming.rmi > > jdk.sctp > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erik.gahlin at oracle.com Thu Nov 19 15:51:31 2015 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Thu, 19 Nov 2015 16:51:31 +0100 Subject: 8143226 (M): Minor updates to Event Based tracing In-Reply-To: References: Message-ID: <564DF003.3080200@oracle.com> Looks good, not a reviewer. Erik On 2015-11-18 14:30, Markus Gronlund wrote: > > Greetings, > > Kindly asking for reviews for the following bug: > > https://bugs.openjdk.java.net/browse/JDK-8143226 > > Webrev: > > http://cr.openjdk.java.net/~mgronlun/8143226/webrev01/ > > > Summary: > > > - Remove obsoleted files from make system > - Consolidate to a single Thread abstraction > - Expand relational keys to u8's to introduce uniqueness property > - Tracing hook to be notified on out of memory > - Tracing hook in early vm startup > - Event "starttime" to become the primary time stamp for the event > (previously it was "endtime") > > Thanks in advance > > Markus > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mandy.chung at oracle.com Thu Nov 19 18:31:25 2015 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 19 Nov 2015 10:31:25 -0800 Subject: RFR(XS): 8143235: Remove libjfr mapfile In-Reply-To: References: Message-ID: <554CB69C-7CCE-4865-B725-4002542065EB@oracle.com> Looks okay to me. Mandy > On Nov 18, 2015, at 12:09 PM, Markus Gronlund wrote: > > Greetings, > > Kindly asking for reviews for the following: > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8143235 > > Summary: > > Remove libjfr mapfile (cleanup) + adds ?jdk.management.jfr? to boot modules > > > > Patch is attached and also unrolled here: > > # HG changeset patch > # User mgronlun > # Date 1447765226 -3600 > # Tue Nov 17 14:00:26 2015 +0100 > # Node ID 13e7746e92f9b9651c0f77ed04057f382cc9e477 > # Parent 85c9ddeda1c3fe13ef64ce1005c09af8e0bf1f42 > [mq]: jdk_src_open_remove_mapfile > > diff --git a/make/mapfiles/libjfr/mapfile-vers b/make/mapfiles/libjfr/mapfile-vers > deleted file mode 100644 > --- a/make/mapfiles/libjfr/mapfile-vers > +++ /dev/null > @@ -1,45 +0,0 @@ > -# > -# Copyright (c) 2012, 2013, Oracle and/or its affiliates. All rights reserved. > -# ORACLE PROPRIETARY/CONFIDENTIAL. Use is subject to license terms. > -# > - > -# Define library interface. > - > -SUNWprivate_1.1 { > - global: > - Java_oracle_jrockit_jfr_Process_getpid; > - Java_oracle_jrockit_jfr_Timing_counterTime; > - Java_oracle_jrockit_jfr_Timing_init; > - Java_oracle_jrockit_jfr_Logger_output0; > - Java_oracle_jrockit_jfr_JFR_isCommercialFeaturesUnlocked; > - Java_oracle_jrockit_jfr_JFR_isStarted; > - Java_oracle_jrockit_jfr_JFR_isSupportedInVM; > - Java_oracle_jrockit_jfr_JFR_startFlightRecorder; > - Java_oracle_jrockit_jfr_JFR_isDisabledOnCommandLine; > - Java_oracle_jrockit_jfr_JFR_isEnabled; > - Java_oracle_jrockit_jfr_VMJFR_options; > - Java_oracle_jrockit_jfr_VMJFR_init; > - Java_oracle_jrockit_jfr_VMJFR_addConstPool; > - Java_oracle_jrockit_jfr_VMJFR_removeConstPool; > - Java_oracle_jrockit_jfr_VMJFR_storeConstPool; > - Java_oracle_jrockit_jfr_VMJFR_classID0; > - Java_oracle_jrockit_jfr_VMJFR_stackTraceID; > - Java_oracle_jrockit_jfr_VMJFR_threadID; > - Java_oracle_jrockit_jfr_VMJFR_rotate; > - Java_oracle_jrockit_jfr_VMJFR_shutdown; > - Java_oracle_jrockit_jfr_VMJFR_start; > - Java_oracle_jrockit_jfr_VMJFR_stop; > - Java_oracle_jrockit_jfr_VMJFR_buffer; > - Java_oracle_jrockit_jfr_VMJFR_flush; > - Java_oracle_jrockit_jfr_VMJFR_write; > - Java_oracle_jrockit_jfr_VMJFR_add; > - Java_oracle_jrockit_jfr_VMJFR_remove; > - Java_oracle_jrockit_jfr_VMJFR_setThreshold; > - Java_oracle_jrockit_jfr_VMJFR_setPeriod; > - Java_oracle_jrockit_jfr_VMJFR_getPeriod; > - Java_oracle_jrockit_jfr_VMJFR_descriptors; > - Java_oracle_jrockit_jfr_VMJFR_retransformClasses0; > - JNI_OnLoad; > - local: > - *; > -}; > diff --git a/make/src/classes/build/tools/module/boot.modules b/make/src/classes/build/tools/module/boot.modules > --- a/make/src/classes/build/tools/module/boot.modules > +++ b/make/src/classes/build/tools/module/boot.modules > @@ -23,6 +23,7 @@ > jdk.jfr > jdk.management > jdk.management.cmm > +jdk.management.jfr > jdk.management.resource > jdk.naming.rmi > jdk.sctp > From rachel.protacio at oracle.com Thu Nov 19 18:46:35 2015 From: rachel.protacio at oracle.com (Rachel Protacio) Date: Thu, 19 Nov 2015 13:46:35 -0500 Subject: RFR: 8143157: Convert TraceVMOperation to Unified Logging In-Reply-To: <2DE32236-0866-4BB5-87EB-3946039B05A0@gmail.com> References: <564B8517.7080105@oracle.com> <564C4366.4070803@oracle.com> <564CB632.9060201@oracle.com> <564CC63D.2000509@oracle.com> <564CF671.9060400@oracle.com> <2DE32236-0866-4BB5-87EB-3946039B05A0@gmail.com> Message-ID: <564E190B.2030606@oracle.com> Hello! On 11/19/2015 3:48 AM, kirk.pepperdine at gmail.com wrote: > Hi David, > >>> You enable the logging with "-Xlog:vmoperation=debug". If you leave of >>> the "=" portion, it is by default parsed as "=info". We don't >>> believe this vmoperation logging needs to come out by default in the >>> case where someone asks for "-Xlog:all", so we put it one level below. >> So does -Xlog:all signify all tags at info level or ??? >> >> I think this is real conceptual problem with the UL framework. I should be able to define the info/trace/debug levels for a given tag, without having to think about how they interact with a "log all" request. > This has been one of my concerns about UL since the initial JEP and why I was speaking towards UL being tag based instead of level based. Now, IMO, UL is backwards in that it is geared towards levels but that is confused with tags. Being geared towards tags with levels would have created less of a conceptual problem. And, it is more inline with how other messaging systems work (given that logging is a specialization of messaging). > >>> Thanks for voicing your concern. The justification for this tag being >>> product is that it could be useful to helping diagnose user problems, it >>> does not slow performance when off, and it minimally increases the size >>> of the code. If any of these prove to be false in aggregated logging >>> options, we can move it to develop level during code freeze. >> You could make that argument for every piece of logging and end up with a very simple flat logging system. :) It is all very subjective of course. > Agreed, slippery slope but in this case I?m happy that we will now have access to this type of information in production environments. What we currently have are indications of a problem without any real means of seeing what the problem is. > > My rule of thumb for using log levels is > debug - detailed information that a developer needs. It would have limited value to diagnosing problems in production systems > warning - defensive action needed to be taken, or an action couldn?t be taken to avert something bad from happening > info - an event occurred (such as jvm operation, GC, safe-point, CHA event?..) > > I would not want ?all" to include ?debug?. So, not exactly the hierarchical escalation that is being forced on us. Just want to make sure we're all on the same page. Your rules of thumb are pretty accurate. The levels are as follows (directly from logLevel.hpp) and aggregate the levels below them: // develop - A non-product level that is finer than trace. // Should be used for really expensive and/or // extensive logging, or logging that shouldn't // or can't be included in a product build. // // trace - Finest level of logging in product builds. // Use for extensive/noisy logging that can // give slow-down when enabled. // // debug - A finer level of logging. Use for semi-noisy // logging that is does not fit the info level. // // info - General level of logging. Use for significant // events and/or informative summaries. // // warning - Important messages that are not strictly errors. // // error - Critical messages caused by errors. "Info" is default, which would include only the levels "warning", "error", and "info". I used "-Xlog:all" as an example; it's not inherently central to the purpose of levels. It means print logging for all tags. But by not specifying a level, you get "info", which does not have "debug." Rachel > > Kind regards, > Kirk > -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.holmes at oracle.com Thu Nov 19 22:32:29 2015 From: david.holmes at oracle.com (David Holmes) Date: Fri, 20 Nov 2015 08:32:29 +1000 Subject: RFR: 8143157: Convert TraceVMOperation to Unified Logging In-Reply-To: <564DBF34.7060003@oracle.com> References: <564B8517.7080105@oracle.com> <564C4366.4070803@oracle.com> <564CB632.9060201@oracle.com> <564CC63D.2000509@oracle.com> <564CF671.9060400@oracle.com> <564DBF34.7060003@oracle.com> Message-ID: <564E4DFD.1090303@oracle.com> Hi Marcus, On 19/11/2015 10:23 PM, Marcus Larsson wrote: > Hi, > > On 2015-11-18 23:06, David Holmes wrote: >> Hi Rachel, >> >> On 19/11/2015 4:41 AM, Rachel Protacio wrote: >>> Hi, >>> >>> On 11/18/2015 12:32 PM, Max Ockner wrote: >>>> I think the issue is that doit() happens regardless of what is being >>>> logged. I don't think we can reorder the logging statements relative >>>> to doit(). In that case, we would be forced to split into 2 >>>> conditionals for the logging. >>>> >>>> Still I don't think it is necessary to call log_is_enabled twice, and >>>> I don't think it is necessary to always define the outputStream. If >>>> this turns out to be a problem, we could flip things around a bit: >>>> >>>> void VM_Operation::evaluate() { >>>> ResourceMark rm; >>>> ! bool enabled = false; >>>> ! if (log_is_enabled(Debug, vmoperation)) { >>>> ! outputStream* debugstream = >>>> LogHandle(vmoperation)::debug_stream(); >>>> ! enabled = true; >>>> ! if (enabled) { >>>> ! print_on_error(debugstream); >>>> ! debugstream->cr(); >>>> } >>>> doit(); >>>> ! if (enabled) { >>>> ! print_on_error(debugstream); >>>> ! debugstream->print_cr("]"); >>>> } >>>> } >>>> >>> I'm fine with moving the definition into a conditional (but I can't move >>> the declaration for scope reasons). It will look like this >>> >>> outputStream* debugstream; >>> if (log_is_enabled(Debug, vmoperation)) { >>> debugstream = LogHandle(vmoperation)::debug_stream(); >> >> You may run afoul of the static analyzer when you use debugstream in >> the logging clause after the doit(). It won't know that log_is_enabled >> has to return the same answer both times. The use of the enabled flag >> would address that. > > The log configuration can change at runtime, so in this case the two > log_is_enabled() tests might in fact give different answers. In that case I would argue for the caching of the initial value of the log setting. While disabling the log at runtime would be racey wrt existing logging actions, I think a bracketing log action like this should either be complete or else fully absent, but not just the first half or second half. David ------ >> >>> To be honest, I'm not sure whether evaluating log_is_enabled twice is >>> worse than adding a boolean check and cluttering the code and making it >>> less readable. The argument for changing log_is_enabled() to a boolean >>> could be extrapolated to say that it should be a boolean for every check >>> for every tag. And that might be a good idea. But I feel like the >>> problem with it here is that it comes across as confusing to add >>> "enabled", which only actually needs to be used once. If we want it >>> changed, maybe we should change the macro? Or put it outside the >>> function, even, to make it only evaluate once? >> >> I would argue that within a function log_is_enabled should only be >> called once per tag/level combination. I don't see it impeding >> readability at all - quite the opposite. (The new logging code in the >> current example, looks worse to me, than the original, from a >> clutter/readability perspective.) >> >>>> There really is nothing quite like typing code into an editor with >>>> misaligned columns. >>>> Anyway, the change looks pretty good to me. >>>> -Max >>>> >>>> On 11/18/2015 4:22 AM, David Holmes wrote: >>>>> Hi Rachel, >>>>> >>>>> On 18/11/2015 5:50 AM, Rachel Protacio wrote: >>>>>> Hi, >>>>>> >>>>>> Please review the following small logging enhancement. >>>>>> >>>>>> Summary: The former -XX:+TraceVMOperation flag is updated to the >>>>>> unified >>>>>> logging framework and is now replaced with -Xlog:vmoperation in >>>>>> product >>>>>> mode. >>>>>> >>>>>> Open webrev: http://cr.openjdk.java.net/~rprotacio/8143157/ >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8143157 >>>>>> Testing: Passes jtreg, JPRT, RBT quick tests, and refworkload >>>>>> performance tests. >>>>> >>>>> Meta-question: the logging framework is safe to be called when at a >>>>> safepoint isn't it? >>> I haven't seen any problems, but I also don't know what would cause it >>> to be unsafe. I'd appreciate any guidance in figuring it out! >> >> The main thing would be locking of course. > > Naturally, logging is and should be available for use at any time, > including during safepoints. > >> >>>>> >>>>> --- >>>>> >>>>> src/share/vm/runtime/vm_operations.cpp >>>>> >>>>> void VM_Operation::evaluate() { >>>>> ResourceMark rm; >>>>> ! outputStream* debugstream = >>>>> LogHandle(vmoperation)::debug_stream(); >>>>> ! if (log_is_enabled(Debug, vmoperation)) { >>>>> ! debugstream->print("["); >>>>> ! print_on_error(debugstream); >>>>> ! debugstream->cr(); >>>>> } >>>>> doit(); >>>>> ! if (log_is_enabled(Debug, vmoperation)) { >>>>> ! print_on_error(debugstream); >>>>> ! debugstream->print_cr("]"); >>>>> } >>>>> } >>>>> >>>>> Why are you calling print_on_error twice? >>> The problem with switching VMOperation to logging is that in the current >>> system, the output goes straight to tty and so can be concatenated. >>> However, since the output triggered by the flag needs to be printed in a >>> log stream (which is not necessarily tty and which has bracketed >>> decorations), the "[" portion comes out in that >>> stream, which could then be broken be other output, making it confusing >>> to have a closing "]" sitting around later. Coleen and I agreed that it >>> would improve output readability to be able to match the closing bracket >>> by restating the VM operation info. In other words: >>> >>> [VM_Operation (0x00007fa3b44ce700): G1CollectFull, mode: safepoint, >>> requested by thread 0x00007fa3ac018000 >>> >>> ] >>> >>> becomes >>> >>> [0.257s][debug ][vmoperation] [VM_Operation (0x00007f74e589c700): >>> G1CollectFull, mode: safepoint, requested by thread >>> 0x00007f74dc018000 >>> >>> [0.272s][debug ][vmoperation] VM_Operation (0x00007f74e589c700): >>> G1CollectFull, mode: safepoint, requested by thread >>> 0x00007f74dc018000] >>> >>> rather than >>> >>> [0.257s][debug ][vmoperation] [VM_Operation (0x00007f74e589c700): >>> G1CollectFull, mode: safepoint, requested by thread >>> 0x00007f74dc018000 >>> >>> [0.272s][debug ][vmoperation] ] >>> >>> It should be easier for users to parse this. >> >> Okay but in that case each part should have the trailing ] >> >> Or, perhaps better, given all the other decoration why not just drop >> the [ ] bracketing altogether? The intent to was capture all output >> that occurred while the VMOperation was active, but that is now much >> clearer in the output anyway. Though perhaps a "start" and "end" >> indicator would aid parsing - especially when nested VMOperations >> might occur. > > I agree. Adding some sort of end message and skipping the brackets > altogether seems clearer to me. > >> >>>>> >>>>> Why is the only logging level for this tag the "debug" level? I think >>>>> I may be missing part of the way UL works here - can you enable >>>>> logging both by explicit tag (ie vmoperation) and the level (ie >>>>> debug)? >>>>> >>> You enable the logging with "-Xlog:vmoperation=debug". If you leave of >>> the "=" portion, it is by default parsed as "=info". We don't >>> believe this vmoperation logging needs to come out by default in the >>> case where someone asks for "-Xlog:all", so we put it one level below. >> >> So does -Xlog:all signify all tags at info level or ??? > > Yes. > >> >> I think this is real conceptual problem with the UL framework. I >> should be able to define the info/trace/debug levels for a given tag, >> without having to think about how they interact with a "log all" request. > > I believe focus should be on classifying each log message according to > its component & type (tags) and verbosity & cost (level), and less about > which users will get it using what command line. > The log configuration is very flexible to allow users full control over > the logging they want, all we need to do on the development side is to > keep the logging logically classified and ordered so that the users can > easily select from it. > > Thanks, > Marcus > >> >>>>> And I know I sound like a broken record but I'm concerned about the >>>>> overhead of the logging actions when it is not enabled ie: >>>>> >>>>> outputStream* debugstream = LogHandle(vmoperation)::debug_stream(); >>>>> >>>>> always gets executed - and we evaluate is_enabled twice. >>>>> >>> See my reply to Max above. >>>>> --- >>>>> >>>>> test/runtime/logging/VMOperationTestMain.java >>>>> >>>>> Can you add a comment explaining what the logic is attempting to do >>>>> please. I'm curious how many times the loop executes before you get a >>>>> safepoint-based GC (and how it varies for different GC's). >>>>> >>>>> Style nit: while( -> while ( >>> Yes, will fix. >>>>> >>>>>> A compatability request has been accepted with regard to this change. >>>>> >>>>> I'll record my objections again to the conversion of develop flags to >>>>> product. >>> Thanks for voicing your concern. The justification for this tag being >>> product is that it could be useful to helping diagnose user problems, it >>> does not slow performance when off, and it minimally increases the size >>> of the code. If any of these prove to be false in aggregated logging >>> options, we can move it to develop level during code freeze. >> >> You could make that argument for every piece of logging and end up >> with a very simple flat logging system. :) It is all very subjective >> of course. >> >> Thanks, >> David >> >>> Thanks, >>> Rachel >>>>> >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>>> Thank you very much, >>>>>> Rachel >>>> >>> > From jaroslav.bachorik at oracle.com Fri Nov 20 09:25:01 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Fri, 20 Nov 2015 10:25:01 +0100 Subject: 8143226 (M): Minor updates to Event Based tracing In-Reply-To: References: Message-ID: <564EE6ED.7020606@oracle.com> On 18.11.2015 14:30, Markus Gronlund wrote: > Greetings, > > Kindly asking for reviews for the following bug: > > https://bugs.openjdk.java.net/browse/JDK-8143226 > > Webrev: > > http://cr.openjdk.java.net/~mgronlun/8143226/webrev01/ > > Summary: > > > - Remove obsoleted files from make system > - Consolidate to a single Thread abstraction > - Expand relational keys to u8's to introduce uniqueness property > - Tracing hook to be notified on out of memory > - Tracing hook in early vm startup > - Event "starttime" to become the primary time stamp for the event > (previously it was "endtime") > > Thanks in advance Looks good! -JB- > > Markus > From jaroslav.bachorik at oracle.com Fri Nov 20 10:03:19 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Fri, 20 Nov 2015 11:03:19 +0100 Subject: RFR(XS): 8143235: Remove libjfr mapfile In-Reply-To: References: Message-ID: <564EEFE7.6030308@oracle.com> Looks good! -JB- On 18.11.2015 21:09, Markus Gronlund wrote: > Greetings, > > Kindly asking for reviews for the following: > > Bug: > > https://bugs.openjdk.java.net/browse/JDK-8143235 > > Summary: > > Remove libjfr mapfile (cleanup) + adds ?jdk.management.jfr? to boot modules > > Patch is attached and also unrolled here: > > # HG changeset patch > > # User mgronlun > > # Date 1447765226 -3600 > > # Tue Nov 17 14:00:26 2015 +0100 > > # Node ID 13e7746e92f9b9651c0f77ed04057f382cc9e477 > > # Parent 85c9ddeda1c3fe13ef64ce1005c09af8e0bf1f42 > > [mq]: jdk_src_open_remove_mapfile > > diff --git a/make/mapfiles/libjfr/mapfile-vers > b/make/mapfiles/libjfr/mapfile-vers > > deleted file mode 100644 > > --- a/make/mapfiles/libjfr/mapfile-vers > > +++ /dev/null > > @@ -1,45 +0,0 @@ > > -# > > -# Copyright (c) 2012, 2013, Oracle and/or its affiliates. All rights > reserved. > > -# ORACLE PROPRIETARY/CONFIDENTIAL. Use is subject to license terms. > > -# > > - > > -# Define library interface. > > - > > -SUNWprivate_1.1 { > > - global: > > - Java_oracle_jrockit_jfr_Process_getpid; > > - Java_oracle_jrockit_jfr_Timing_counterTime; > > - Java_oracle_jrockit_jfr_Timing_init; > > - Java_oracle_jrockit_jfr_Logger_output0; > > - Java_oracle_jrockit_jfr_JFR_isCommercialFeaturesUnlocked; > > - Java_oracle_jrockit_jfr_JFR_isStarted; > > - Java_oracle_jrockit_jfr_JFR_isSupportedInVM; > > - Java_oracle_jrockit_jfr_JFR_startFlightRecorder; > > - Java_oracle_jrockit_jfr_JFR_isDisabledOnCommandLine; > > - Java_oracle_jrockit_jfr_JFR_isEnabled; > > - Java_oracle_jrockit_jfr_VMJFR_options; > > - Java_oracle_jrockit_jfr_VMJFR_init; > > - Java_oracle_jrockit_jfr_VMJFR_addConstPool; > > - Java_oracle_jrockit_jfr_VMJFR_removeConstPool; > > - Java_oracle_jrockit_jfr_VMJFR_storeConstPool; > > - Java_oracle_jrockit_jfr_VMJFR_classID0; > > - Java_oracle_jrockit_jfr_VMJFR_stackTraceID; > > - Java_oracle_jrockit_jfr_VMJFR_threadID; > > - Java_oracle_jrockit_jfr_VMJFR_rotate; > > - Java_oracle_jrockit_jfr_VMJFR_shutdown; > > - Java_oracle_jrockit_jfr_VMJFR_start; > > - Java_oracle_jrockit_jfr_VMJFR_stop; > > - Java_oracle_jrockit_jfr_VMJFR_buffer; > > - Java_oracle_jrockit_jfr_VMJFR_flush; > > - Java_oracle_jrockit_jfr_VMJFR_write; > > - Java_oracle_jrockit_jfr_VMJFR_add; > > - Java_oracle_jrockit_jfr_VMJFR_remove; > > - Java_oracle_jrockit_jfr_VMJFR_setThreshold; > > - Java_oracle_jrockit_jfr_VMJFR_setPeriod; > > - Java_oracle_jrockit_jfr_VMJFR_getPeriod; > > - Java_oracle_jrockit_jfr_VMJFR_descriptors; > > - Java_oracle_jrockit_jfr_VMJFR_retransformClasses0; > > - JNI_OnLoad; > > - local: > > - *; > > -}; > > diff --git a/make/src/classes/build/tools/module/boot.modules > b/make/src/classes/build/tools/module/boot.modules > > --- a/make/src/classes/build/tools/module/boot.modules > > +++ b/make/src/classes/build/tools/module/boot.modules > > @@ -23,6 +23,7 @@ > > jdk.jfr > > jdk.management > > jdk.management.cmm > > +jdk.management.jfr > > jdk.management.resource > > jdk.naming.rmi > > jdk.sctp > From marcus.larsson at oracle.com Fri Nov 20 11:58:02 2015 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Fri, 20 Nov 2015 12:58:02 +0100 Subject: RFR: 8143229: Replace the develop level with develop macros in Unified Logging Message-ID: <564F0ACA.4050604@oracle.com> Hi, Please review the following patch to replace the previous 'develop' level with macros, in the unified logging framework. The macros are essentially regular log calls inside a #ifndef PRODUCT. In the product they do nothing / return false. This allows the regular log levels to be used for develop-only logging. Webrev: http://cr.openjdk.java.net/~mlarsson/8143229/webrev.00/ Issue: https://bugs.openjdk.java.net/browse/JDK-8143229 Thanks, Marcus From bengt.rutisson at oracle.com Fri Nov 20 12:09:30 2015 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Fri, 20 Nov 2015 13:09:30 +0100 Subject: RFR: 8143229: Replace the develop level with develop macros in Unified Logging In-Reply-To: <564F0ACA.4050604@oracle.com> References: <564F0ACA.4050604@oracle.com> Message-ID: <564F0D7A.8050501@oracle.com> Hi Marcus, On 2015-11-20 12:58, Marcus Larsson wrote: > Hi, > > Please review the following patch to replace the previous 'develop' > level with macros, in the unified logging framework. > The macros are essentially regular log calls inside a #ifndef PRODUCT. > In the product they do nothing / return false. > This allows the regular log levels to be used for develop-only logging. > > Webrev: > http://cr.openjdk.java.net/~mlarsson/8143229/webrev.00/ > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8143229 Looks good. Bengt > > Thanks, > Marcus From marcus.larsson at oracle.com Fri Nov 20 12:29:22 2015 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Fri, 20 Nov 2015 13:29:22 +0100 Subject: [PING] Re: RFR (XS): 8142483: Unified logging log instances cause warnings on windows when only static functions are used In-Reply-To: <5645AB4F.40907@oracle.com> References: <5645AB4F.40907@oracle.com> Message-ID: <564F1222.6000007@oracle.com> Need another review for this! Thanks, Marcus On 2015-11-13 10:20, Marcus Larsson wrote: > Hi, > > Please review the following small patch to avoid 'unused variable' > warnings on windows when only static functions are used on a log > instance. Using log instances like this saves us from specifying the > log tags for every function call, so it is something we should not > warn about. > > The patch adds an empty constructor, which means that at least one > non-static function is called for every log instance, and the warning > is avoided. > > Webrev: > http://cr.openjdk.java.net/~mlarsson/8142483/webrev.00/ > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8142483 > > Thanks, > Marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: From staffan.larsen at oracle.com Fri Nov 20 12:38:47 2015 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Fri, 20 Nov 2015 13:38:47 +0100 Subject: [PING] RFR (XS): 8142483: Unified logging log instances cause warnings on windows when only static functions are used In-Reply-To: <564F1222.6000007@oracle.com> References: <5645AB4F.40907@oracle.com> <564F1222.6000007@oracle.com> Message-ID: Looks good! Thanks, /Staffan > On 20 nov. 2015, at 13:29, Marcus Larsson wrote: > > Need another review for this! > > Thanks, > Marcus > > On 2015-11-13 10:20, Marcus Larsson wrote: >> Hi, >> >> Please review the following small patch to avoid 'unused variable' warnings on windows when only static functions are used on a log instance. Using log instances like this saves us from specifying the log tags for every function call, so it is something we should not warn about. >> >> The patch adds an empty constructor, which means that at least one non-static function is called for every log instance, and the warning is avoided. >> >> Webrev: >> http://cr.openjdk.java.net/~mlarsson/8142483/webrev.00/ >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8142483 >> >> Thanks, >> Marcus > -------------- next part -------------- An HTML attachment was scrubbed... URL: From marcus.larsson at oracle.com Fri Nov 20 14:22:52 2015 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Fri, 20 Nov 2015 15:22:52 +0100 Subject: RFR: 8143229: Replace the develop level with develop macros in Unified Logging In-Reply-To: <564F0D7A.8050501@oracle.com> References: <564F0ACA.4050604@oracle.com> <564F0D7A.8050501@oracle.com> Message-ID: <564F2CBC.6090200@oracle.com> Thank you for reviewing, Bengt! Marcus On 2015-11-20 13:09, Bengt Rutisson wrote: > > Hi Marcus, > > On 2015-11-20 12:58, Marcus Larsson wrote: >> Hi, >> >> Please review the following patch to replace the previous 'develop' >> level with macros, in the unified logging framework. >> The macros are essentially regular log calls inside a #ifndef >> PRODUCT. In the product they do nothing / return false. >> This allows the regular log levels to be used for develop-only logging. >> >> Webrev: >> http://cr.openjdk.java.net/~mlarsson/8143229/webrev.00/ >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8143229 > > Looks good. > > Bengt > >> >> Thanks, >> Marcus > From markus.gronlund at oracle.com Fri Nov 20 18:27:57 2015 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Fri, 20 Nov 2015 10:27:57 -0800 (PST) Subject: RFR(L): 8140485: Class load and creation clean up In-Reply-To: <564E98A8.40209@oracle.com> References: <8e840172-533d-4122-a231-5aeac6730445@default> <2540c70d-5d78-4276-a41e-4057fbfb5528@default> <564E98A8.40209@oracle.com> Message-ID: <1bee9d19-50f9-4602-9062-92793d6dc21f@default> Hi Coleen, Many, many thanks for taking on this large change. Thanks for the feedback, I have updated like this: ClassFactory::create(...) -> KlassFactory::create_from_stream(...) // "Klass" better signifies internal representation (metadata) compared to "Class" which has a more classfile contextual association. Fixed. // In addition, I have removed all overloaded create_from_stream() functions, reducing them to a single create_from_stream(). The caller will // have to pass NULL on unused parameters with this tradeoff. // renaming ClassFileParser::instance_klass(...) -> ClassFileParser::create_instance_klass(...) ClassFileParser::fill_instance(...) -> ClassFileParser::fill_instance_klass(...) // moved the parse_stream() and post_process_stream() into the ClassFileParser constructor (process() removed) - thanks! Here is an updated webrev: http://cr.openjdk.java.net/~mgronlun/8140485/unified/unified_review/webrev03/ Many thanks again Markus -----Original Message----- From: Coleen Phillimore Sent: den 20 november 2015 04:51 To: hotspot-runtime-dev at openjdk.java.net Subject: Re: RFR(L): 8140485: Class load and creation clean up Hi, More comments on classFileParser. It seems like the ClassFileParser::process is unnecessary as a function. I think the 3 lines should be imported into ClassFileParser::ClassFileParser and it makes sense there. > Summary: > > Reduce the visibility of CFP impl methods by internal linkage instead of external linkage. > > Comment: > > In my opinion, we should attempt to break the pattern of having private functions declared in header files when the private function does not need to reach inside "this". Yes, I agree and appreciate moving all of these functions static and internal in CFP. Coleen On 11/12/15 7:08 AM, Markus Gronlund wrote: > Hi again, > > I have reworked and simplified this clean up/refactoring suggestion: > > Updated high-level description: > http://cr.openjdk.java.net/~mgronlun/8140485/unified/unified_review/81 > 40485_updated.jpg > > Updated webrev: > http://cr.openjdk.java.net/~mgronlun/8140485/unified/unified_review/we > brev02/ > > Updated patch: > http://cr.openjdk.java.net/~mgronlun/8140485/unified/unified_review/81 > 40485_open_unified_updated.patch > > Thanks in advance > Markus > > > -----Original Message----- > From: Markus Gronlund > Sent: den 27 oktober 2015 13:21 > To: hotspot-runtime-dev at openjdk.java.net; > serviceability-dev at openjdk.net > Subject: RFR(L): 8140485: Class load and creation clean up > > Greetings, > > > > I have spent some time refactoring and cleaning up parts of the class load and creation sequence and code. > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8140485 > > Overview: > > There are quite a lot of changes here - hopefully it will not be a deterrent of taking this on- first of all, a lot of changes are mechanical and applicable to "form" more than content- for example, variable, parameter and member function "constness". Same applies to other "domino changes", for example when moving an included header from a .hpp file (maybe replacing with a fwd decl instead). > > > > I have tried to stage the changes for easier overview / review (including a high level diagram). The patches/webrevs are split into a patch series of 7 patches (there is also a unified patch). > > > > Use case: > > My initial use case for starting all of this clean up / refactoring work is a need to reuse mechanics of the class parsing and creation sequence. > > > > Cleanup / refactoring objectives: > > > > - Allow for class parsing implementation reuse > > - Decouple class parsing and Klass creation > > - Decouple JVMTI class file load hook from ClassFileParser > > - Add compiler assisted read-only intent in order to assist future refactoring attempts > > - "componentify" ClassFileParser / apply to Klass/IK/AK creations > > - Take advantage of some optimizations opportunities (constructors and underlying "calloc" memory) > > - Maintain class load performance > > > > High level diagram explaining the updated suggested sequence (high level): > > http://cr.openjdk.java.net/~mgronlun/8140485/8140485_Class_load_and_cr > eation_cleanup_high_level_sequence.jpg > > > > Split webrevs/patches for easier overview / review (do note that not all of these splits will compile individually - there is a unified patch link that includes everything, pls see below): > > > > Sequence: "first" (order of changes) > > Name: "8140485_1_unified entry point" > > Link: > http://cr.openjdk.java.net/~mgronlun/8140485/split/first/webrev01/ > > Summary: > > Split out JVMTI agent CFLH from CFP - moved to SD::create_vm_representation_prologue(). > > Channel all class loading via a unified entry point - > SD::create_vm_representation() > > Split out ClassPathEntry into own header. > > Creation of new class ClassPathFileStream (specialized ClassPathStream) to retain ClassLoaderExt::Context functionality in the refactored code. > > "Need verify" property is carried within the CFS itself. > > Comment: > > SystemDictionary::load_instance_class() will now have a call to > "load_class_from_classpath" - the relevance of this name might need to > be modified (modules etc?) > > > > Sequence: "second" > > Name: "8140485_2_read_only_class_file_stream" > > Link: > http://cr.openjdk.java.net/~mgronlun/8140485/split/second/webrev01/ > > Summary: > > The underlying bytestream (buffer) in a CFS representation should not be altered (advertently or inadvertently) once created, made read-only. > > Comment: > > Modifications should work with copies. > > > > Sequence: "third" > > Name: "8140485_3_componentify_class_file_parser" > > Link: > http://cr.openjdk.java.net/~mgronlun/8140485/split/third/webrev01/ > > Summary: > > In order to allow for code refactoring of CFP - stack allocated variables are modified into fields. The entire CFP acts as a generic RAII class for the allocated structures (note CHECK macros). > > I have not fulfilled refactoring of some of the longest methods since it's is hard to really understand the impact of the inlining I guess we are after here.. > > The performance work done so far indicated improvements in still passing parameters instead of reading the fields directly (so you might still see "fat" methods taking a lot of parameters even though the fields are set). > > > "Consts everywhere.." > > One of the real hard issues trying to refactor this code is the lack of expressed read-only intent - it's hard to determine (both on reading but more importantly when compiling) what can/should change. > > Yes, I have entered the "const" rabbit hole related to this code (in some cases it might even be termed gratuitous (event 'consting' function parameter pointers in some cases for example)) - hopefully this will help in upcoming attempts to refactor/simplify this code. > > > > Accessors - CFP becomes a data container to be used in constructors > (see change seven). In addition, you will now have a possibility to > fetch data from a parser without creating a fullblown Klass > > > > Comment: > > I think the code could be improved and more refactoring could be > accomplished by introducing a Metaspace allocation RAII class - this > would allow the functions to decouple even more that is currently > (might be next steps) > > > > Sequence: "fourth" > > Name: "8140485_4_hide_impl_details_class_file_parser" > > Link: > http://cr.openjdk.java.net/~mgronlun/8140485/split/fourth/webrev01/ > > Summary: > > Reduce the visibility of CFP impl methods by internal linkage instead of external linkage. > > Comment: > > In my opinion, we should attempt to break the pattern of having private functions declared in header files when the private function does not need to reach inside "this". > > (that is, it can use an external interface alt. might not need access > at all (helpers)) > > > > Sequence: "fifth" > > Name: "8140485_5_update_entry_point" > > Link: > http://cr.openjdk.java.net/~mgronlun/8140485/split/fifth/webrev01/ > > Summary: > > Update signatures. Remove the parameter passing where not needed > ("parsed_name" reference for example) > > > > Sequence: "sixth" > > Name: "8140485_6_types_const_includes_repercussions" > > Link: > http://cr.openjdk.java.net/~mgronlun/8140485/split/sixth/webrev01/ > > Summary: > > Rippling effects of stronger constness. > > Forward includes where can be applied. This revealed other code modules not having includes -added. > > Downcasting of const qualified types (applied Scott Meyers idiom of > dual const_cast additions (see Effective C++)) > > > > Sequence: "seventh" > > Name: "8140485_7_applied_use_in_constructors_and_some_optimizations" > > Link: > http://cr.openjdk.java.net/~mgronlun/8140485/split/seventh/webrev01/ > > Summary: > > Take advantage of above modifications to gain direct upshots in other areas. Use CFP as the data container when constructing Klass/IK/AK. > > Optimizations: Reduce/optimize Klass/InstanceKlass/ArrayKlass/ConstantPool constructors by using underlying invariant on "calloc" equivalent MetaspaceObj allocations. > > > > Unified change/patch: > > Webrev: http://cr.openjdk.java.net/~mgronlun/8140485/unified/webrev01/ > > Patch: > http://cr.openjdk.java.net/~mgronlun/8140485/unified/8140485_unified.p > atch > > Summary: > > Unified (folded) patch for easier application/imports, for > compilation/try out > > > > Testing: > > I have done targeted testing during development especially using the > "runtime/ParallelClassLoading" test suite in order to ensure > functional equivalency. JPRT testing. Performance tests (see below) > > > > Performance: > > Unstructured: > > I have used Java Flight Recorder (JFR), which envelops the class loading sequence with high resolution timers - I have used this information to ensure performance has been stable regarding the changes. > > > > Structured: > > Startup/footprint-benchmarks (Linux32, Linux64, Windows32, Windows64, > Solaris SPARC-T) > > > > It is hard to read anything decisive around these (the effects most > likely hides in noise), but at least determine that the changes do not > introduce anything statistically significant (+/-) (both time and > space) > > Thanks in advance > > Markus > > From coleen.phillimore at oracle.com Fri Nov 20 19:02:01 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 20 Nov 2015 14:02:01 -0500 Subject: RFR(L): 8140485: Class load and creation clean up In-Reply-To: <1bee9d19-50f9-4602-9062-92793d6dc21f@default> References: <8e840172-533d-4122-a231-5aeac6730445@default> <2540c70d-5d78-4276-a41e-4057fbfb5528@default> <564E98A8.40209@oracle.com> <1bee9d19-50f9-4602-9062-92793d6dc21f@default> Message-ID: <564F6E29.3090902@oracle.com> Hi Markus, I ran your n-1 change through my local tests and they all passed. The only thing left is I think adding comments would be good to KlassFactory.hpp/cpp. Thanks, Coleen On 11/20/15 1:27 PM, Markus Gronlund wrote: > Hi Coleen, > > Many, many thanks for taking on this large change. > > Thanks for the feedback, I have updated like this: > > ClassFactory::create(...) -> KlassFactory::create_from_stream(...) > > // "Klass" better signifies internal representation (metadata) compared to "Class" which has a more classfile contextual association. Fixed. > // In addition, I have removed all overloaded create_from_stream() functions, reducing them to a single create_from_stream(). The caller will > // have to pass NULL on unused parameters with this tradeoff. > > // renaming > ClassFileParser::instance_klass(...) -> ClassFileParser::create_instance_klass(...) > > ClassFileParser::fill_instance(...) -> ClassFileParser::fill_instance_klass(...) > > // moved the parse_stream() and post_process_stream() into the ClassFileParser constructor (process() removed) - thanks! > > Here is an updated webrev: > > http://cr.openjdk.java.net/~mgronlun/8140485/unified/unified_review/webrev03/ > > Many thanks again > Markus > > > -----Original Message----- > From: Coleen Phillimore > Sent: den 20 november 2015 04:51 > To: hotspot-runtime-dev at openjdk.java.net > Subject: Re: RFR(L): 8140485: Class load and creation clean up > > > Hi, More comments on classFileParser. > > It seems like the ClassFileParser::process is unnecessary as a > function. I think the 3 lines should be imported into > ClassFileParser::ClassFileParser and it makes sense there. > >> Summary: >> >> Reduce the visibility of CFP impl methods by internal linkage instead of external linkage. >> >> Comment: >> >> In my opinion, we should attempt to break the pattern of having private functions declared in header files when the private function does not need to reach inside "this". > Yes, I agree and appreciate moving all of these functions static and internal in CFP. > > Coleen > > > On 11/12/15 7:08 AM, Markus Gronlund wrote: >> Hi again, >> >> I have reworked and simplified this clean up/refactoring suggestion: >> >> Updated high-level description: >> http://cr.openjdk.java.net/~mgronlun/8140485/unified/unified_review/81 >> 40485_updated.jpg >> >> Updated webrev: >> http://cr.openjdk.java.net/~mgronlun/8140485/unified/unified_review/we >> brev02/ >> >> Updated patch: >> http://cr.openjdk.java.net/~mgronlun/8140485/unified/unified_review/81 >> 40485_open_unified_updated.patch >> >> Thanks in advance >> Markus >> >> >> -----Original Message----- >> From: Markus Gronlund >> Sent: den 27 oktober 2015 13:21 >> To: hotspot-runtime-dev at openjdk.java.net; >> serviceability-dev at openjdk.net >> Subject: RFR(L): 8140485: Class load and creation clean up >> >> Greetings, >> >> >> >> I have spent some time refactoring and cleaning up parts of the class load and creation sequence and code. >> >> >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8140485 >> >> Overview: >> >> There are quite a lot of changes here - hopefully it will not be a deterrent of taking this on- first of all, a lot of changes are mechanical and applicable to "form" more than content- for example, variable, parameter and member function "constness". Same applies to other "domino changes", for example when moving an included header from a .hpp file (maybe replacing with a fwd decl instead). >> >> >> >> I have tried to stage the changes for easier overview / review (including a high level diagram). The patches/webrevs are split into a patch series of 7 patches (there is also a unified patch). >> >> >> >> Use case: >> >> My initial use case for starting all of this clean up / refactoring work is a need to reuse mechanics of the class parsing and creation sequence. >> >> >> >> Cleanup / refactoring objectives: >> >> >> >> - Allow for class parsing implementation reuse >> >> - Decouple class parsing and Klass creation >> >> - Decouple JVMTI class file load hook from ClassFileParser >> >> - Add compiler assisted read-only intent in order to assist future refactoring attempts >> >> - "componentify" ClassFileParser / apply to Klass/IK/AK creations >> >> - Take advantage of some optimizations opportunities (constructors and underlying "calloc" memory) >> >> - Maintain class load performance >> >> >> >> High level diagram explaining the updated suggested sequence (high level): >> >> http://cr.openjdk.java.net/~mgronlun/8140485/8140485_Class_load_and_cr >> eation_cleanup_high_level_sequence.jpg >> >> >> >> Split webrevs/patches for easier overview / review (do note that not all of these splits will compile individually - there is a unified patch link that includes everything, pls see below): >> >> >> >> Sequence: "first" (order of changes) >> >> Name: "8140485_1_unified entry point" >> >> Link: >> http://cr.openjdk.java.net/~mgronlun/8140485/split/first/webrev01/ >> >> Summary: >> >> Split out JVMTI agent CFLH from CFP - moved to SD::create_vm_representation_prologue(). >> >> Channel all class loading via a unified entry point - >> SD::create_vm_representation() >> >> Split out ClassPathEntry into own header. >> >> Creation of new class ClassPathFileStream (specialized ClassPathStream) to retain ClassLoaderExt::Context functionality in the refactored code. >> >> "Need verify" property is carried within the CFS itself. >> >> Comment: >> >> SystemDictionary::load_instance_class() will now have a call to >> "load_class_from_classpath" - the relevance of this name might need to >> be modified (modules etc?) >> >> >> >> Sequence: "second" >> >> Name: "8140485_2_read_only_class_file_stream" >> >> Link: >> http://cr.openjdk.java.net/~mgronlun/8140485/split/second/webrev01/ >> >> Summary: >> >> The underlying bytestream (buffer) in a CFS representation should not be altered (advertently or inadvertently) once created, made read-only. >> >> Comment: >> >> Modifications should work with copies. >> >> >> >> Sequence: "third" >> >> Name: "8140485_3_componentify_class_file_parser" >> >> Link: >> http://cr.openjdk.java.net/~mgronlun/8140485/split/third/webrev01/ >> >> Summary: >> >> In order to allow for code refactoring of CFP - stack allocated variables are modified into fields. The entire CFP acts as a generic RAII class for the allocated structures (note CHECK macros). >> >> I have not fulfilled refactoring of some of the longest methods since it's is hard to really understand the impact of the inlining I guess we are after here.. >> >> The performance work done so far indicated improvements in still passing parameters instead of reading the fields directly (so you might still see "fat" methods taking a lot of parameters even though the fields are set). >> >> >> "Consts everywhere.." >> >> One of the real hard issues trying to refactor this code is the lack of expressed read-only intent - it's hard to determine (both on reading but more importantly when compiling) what can/should change. >> >> Yes, I have entered the "const" rabbit hole related to this code (in some cases it might even be termed gratuitous (event 'consting' function parameter pointers in some cases for example)) - hopefully this will help in upcoming attempts to refactor/simplify this code. >> >> >> >> Accessors - CFP becomes a data container to be used in constructors >> (see change seven). In addition, you will now have a possibility to >> fetch data from a parser without creating a fullblown Klass >> >> >> >> Comment: >> >> I think the code could be improved and more refactoring could be >> accomplished by introducing a Metaspace allocation RAII class - this >> would allow the functions to decouple even more that is currently >> (might be next steps) >> >> >> >> Sequence: "fourth" >> >> Name: "8140485_4_hide_impl_details_class_file_parser" >> >> Link: >> http://cr.openjdk.java.net/~mgronlun/8140485/split/fourth/webrev01/ >> >> Summary: >> >> Reduce the visibility of CFP impl methods by internal linkage instead of external linkage. >> >> Comment: >> >> In my opinion, we should attempt to break the pattern of having private functions declared in header files when the private function does not need to reach inside "this". >> >> (that is, it can use an external interface alt. might not need access >> at all (helpers)) >> >> >> >> Sequence: "fifth" >> >> Name: "8140485_5_update_entry_point" >> >> Link: >> http://cr.openjdk.java.net/~mgronlun/8140485/split/fifth/webrev01/ >> >> Summary: >> >> Update signatures. Remove the parameter passing where not needed >> ("parsed_name" reference for example) >> >> >> >> Sequence: "sixth" >> >> Name: "8140485_6_types_const_includes_repercussions" >> >> Link: >> http://cr.openjdk.java.net/~mgronlun/8140485/split/sixth/webrev01/ >> >> Summary: >> >> Rippling effects of stronger constness. >> >> Forward includes where can be applied. This revealed other code modules not having includes -added. >> >> Downcasting of const qualified types (applied Scott Meyers idiom of >> dual const_cast additions (see Effective C++)) >> >> >> >> Sequence: "seventh" >> >> Name: "8140485_7_applied_use_in_constructors_and_some_optimizations" >> >> Link: >> http://cr.openjdk.java.net/~mgronlun/8140485/split/seventh/webrev01/ >> >> Summary: >> >> Take advantage of above modifications to gain direct upshots in other areas. Use CFP as the data container when constructing Klass/IK/AK. >> >> Optimizations: Reduce/optimize Klass/InstanceKlass/ArrayKlass/ConstantPool constructors by using underlying invariant on "calloc" equivalent MetaspaceObj allocations. >> >> >> >> Unified change/patch: >> >> Webrev: http://cr.openjdk.java.net/~mgronlun/8140485/unified/webrev01/ >> >> Patch: >> http://cr.openjdk.java.net/~mgronlun/8140485/unified/8140485_unified.p >> atch >> >> Summary: >> >> Unified (folded) patch for easier application/imports, for >> compilation/try out >> >> >> >> Testing: >> >> I have done targeted testing during development especially using the >> "runtime/ParallelClassLoading" test suite in order to ensure >> functional equivalency. JPRT testing. Performance tests (see below) >> >> >> >> Performance: >> >> Unstructured: >> >> I have used Java Flight Recorder (JFR), which envelops the class loading sequence with high resolution timers - I have used this information to ensure performance has been stable regarding the changes. >> >> >> >> Structured: >> >> Startup/footprint-benchmarks (Linux32, Linux64, Windows32, Windows64, >> Solaris SPARC-T) >> >> >> >> It is hard to read anything decisive around these (the effects most >> likely hides in noise), but at least determine that the changes do not >> introduce anything statistically significant (+/-) (both time and >> space) >> >> Thanks in advance >> >> Markus >> >> From rachel.protacio at oracle.com Fri Nov 20 19:27:43 2015 From: rachel.protacio at oracle.com (Rachel Protacio) Date: Fri, 20 Nov 2015 14:27:43 -0500 Subject: RFR: 8143157: Convert TraceVMOperation to Unified Logging In-Reply-To: <564B8517.7080105@oracle.com> References: <564B8517.7080105@oracle.com> Message-ID: <564F742F.8010109@oracle.com> Thank you, David, Kirk, Marcus, Max, and Coleen, for all your comments. Updated webrev: http://cr.openjdk.java.net/~rprotacio/8143157.01/ Changes: - moved initialization of "outputStream* debugstream" to inside vmoperation conditional - added boolean for log_is_enabled conditional to save duplicate operation and avoid possibility that flag has been enabled/disabled between calls - removed "[" and "]" from the beginning and end delimiters of the logging output. Added "begin" and "end" as lead-in to log VMOperation thread info statements instead (could not insert in middle of string because print_on_error is used in other places).* - added memory heap limit to test (ProcessBuilder call in VMOperationTest) and added comment to VMOperationTestMain Thank you, Rachel *Sample output: $ java -Xlog:vmoperation=debug VMOperationTestMain [0.262s][debug ][vmoperation] begin VM_Operation (0x00007f01a029a700): G1CollectFull, mode: safepoint, requested by thread 0x00007f0198018000 [0.278s][debug ][vmoperation] end VM_Operation (0x00007f01a029a700): G1CollectFull, mode: safepoint, requested by thread 0x00007f0198018000 On 11/17/2015 2:50 PM, Rachel Protacio wrote: > Hi, > > Please review the following small logging enhancement. > > Summary: The former -XX:+TraceVMOperation flag is updated to the > unified logging framework and is now replaced with -Xlog:vmoperation > in product mode. > > Open webrev: http://cr.openjdk.java.net/~rprotacio/8143157/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8143157 > Testing: Passes jtreg, JPRT, RBT quick tests, and refworkload > performance tests. > > A compatability request has been accepted with regard to this change. > > Thank you very much, > Rachel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jini.george at oracle.com Mon Nov 23 02:35:44 2015 From: jini.george at oracle.com (Jini George) Date: Sun, 22 Nov 2015 18:35:44 -0800 (PST) Subject: RFR(S): JDK-7041183 Improve error handling in Improve error handling in src/share/vm/services/management.cpp In-Reply-To: <5636BF1D.8030903@oracle.com> References: <56334E21.4020500@oracle.com> <5636BF1D.8030903@oracle.com> Message-ID: <6b191613-77f7-4eaa-8152-dc3816834d7c@default> Hi David, Thanks much for the review and for the suggestions offline. I have addressed your comments and have a new webrev at: http://cr.openjdk.java.net/~dsamersoff/sponsorship/jingeorg/JDK-7041183/webrev.02/ (Thanks, Dmitry for hosting). Further comments are inlined below. > -----Original Message----- > > On 30/10/2015 9:01 PM, Dmitry Samersoff wrote: > > Everybody, > > > > (* On behalf of Jini George *) > > > > Please review the fix > > > > http://cr.openjdk.java.net/~dsamersoff/sponsorship/jingeorg/JDK- > 704118 > > 3/webrev.01/ > > If you throw IllegalArgumentException it should be obvious at the Java > level which argument is illegal and why. I can't tell at the Java level > how that can come about - these seem to be methods invoked on a > MemoryPoolMXBean - see for example: > > https://bugs.openjdk.java.net/browse/JDK-8025089 > > so the "illegal argument" would seem to be "this" ??? [[Jini]] This has been changed to the more valid IllegalStateException as you suggested. JDK-8025089 is not reproducible anymore and has been closed now. > I don't see any changes to the code that would currently throw Internal > Error ?? [[Jini]] I removed the code throwing the Internal Error from the upper layer now since this would have masked the exception thrown from the inner layer. > The asserts for pool!=NULL seem rather pointless as the method has been > changed to never return null. If you really want the assert add it to > the end of the method being called, rather than placing at all the call > sites. [[Jini]] I removed these asserts now. Thanks, Jini. From david.holmes at oracle.com Mon Nov 23 04:30:13 2015 From: david.holmes at oracle.com (David Holmes) Date: Mon, 23 Nov 2015 14:30:13 +1000 Subject: RFR: 8143157: Convert TraceVMOperation to Unified Logging In-Reply-To: <564F742F.8010109@oracle.com> References: <564B8517.7080105@oracle.com> <564F742F.8010109@oracle.com> Message-ID: <56529655.508@oracle.com> Hi Rachel, Code changes seem okay - thanks. We will continue discussion re info vs debug elsewhere. :) I think we will also have to address the problem of no-atomic logging of multi-part logging statements sometime as well. Not a new problem but perhaps exacerbated in UL and compounded by inclusion of this particular multi-part logging in product mode. Only issue with regard to the test is the occurrence of the "CollectFull" string - is that dependent on which GC is being used? I turned on TraceVMOperation with code that called System.gc() and only saw: [VM_Operation (0xf6773e40): GenCollectForAllocation, mode: safepoint, requested by thread 0xf660f400] Thanks, David On 21/11/2015 5:27 AM, Rachel Protacio wrote: > Thank you, David, Kirk, Marcus, Max, and Coleen, for all your comments. > > Updated webrev: http://cr.openjdk.java.net/~rprotacio/8143157.01/ > Changes: > - moved initialization of "outputStream* debugstream" to inside > vmoperation conditional > - added boolean for log_is_enabled conditional to save duplicate > operation and avoid possibility that flag has been enabled/disabled > between calls > - removed "[" and "]" from the beginning and end delimiters of the > logging output. Added "begin" and "end" as lead-in to log VMOperation > thread info statements instead (could not insert in middle of string > because print_on_error is used in other places).* > - added memory heap limit to test (ProcessBuilder call in > VMOperationTest) and added comment to VMOperationTestMain > > Thank you, > Rachel > > *Sample output: > > $ java -Xlog:vmoperation=debug VMOperationTestMain > [0.262s][debug ][vmoperation] begin VM_Operation > (0x00007f01a029a700): G1CollectFull, mode: safepoint, requested by > thread 0x00007f0198018000 > [0.278s][debug ][vmoperation] end VM_Operation > (0x00007f01a029a700): G1CollectFull, mode: safepoint, requested by > thread 0x00007f0198018000 > > > > On 11/17/2015 2:50 PM, Rachel Protacio wrote: >> Hi, >> >> Please review the following small logging enhancement. >> >> Summary: The former -XX:+TraceVMOperation flag is updated to the >> unified logging framework and is now replaced with -Xlog:vmoperation >> in product mode. >> >> Open webrev: http://cr.openjdk.java.net/~rprotacio/8143157/ >> Bug: https://bugs.openjdk.java.net/browse/JDK-8143157 >> Testing: Passes jtreg, JPRT, RBT quick tests, and refworkload >> performance tests. >> >> A compatability request has been accepted with regard to this change. >> >> Thank you very much, >> Rachel > From david.holmes at oracle.com Mon Nov 23 06:44:01 2015 From: david.holmes at oracle.com (David Holmes) Date: Mon, 23 Nov 2015 16:44:01 +1000 Subject: RFR(S): JDK-7041183 Improve error handling in Improve error handling in src/share/vm/services/management.cpp In-Reply-To: <6b191613-77f7-4eaa-8152-dc3816834d7c@default> References: <56334E21.4020500@oracle.com> <5636BF1D.8030903@oracle.com> <6b191613-77f7-4eaa-8152-dc3816834d7c@default> Message-ID: <5652B5B1.8010603@oracle.com> On 23/11/2015 12:35 PM, Jini George wrote: > Hi David, > > Thanks much for the review and for the suggestions offline. I have addressed your comments and have a new webrev at: > > http://cr.openjdk.java.net/~dsamersoff/sponsorship/jingeorg/JDK-7041183/webrev.02/ src/share/vm/services/management.cpp 424 THROW_MSG_(vmSymbols::java_lang_IllegalStateException(), 425 "Memory Pool should exist", 426 NULL); why did you use the three arg THROW_MSG_ instead of the two-arg THROW_MSG? 428 return (MemoryPool*) mpool; Cast is not needed. That aside I think we're still miscomunicating on this. Here's what I last said in our IM chat: (4:24:02 PM) david.holmes: Basic premise: MemoryPoolMXBeans should never be invalid. VM code returns NULL if they are found to be invalid. JMM library code converts NULL to InternalError because it should not happen. If it does it is a VM bug. You found a situation where they were invalid so the cause of that needs to be determined and either fixed, or else the specifications and code updated to reflect that there are circumstances where they can be invalid. --- Changing the inner layer to throw IllegalStateException does not change the basic presumption that these pools should never be invalid - in which case the InternalError seems far more appropriate. Otherwise, as I said, you would need to update the specification and the code to allow for this IllegalStateException - you can't just change the code. David ----- > (Thanks, Dmitry for hosting). Further comments are inlined below. > >> -----Original Message----- >> >> On 30/10/2015 9:01 PM, Dmitry Samersoff wrote: >>> Everybody, >>> >>> (* On behalf of Jini George *) >>> >>> Please review the fix >>> >>> http://cr.openjdk.java.net/~dsamersoff/sponsorship/jingeorg/JDK- >> 704118 >>> 3/webrev.01/ >> >> If you throw IllegalArgumentException it should be obvious at the Java >> level which argument is illegal and why. I can't tell at the Java level >> how that can come about - these seem to be methods invoked on a >> MemoryPoolMXBean - see for example: >> >> https://bugs.openjdk.java.net/browse/JDK-8025089 >> >> so the "illegal argument" would seem to be "this" ??? > > [[Jini]] This has been changed to the more valid IllegalStateException as you suggested. JDK-8025089 is not reproducible anymore and has been closed now. > >> I don't see any changes to the code that would currently throw Internal >> Error ?? > > [[Jini]] I removed the code throwing the Internal Error from the upper layer now since this would have masked the exception thrown from the inner layer. > >> The asserts for pool!=NULL seem rather pointless as the method has been >> changed to never return null. If you really want the assert add it to >> the end of the method being called, rather than placing at all the call >> sites. > [[Jini]] I removed these asserts now. > > Thanks, > Jini. > From amy.lu at oracle.com Mon Nov 23 07:10:23 2015 From: amy.lu at oracle.com (Amy Lu) Date: Mon, 23 Nov 2015 15:10:23 +0800 Subject: JDK 9 RFR of JDK-8143583: Several tests don't work with latest jtreg due to non-existing files in @build Message-ID: <5652BBDF.1090503@oracle.com> Below tests failed with latest nightly jtreg due to non-existing files in @build com/sun/management/HotSpotDiagnosticMXBean/DumpHeap.java sun/tools/jmap/BasicJMapTest.java com/sun/jdi/DoubleAgentTest.java com/sun/jdi/SuspendNoFlagTest.java Please review this patch to fix the typo in @build bug: https://bugs.openjdk.java.net/browse/JDK-8143583 webrev: http://cr.openjdk.java.net/~amlu/8143583/webrev.00/ Thanks, Amy --- old/test/com/sun/jdi/DoubleAgentTest.java 2015-11-23 13:36:04.000000000 +0800 +++ new/test/com/sun/jdi/DoubleAgentTest.java 2015-11-23 13:36:04.000000000 +0800 @@ -31,7 +31,7 @@ * * @library /lib/testlibrary * @modules java.management - * @build jdk.testlibarary.* + * @build jdk.testlibrary.* * @build DoubleAgentTest Exit0 * @run driver DoubleAgentTest */ --- old/test/com/sun/jdi/SuspendNoFlagTest.java 2015-11-23 13:36:06.000000000 +0800 +++ new/test/com/sun/jdi/SuspendNoFlagTest.java 2015-11-23 13:36:05.000000000 +0800 @@ -29,7 +29,7 @@ * @summary Test for JDWP: -agentlib:jdwp=suspend=n hanging * @library /lib/testlibrary * @modules java.management - * @build jdk.testlibarary.* + * @build jdk.testlibrary.* * @compile -g HelloWorld.java * @run driver SuspendNoFlagTest */ --- old/test/com/sun/management/HotSpotDiagnosticMXBean/DumpHeap.java 2015-11-23 13:36:07.000000000 +0800 +++ new/test/com/sun/management/HotSpotDiagnosticMXBean/DumpHeap.java 2015-11-23 13:36:07.000000000 +0800 @@ -41,9 +41,9 @@ * @library /test/lib/share/classes * @build jdk.testlibrary.* * @build jdk.test.lib.hprof.* - * @build jdk.test.lib.hprof.module.* + * @build jdk.test.lib.hprof.model.* * @build jdk.test.lib.hprof.parser.* - * @build jdk.test.lib.hprof.utils.* + * @build jdk.test.lib.hprof.util.* * @run main DumpHeap */ public class DumpHeap { --- old/test/sun/tools/jmap/BasicJMapTest.java 2015-11-23 13:36:08.000000000 +0800 +++ new/test/sun/tools/jmap/BasicJMapTest.java 2015-11-23 13:36:08.000000000 +0800 @@ -42,9 +42,9 @@ * @modules java.management * @build jdk.testlibrary.* * @build jdk.test.lib.hprof.* - * @build jdk.test.lib.hprof.module.* + * @build jdk.test.lib.hprof.model.* * @build jdk.test.lib.hprof.parser.* - * @build jdk.test.lib.hprof.utils.* + * @build jdk.test.lib.hprof.util.* * @run main/timeout=240 BasicJMapTest */ public class BasicJMapTest { From staffan.larsen at oracle.com Mon Nov 23 07:18:01 2015 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Mon, 23 Nov 2015 08:18:01 +0100 Subject: JDK 9 RFR of JDK-8143583: Several tests don't work with latest jtreg due to non-existing files in @build In-Reply-To: <5652BBDF.1090503@oracle.com> References: <5652BBDF.1090503@oracle.com> Message-ID: Looks good! Thanks, /Staffan > On 23 nov. 2015, at 08:10, Amy Lu wrote: > > Below tests failed with latest nightly jtreg due to non-existing files in @build > > com/sun/management/HotSpotDiagnosticMXBean/DumpHeap.java > sun/tools/jmap/BasicJMapTest.java > com/sun/jdi/DoubleAgentTest.java > com/sun/jdi/SuspendNoFlagTest.java > > Please review this patch to fix the typo in @build > > bug: https://bugs.openjdk.java.net/browse/JDK-8143583 > webrev: http://cr.openjdk.java.net/~amlu/8143583/webrev.00/ > > Thanks, > Amy > > > --- old/test/com/sun/jdi/DoubleAgentTest.java 2015-11-23 13:36:04.000000000 +0800 > +++ new/test/com/sun/jdi/DoubleAgentTest.java 2015-11-23 13:36:04.000000000 +0800 > @@ -31,7 +31,7 @@ > * > * @library /lib/testlibrary > * @modules java.management > - * @build jdk.testlibarary.* > + * @build jdk.testlibrary.* > * @build DoubleAgentTest Exit0 > * @run driver DoubleAgentTest > */ > --- old/test/com/sun/jdi/SuspendNoFlagTest.java 2015-11-23 13:36:06.000000000 +0800 > +++ new/test/com/sun/jdi/SuspendNoFlagTest.java 2015-11-23 13:36:05.000000000 +0800 > @@ -29,7 +29,7 @@ > * @summary Test for JDWP: -agentlib:jdwp=suspend=n hanging > * @library /lib/testlibrary > * @modules java.management > - * @build jdk.testlibarary.* > + * @build jdk.testlibrary.* > * @compile -g HelloWorld.java > * @run driver SuspendNoFlagTest > */ > --- old/test/com/sun/management/HotSpotDiagnosticMXBean/DumpHeap.java 2015-11-23 13:36:07.000000000 +0800 > +++ new/test/com/sun/management/HotSpotDiagnosticMXBean/DumpHeap.java 2015-11-23 13:36:07.000000000 +0800 > @@ -41,9 +41,9 @@ > * @library /test/lib/share/classes > * @build jdk.testlibrary.* > * @build jdk.test.lib.hprof.* > - * @build jdk.test.lib.hprof.module.* > + * @build jdk.test.lib.hprof.model.* > * @build jdk.test.lib.hprof.parser.* > - * @build jdk.test.lib.hprof.utils.* > + * @build jdk.test.lib.hprof.util.* > * @run main DumpHeap > */ > public class DumpHeap { > --- old/test/sun/tools/jmap/BasicJMapTest.java 2015-11-23 13:36:08.000000000 +0800 > +++ new/test/sun/tools/jmap/BasicJMapTest.java 2015-11-23 13:36:08.000000000 +0800 > @@ -42,9 +42,9 @@ > * @modules java.management > * @build jdk.testlibrary.* > * @build jdk.test.lib.hprof.* > - * @build jdk.test.lib.hprof.module.* > + * @build jdk.test.lib.hprof.model.* > * @build jdk.test.lib.hprof.parser.* > - * @build jdk.test.lib.hprof.utils.* > + * @build jdk.test.lib.hprof.util.* > * @run main/timeout=240 BasicJMapTest > */ > public class BasicJMapTest { > > > > From Alan.Bateman at oracle.com Mon Nov 23 07:47:32 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 23 Nov 2015 07:47:32 +0000 Subject: JDK 9 RFR of JDK-8143583: Several tests don't work with latest jtreg due to non-existing files in @build In-Reply-To: <5652BBDF.1090503@oracle.com> References: <5652BBDF.1090503@oracle.com> Message-ID: <5652C494.3090500@oracle.com> On 23/11/2015 07:10, Amy Lu wrote: > Below tests failed with latest nightly jtreg due to non-existing files > in @build > > com/sun/management/HotSpotDiagnosticMXBean/DumpHeap.java > sun/tools/jmap/BasicJMapTest.java > com/sun/jdi/DoubleAgentTest.java > com/sun/jdi/SuspendNoFlagTest.java > > Please review this patch to fix the typo in @build > > bug: https://bugs.openjdk.java.net/browse/JDK-8143583 > webrev: http://cr.openjdk.java.net/~amlu/8143583/webrev.00/ Looks good. -Alan From jini.george at oracle.com Mon Nov 23 08:03:14 2015 From: jini.george at oracle.com (Jini George) Date: Mon, 23 Nov 2015 00:03:14 -0800 (PST) Subject: RFR(S): JDK-7041183 Improve error handling in Improve error handling in src/share/vm/services/management.cpp In-Reply-To: <5652B5B1.8010603@oracle.com> References: <56334E21.4020500@oracle.com> <5636BF1D.8030903@oracle.com> <6b191613-77f7-4eaa-8152-dc3816834d7c@default> <5652B5B1.8010603@oracle.com> Message-ID: <3337766e-4bbb-4e65-a8a2-4a6691bb904d@default> Thank you David for the review. The bug report (JDK-7041183) mentions this: =======Begin bug report snippet ================================ JNIEXPORT jobject JNICALL Java_sun_management_MemoryPoolImpl_getUsage0 (JNIEnv *env, jobject pool) { jobject usage = jmm_interface->GetMemoryPoolUsage(env, pool); if (usage == NULL) { // Throw internal error since this implementation expects the // pool will never become invalid. JNU_ThrowInternalError(env, "Memory Pool not found"); } return usage; } This function as well as Java_sun_management_MemoryPoolImpl_getMemoryManagers0, Java_sun_management_MemoryPoolImpl_getPeakUsage0 should propagate the exception that may be returned by the jmm call rather than throw InternalError. =======End bug report snippet ================================== I could not find a scenario where the MemoryPool could not be found, due to some GC related fixes having gone in in the past. (Hence the closure of the related https://bugs.openjdk.java.net/browse/JDK-8025089 ). As you said, that is a scenario which shouldn?t exist. My changes made were to cater to having the exception thrown from the jmm layer being seen, as mentioned in the bug report (if in case, the MemoryPool is not found, maybe due to some corruption or some unforeseen bug which might get introduced later). Still, having said that, if you and others feel that it is better to retain the Internal Error, I can close the bug as no change. Thanks, Jini. > -----Original Message----- > From: David Holmes > Sent: Monday, November 23, 2015 12:14 PM > To: Jini George; serviceability-dev at openjdk.java.net; Dmitry Samersoff > Subject: Re: RFR(S): JDK-7041183 Improve error handling in Improve > error handling in src/share/vm/services/management.cpp > > On 23/11/2015 12:35 PM, Jini George wrote: > > Hi David, > > > > Thanks much for the review and for the suggestions offline. I have > addressed your comments and have a new webrev at: > > > > http://cr.openjdk.java.net/~dsamersoff/sponsorship/jingeorg/JDK- > 704118 > > 3/webrev.02/ > > src/share/vm/services/management.cpp > > 424 THROW_MSG_(vmSymbols::java_lang_IllegalStateException(), > 425 "Memory Pool should exist", > 426 NULL); > > why did you use the three arg THROW_MSG_ instead of the two-arg > THROW_MSG? > > 428 return (MemoryPool*) mpool; > > Cast is not needed. > > That aside I think we're still miscomunicating on this. Here's what I > last said in our IM chat: > > (4:24:02 PM) david.holmes: Basic premise: MemoryPoolMXBeans should > never be invalid. > VM code returns NULL if they are found to be invalid. > JMM library code converts NULL to InternalError because it should not > happen. If it does it is a VM bug. > You found a situation where they were invalid so the cause of that > needs to be determined and either fixed, or else the specifications and > code updated to reflect that there are circumstances where they can be > invalid. > > --- > > Changing the inner layer to throw IllegalStateException does not change > the basic presumption that these pools should never be invalid - in > which case the InternalError seems far more appropriate. Otherwise, as > I > said, you would need to update the specification and the code to allow > for this IllegalStateException - you can't just change the code. > > David > ----- > > > (Thanks, Dmitry for hosting). Further comments are inlined below. > > > >> -----Original Message----- > >> > >> On 30/10/2015 9:01 PM, Dmitry Samersoff wrote: > >>> Everybody, > >>> > >>> (* On behalf of Jini George *) > >>> > >>> Please review the fix > >>> > >>> http://cr.openjdk.java.net/~dsamersoff/sponsorship/jingeorg/JDK- > >> 704118 > >>> 3/webrev.01/ > >> > >> If you throw IllegalArgumentException it should be obvious at the > Java > >> level which argument is illegal and why. I can't tell at the Java > level > >> how that can come about - these seem to be methods invoked on a > >> MemoryPoolMXBean - see for example: > >> > >> https://bugs.openjdk.java.net/browse/JDK-8025089 > >> > >> so the "illegal argument" would seem to be "this" ??? > > > > [[Jini]] This has been changed to the more valid > IllegalStateException as you suggested. JDK-8025089 is not reproducible > anymore and has been closed now. > > > >> I don't see any changes to the code that would currently throw > Internal > >> Error ?? > > > > [[Jini]] I removed the code throwing the Internal Error from the > upper layer now since this would have masked the exception thrown from > the inner layer. > > > >> The asserts for pool!=NULL seem rather pointless as the method has > been > >> changed to never return null. If you really want the assert add it > to > >> the end of the method being called, rather than placing at all the > call > >> sites. > > [[Jini]] I removed these asserts now. > > > > Thanks, > > Jini. > > From marcus.larsson at oracle.com Mon Nov 23 08:32:21 2015 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Mon, 23 Nov 2015 09:32:21 +0100 Subject: RFR: 8143229: Replace the develop level with develop macros in Unified Logging In-Reply-To: <56529C87.8090204@oracle.com> References: <564F0ACA.4050604@oracle.com> <564F976F.3020103@oracle.com> <56529C87.8090204@oracle.com> Message-ID: <5652CF15.9060407@oracle.com> Hey all, You're right, keeping this on info level makes more sense. I'll fix this before pushing. Thanks for reviewing! Regards, Marcus On 2015-11-23 05:56, David Holmes wrote: > On 21/11/2015 7:58 AM, Coleen Phillimore wrote: >> >> Hi Marcus, >> >> This looks good, except I have one question: >> http://cr.openjdk.java.net/~mlarsson/8143229/webrev.00/src/share/vm/logging/logConfiguration.cpp.udiff.html >> >> >> >> Shouldn't this be log_develop_info(logging) not trace? Why did you pick >> trace? > > I agree it should be info. The availability of develop logging should > be reported as a "first-class" piece of information in a develop build. > > Otherwise seems okay - though I'm still trying to come to grips with > UL as a whole. :) > > Thanks, > David > > >> thanks, >> Coleen >> >> On 11/20/15 6:58 AM, Marcus Larsson wrote: >>> Hi, >>> >>> Please review the following patch to replace the previous 'develop' >>> level with macros, in the unified logging framework. >>> The macros are essentially regular log calls inside a #ifndef PRODUCT. >>> In the product they do nothing / return false. >>> This allows the regular log levels to be used for develop-only logging. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~mlarsson/8143229/webrev.00/ >>> >>> Issue: >>> https://bugs.openjdk.java.net/browse/JDK-8143229 >>> >>> Thanks, >>> Marcus >> From david.holmes at oracle.com Mon Nov 23 08:33:46 2015 From: david.holmes at oracle.com (David Holmes) Date: Mon, 23 Nov 2015 18:33:46 +1000 Subject: RFR(S): JDK-7041183 Improve error handling in Improve error handling in src/share/vm/services/management.cpp In-Reply-To: <3337766e-4bbb-4e65-a8a2-4a6691bb904d@default> References: <56334E21.4020500@oracle.com> <5636BF1D.8030903@oracle.com> <6b191613-77f7-4eaa-8152-dc3816834d7c@default> <5652B5B1.8010603@oracle.com> <3337766e-4bbb-4e65-a8a2-4a6691bb904d@default> Message-ID: <5652CF6A.1030000@oracle.com> On 23/11/2015 6:03 PM, Jini George wrote: > Thank you David for the review. > > The bug report (JDK-7041183) mentions this: > > > =======Begin bug report snippet ================================ > JNIEXPORT jobject JNICALL > Java_sun_management_MemoryPoolImpl_getUsage0 > (JNIEnv *env, jobject pool) > { > jobject usage = jmm_interface->GetMemoryPoolUsage(env, pool); > if (usage == NULL) { > // Throw internal error since this implementation expects the > // pool will never become invalid. > JNU_ThrowInternalError(env, "Memory Pool not found"); > } > return usage; > } > > This function as well as Java_sun_management_MemoryPoolImpl_getMemoryManagers0, Java_sun_management_MemoryPoolImpl_getPeakUsage0 should propagate the exception that may be returned by the jmm call rather than throw InternalError. > > =======End bug report snippet ================================== That assumes there are expected exceptions coming from the lower layer and was likely based on the fact that these InternalErrors were being thrown. But as I keep saying either it is a bug if the memory pools are invalid (in which case InternalError should be thrown) or we have to identify the circumstances under which they can be invalid and then update the spec to reflect that and to state what exceptions should be thrown. > I could not find a scenario where the MemoryPool could not be found, due to some GC related fixes having gone in in the past. (Hence the closure of the related https://bugs.openjdk.java.net/browse/JDK-8025089 ). As you said, that is a scenario which shouldn?t exist. My changes made were to cater to having the exception thrown from the jmm layer being seen, as mentioned in the bug report (if in case, the MemoryPool is not found, maybe due to some corruption or some unforeseen bug which might get introduced later). > > Still, having said that, if you and others feel that it is better to retain the Internal Error, I can close the bug as no change. If we have not identified legitimate reasons for the pool being invalid then I would close this as not a bug. Cheers, David ----- > Thanks, > Jini. > > >> -----Original Message----- >> From: David Holmes >> Sent: Monday, November 23, 2015 12:14 PM >> To: Jini George; serviceability-dev at openjdk.java.net; Dmitry Samersoff >> Subject: Re: RFR(S): JDK-7041183 Improve error handling in Improve >> error handling in src/share/vm/services/management.cpp >> >> On 23/11/2015 12:35 PM, Jini George wrote: >>> Hi David, >>> >>> Thanks much for the review and for the suggestions offline. I have >> addressed your comments and have a new webrev at: >>> >>> http://cr.openjdk.java.net/~dsamersoff/sponsorship/jingeorg/JDK- >> 704118 >>> 3/webrev.02/ >> >> src/share/vm/services/management.cpp >> >> 424 THROW_MSG_(vmSymbols::java_lang_IllegalStateException(), >> 425 "Memory Pool should exist", >> 426 NULL); >> >> why did you use the three arg THROW_MSG_ instead of the two-arg >> THROW_MSG? >> >> 428 return (MemoryPool*) mpool; >> >> Cast is not needed. >> >> That aside I think we're still miscomunicating on this. Here's what I >> last said in our IM chat: >> >> (4:24:02 PM) david.holmes: Basic premise: MemoryPoolMXBeans should >> never be invalid. >> VM code returns NULL if they are found to be invalid. >> JMM library code converts NULL to InternalError because it should not >> happen. If it does it is a VM bug. >> You found a situation where they were invalid so the cause of that >> needs to be determined and either fixed, or else the specifications and >> code updated to reflect that there are circumstances where they can be >> invalid. >> >> --- >> >> Changing the inner layer to throw IllegalStateException does not change >> the basic presumption that these pools should never be invalid - in >> which case the InternalError seems far more appropriate. Otherwise, as >> I >> said, you would need to update the specification and the code to allow >> for this IllegalStateException - you can't just change the code. >> >> David >> ----- >> >>> (Thanks, Dmitry for hosting). Further comments are inlined below. >>> >>>> -----Original Message----- >>>> >>>> On 30/10/2015 9:01 PM, Dmitry Samersoff wrote: >>>>> Everybody, >>>>> >>>>> (* On behalf of Jini George *) >>>>> >>>>> Please review the fix >>>>> >>>>> http://cr.openjdk.java.net/~dsamersoff/sponsorship/jingeorg/JDK- >>>> 704118 >>>>> 3/webrev.01/ >>>> >>>> If you throw IllegalArgumentException it should be obvious at the >> Java >>>> level which argument is illegal and why. I can't tell at the Java >> level >>>> how that can come about - these seem to be methods invoked on a >>>> MemoryPoolMXBean - see for example: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8025089 >>>> >>>> so the "illegal argument" would seem to be "this" ??? >>> >>> [[Jini]] This has been changed to the more valid >> IllegalStateException as you suggested. JDK-8025089 is not reproducible >> anymore and has been closed now. >>> >>>> I don't see any changes to the code that would currently throw >> Internal >>>> Error ?? >>> >>> [[Jini]] I removed the code throwing the Internal Error from the >> upper layer now since this would have masked the exception thrown from >> the inner layer. >>> >>>> The asserts for pool!=NULL seem rather pointless as the method has >> been >>>> changed to never return null. If you really want the assert add it >> to >>>> the end of the method being called, rather than placing at all the >> call >>>> sites. >>> [[Jini]] I removed these asserts now. >>> >>> Thanks, >>> Jini. >>> From serguei.spitsyn at oracle.com Mon Nov 23 10:00:55 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 23 Nov 2015 02:00:55 -0800 Subject: RFR(L): 8140485: Class load and creation clean up In-Reply-To: <564F6E29.3090902@oracle.com> References: <8e840172-533d-4122-a231-5aeac6730445@default> <2540c70d-5d78-4276-a41e-4057fbfb5528@default> <564E98A8.40209@oracle.com> <1bee9d19-50f9-4602-9062-92793d6dc21f@default> <564F6E29.3090902@oracle.com> Message-ID: <5652E3D7.9010800@oracle.com> Hi Markus, I'm reviewing this too. Thanks, Serguei On 11/20/15 11:02, Coleen Phillimore wrote: > > Hi Markus, > > I ran your n-1 change through my local tests and they all passed. > > The only thing left is I think adding comments would be good to > KlassFactory.hpp/cpp. > > Thanks, > Coleen > > On 11/20/15 1:27 PM, Markus Gronlund wrote: >> Hi Coleen, >> >> Many, many thanks for taking on this large change. >> >> Thanks for the feedback, I have updated like this: >> >> ClassFactory::create(...) -> KlassFactory::create_from_stream(...) >> >> // "Klass" better signifies internal representation (metadata) >> compared to "Class" which has a more classfile contextual >> association. Fixed. >> // In addition, I have removed all overloaded create_from_stream() >> functions, reducing them to a single create_from_stream(). The caller >> will >> // have to pass NULL on unused parameters with this tradeoff. >> >> // renaming >> ClassFileParser::instance_klass(...) -> >> ClassFileParser::create_instance_klass(...) >> >> ClassFileParser::fill_instance(...) -> >> ClassFileParser::fill_instance_klass(...) >> >> // moved the parse_stream() and post_process_stream() into the >> ClassFileParser constructor (process() removed) - thanks! >> >> Here is an updated webrev: >> >> http://cr.openjdk.java.net/~mgronlun/8140485/unified/unified_review/webrev03/ >> >> >> Many thanks again >> Markus >> >> >> -----Original Message----- >> From: Coleen Phillimore >> Sent: den 20 november 2015 04:51 >> To: hotspot-runtime-dev at openjdk.java.net >> Subject: Re: RFR(L): 8140485: Class load and creation clean up >> >> >> Hi, More comments on classFileParser. >> >> It seems like the ClassFileParser::process is unnecessary as a >> function. I think the 3 lines should be imported into >> ClassFileParser::ClassFileParser and it makes sense there. >> >>> Summary: >>> >>> Reduce the visibility of CFP impl methods by internal linkage >>> instead of external linkage. >>> >>> Comment: >>> >>> In my opinion, we should attempt to break the pattern of having >>> private functions declared in header files when the private function >>> does not need to reach inside "this". >> Yes, I agree and appreciate moving all of these functions static and >> internal in CFP. >> >> Coleen >> >> >> On 11/12/15 7:08 AM, Markus Gronlund wrote: >>> Hi again, >>> >>> I have reworked and simplified this clean up/refactoring suggestion: >>> >>> Updated high-level description: >>> http://cr.openjdk.java.net/~mgronlun/8140485/unified/unified_review/81 >>> 40485_updated.jpg >>> >>> Updated webrev: >>> http://cr.openjdk.java.net/~mgronlun/8140485/unified/unified_review/we >>> brev02/ >>> >>> Updated patch: >>> http://cr.openjdk.java.net/~mgronlun/8140485/unified/unified_review/81 >>> 40485_open_unified_updated.patch >>> >>> Thanks in advance >>> Markus >>> >>> >>> -----Original Message----- >>> From: Markus Gronlund >>> Sent: den 27 oktober 2015 13:21 >>> To: hotspot-runtime-dev at openjdk.java.net; >>> serviceability-dev at openjdk.net >>> Subject: RFR(L): 8140485: Class load and creation clean up >>> >>> Greetings, >>> >>> >>> I have spent some time refactoring and cleaning up parts of the >>> class load and creation sequence and code. >>> >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8140485 >>> >>> Overview: >>> >>> There are quite a lot of changes here - hopefully it will not be a >>> deterrent of taking this on- first of all, a lot of changes are >>> mechanical and applicable to "form" more than content- for example, >>> variable, parameter and member function "constness". Same applies to >>> other "domino changes", for example when moving an included header >>> from a .hpp file (maybe replacing with a fwd decl instead). >>> >>> >>> I have tried to stage the changes for easier overview / review >>> (including a high level diagram). The patches/webrevs are split into >>> a patch series of 7 patches (there is also a unified patch). >>> >>> >>> Use case: >>> >>> My initial use case for starting all of this clean up / refactoring >>> work is a need to reuse mechanics of the class parsing and creation >>> sequence. >>> >>> >>> Cleanup / refactoring objectives: >>> >>> >>> - Allow for class parsing implementation reuse >>> >>> - Decouple class parsing and Klass creation >>> >>> - Decouple JVMTI class file load hook from ClassFileParser >>> >>> - Add compiler assisted read-only intent in order to assist >>> future refactoring attempts >>> >>> - "componentify" ClassFileParser / apply to Klass/IK/AK >>> creations >>> >>> - Take advantage of some optimizations opportunities >>> (constructors and underlying "calloc" memory) >>> >>> - Maintain class load performance >>> >>> >>> High level diagram explaining the updated suggested sequence (high >>> level): >>> >>> http://cr.openjdk.java.net/~mgronlun/8140485/8140485_Class_load_and_cr >>> eation_cleanup_high_level_sequence.jpg >>> >>> >>> Split webrevs/patches for easier overview / review (do note that not >>> all of these splits will compile individually - there is a unified >>> patch link that includes everything, pls see below): >>> >>> >>> Sequence: "first" (order of changes) >>> >>> Name: "8140485_1_unified entry point" >>> >>> Link: >>> http://cr.openjdk.java.net/~mgronlun/8140485/split/first/webrev01/ >>> >>> Summary: >>> >>> Split out JVMTI agent CFLH from CFP - moved to >>> SD::create_vm_representation_prologue(). >>> >>> Channel all class loading via a unified entry point - >>> SD::create_vm_representation() >>> >>> Split out ClassPathEntry into own header. >>> >>> Creation of new class ClassPathFileStream (specialized >>> ClassPathStream) to retain ClassLoaderExt::Context functionality in >>> the refactored code. >>> >>> "Need verify" property is carried within the CFS itself. >>> >>> Comment: >>> >>> SystemDictionary::load_instance_class() will now have a call to >>> "load_class_from_classpath" - the relevance of this name might need to >>> be modified (modules etc?) >>> >>> >>> Sequence: "second" >>> >>> Name: "8140485_2_read_only_class_file_stream" >>> >>> Link: >>> http://cr.openjdk.java.net/~mgronlun/8140485/split/second/webrev01/ >>> >>> Summary: >>> >>> The underlying bytestream (buffer) in a CFS representation should >>> not be altered (advertently or inadvertently) once created, made >>> read-only. >>> >>> Comment: >>> >>> Modifications should work with copies. >>> >>> >>> Sequence: "third" >>> >>> Name: "8140485_3_componentify_class_file_parser" >>> >>> Link: >>> http://cr.openjdk.java.net/~mgronlun/8140485/split/third/webrev01/ >>> >>> Summary: >>> >>> In order to allow for code refactoring of CFP - stack allocated >>> variables are modified into fields. The entire CFP acts as a >>> generic RAII class for the allocated structures (note CHECK macros). >>> >>> I have not fulfilled refactoring of some of the longest methods >>> since it's is hard to really understand the impact of the inlining I >>> guess we are after here.. >>> >>> The performance work done so far indicated improvements in still >>> passing parameters instead of reading the fields directly (so you >>> might still see "fat" methods taking a lot of parameters even though >>> the fields are set). >>> >>> >>> "Consts everywhere.." >>> >>> One of the real hard issues trying to refactor this code is the lack >>> of expressed read-only intent - it's hard to determine (both on >>> reading but more importantly when compiling) what can/should change. >>> >>> Yes, I have entered the "const" rabbit hole related to this code (in >>> some cases it might even be termed gratuitous (event 'consting' >>> function parameter pointers in some cases for example)) - hopefully >>> this will help in upcoming attempts to refactor/simplify this code. >>> >>> >>> Accessors - CFP becomes a data container to be used in constructors >>> (see change seven). In addition, you will now have a possibility to >>> fetch data from a parser without creating a fullblown Klass >>> >>> >>> Comment: >>> >>> I think the code could be improved and more refactoring could be >>> accomplished by introducing a Metaspace allocation RAII class - this >>> would allow the functions to decouple even more that is currently >>> (might be next steps) >>> >>> >>> Sequence: "fourth" >>> >>> Name: "8140485_4_hide_impl_details_class_file_parser" >>> >>> Link: >>> http://cr.openjdk.java.net/~mgronlun/8140485/split/fourth/webrev01/ >>> >>> Summary: >>> >>> Reduce the visibility of CFP impl methods by internal linkage >>> instead of external linkage. >>> >>> Comment: >>> >>> In my opinion, we should attempt to break the pattern of having >>> private functions declared in header files when the private function >>> does not need to reach inside "this". >>> >>> (that is, it can use an external interface alt. might not need access >>> at all (helpers)) >>> >>> >>> Sequence: "fifth" >>> >>> Name: "8140485_5_update_entry_point" >>> >>> Link: >>> http://cr.openjdk.java.net/~mgronlun/8140485/split/fifth/webrev01/ >>> >>> Summary: >>> >>> Update signatures. Remove the parameter passing where not needed >>> ("parsed_name" reference for example) >>> >>> >>> Sequence: "sixth" >>> >>> Name: "8140485_6_types_const_includes_repercussions" >>> >>> Link: >>> http://cr.openjdk.java.net/~mgronlun/8140485/split/sixth/webrev01/ >>> >>> Summary: >>> >>> Rippling effects of stronger constness. >>> >>> Forward includes where can be applied. This revealed other code >>> modules not having includes -added. >>> >>> Downcasting of const qualified types (applied Scott Meyers idiom of >>> dual const_cast additions (see Effective C++)) >>> >>> >>> Sequence: "seventh" >>> >>> Name: "8140485_7_applied_use_in_constructors_and_some_optimizations" >>> >>> Link: >>> http://cr.openjdk.java.net/~mgronlun/8140485/split/seventh/webrev01/ >>> >>> Summary: >>> >>> Take advantage of above modifications to gain direct upshots in >>> other areas. Use CFP as the data container when constructing >>> Klass/IK/AK. >>> >>> Optimizations: Reduce/optimize >>> Klass/InstanceKlass/ArrayKlass/ConstantPool constructors by using >>> underlying invariant on "calloc" equivalent MetaspaceObj allocations. >>> >>> >>> Unified change/patch: >>> >>> Webrev: http://cr.openjdk.java.net/~mgronlun/8140485/unified/webrev01/ >>> >>> Patch: >>> http://cr.openjdk.java.net/~mgronlun/8140485/unified/8140485_unified.p >>> atch >>> >>> Summary: >>> >>> Unified (folded) patch for easier application/imports, for >>> compilation/try out >>> >>> >>> Testing: >>> >>> I have done targeted testing during development especially using the >>> "runtime/ParallelClassLoading" test suite in order to ensure >>> functional equivalency. JPRT testing. Performance tests (see below) >>> >>> >>> Performance: >>> >>> Unstructured: >>> >>> I have used Java Flight Recorder (JFR), which envelops the class >>> loading sequence with high resolution timers - I have used this >>> information to ensure performance has been stable regarding the >>> changes. >>> >>> >>> Structured: >>> >>> Startup/footprint-benchmarks (Linux32, Linux64, Windows32, Windows64, >>> Solaris SPARC-T) >>> >>> >>> It is hard to read anything decisive around these (the effects most >>> likely hides in noise), but at least determine that the changes do not >>> introduce anything statistically significant (+/-) (both time and >>> space) >>> >>> Thanks in advance >>> >>> Markus >>> > From alexander.kulyakhtin at oracle.com Mon Nov 23 11:19:41 2015 From: alexander.kulyakhtin at oracle.com (Alexander Kulyakhtin) Date: Mon, 23 Nov 2015 03:19:41 -0800 (PST) Subject: RFR: 8143121: javax/management/remote/mandatory/loading/MethodResultTest.java fails intermittently Message-ID: <433d55f4-a441-4a78-8556-b2770983324d@default> Hi, Could you, please, review this small, test-only, change: CR: https://bugs.openjdk.java.net/browse/JDK-8143121 Webrev: http://cr.openjdk.java.net/~akulyakh/8143121/test/javax/management/remote/mandatory/loading/MethodResultTest.java.udiff.html The precondition for this test is a JMXConnector having established a successful connection. On some environments the test sometimes can not connect at the first attempt, because the target application is not yet ready. We are changing the test so that it tries to connect again if it gets IOException during the connection. Best regards, Alexander From daniel.fuchs at oracle.com Mon Nov 23 11:32:29 2015 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Mon, 23 Nov 2015 12:32:29 +0100 Subject: RFR: 8143121: javax/management/remote/mandatory/loading/MethodResultTest.java fails intermittently In-Reply-To: <433d55f4-a441-4a78-8556-b2770983324d@default> References: <433d55f4-a441-4a78-8556-b2770983324d@default> Message-ID: <5652F94D.8080109@oracle.com> Hi Alexander, This looks a bit dangerous to me - it could create a busy loop if for some reason the connection can never go through. I would suggest retrying only once (or retrying a fixed number of times) - and possibly introducing a small delay (Thread.sleep) before retrying. best regards, -- daniel On 23/11/15 12:19, Alexander Kulyakhtin wrote: > Hi, > > Could you, please, review this small, test-only, change: > > CR: https://bugs.openjdk.java.net/browse/JDK-8143121 > Webrev: http://cr.openjdk.java.net/~akulyakh/8143121/test/javax/management/remote/mandatory/loading/MethodResultTest.java.udiff.html > > The precondition for this test is a JMXConnector having established a successful connection. > On some environments the test sometimes can not connect at the first attempt, because the target application is not yet ready. > We are changing the test so that it tries to connect again if it gets IOException during the connection. > > Best regards, > Alexander > From shanliang.jiang at oracle.com Mon Nov 23 12:05:10 2015 From: shanliang.jiang at oracle.com (Shanliang JIANG) Date: Mon, 23 Nov 2015 13:05:10 +0100 Subject: RFR: 8143121: javax/management/remote/mandatory/loading/MethodResultTest.java fails intermittently In-Reply-To: <5652F94D.8080109@oracle.com> References: <433d55f4-a441-4a78-8556-b2770983324d@default> <5652F94D.8080109@oracle.com> Message-ID: We could repeat retrying until the test harness timeout, but add a sleep is definitively useful between retrying. Shanliang > On 23 Nov 2015, at 12:32, Daniel Fuchs wrote: > > Hi Alexander, > > This looks a bit dangerous to me - it could create a busy loop > if for some reason the connection can never go through. > > I would suggest retrying only once (or retrying a fixed number > of times) - and possibly introducing a small delay (Thread.sleep) > before retrying. > > best regards, > > -- daniel > > On 23/11/15 12:19, Alexander Kulyakhtin wrote: >> Hi, >> >> Could you, please, review this small, test-only, change: >> >> CR: https://bugs.openjdk.java.net/browse/JDK-8143121 >> Webrev: http://cr.openjdk.java.net/~akulyakh/8143121/test/javax/management/remote/mandatory/loading/MethodResultTest.java.udiff.html >> >> The precondition for this test is a JMXConnector having established a successful connection. >> On some environments the test sometimes can not connect at the first attempt, because the target application is not yet ready. >> We are changing the test so that it tries to connect again if it gets IOException during the connection. >> >> Best regards, >> Alexander >> > From alexander.kulyakhtin at oracle.com Mon Nov 23 12:09:17 2015 From: alexander.kulyakhtin at oracle.com (Alexander Kulyakhtin) Date: Mon, 23 Nov 2015 04:09:17 -0800 (PST) Subject: RFR: 8143121: javax/management/remote/mandatory/loading/MethodResultTest.java fails intermittently Message-ID: <1b0b203d-8eb5-4263-9916-028a6f1cf7ea@default> Hi Daniel, Thank you very much for the review. I presumed that in such case the jtreg will terminate the test by timeout and so I wanted to avoid any specific delays and counters. If it's not safe to presume that, then I'm going to limit the number of retries to 5 times introducing a delay of 3 seconds between the reties, or I can use any other values if there are more preferable values. Best regards, Alexander ----- Original Message ----- From: daniel.fuchs at oracle.com To: alexander.kulyakhtin at oracle.com, serviceability-dev at openjdk.java.net Sent: Monday, November 23, 2015 2:32:27 PM GMT +03:00 Iraq Subject: Re: RFR: 8143121: javax/management/remote/mandatory/loading/MethodResultTest.java fails intermittently Hi Alexander, This looks a bit dangerous to me - it could create a busy loop if for some reason the connection can never go through. I would suggest retrying only once (or retrying a fixed number of times) - and possibly introducing a small delay (Thread.sleep) before retrying. best regards, -- daniel On 23/11/15 12:19, Alexander Kulyakhtin wrote: > Hi, > > Could you, please, review this small, test-only, change: > > CR: https://bugs.openjdk.java.net/browse/JDK-8143121 > Webrev: http://cr.openjdk.java.net/~akulyakh/8143121/test/javax/management/remote/mandatory/loading/MethodResultTest.java.udiff.html > > The precondition for this test is a JMXConnector having established a successful connection. > On some environments the test sometimes can not connect at the first attempt, because the target application is not yet ready. > We are changing the test so that it tries to connect again if it gets IOException during the connection. > > Best regards, > Alexander > From serguei.spitsyn at oracle.com Mon Nov 23 12:48:05 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 23 Nov 2015 04:48:05 -0800 Subject: RFR(L): 8140485: Class load and creation clean up In-Reply-To: <564F6E29.3090902@oracle.com> References: <8e840172-533d-4122-a231-5aeac6730445@default> <2540c70d-5d78-4276-a41e-4057fbfb5528@default> <564E98A8.40209@oracle.com> <1bee9d19-50f9-4602-9062-92793d6dc21f@default> <564F6E29.3090902@oracle.com> Message-ID: <56530B05.9000301@oracle.com> Hi Markus, These are the comments I have at the moment. webrev03/src/share/vm/oops/constantPool.cpp 67 static bool tag_array_is_zero_initialized(Array* tags) { 68 assert(tags != NULL, "invariant"); 69 const int length = tags->length(); 70 for (int index = 0; index < length; index++) { 71 if (JVM_CONSTANT_Invalid != tags->at(index)) { 72 return false; 73 } 74 } 75 return true; 76 } 77 78 #endif 79 80 ConstantPool::ConstantPool(Array* tags) : 81 _length(tags->length()), 82 _flags(0) { 83 assert(tags != NULL, "invariant"); 84 assert(tags->length() == _length, "invariant"); 85 assert(tag_array_is_zero_initialized(tags), "invariant"); 86 set_tags(tags); 87 88 // only set to non-zero if constant pool is merged by RedefineClasses 89 set_version(0); 90 91 assert(NULL == _cache, "invariant"); 92 assert(NULL == _reference_map, "invariant"); 93 assert(NULL == _resolved_references, "invariant"); 94 assert(NULL == _operands, "invariant"); 95 assert(NULL == _pool_holder, "invariant"); 96 // maybe its enough to just check one of the above 97 // fields to verify underlying calloc allocation 98 } Both checks at the lines 83 ans 69 are late as tags is dereferenced at the line 81. Would it better to remove the line 81 and initialize the _length in the 'tag_array_..' after the check at 68? I'd also suggest to follow the comment at 96-97: to leave just one of the asserts 91-95 and remove the comment. webrev03/src/share/vm/oops/arrayKlass.hpp 93 // Casting from Klass* 94 static const ArrayKlass* cast(const Klass* k) { 95 assert(k->is_array_klass(), "cast to ArrayKlass"); 96 return static_cast(k); 97 } Minor: The comment at line 93 is not fully correct as the casting is from 'const Klass*'. The same issue is in some other updated files: instanceKlass.hpp, objArrayKlass.hpp, typeArrayKlass.hpp webrev03/src/share/vm/runtime/vmStructs.cpp Minor: Line 323: Unaligned '\' at the end. Thanks, Serguei On 11/20/15 11:02, Coleen Phillimore wrote: > > Hi Markus, > > I ran your n-1 change through my local tests and they all passed. > > The only thing left is I think adding comments would be good to > KlassFactory.hpp/cpp. > > Thanks, > Coleen > > On 11/20/15 1:27 PM, Markus Gronlund wrote: >> Hi Coleen, >> >> Many, many thanks for taking on this large change. >> >> Thanks for the feedback, I have updated like this: >> >> ClassFactory::create(...) -> KlassFactory::create_from_stream(...) >> >> // "Klass" better signifies internal representation (metadata) >> compared to "Class" which has a more classfile contextual >> association. Fixed. >> // In addition, I have removed all overloaded create_from_stream() >> functions, reducing them to a single create_from_stream(). The caller >> will >> // have to pass NULL on unused parameters with this tradeoff. >> >> // renaming >> ClassFileParser::instance_klass(...) -> >> ClassFileParser::create_instance_klass(...) >> >> ClassFileParser::fill_instance(...) -> >> ClassFileParser::fill_instance_klass(...) >> >> // moved the parse_stream() and post_process_stream() into the >> ClassFileParser constructor (process() removed) - thanks! >> >> Here is an updated webrev: >> >> http://cr.openjdk.java.net/~mgronlun/8140485/unified/unified_review/webrev03/ >> >> >> Many thanks again >> Markus >> >> >> -----Original Message----- >> From: Coleen Phillimore >> Sent: den 20 november 2015 04:51 >> To: hotspot-runtime-dev at openjdk.java.net >> Subject: Re: RFR(L): 8140485: Class load and creation clean up >> >> >> Hi, More comments on classFileParser. >> >> It seems like the ClassFileParser::process is unnecessary as a >> function. I think the 3 lines should be imported into >> ClassFileParser::ClassFileParser and it makes sense there. >> >>> Summary: >>> >>> Reduce the visibility of CFP impl methods by internal linkage >>> instead of external linkage. >>> >>> Comment: >>> >>> In my opinion, we should attempt to break the pattern of having >>> private functions declared in header files when the private function >>> does not need to reach inside "this". >> Yes, I agree and appreciate moving all of these functions static and >> internal in CFP. >> >> Coleen >> >> >> On 11/12/15 7:08 AM, Markus Gronlund wrote: >>> Hi again, >>> >>> I have reworked and simplified this clean up/refactoring suggestion: >>> >>> Updated high-level description: >>> http://cr.openjdk.java.net/~mgronlun/8140485/unified/unified_review/81 >>> 40485_updated.jpg >>> >>> Updated webrev: >>> http://cr.openjdk.java.net/~mgronlun/8140485/unified/unified_review/we >>> brev02/ >>> >>> Updated patch: >>> http://cr.openjdk.java.net/~mgronlun/8140485/unified/unified_review/81 >>> 40485_open_unified_updated.patch >>> >>> Thanks in advance >>> Markus >>> >>> >>> -----Original Message----- >>> From: Markus Gronlund >>> Sent: den 27 oktober 2015 13:21 >>> To: hotspot-runtime-dev at openjdk.java.net; >>> serviceability-dev at openjdk.net >>> Subject: RFR(L): 8140485: Class load and creation clean up >>> >>> Greetings, >>> >>> >>> I have spent some time refactoring and cleaning up parts of the >>> class load and creation sequence and code. >>> >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8140485 >>> >>> Overview: >>> >>> There are quite a lot of changes here - hopefully it will not be a >>> deterrent of taking this on- first of all, a lot of changes are >>> mechanical and applicable to "form" more than content- for example, >>> variable, parameter and member function "constness". Same applies to >>> other "domino changes", for example when moving an included header >>> from a .hpp file (maybe replacing with a fwd decl instead). >>> >>> >>> I have tried to stage the changes for easier overview / review >>> (including a high level diagram). The patches/webrevs are split into >>> a patch series of 7 patches (there is also a unified patch). >>> >>> >>> Use case: >>> >>> My initial use case for starting all of this clean up / refactoring >>> work is a need to reuse mechanics of the class parsing and creation >>> sequence. >>> >>> >>> Cleanup / refactoring objectives: >>> >>> >>> - Allow for class parsing implementation reuse >>> >>> - Decouple class parsing and Klass creation >>> >>> - Decouple JVMTI class file load hook from ClassFileParser >>> >>> - Add compiler assisted read-only intent in order to assist >>> future refactoring attempts >>> >>> - "componentify" ClassFileParser / apply to Klass/IK/AK >>> creations >>> >>> - Take advantage of some optimizations opportunities >>> (constructors and underlying "calloc" memory) >>> >>> - Maintain class load performance >>> >>> >>> High level diagram explaining the updated suggested sequence (high >>> level): >>> >>> http://cr.openjdk.java.net/~mgronlun/8140485/8140485_Class_load_and_cr >>> eation_cleanup_high_level_sequence.jpg >>> >>> >>> Split webrevs/patches for easier overview / review (do note that not >>> all of these splits will compile individually - there is a unified >>> patch link that includes everything, pls see below): >>> >>> >>> Sequence: "first" (order of changes) >>> >>> Name: "8140485_1_unified entry point" >>> >>> Link: >>> http://cr.openjdk.java.net/~mgronlun/8140485/split/first/webrev01/ >>> >>> Summary: >>> >>> Split out JVMTI agent CFLH from CFP - moved to >>> SD::create_vm_representation_prologue(). >>> >>> Channel all class loading via a unified entry point - >>> SD::create_vm_representation() >>> >>> Split out ClassPathEntry into own header. >>> >>> Creation of new class ClassPathFileStream (specialized >>> ClassPathStream) to retain ClassLoaderExt::Context functionality in >>> the refactored code. >>> >>> "Need verify" property is carried within the CFS itself. >>> >>> Comment: >>> >>> SystemDictionary::load_instance_class() will now have a call to >>> "load_class_from_classpath" - the relevance of this name might need to >>> be modified (modules etc?) >>> >>> >>> Sequence: "second" >>> >>> Name: "8140485_2_read_only_class_file_stream" >>> >>> Link: >>> http://cr.openjdk.java.net/~mgronlun/8140485/split/second/webrev01/ >>> >>> Summary: >>> >>> The underlying bytestream (buffer) in a CFS representation should >>> not be altered (advertently or inadvertently) once created, made >>> read-only. >>> >>> Comment: >>> >>> Modifications should work with copies. >>> >>> >>> Sequence: "third" >>> >>> Name: "8140485_3_componentify_class_file_parser" >>> >>> Link: >>> http://cr.openjdk.java.net/~mgronlun/8140485/split/third/webrev01/ >>> >>> Summary: >>> >>> In order to allow for code refactoring of CFP - stack allocated >>> variables are modified into fields. The entire CFP acts as a >>> generic RAII class for the allocated structures (note CHECK macros). >>> >>> I have not fulfilled refactoring of some of the longest methods >>> since it's is hard to really understand the impact of the inlining I >>> guess we are after here.. >>> >>> The performance work done so far indicated improvements in still >>> passing parameters instead of reading the fields directly (so you >>> might still see "fat" methods taking a lot of parameters even though >>> the fields are set). >>> >>> >>> "Consts everywhere.." >>> >>> One of the real hard issues trying to refactor this code is the lack >>> of expressed read-only intent - it's hard to determine (both on >>> reading but more importantly when compiling) what can/should change. >>> >>> Yes, I have entered the "const" rabbit hole related to this code (in >>> some cases it might even be termed gratuitous (event 'consting' >>> function parameter pointers in some cases for example)) - hopefully >>> this will help in upcoming attempts to refactor/simplify this code. >>> >>> >>> Accessors - CFP becomes a data container to be used in constructors >>> (see change seven). In addition, you will now have a possibility to >>> fetch data from a parser without creating a fullblown Klass >>> >>> >>> Comment: >>> >>> I think the code could be improved and more refactoring could be >>> accomplished by introducing a Metaspace allocation RAII class - this >>> would allow the functions to decouple even more that is currently >>> (might be next steps) >>> >>> >>> Sequence: "fourth" >>> >>> Name: "8140485_4_hide_impl_details_class_file_parser" >>> >>> Link: >>> http://cr.openjdk.java.net/~mgronlun/8140485/split/fourth/webrev01/ >>> >>> Summary: >>> >>> Reduce the visibility of CFP impl methods by internal linkage >>> instead of external linkage. >>> >>> Comment: >>> >>> In my opinion, we should attempt to break the pattern of having >>> private functions declared in header files when the