From david.holmes at oracle.com Fri Mar 1 01:07:28 2019 From: david.holmes at oracle.com (David Holmes) Date: Fri, 1 Mar 2019 11:07:28 +1000 Subject: RFR(S): 8214854: JDWP: Unforseen output truncation in logging In-Reply-To: <6b080c3a-27c9-fff5-147c-c5a3baa44c5c@bell-sw.com> References: <6b080c3a-27c9-fff5-147c-c5a3baa44c5c@bell-sw.com> Message-ID: <831d28ed-1fd8-90d3-1090-b434e5a6eeb2@oracle.com> Hi Dmitry, On 1/03/2019 6:15 am, Dmitry Chuyko wrote: > Hello, > > Please review a small fix for GCC 8.x warning in log_messages.c. Buffer > sizes for parts of timestamp string can be adjusted to not exceed > maximum buffer size when combined, while being able to hold necessary > information. > > webrev: http://cr.openjdk.java.net/~dchuyko/8214854/webrev.00/ > bug: https://bugs.openjdk.java.net/browse/JDK-8214854 > testing: after the fix GCC 8 compiles working OpenJDK on Linux. > Dev-submit job started. This looks good. One query: 51 typedef char timestamp_buffer[TIMESTAMP_SIZE]; Is this needed so that the size of the buffer is known by gcc inside get_time_stamp? (Generally I prefer to see explicitly when an array is being stack allocated, not have it hidden behind a typedef.) Thanks, David > -Dmitry > From zanglin5 at jd.com Fri Mar 1 03:33:39 2019 From: zanglin5 at jd.com (=?iso-2022-jp?B?GyRCZ0lOVhsoQg==?=) Date: Fri, 1 Mar 2019 03:33:39 +0000 Subject: Protocol version of Attach API In-Reply-To: <7965FB48-52AE-46C1-964A-9FDF78B0D644@amazon.com> References: <3dabf14d-e0bd-1ed1-4e37-f81332c1d68b@gmail.com> <01c706d8-600d-d823-6ebc-2d7e153c305f@oracle.com> <04aadf26-d2c9-6f65-9815-06ad36c53219@gmail.com>

<4a5f251d-67ef-a81a-ef73-48f9aa9c853e@gmail.com> <0e7c23fec044419d845c5608b0586fcb@jd.com> <3dab08619bbd4254943361d2cdc1298b@jd.com>

<077ca61fe91f46ef93c4472accdf0337@jd.com> <8f6ec4c6-18f1-4f61-7ab0-6e1b40acf544@oracle.com> <22427d1123114a0d869eaade059fcd12@jd.com> <66511977ba394d7084d94a356f68f4b7@jd.com> <33bb67d77da949ab9410ea3cdb5dd1ce@jd.com>, <650f2814-9bb6-39c1-87ab-583a206b6486@oracle.com> Message-ID: <5d2d51087deb43988867ae66918482aa@jd.com> Hi David, My understanding about those "arg" is that they are trying to make limitation of the overall message length and make it convince to communicate with sockets. I will do more test try to see whether changing back to 3 makes everything fine, and then prepare a webrev. Thanks. Lin ________________________________________ From: David Holmes Sent: Thursday, February 28, 2019 7:55:01 PM To: ??; Yasumasa Suenaga Cc: serviceability-dev at openjdk.java.net serviceability-dev at openjdk.java.net Subject: Re: Protocol version of Attach API Hi Lin, On 28/02/2019 7:30 pm, ?? wrote: > Hi David, > I am a little confused, do you think it is proper to made the patch as a fix of https://bugs.openjdk.java.net/browse/JDK-8219721 so that we don't need to backout and REDO? Generally I'd prefer to do the backout and then apply the revised fix as it will make the changes easier to track. However, if you are saying that everything works fine just by changing the 4 back to 3 everywhere, then that does seem a very simple fix to apply directly. I admit that if that does work then I really don't understand what these "arg" values actually means. :( Though it would explain why windows appears to work fine even though it was left at 3. Thanks, David > Thanks, > LIn > ________________________________________ > From: ?? > Sent: Thursday, February 28, 2019 4:50:12 PM > To: David Holmes; Yasumasa Suenaga > Cc: serviceability-dev at openjdk.java.net serviceability-dev at openjdk.java.net > Subject: Re: Protocol version of Attach API > > Dear All, > I have tried simply recover the max argument count makes jmap -histo works and also makes the compatibility works fine. > Following are the changes I made: > > diff -r 07dd34f487d4 src/hotspot/share/services/attachListener.hpp > --- a/src/hotspot/share/services/attachListener.hpp Thu Feb 28 02:47:39 2019 +0100 > +++ b/src/hotspot/share/services/attachListener.hpp Thu Feb 28 16:48:19 2019 +0800 > @@ -106,7 +106,7 @@ > enum { > name_length_max = 16, // maximum length of name > arg_length_max = 1024, // maximum length of argument > - arg_count_max = 4 // maximum number of arguments > + arg_count_max = 3 // maximum number of arguments > }; > > // name of special operation that can be enqueued when all > diff -r 07dd34f487d4 src/jdk.attach/aix/classes/sun/tools/attach/VirtualMachineImpl.java > --- a/src/jdk.attach/aix/classes/sun/tools/attach/VirtualMachineImpl.java Thu Feb 28 02:47:39 2019 +0100 > +++ b/src/jdk.attach/aix/classes/sun/tools/attach/VirtualMachineImpl.java Thu Feb 28 16:48:19 2019 +0800 > @@ -138,7 +138,7 @@ > * Execute the given command in the target VM. > */ > InputStream execute(String cmd, Object ... args) throws AgentLoadException, IOException { > - assert args.length <= 4; // includes null > + assert args.length <= 3; // includes null > > // did we detach? > synchronized (this) { > @@ -166,7 +166,7 @@ > writeString(s, PROTOCOL_VERSION); > writeString(s, cmd); > > - for (int i = 0; i < 4; i++) { > + for (int i = 0; i < 3; i++) { > if (i < args.length && args[i] != null) { > writeString(s, (String)args[i]); > } else { > diff -r 07dd34f487d4 src/jdk.attach/linux/classes/sun/tools/attach/VirtualMachineImpl.java > --- a/src/jdk.attach/linux/classes/sun/tools/attach/VirtualMachineImpl.java Thu Feb 28 02:47:39 2019 +0100 > +++ b/src/jdk.attach/linux/classes/sun/tools/attach/VirtualMachineImpl.java Thu Feb 28 16:48:19 2019 +0800 > @@ -143,7 +143,7 @@ > * Execute the given command in the target VM. > */ > InputStream execute(String cmd, Object ... args) throws AgentLoadException, IOException { > - assert args.length <= 4; // includes null > + assert args.length <= 3; // includes null > > // did we detach? > synchronized (this) { > diff -r 07dd34f487d4 src/jdk.attach/macosx/classes/sun/tools/attach/VirtualMachineImpl.java > --- a/src/jdk.attach/macosx/classes/sun/tools/attach/VirtualMachineImpl.java Thu Feb 28 02:47:39 2019 +0100 > +++ b/src/jdk.attach/macosx/classes/sun/tools/attach/VirtualMachineImpl.java Thu Feb 28 16:48:19 2019 +0800 > @@ -139,7 +139,7 @@ > * Execute the given command in the target VM. > */ > InputStream execute(String cmd, Object ... args) throws AgentLoadException, IOException { > - assert args.length <= 4; // includes null > + assert args.length <= 3; // includes null > > // did we detach? > synchronized (this) { > diff -r 07dd34f487d4 src/jdk.attach/solaris/classes/sun/tools/attach/VirtualMachineImpl.java > --- a/src/jdk.attach/solaris/classes/sun/tools/attach/VirtualMachineImpl.java Thu Feb 28 02:47:39 2019 +0100 > +++ b/src/jdk.attach/solaris/classes/sun/tools/attach/VirtualMachineImpl.java Thu Feb 28 16:48:19 2019 +0800 > @@ -126,7 +126,7 @@ > * Execute the given command in the target VM. > */ > InputStream execute(String cmd, Object ... args) throws AgentLoadException, IOException { > - assert args.length <= 4; // includes null > + assert args.length <= 3; // includes null > > // first check that we are still attached > int door; > diff -r 07dd34f487d4 src/jdk.attach/windows/classes/sun/tools/attach/VirtualMachineImpl.java > --- a/src/jdk.attach/windows/classes/sun/tools/attach/VirtualMachineImpl.java Thu Feb 28 02:47:39 2019 +0100 > +++ b/src/jdk.attach/windows/classes/sun/tools/attach/VirtualMachineImpl.java Thu Feb 28 16:48:19 2019 +0800 > @@ -77,7 +77,7 @@ > InputStream execute(String cmd, Object ... args) > throws AgentLoadException, IOException > { > - assert args.length <= 4; // includes null > + assert args.length <= 3; // includes null > > // create a pipe using a random name > Random rnd = new Random(); > > > > Thanks, > Lin > ________________________________________ > From: ?? > Sent: Thursday, February 28, 2019 3:24:52 PM > To: David Holmes; Yasumasa Suenaga > Cc: serviceability-dev at openjdk.java.net serviceability-dev at openjdk.java.net > Subject: RE: Protocol version of Attach API > > Hi David, > Since I don't have the access to JBS, may I ask your help to ceate sub-task? Thanks. > > BRs, > Lin > >> -----Original Message----- >> From: David Holmes >> Sent: Thursday, February 28, 2019 3:16 PM >> To: ?? ; Yasumasa Suenaga >> Cc: serviceability-dev at openjdk.java.net serviceability-dev at openjdk.java.net >> >> Subject: Re: Protocol version of Attach API >> >> Hi Lin, >> >> On 28/02/2019 4:49 pm, ?? wrote: >>> Hi David, >>> Your are right and thanks for pointing it out. when I worte that patch, I >> was considering implement -filename and -incremental together. and I must >> be too stupid to forget recover it when I divided the patch into two. >>> And it seems a good solution is to refine the original patch of jmap histo, >> and try to composite all args as one when passing it to socket and let >> attachlistener to handle the analyze. >>> I will try that. >>> One more, do I need to consider changing the jmap -dump also? >> >> I'm assuming -dump already works fine, so I'm just expecting to see -histo >> handle the file in a similar manner. >> >> If you find this works I suggest creating a sub-task of 8215622 to first backout >> the original changes (hg backout), and a second sub-task to REDO with the >> new implementation. Each will need reviewing separately in their own RFR >> thread. >> >> Thanks, >> David >> >>> BRs, >>> Lin >>> ________________________________________ >>> From: David Holmes >>> Sent: Thursday, February 28, 2019 12:59:28 PM >>> To: ??; Yasumasa Suenaga >>> Cc: serviceability-dev at openjdk.java.net >>> serviceability-dev at openjdk.java.net >>> Subject: Re: Protocol version of Attach API >>> >>> Sorry I'm going to pick up on the rollback and re-do option here as I >>> just had a closer look at jmap. Given jmap -dump already has more >>> options than -histo does, why was any change to the "maximum number of >>> args" needed in the first place ??? >>> >>> David >>> >>> On 28/02/2019 2:43 pm, David Holmes wrote: >>>> Hi everyone, >>>> >>>> I'm not sure we're converging on a suitable solution here, but to >>>> address the issues flagged by Lin below ... >>>> >>>> On 28/02/2019 12:39 pm, ?? wrote: >>>>> Hi Suenaga, >>>>> >>>>> Thanks for your expaination about the arg_length_max, I >>>>> generally agree with you that it is better to consider using dynamic >>>>> memory, and that would be handled carefully to aviod introduce >>>>> compatibility issue, plus it would be a big change. So let?s see >>>>> what others suggest. >>>>> >>>>> Hi All, >>>>> >>>>> It seems for me that there are basically three problems forked by >>>>> this >>>>> thread? >>>>> >>>>> ?Compatibility issue with old jcmd alike tools with attachListener?s >>>>> change. >>>> >>>> This is issue: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8219721 >>>> >>>>> ?Only 3 arguments limited to passed by socket to attachListener for >>>>> Windows, which cause 8215622 work abnormally on Windows. >>>> >>>> I have filed a new bug for this: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8219895 >>>> >>>>> ?The arg_length_max may not be enough for handling filename. >>>> >>>> I have filed a new bug for this: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8219896 >>>> >>>> though it seems very related to this issue. >>>> >>>>> So I suggest we keep the first problem discussed in this thread, and >>>>> create separate thread for the others. Do you agree? >>>> >>>> There is some overlap but yes this can be broken down somewhat - >>>> though dealing with the variable length "packet" is going to have to >>>> consider that what is received is in fact much larger than the >>>> purported maximum packet size if these long paths are expected and >> accepted. >>>> >>>> FWIW I don't see crashes or anything drastic if the arguments are too >>>> long - the operations just fail (in somewhat obscure ways sometimes). >>>> >>>>> >>>>> For me, I will refine my patch to use timeout as a fix for the first >>>>> one, and update it in this thread. And I will try to fix the second >>>>> one for Windows, and create a separate thread for discussing. And if >>>>> possible, I can help to fix the third one. >>>>> >>>>> What?s your opinion? >>>> >>>> That sounds fine ... >>>> >>>> Or you could choose to rollback JDK-8215622 and see how to solve that >>>> without increasing the arg count. Given this usage: >>>> >>>> jmap -histo:live,file=foo.txt >>>> >>>> I'm not sure why this is sent to the VM as multiple args rather than >>>> as a single composite arg that can then be parsed again by the actual >>>> "jmap" logic. There would be some double-up perhaps if the front-end >>>> tool wants to perform the command-line validation, but it would be >>>> easy enough I think to do that checking then send the original composite >> arg. >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>> >>>>> BRs, >>>>> >>>>> Lin >>>>> >>>>> *From:*Yasumasa Suenaga >>>>> *Sent:* Thursday, February 28, 2019 8:39 AM >>>>> *To:* ?? >>>>> *Cc:* David Holmes ; >>>>> serviceability-dev at openjdk.java.net >>>>> serviceability-dev at openjdk.java.net >>>>> >>>>> *Subject:* Re: Protocol version of Attach API >>>>> >>>>> 2019?2?28?(?) 0:04 ?? > >: >>>>> >>>>> Dear Suenaga, >>>>> Thanks for your reviewing. I will try to refine the patch. >>>>> For the argument length you mentioned, do you mean the >>>>> "arg_length_max" should be large enough to accept the max filename >>>>> length? >>>>> >>>>> Yes, but it is not enough. >>>>> >>>>> For example, jcmd VM.log might pass 2 or more paths to define logs. >>>>> >>>>> IMHO, all the handling of the argument length is at receiver >>>>> side in the attachListener, such as >>>>> >>>>> >> http://hg.openjdk.java.net/jdk/jdk/file/df3d253aaf81/src/hotspot/os/linux/att >> achListener_linux.cpp#l322, >>>>> for me it means that the VM side limits the argments length less >>>>> than arg_length_max, otherwise it will return NULL, which cause the >>>>> sender side (tools like jcmd and jmap) exits with error message. so >>>>> I think there may be no need to limit the argument size in tool side. >>>>> >>>>> IMHO all programs which use filesystem should support any locations >>>>> on it. >>>>> >>>>> So I think we should use dynamic memory (or GrowableArray) for it if >>>>> we do not change client side for compatibility. >>>>> >>>>> And from my experiment with jmap, the arguments send to >>>>> sockets are not arg0 only. as you can see in >>>>> >>>>> http://hg.openjdk.java.net/jdk/jdk/file/df3d253aaf81/src/jdk.jcmd/sh >>>>> are/classes/sun/tools/jmap/JMap.java#l193 >>>>> >>>>> and >>>>> >>>>> http://hg.openjdk.java.net/jdk/jdk/file/df3d253aaf81/src/jdk.jcmd/sh >>>>> are/classes/sun/tools/jmap/JMap.java#l133, >>>>> >>>>> jmap can pass arg0 as "filename", and arg1 as "-live", and both of >>>>> them can be NULL. so 00000 can be >>>>> 0000, and file can be null. so 00 may not >>>>> indicate it reach the end. >>>>> >>>>> We should consider for other tools - jstack and jinfo. >>>>> >>>>> (jstack is ok because it will not have long arguments) >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> BRs, >>>>> Lin >>>>> ________________________________________ >>>>> From: Yasumasa Suenaga >>>> > >>>>> Sent: Wednesday, February 27, 2019 8:10:14 PM >>>>> To: ?? >>>>> Cc: David Holmes; serviceability-dev at openjdk.java.net >>>>> >>>>> Subject: Re: Protocol version of Attach API >>>>> >>>>> Hi Lin, >>>>> >>>>> I think we need to research more about this. >>>>> IMHO we need to match length of arguments between >>>>> server (AttachListener) and client (serviceability tools) at least. >>>>> (please see previous email from me). >>>>> >>>>> I have some comments for your change: >>>>> >>>>> On 2019/02/27 18:22, ?? wrote: >>>>> > Dear All, >>>>> >? Here I have figured out one solution based on timeout. would >>>>> you like help to see whether this is OK? >>>>> > --- a/src/hotspot/os/linux/attachListener_linux.cpp Tue Feb >>>>> 26 14:57:23 2019 +0530 >>>>> > +++ b/src/hotspot/os/linux/attachListener_linux.cpp Wed Feb >>>>> 27 17:21:48 2019 +0800 >>>>> > @@ -263,9 +263,29 @@ >>>>> >? int off = 0; >>>>> >? int left = max_len; >>>>> > >>>>> > + memset(buf, 0, max_len); >>>>> > + // set timeout for read >>>>> > + struct timeval timeout; >>>>> > + timeout.tv_sec = 3; >>>>> > + timeout.tv_usec = 0; >>>>> >>>>> I think timeout value should be configurable. >>>>> For example, we can introduce new flag in globals.hpp . >>>>> >>>>> >>>>> > + if(setsockopt(s, SOL_SOCKET, SO_RCVTIMEO, (struct >>>>> timeval*)&timeout, sizeof(timeout))) { >>>>> > + log_debug(attach)("Failed to set socket option SO_RCVTIMEO: >>>>> %s\n", strerror(errorno)); >>>>> > + printf("Warning: Failed to set socket option SO_RCVTIMEO: >>>>> %s!!!\n", strerror(errno)); >>>>> >>>>> You should not use printf(), and do you need to pass '\n' to >>>>> log_debug()? >>>>> >>>>> >>>>> > + } >>>>> >? do { >>>>> >? int n; >>>>> > - RESTARTABLE(read(s, buf+off, left), n); >>>>> > + log_debug(attach)("start reading data from socket....\n"); >>>>> > + n = read(s, buf+off, left); >>>>> >>>>> You should use RESTARTABLE macro. >>>>> read(2) might be interrupted by signal. >>>>> >>>>> >>>>> > + if (n < 0) { >>>>> > + if (errno == EWOULDBLOCK) { >>>>> >>>>> According to man page, read(2) sets EWOULDBLOCK or EAGAIN. >>>>> So you should check both errno. >>>>> >>>>> >>>>> > + for (int i = str_count; i < expected_str_count; i++) { >>>>> > + //timeout, fill reminded arguments with \0; >>>>> > + buf[off+i] = '\0'; >>>>> > + str_count++; >>>>> > + } >>>>> >>>>> You set zero to buf[] in above. >>>>> So you can remove this loop, and set str_count to expected_str_count >>>>> without manipulating buf[]. >>>>> >>>>> In addition, I prefer to add log_debug() at this >>>>> to record NULL arguments are added. >>>>> >>>>> >>>>> > + break;; >>>>> > + } >>>>> > + } >>>>> >? assert(n <= left, "buffer was too small, impossible!"); >>>>> >? buf[max_len - 1] = '\0'; >>>>> >? if (n == -1) { >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> > Thanks. >>>>> > Lin >>>>> > >>>>> > ________________________________________ >>>>> > From: Yasumasa Suenaga >>>> > >>>>> > Sent: Wednesday, February 27, 2019 15:15 >>>>> > To: David Holmes; ?? >>>>> > Cc: serviceability-dev at openjdk.java.net >>>>> >>>>> > Subject: Re: Protocol version of Attach API >>>>> > >>>>> > On 2019/02/27 15:59, David Holmes wrote: >>>>> >> On 27/02/2019 4:10 pm, Yasumasa Suenaga wrote: >>>>> >>> Hi, >>>>> >>> >>>>> >>> Buffer size for receiving packets from client is >>>>> determined at [1]. >>>>> >> >>>>> >> Maximum buffer size, yes. >>>>> >> >>>>> >>> It defines length of command name and of argument. >>>>> >>> It is passed via Unix domain, so we fill '\0' to remaining >>>>> bytes and >>>>> >>> might be able to assume all arguments are passed with >>>>> empty string. >>>>> >> >>>>> >> Not sure what you mean. >>>>> >> >>>>> >> // The buffer is expected to be formatted as follows: >>>>> >> // 00000 >>>>> >> >>>>> >> so we can expect to read at least two things - the ver and cmd. >>>>> If we encounter 00 we can infer we reached the end. But we may not >>>>> have read the full data into the buffer, so can't tell if another >>>>> call to read() is needed yet - you only know after you've read >>>>> the 00. >>>>> >> >>>>> >>> BTW length of arguments is defined to 1024 in [2]. >>>>> >>> jcmd and jmap might pas file path - it might be JVM_MAXPATHLEN >>>>> (4097 bytes). >>>>> >>> Buffer size which is used in AttachListener seems not to >>>>> be enough. >>>>> >> >>>>> >> One has to assume/hope that the code sending the data is working >>>>> to the same protocol rules as the receiver! Otherwise this is just >>>>> completely broken. >>>>> > >>>>> > On Linux, client (VirtualMachineImpl) seems not to check length >>>>> of arguments: >>>>> > >>>>> >>>>> http://hg.openjdk.java.net/jdk/jdk/file/df3d253aaf81/src/jdk.attach/ >>>>> linux/classes/sun/tools/attach/VirtualMachineImpl.java#l168 >>>>> >>>>> > >>>>> > In case of jcmd, all options are passed to arg #1. It seems not >>>>> to check the length: >>>>> > >>>>> >>>>> http://hg.openjdk.java.net/jdk/jdk/file/df3d253aaf81/src/jdk.jcmd/sh >>>>> are/classes/sun/tools/jcmd/JCmd.java#l111 >>>>> >>>>> > >>>>> > >>>>> > I guess other tools (jstack, jmap, etc) which connect to >>>>> AttachListener are same. >>>>> > So we can fix both Attach API and AttachListener (it will be big >>>>> change!), >>>>> > but I concern we can keep protocol version... >>>>> > >>>>> > >>>>> > Thanks, >>>>> > >>>>> > Yasumasa >>>>> > >>>>> > >>>>> >> David >>>>> >> ----- >>>>> >> >>>>> >>> We might have to change more. >>>>> >>> >>>>> >>> >>>>> >>> Thanks, >>>>> >>> >>>>> >>> Yasumasa >>>>> >>> >>>>> >>> >>>>> >>> [1] >>>>> >>>>> http://hg.openjdk.java.net/jdk/jdk/file/df3d253aaf81/src/hotspot/os/ >>>>> linux/attachListener_linux.cpp#l254 >>>>> >>>>> >>> [2] >>>>> >>>>> http://hg.openjdk.java.net/jdk/jdk/file/df3d253aaf81/src/hotspot/sha >>>>> re/services/attachListener.hpp#l106 >>>>> >>>>> >>> >>>>> >>> >>>>> >>> On 2019/02/27 15:00, ?? wrote: >>>>> >>>> Another solution I can figure out is try to add timeout for >>>>> socket read. I will also investigate whether is works. >>>>> >>>> >>>>> >>>> Cheers, >>>>> >>>> Lin >>>>> >>>> >>>>> >>>> >>>>> >>>>> -----Original Message----- >>>>> >>>>> From: ?? >>>>> >>>>> Sent: Wednesday, February 27, 2019 1:52 PM >>>>> >>>>> To: 'David Holmes' >>>> >; Yasumasa Suenaga >>>>> >>>>> > >>>>> >>>>> Cc: serviceability-dev at openjdk.java.net >>>>> >>>>> >>>>> Subject: RE: Protocol version of Attach API >>>>> >>>>> >>>>> >>>>> Dear David, Yasumasa, >>>>> >>>>>? I think it is hard to know how long the buffer is >>>>> passed from socket >>>>> >>>>> without changing the info of the message from the >>>>> receiver side. >>>>> >>>>>? So how about when str_count reach to 3, we test it >>>>> with non_blocking >>>>> >>>>> read? >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> Cheers, >>>>> >>>>> Lin >>>>> >>>>>> -----Original Message----- >>>>> >>>>>> From: David Holmes >>>> > >>>>> >>>>>> Sent: Wednesday, February 27, 2019 1:44 PM >>>>> >>>>>> To: Yasumasa Suenaga >>>> >; ?? >>>> > >>>>> >>>>>> Cc: serviceability-dev at openjdk.java.net >>>>> >>>>> >>>>>> Subject: Re: Protocol version of Attach API >>>>> >>>>>> >>>>> >>>>>> Hi Yasumasa, >>>>> >>>>>> >>>>> >>>>>> On 27/02/2019 1:05 pm, Yasumasa Suenaga wrote: >>>>> >>>>>>> Hi Lin, >>>>> >>>>>>> >>>>> >>>>>>> My proposal is a just idea, so you need to tweak it. >>>>> >>>>>>> >>>>> >>>>>>> AttachListener receives raw command. >>>>> >>>>>>> For example, jcmd is `jcmd\0`. Please see >>>>> >>>>>>> HotSpotVirtualMachine and extended classes. >>>>> >>>>>>> >>>>> >>>>>>> In case of jcmd, I guess AttachListener will receive message >>>>> >>>>>>> `\0jcmd\0\0\0\0` (I did not check it well). >>>>> >>>>>>> So I guess we can add '\0' when `str_count` does not reach >>>>> to maximum. >>>>> >>>>>> >>>>> >>>>>> I don't see how this approach can work. You have to know >> how >>>>> many >>>>> >>>>>> arguments are coming in the "packet", but that information >>>>> is not >>>>> >>>>>> available in the current Linux implementation.Without it you >>>>> can't >>>>> >>>>>> know when to stop calling read(). >>>>> >>>>>> >>>>> >>>>>> The protocol would need to be changed to send the "packet" >>>>> size, but >>>>> >>>>>> that's not compatible with older JDKs. >>>>> >>>>>> >>>>> >>>>>> Otherwise I think we have no choice but to use a >>>>> non-blocking read ... >>>>> >>>>>> though I'm still unsure if you can know for certain >>>>> that you've >>>>> >>>>>> reached the end of the "packet" ... >>>>> >>>>>> >>>>> >>>>>> Thanks, >>>>> >>>>>> David >>>>> >>>>>> >>>>> >>>>>>> >>>>> >>>>>>> Thanks, >>>>> >>>>>>> >>>>> >>>>>>> Yasumasa >>>>> >>>>>>> >>>>> >>>>>>> >>>>> >>>>>>> On 2019/02/27 11:50, zanglin5 at jd.com >>>>> wrote: >>>>> >>>>>>>> Dear Yasumasa, >>>>> >>>>>>>>? The fix you mentioned seems not work as expected. >>>>> >>>>>>>>? I have done an experiment that use jdk1.8's "jcmd >>>>> help" to >>>>> >>>>>>>> attach to latest jdk. >>>>> >>>>>>>>? it seem the whole "jcmd help"? buffer is not >>>>> >>>>>>>>? read completely at one time. in my case it is "jcmd", >>>>> "", >>>>> >>>>>>>> "help", and then block at while(). >>>>> >>>>>>>>? After applied your change, it seems only the "jcmd" is >>>>> >>>>>>>> processed, the "", "help" is replaced by '\0'. >>>>> >>>>>>>> >>>>> >>>>>>>> BRs, >>>>> >>>>>>>> Lin >>>>> >>>>>>>> >>>>> From yasuenag at gmail.com Fri Mar 1 03:54:54 2019 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Fri, 1 Mar 2019 12:54:54 +0900 Subject: Protocol version of Attach API In-Reply-To: <5d2d51087deb43988867ae66918482aa@jd.com> References: <3dabf14d-e0bd-1ed1-4e37-f81332c1d68b@gmail.com> <01c706d8-600d-d823-6ebc-2d7e153c305f@oracle.com> <04aadf26-d2c9-6f65-9815-06ad36c53219@gmail.com>

<4a5f251d-67ef-a81a-ef73-48f9aa9c853e@gmail.com> <0e7c23fec044419d845c5608b0586fcb@jd.com> <3dab08619bbd4254943361d2cdc1298b@jd.com>

<077ca61fe91f46ef93c4472accdf0337@jd.com> <8f6ec4c6-18f1-4f61-7ab0-6e1b40acf544@oracle.com> <22427d1123114a0d869eaade059fcd12@jd.com> <66511977ba394d7084d94a356f68f4b7@jd.com> <33bb67d77da949ab9410ea3cdb5dd1ce@jd.com> <650f2814-9bb6-39c1-87ab-583a206b6486@oracle.com> <5d2d51087deb43988867ae66918482aa@jd.com> Message-ID: Hi, I agree with David. I think we should backout 8215622. We should re-do 8215622 completely for all platforms (includes Windows) and support receiving requests from earlier serviceability tools. Problems about arguments of AttachListener should be worked as another issues. Thanks, Yasumasa 2019?3?1?(?) 12:39 ?? : > > Hi David, > My understanding about those "arg" is that they are trying to make limitation of the overall message length and make it convince to communicate with sockets. > I will do more test try to see whether changing back to 3 makes everything fine, and then prepare a webrev. > Thanks. > > Lin > ________________________________________ > From: David Holmes > Sent: Thursday, February 28, 2019 7:55:01 PM > To: ??; Yasumasa Suenaga > Cc: serviceability-dev at openjdk.java.net serviceability-dev at openjdk.java.net > Subject: Re: Protocol version of Attach API > > Hi Lin, > > On 28/02/2019 7:30 pm, ?? wrote: > > Hi David, > > I am a little confused, do you think it is proper to made the patch as a fix of https://bugs.openjdk.java.net/browse/JDK-8219721 so that we don't need to backout and REDO? > > Generally I'd prefer to do the backout and then apply the revised fix as > it will make the changes easier to track. > > However, if you are saying that everything works fine just by changing > the 4 back to 3 everywhere, then that does seem a very simple fix to > apply directly. > > I admit that if that does work then I really don't understand what these > "arg" values actually means. :( Though it would explain why windows > appears to work fine even though it was left at 3. > > Thanks, > David > > > Thanks, > > LIn > > ________________________________________ > > From: ?? > > Sent: Thursday, February 28, 2019 4:50:12 PM > > To: David Holmes; Yasumasa Suenaga > > Cc: serviceability-dev at openjdk.java.net serviceability-dev at openjdk.java.net > > Subject: Re: Protocol version of Attach API > > > > Dear All, > > I have tried simply recover the max argument count makes jmap -histo works and also makes the compatibility works fine. > > Following are the changes I made: > > > > diff -r 07dd34f487d4 src/hotspot/share/services/attachListener.hpp > > --- a/src/hotspot/share/services/attachListener.hpp Thu Feb 28 02:47:39 2019 +0100 > > +++ b/src/hotspot/share/services/attachListener.hpp Thu Feb 28 16:48:19 2019 +0800 > > @@ -106,7 +106,7 @@ > > enum { > > name_length_max = 16, // maximum length of name > > arg_length_max = 1024, // maximum length of argument > > - arg_count_max = 4 // maximum number of arguments > > + arg_count_max = 3 // maximum number of arguments > > }; > > > > // name of special operation that can be enqueued when all > > diff -r 07dd34f487d4 src/jdk.attach/aix/classes/sun/tools/attach/VirtualMachineImpl.java > > --- a/src/jdk.attach/aix/classes/sun/tools/attach/VirtualMachineImpl.java Thu Feb 28 02:47:39 2019 +0100 > > +++ b/src/jdk.attach/aix/classes/sun/tools/attach/VirtualMachineImpl.java Thu Feb 28 16:48:19 2019 +0800 > > @@ -138,7 +138,7 @@ > > * Execute the given command in the target VM. > > */ > > InputStream execute(String cmd, Object ... args) throws AgentLoadException, IOException { > > - assert args.length <= 4; // includes null > > + assert args.length <= 3; // includes null > > > > // did we detach? > > synchronized (this) { > > @@ -166,7 +166,7 @@ > > writeString(s, PROTOCOL_VERSION); > > writeString(s, cmd); > > > > - for (int i = 0; i < 4; i++) { > > + for (int i = 0; i < 3; i++) { > > if (i < args.length && args[i] != null) { > > writeString(s, (String)args[i]); > > } else { > > diff -r 07dd34f487d4 src/jdk.attach/linux/classes/sun/tools/attach/VirtualMachineImpl.java > > --- a/src/jdk.attach/linux/classes/sun/tools/attach/VirtualMachineImpl.java Thu Feb 28 02:47:39 2019 +0100 > > +++ b/src/jdk.attach/linux/classes/sun/tools/attach/VirtualMachineImpl.java Thu Feb 28 16:48:19 2019 +0800 > > @@ -143,7 +143,7 @@ > > * Execute the given command in the target VM. > > */ > > InputStream execute(String cmd, Object ... args) throws AgentLoadException, IOException { > > - assert args.length <= 4; // includes null > > + assert args.length <= 3; // includes null > > > > // did we detach? > > synchronized (this) { > > diff -r 07dd34f487d4 src/jdk.attach/macosx/classes/sun/tools/attach/VirtualMachineImpl.java > > --- a/src/jdk.attach/macosx/classes/sun/tools/attach/VirtualMachineImpl.java Thu Feb 28 02:47:39 2019 +0100 > > +++ b/src/jdk.attach/macosx/classes/sun/tools/attach/VirtualMachineImpl.java Thu Feb 28 16:48:19 2019 +0800 > > @@ -139,7 +139,7 @@ > > * Execute the given command in the target VM. > > */ > > InputStream execute(String cmd, Object ... args) throws AgentLoadException, IOException { > > - assert args.length <= 4; // includes null > > + assert args.length <= 3; // includes null > > > > // did we detach? > > synchronized (this) { > > diff -r 07dd34f487d4 src/jdk.attach/solaris/classes/sun/tools/attach/VirtualMachineImpl.java > > --- a/src/jdk.attach/solaris/classes/sun/tools/attach/VirtualMachineImpl.java Thu Feb 28 02:47:39 2019 +0100 > > +++ b/src/jdk.attach/solaris/classes/sun/tools/attach/VirtualMachineImpl.java Thu Feb 28 16:48:19 2019 +0800 > > @@ -126,7 +126,7 @@ > > * Execute the given command in the target VM. > > */ > > InputStream execute(String cmd, Object ... args) throws AgentLoadException, IOException { > > - assert args.length <= 4; // includes null > > + assert args.length <= 3; // includes null > > > > // first check that we are still attached > > int door; > > diff -r 07dd34f487d4 src/jdk.attach/windows/classes/sun/tools/attach/VirtualMachineImpl.java > > --- a/src/jdk.attach/windows/classes/sun/tools/attach/VirtualMachineImpl.java Thu Feb 28 02:47:39 2019 +0100 > > +++ b/src/jdk.attach/windows/classes/sun/tools/attach/VirtualMachineImpl.java Thu Feb 28 16:48:19 2019 +0800 > > @@ -77,7 +77,7 @@ > > InputStream execute(String cmd, Object ... args) > > throws AgentLoadException, IOException > > { > > - assert args.length <= 4; // includes null > > + assert args.length <= 3; // includes null > > > > // create a pipe using a random name > > Random rnd = new Random(); > > > > > > > > Thanks, > > Lin > > ________________________________________ > > From: ?? > > Sent: Thursday, February 28, 2019 3:24:52 PM > > To: David Holmes; Yasumasa Suenaga > > Cc: serviceability-dev at openjdk.java.net serviceability-dev at openjdk.java.net > > Subject: RE: Protocol version of Attach API > > > > Hi David, > > Since I don't have the access to JBS, may I ask your help to ceate sub-task? Thanks. > > > > BRs, > > Lin > > > >> -----Original Message----- > >> From: David Holmes > >> Sent: Thursday, February 28, 2019 3:16 PM > >> To: ?? ; Yasumasa Suenaga > >> Cc: serviceability-dev at openjdk.java.net serviceability-dev at openjdk.java.net > >> > >> Subject: Re: Protocol version of Attach API > >> > >> Hi Lin, > >> > >> On 28/02/2019 4:49 pm, ?? wrote: > >>> Hi David, > >>> Your are right and thanks for pointing it out. when I worte that patch, I > >> was considering implement -filename and -incremental together. and I must > >> be too stupid to forget recover it when I divided the patch into two. > >>> And it seems a good solution is to refine the original patch of jmap histo, > >> and try to composite all args as one when passing it to socket and let > >> attachlistener to handle the analyze. > >>> I will try that. > >>> One more, do I need to consider changing the jmap -dump also? > >> > >> I'm assuming -dump already works fine, so I'm just expecting to see -histo > >> handle the file in a similar manner. > >> > >> If you find this works I suggest creating a sub-task of 8215622 to first backout > >> the original changes (hg backout), and a second sub-task to REDO with the > >> new implementation. Each will need reviewing separately in their own RFR > >> thread. > >> > >> Thanks, > >> David > >> > >>> BRs, > >>> Lin > >>> ________________________________________ > >>> From: David Holmes > >>> Sent: Thursday, February 28, 2019 12:59:28 PM > >>> To: ??; Yasumasa Suenaga > >>> Cc: serviceability-dev at openjdk.java.net > >>> serviceability-dev at openjdk.java.net > >>> Subject: Re: Protocol version of Attach API > >>> > >>> Sorry I'm going to pick up on the rollback and re-do option here as I > >>> just had a closer look at jmap. Given jmap -dump already has more > >>> options than -histo does, why was any change to the "maximum number of > >>> args" needed in the first place ??? > >>> > >>> David > >>> > >>> On 28/02/2019 2:43 pm, David Holmes wrote: > >>>> Hi everyone, > >>>> > >>>> I'm not sure we're converging on a suitable solution here, but to > >>>> address the issues flagged by Lin below ... > >>>> > >>>> On 28/02/2019 12:39 pm, ?? wrote: > >>>>> Hi Suenaga, > >>>>> > >>>>> Thanks for your expaination about the arg_length_max, I > >>>>> generally agree with you that it is better to consider using dynamic > >>>>> memory, and that would be handled carefully to aviod introduce > >>>>> compatibility issue, plus it would be a big change. So let?s see > >>>>> what others suggest. > >>>>> > >>>>> Hi All, > >>>>> > >>>>> It seems for me that there are basically three problems forked by > >>>>> this > >>>>> thread? > >>>>> > >>>>> ?Compatibility issue with old jcmd alike tools with attachListener?s > >>>>> change. > >>>> > >>>> This is issue: > >>>> > >>>> https://bugs.openjdk.java.net/browse/JDK-8219721 > >>>> > >>>>> ?Only 3 arguments limited to passed by socket to attachListener for > >>>>> Windows, which cause 8215622 work abnormally on Windows. > >>>> > >>>> I have filed a new bug for this: > >>>> > >>>> https://bugs.openjdk.java.net/browse/JDK-8219895 > >>>> > >>>>> ?The arg_length_max may not be enough for handling filename. > >>>> > >>>> I have filed a new bug for this: > >>>> > >>>> https://bugs.openjdk.java.net/browse/JDK-8219896 > >>>> > >>>> though it seems very related to this issue. > >>>> > >>>>> So I suggest we keep the first problem discussed in this thread, and > >>>>> create separate thread for the others. Do you agree? > >>>> > >>>> There is some overlap but yes this can be broken down somewhat - > >>>> though dealing with the variable length "packet" is going to have to > >>>> consider that what is received is in fact much larger than the > >>>> purported maximum packet size if these long paths are expected and > >> accepted. > >>>> > >>>> FWIW I don't see crashes or anything drastic if the arguments are too > >>>> long - the operations just fail (in somewhat obscure ways sometimes). > >>>> > >>>>> > >>>>> For me, I will refine my patch to use timeout as a fix for the first > >>>>> one, and update it in this thread. And I will try to fix the second > >>>>> one for Windows, and create a separate thread for discussing. And if > >>>>> possible, I can help to fix the third one. > >>>>> > >>>>> What?s your opinion? > >>>> > >>>> That sounds fine ... > >>>> > >>>> Or you could choose to rollback JDK-8215622 and see how to solve that > >>>> without increasing the arg count. Given this usage: > >>>> > >>>> jmap -histo:live,file=foo.txt > >>>> > >>>> I'm not sure why this is sent to the VM as multiple args rather than > >>>> as a single composite arg that can then be parsed again by the actual > >>>> "jmap" logic. There would be some double-up perhaps if the front-end > >>>> tool wants to perform the command-line validation, but it would be > >>>> easy enough I think to do that checking then send the original composite > >> arg. > >>>> > >>>> Thanks, > >>>> David > >>>> ----- > >>>> > >>>> > >>>>> BRs, > >>>>> > >>>>> Lin > >>>>> > >>>>> *From:*Yasumasa Suenaga > >>>>> *Sent:* Thursday, February 28, 2019 8:39 AM > >>>>> *To:* ?? > >>>>> *Cc:* David Holmes ; > >>>>> serviceability-dev at openjdk.java.net > >>>>> serviceability-dev at openjdk.java.net > >>>>> > >>>>> *Subject:* Re: Protocol version of Attach API > >>>>> > >>>>> 2019?2?28?(?) 0:04 ?? >> >: > >>>>> > >>>>> Dear Suenaga, > >>>>> Thanks for your reviewing. I will try to refine the patch. > >>>>> For the argument length you mentioned, do you mean the > >>>>> "arg_length_max" should be large enough to accept the max filename > >>>>> length? > >>>>> > >>>>> Yes, but it is not enough. > >>>>> > >>>>> For example, jcmd VM.log might pass 2 or more paths to define logs. > >>>>> > >>>>> IMHO, all the handling of the argument length is at receiver > >>>>> side in the attachListener, such as > >>>>> > >>>>> > >> http://hg.openjdk.java.net/jdk/jdk/file/df3d253aaf81/src/hotspot/os/linux/att > >> achListener_linux.cpp#l322, > >>>>> for me it means that the VM side limits the argments length less > >>>>> than arg_length_max, otherwise it will return NULL, which cause the > >>>>> sender side (tools like jcmd and jmap) exits with error message. so > >>>>> I think there may be no need to limit the argument size in tool side. > >>>>> > >>>>> IMHO all programs which use filesystem should support any locations > >>>>> on it. > >>>>> > >>>>> So I think we should use dynamic memory (or GrowableArray) for it if > >>>>> we do not change client side for compatibility. > >>>>> > >>>>> And from my experiment with jmap, the arguments send to > >>>>> sockets are not arg0 only. as you can see in > >>>>> > >>>>> http://hg.openjdk.java.net/jdk/jdk/file/df3d253aaf81/src/jdk.jcmd/sh > >>>>> are/classes/sun/tools/jmap/JMap.java#l193 > >>>>> > >>>>> and > >>>>> > >>>>> http://hg.openjdk.java.net/jdk/jdk/file/df3d253aaf81/src/jdk.jcmd/sh > >>>>> are/classes/sun/tools/jmap/JMap.java#l133, > >>>>> > >>>>> jmap can pass arg0 as "filename", and arg1 as "-live", and both of > >>>>> them can be NULL. so 00000 can be > >>>>> 0000, and file can be null. so 00 may not > >>>>> indicate it reach the end. > >>>>> > >>>>> We should consider for other tools - jstack and jinfo. > >>>>> > >>>>> (jstack is ok because it will not have long arguments) > >>>>> > >>>>> Thanks, > >>>>> > >>>>> Yasumasa > >>>>> > >>>>> > >>>>> BRs, > >>>>> Lin > >>>>> ________________________________________ > >>>>> From: Yasumasa Suenaga >>>>> > > >>>>> Sent: Wednesday, February 27, 2019 8:10:14 PM > >>>>> To: ?? > >>>>> Cc: David Holmes; serviceability-dev at openjdk.java.net > >>>>> > >>>>> Subject: Re: Protocol version of Attach API > >>>>> > >>>>> Hi Lin, > >>>>> > >>>>> I think we need to research more about this. > >>>>> IMHO we need to match length of arguments between > >>>>> server (AttachListener) and client (serviceability tools) at least. > >>>>> (please see previous email from me). > >>>>> > >>>>> I have some comments for your change: > >>>>> > >>>>> On 2019/02/27 18:22, ?? wrote: > >>>>> > Dear All, > >>>>> > Here I have figured out one solution based on timeout. would > >>>>> you like help to see whether this is OK? > >>>>> > --- a/src/hotspot/os/linux/attachListener_linux.cpp Tue Feb > >>>>> 26 14:57:23 2019 +0530 > >>>>> > +++ b/src/hotspot/os/linux/attachListener_linux.cpp Wed Feb > >>>>> 27 17:21:48 2019 +0800 > >>>>> > @@ -263,9 +263,29 @@ > >>>>> > int off = 0; > >>>>> > int left = max_len; > >>>>> > > >>>>> > + memset(buf, 0, max_len); > >>>>> > + // set timeout for read > >>>>> > + struct timeval timeout; > >>>>> > + timeout.tv_sec = 3; > >>>>> > + timeout.tv_usec = 0; > >>>>> > >>>>> I think timeout value should be configurable. > >>>>> For example, we can introduce new flag in globals.hpp . > >>>>> > >>>>> > >>>>> > + if(setsockopt(s, SOL_SOCKET, SO_RCVTIMEO, (struct > >>>>> timeval*)&timeout, sizeof(timeout))) { > >>>>> > + log_debug(attach)("Failed to set socket option SO_RCVTIMEO: > >>>>> %s\n", strerror(errorno)); > >>>>> > + printf("Warning: Failed to set socket option SO_RCVTIMEO: > >>>>> %s!!!\n", strerror(errno)); > >>>>> > >>>>> You should not use printf(), and do you need to pass '\n' to > >>>>> log_debug()? > >>>>> > >>>>> > >>>>> > + } > >>>>> > do { > >>>>> > int n; > >>>>> > - RESTARTABLE(read(s, buf+off, left), n); > >>>>> > + log_debug(attach)("start reading data from socket....\n"); > >>>>> > + n = read(s, buf+off, left); > >>>>> > >>>>> You should use RESTARTABLE macro. > >>>>> read(2) might be interrupted by signal. > >>>>> > >>>>> > >>>>> > + if (n < 0) { > >>>>> > + if (errno == EWOULDBLOCK) { > >>>>> > >>>>> According to man page, read(2) sets EWOULDBLOCK or EAGAIN. > >>>>> So you should check both errno. > >>>>> > >>>>> > >>>>> > + for (int i = str_count; i < expected_str_count; i++) { > >>>>> > + //timeout, fill reminded arguments with \0; > >>>>> > + buf[off+i] = '\0'; > >>>>> > + str_count++; > >>>>> > + } > >>>>> > >>>>> You set zero to buf[] in above. > >>>>> So you can remove this loop, and set str_count to expected_str_count > >>>>> without manipulating buf[]. > >>>>> > >>>>> In addition, I prefer to add log_debug() at this > >>>>> to record NULL arguments are added. > >>>>> > >>>>> > >>>>> > + break;; > >>>>> > + } > >>>>> > + } > >>>>> > assert(n <= left, "buffer was too small, impossible!"); > >>>>> > buf[max_len - 1] = '\0'; > >>>>> > if (n == -1) { > >>>>> > >>>>> > >>>>> Thanks, > >>>>> > >>>>> Yasumasa > >>>>> > >>>>> > >>>>> > Thanks. > >>>>> > Lin > >>>>> > > >>>>> > ________________________________________ > >>>>> > From: Yasumasa Suenaga >>>>> > > >>>>> > Sent: Wednesday, February 27, 2019 15:15 > >>>>> > To: David Holmes; ?? > >>>>> > Cc: serviceability-dev at openjdk.java.net > >>>>> > >>>>> > Subject: Re: Protocol version of Attach API > >>>>> > > >>>>> > On 2019/02/27 15:59, David Holmes wrote: > >>>>> >> On 27/02/2019 4:10 pm, Yasumasa Suenaga wrote: > >>>>> >>> Hi, > >>>>> >>> > >>>>> >>> Buffer size for receiving packets from client is > >>>>> determined at [1]. > >>>>> >> > >>>>> >> Maximum buffer size, yes. > >>>>> >> > >>>>> >>> It defines length of command name and of argument. > >>>>> >>> It is passed via Unix domain, so we fill '\0' to remaining > >>>>> bytes and > >>>>> >>> might be able to assume all arguments are passed with > >>>>> empty string. > >>>>> >> > >>>>> >> Not sure what you mean. > >>>>> >> > >>>>> >> // The buffer is expected to be formatted as follows: > >>>>> >> // 00000 > >>>>> >> > >>>>> >> so we can expect to read at least two things - the ver and cmd. > >>>>> If we encounter 00 we can infer we reached the end. But we may not > >>>>> have read the full data into the buffer, so can't tell if another > >>>>> call to read() is needed yet - you only know after you've read > >>>>> the 00. > >>>>> >> > >>>>> >>> BTW length of arguments is defined to 1024 in [2]. > >>>>> >>> jcmd and jmap might pas file path - it might be JVM_MAXPATHLEN > >>>>> (4097 bytes). > >>>>> >>> Buffer size which is used in AttachListener seems not to > >>>>> be enough. > >>>>> >> > >>>>> >> One has to assume/hope that the code sending the data is working > >>>>> to the same protocol rules as the receiver! Otherwise this is just > >>>>> completely broken. > >>>>> > > >>>>> > On Linux, client (VirtualMachineImpl) seems not to check length > >>>>> of arguments: > >>>>> > > >>>>> > >>>>> http://hg.openjdk.java.net/jdk/jdk/file/df3d253aaf81/src/jdk.attach/ > >>>>> linux/classes/sun/tools/attach/VirtualMachineImpl.java#l168 > >>>>> > >>>>> > > >>>>> > In case of jcmd, all options are passed to arg #1. It seems not > >>>>> to check the length: > >>>>> > > >>>>> > >>>>> http://hg.openjdk.java.net/jdk/jdk/file/df3d253aaf81/src/jdk.jcmd/sh > >>>>> are/classes/sun/tools/jcmd/JCmd.java#l111 > >>>>> > >>>>> > > >>>>> > > >>>>> > I guess other tools (jstack, jmap, etc) which connect to > >>>>> AttachListener are same. > >>>>> > So we can fix both Attach API and AttachListener (it will be big > >>>>> change!), > >>>>> > but I concern we can keep protocol version... > >>>>> > > >>>>> > > >>>>> > Thanks, > >>>>> > > >>>>> > Yasumasa > >>>>> > > >>>>> > > >>>>> >> David > >>>>> >> ----- > >>>>> >> > >>>>> >>> We might have to change more. > >>>>> >>> > >>>>> >>> > >>>>> >>> Thanks, > >>>>> >>> > >>>>> >>> Yasumasa > >>>>> >>> > >>>>> >>> > >>>>> >>> [1] > >>>>> > >>>>> http://hg.openjdk.java.net/jdk/jdk/file/df3d253aaf81/src/hotspot/os/ > >>>>> linux/attachListener_linux.cpp#l254 > >>>>> > >>>>> >>> [2] > >>>>> > >>>>> http://hg.openjdk.java.net/jdk/jdk/file/df3d253aaf81/src/hotspot/sha > >>>>> re/services/attachListener.hpp#l106 > >>>>> > >>>>> >>> > >>>>> >>> > >>>>> >>> On 2019/02/27 15:00, ?? wrote: > >>>>> >>>> Another solution I can figure out is try to add timeout for > >>>>> socket read. I will also investigate whether is works. > >>>>> >>>> > >>>>> >>>> Cheers, > >>>>> >>>> Lin > >>>>> >>>> > >>>>> >>>> > >>>>> >>>>> -----Original Message----- > >>>>> >>>>> From: ?? > >>>>> >>>>> Sent: Wednesday, February 27, 2019 1:52 PM > >>>>> >>>>> To: 'David Holmes' >>>>> >; Yasumasa Suenaga > >>>>> >>>>> > > >>>>> >>>>> Cc: serviceability-dev at openjdk.java.net > >>>>> > >>>>> >>>>> Subject: RE: Protocol version of Attach API > >>>>> >>>>> > >>>>> >>>>> Dear David, Yasumasa, > >>>>> >>>>> I think it is hard to know how long the buffer is > >>>>> passed from socket > >>>>> >>>>> without changing the info of the message from the > >>>>> receiver side. > >>>>> >>>>> So how about when str_count reach to 3, we test it > >>>>> with non_blocking > >>>>> >>>>> read? > >>>>> >>>>> > >>>>> >>>>> > >>>>> >>>>> Cheers, > >>>>> >>>>> Lin > >>>>> >>>>>> -----Original Message----- > >>>>> >>>>>> From: David Holmes >>>>> > > >>>>> >>>>>> Sent: Wednesday, February 27, 2019 1:44 PM > >>>>> >>>>>> To: Yasumasa Suenaga >>>>> >; ?? >>>>> > > >>>>> >>>>>> Cc: serviceability-dev at openjdk.java.net > >>>>> > >>>>> >>>>>> Subject: Re: Protocol version of Attach API > >>>>> >>>>>> > >>>>> >>>>>> Hi Yasumasa, > >>>>> >>>>>> > >>>>> >>>>>> On 27/02/2019 1:05 pm, Yasumasa Suenaga wrote: > >>>>> >>>>>>> Hi Lin, > >>>>> >>>>>>> > >>>>> >>>>>>> My proposal is a just idea, so you need to tweak it. > >>>>> >>>>>>> > >>>>> >>>>>>> AttachListener receives raw command. > >>>>> >>>>>>> For example, jcmd is `jcmd\0`. Please see > >>>>> >>>>>>> HotSpotVirtualMachine and extended classes. > >>>>> >>>>>>> > >>>>> >>>>>>> In case of jcmd, I guess AttachListener will receive message > >>>>> >>>>>>> `\0jcmd\0\0\0\0` (I did not check it well). > >>>>> >>>>>>> So I guess we can add '\0' when `str_count` does not reach > >>>>> to maximum. > >>>>> >>>>>> > >>>>> >>>>>> I don't see how this approach can work. You have to know > >> how > >>>>> many > >>>>> >>>>>> arguments are coming in the "packet", but that information > >>>>> is not > >>>>> >>>>>> available in the current Linux implementation.Without it you > >>>>> can't > >>>>> >>>>>> know when to stop calling read(). > >>>>> >>>>>> > >>>>> >>>>>> The protocol would need to be changed to send the "packet" > >>>>> size, but > >>>>> >>>>>> that's not compatible with older JDKs. > >>>>> >>>>>> > >>>>> >>>>>> Otherwise I think we have no choice but to use a > >>>>> non-blocking read ... > >>>>> >>>>>> though I'm still unsure if you can know for certain > >>>>> that you've > >>>>> >>>>>> reached the end of the "packet" ... > >>>>> >>>>>> > >>>>> >>>>>> Thanks, > >>>>> >>>>>> David > >>>>> >>>>>> > >>>>> >>>>>>> > >>>>> >>>>>>> Thanks, > >>>>> >>>>>>> > >>>>> >>>>>>> Yasumasa > >>>>> >>>>>>> > >>>>> >>>>>>> > >>>>> >>>>>>> On 2019/02/27 11:50, zanglin5 at jd.com > >>>>> wrote: > >>>>> >>>>>>>> Dear Yasumasa, > >>>>> >>>>>>>> The fix you mentioned seems not work as expected. > >>>>> >>>>>>>> I have done an experiment that use jdk1.8's "jcmd > >>>>> help" to > >>>>> >>>>>>>> attach to latest jdk. > >>>>> >>>>>>>> it seem the whole "jcmd help" buffer is not > >>>>> >>>>>>>> read completely at one time. in my case it is "jcmd", > >>>>> "", > >>>>> >>>>>>>> "help", and then block at while(). > >>>>> >>>>>>>> After applied your change, it seems only the "jcmd" is > >>>>> >>>>>>>> processed, the "", "help" is replaced by '\0'. > >>>>> >>>>>>>> > >>>>> >>>>>>>> BRs, > >>>>> >>>>>>>> Lin > >>>>> >>>>>>>> > >>>>> From david.holmes at oracle.com Fri Mar 1 04:22:14 2019 From: david.holmes at oracle.com (David Holmes) Date: Fri, 1 Mar 2019 14:22:14 +1000 Subject: Protocol version of Attach API In-Reply-To: References: <04aadf26-d2c9-6f65-9815-06ad36c53219@gmail.com>

<4a5f251d-67ef-a81a-ef73-48f9aa9c853e@gmail.com> <0e7c23fec044419d845c5608b0586fcb@jd.com> <3dab08619bbd4254943361d2cdc1298b@jd.com>

<077ca61fe91f46ef93c4472accdf0337@jd.com> <8f6ec4c6-18f1-4f61-7ab0-6e1b40acf544@oracle.com> <22427d1123114a0d869eaade059fcd12@jd.com> <66511977ba394d7084d94a356f68f4b7@jd.com> <33bb67d77da949ab9410ea3cdb5dd1ce@jd.com> <650f2814-9bb6-39c1-87ab-583a206b6486@oracle.com> <5d2d51087deb43988867ae66918482aa@jd.com> Message-ID: On 1/03/2019 1:54 pm, Yasumasa Suenaga wrote: > Hi, > > I agree with David. I think we should backout 8215622. Note that I conceded that if Lin's proposal for changing the 4 back to 3 fixed everything, then I'm okay with making that fix rather than a full backout re-do. I'm not at all convinced we need change the number of args at the protocol level, regardless of the number of apparent "args" the command has. Cheers, David > We should re-do 8215622 completely for all platforms (includes > Windows) and support receiving requests from earlier serviceability > tools. > Problems about arguments of AttachListener should be worked as another issues. > > > Thanks, > > Yasumasa > > > 2019?3?1?(?) 12:39 ?? : >> >> Hi David, >> My understanding about those "arg" is that they are trying to make limitation of the overall message length and make it convince to communicate with sockets. >> I will do more test try to see whether changing back to 3 makes everything fine, and then prepare a webrev. >> Thanks. >> >> Lin >> ________________________________________ >> From: David Holmes >> Sent: Thursday, February 28, 2019 7:55:01 PM >> To: ??; Yasumasa Suenaga >> Cc: serviceability-dev at openjdk.java.net serviceability-dev at openjdk.java.net >> Subject: Re: Protocol version of Attach API >> >> Hi Lin, >> >> On 28/02/2019 7:30 pm, ?? wrote: >>> Hi David, >>> I am a little confused, do you think it is proper to made the patch as a fix of https://bugs.openjdk.java.net/browse/JDK-8219721 so that we don't need to backout and REDO? >> >> Generally I'd prefer to do the backout and then apply the revised fix as >> it will make the changes easier to track. >> >> However, if you are saying that everything works fine just by changing >> the 4 back to 3 everywhere, then that does seem a very simple fix to >> apply directly. >> >> I admit that if that does work then I really don't understand what these >> "arg" values actually means. :( Though it would explain why windows >> appears to work fine even though it was left at 3. >> >> Thanks, >> David >> >>> Thanks, >>> LIn >>> ________________________________________ >>> From: ?? >>> Sent: Thursday, February 28, 2019 4:50:12 PM >>> To: David Holmes; Yasumasa Suenaga >>> Cc: serviceability-dev at openjdk.java.net serviceability-dev at openjdk.java.net >>> Subject: Re: Protocol version of Attach API >>> >>> Dear All, >>> I have tried simply recover the max argument count makes jmap -histo works and also makes the compatibility works fine. >>> Following are the changes I made: >>> >>> diff -r 07dd34f487d4 src/hotspot/share/services/attachListener.hpp >>> --- a/src/hotspot/share/services/attachListener.hpp Thu Feb 28 02:47:39 2019 +0100 >>> +++ b/src/hotspot/share/services/attachListener.hpp Thu Feb 28 16:48:19 2019 +0800 >>> @@ -106,7 +106,7 @@ >>> enum { >>> name_length_max = 16, // maximum length of name >>> arg_length_max = 1024, // maximum length of argument >>> - arg_count_max = 4 // maximum number of arguments >>> + arg_count_max = 3 // maximum number of arguments >>> }; >>> >>> // name of special operation that can be enqueued when all >>> diff -r 07dd34f487d4 src/jdk.attach/aix/classes/sun/tools/attach/VirtualMachineImpl.java >>> --- a/src/jdk.attach/aix/classes/sun/tools/attach/VirtualMachineImpl.java Thu Feb 28 02:47:39 2019 +0100 >>> +++ b/src/jdk.attach/aix/classes/sun/tools/attach/VirtualMachineImpl.java Thu Feb 28 16:48:19 2019 +0800 >>> @@ -138,7 +138,7 @@ >>> * Execute the given command in the target VM. >>> */ >>> InputStream execute(String cmd, Object ... args) throws AgentLoadException, IOException { >>> - assert args.length <= 4; // includes null >>> + assert args.length <= 3; // includes null >>> >>> // did we detach? >>> synchronized (this) { >>> @@ -166,7 +166,7 @@ >>> writeString(s, PROTOCOL_VERSION); >>> writeString(s, cmd); >>> >>> - for (int i = 0; i < 4; i++) { >>> + for (int i = 0; i < 3; i++) { >>> if (i < args.length && args[i] != null) { >>> writeString(s, (String)args[i]); >>> } else { >>> diff -r 07dd34f487d4 src/jdk.attach/linux/classes/sun/tools/attach/VirtualMachineImpl.java >>> --- a/src/jdk.attach/linux/classes/sun/tools/attach/VirtualMachineImpl.java Thu Feb 28 02:47:39 2019 +0100 >>> +++ b/src/jdk.attach/linux/classes/sun/tools/attach/VirtualMachineImpl.java Thu Feb 28 16:48:19 2019 +0800 >>> @@ -143,7 +143,7 @@ >>> * Execute the given command in the target VM. >>> */ >>> InputStream execute(String cmd, Object ... args) throws AgentLoadException, IOException { >>> - assert args.length <= 4; // includes null >>> + assert args.length <= 3; // includes null >>> >>> // did we detach? >>> synchronized (this) { >>> diff -r 07dd34f487d4 src/jdk.attach/macosx/classes/sun/tools/attach/VirtualMachineImpl.java >>> --- a/src/jdk.attach/macosx/classes/sun/tools/attach/VirtualMachineImpl.java Thu Feb 28 02:47:39 2019 +0100 >>> +++ b/src/jdk.attach/macosx/classes/sun/tools/attach/VirtualMachineImpl.java Thu Feb 28 16:48:19 2019 +0800 >>> @@ -139,7 +139,7 @@ >>> * Execute the given command in the target VM. >>> */ >>> InputStream execute(String cmd, Object ... args) throws AgentLoadException, IOException { >>> - assert args.length <= 4; // includes null >>> + assert args.length <= 3; // includes null >>> >>> // did we detach? >>> synchronized (this) { >>> diff -r 07dd34f487d4 src/jdk.attach/solaris/classes/sun/tools/attach/VirtualMachineImpl.java >>> --- a/src/jdk.attach/solaris/classes/sun/tools/attach/VirtualMachineImpl.java Thu Feb 28 02:47:39 2019 +0100 >>> +++ b/src/jdk.attach/solaris/classes/sun/tools/attach/VirtualMachineImpl.java Thu Feb 28 16:48:19 2019 +0800 >>> @@ -126,7 +126,7 @@ >>> * Execute the given command in the target VM. >>> */ >>> InputStream execute(String cmd, Object ... args) throws AgentLoadException, IOException { >>> - assert args.length <= 4; // includes null >>> + assert args.length <= 3; // includes null >>> >>> // first check that we are still attached >>> int door; >>> diff -r 07dd34f487d4 src/jdk.attach/windows/classes/sun/tools/attach/VirtualMachineImpl.java >>> --- a/src/jdk.attach/windows/classes/sun/tools/attach/VirtualMachineImpl.java Thu Feb 28 02:47:39 2019 +0100 >>> +++ b/src/jdk.attach/windows/classes/sun/tools/attach/VirtualMachineImpl.java Thu Feb 28 16:48:19 2019 +0800 >>> @@ -77,7 +77,7 @@ >>> InputStream execute(String cmd, Object ... args) >>> throws AgentLoadException, IOException >>> { >>> - assert args.length <= 4; // includes null >>> + assert args.length <= 3; // includes null >>> >>> // create a pipe using a random name >>> Random rnd = new Random(); >>> >>> >>> >>> Thanks, >>> Lin >>> ________________________________________ >>> From: ?? >>> Sent: Thursday, February 28, 2019 3:24:52 PM >>> To: David Holmes; Yasumasa Suenaga >>> Cc: serviceability-dev at openjdk.java.net serviceability-dev at openjdk.java.net >>> Subject: RE: Protocol version of Attach API >>> >>> Hi David, >>> Since I don't have the access to JBS, may I ask your help to ceate sub-task? Thanks. >>> >>> BRs, >>> Lin >>> >>>> -----Original Message----- >>>> From: David Holmes >>>> Sent: Thursday, February 28, 2019 3:16 PM >>>> To: ?? ; Yasumasa Suenaga >>>> Cc: serviceability-dev at openjdk.java.net serviceability-dev at openjdk.java.net >>>> >>>> Subject: Re: Protocol version of Attach API >>>> >>>> Hi Lin, >>>> >>>> On 28/02/2019 4:49 pm, ?? wrote: >>>>> Hi David, >>>>> Your are right and thanks for pointing it out. when I worte that patch, I >>>> was considering implement -filename and -incremental together. and I must >>>> be too stupid to forget recover it when I divided the patch into two. >>>>> And it seems a good solution is to refine the original patch of jmap histo, >>>> and try to composite all args as one when passing it to socket and let >>>> attachlistener to handle the analyze. >>>>> I will try that. >>>>> One more, do I need to consider changing the jmap -dump also? >>>> >>>> I'm assuming -dump already works fine, so I'm just expecting to see -histo >>>> handle the file in a similar manner. >>>> >>>> If you find this works I suggest creating a sub-task of 8215622 to first backout >>>> the original changes (hg backout), and a second sub-task to REDO with the >>>> new implementation. Each will need reviewing separately in their own RFR >>>> thread. >>>> >>>> Thanks, >>>> David >>>> >>>>> BRs, >>>>> Lin >>>>> ________________________________________ >>>>> From: David Holmes >>>>> Sent: Thursday, February 28, 2019 12:59:28 PM >>>>> To: ??; Yasumasa Suenaga >>>>> Cc: serviceability-dev at openjdk.java.net >>>>> serviceability-dev at openjdk.java.net >>>>> Subject: Re: Protocol version of Attach API >>>>> >>>>> Sorry I'm going to pick up on the rollback and re-do option here as I >>>>> just had a closer look at jmap. Given jmap -dump already has more >>>>> options than -histo does, why was any change to the "maximum number of >>>>> args" needed in the first place ??? >>>>> >>>>> David >>>>> >>>>> On 28/02/2019 2:43 pm, David Holmes wrote: >>>>>> Hi everyone, >>>>>> >>>>>> I'm not sure we're converging on a suitable solution here, but to >>>>>> address the issues flagged by Lin below ... >>>>>> >>>>>> On 28/02/2019 12:39 pm, ?? wrote: >>>>>>> Hi Suenaga, >>>>>>> >>>>>>> Thanks for your expaination about the arg_length_max, I >>>>>>> generally agree with you that it is better to consider using dynamic >>>>>>> memory, and that would be handled carefully to aviod introduce >>>>>>> compatibility issue, plus it would be a big change. So let?s see >>>>>>> what others suggest. >>>>>>> >>>>>>> Hi All, >>>>>>> >>>>>>> It seems for me that there are basically three problems forked by >>>>>>> this >>>>>>> thread? >>>>>>> >>>>>>> ?Compatibility issue with old jcmd alike tools with attachListener?s >>>>>>> change. >>>>>> >>>>>> This is issue: >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8219721 >>>>>> >>>>>>> ?Only 3 arguments limited to passed by socket to attachListener for >>>>>>> Windows, which cause 8215622 work abnormally on Windows. >>>>>> >>>>>> I have filed a new bug for this: >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8219895 >>>>>> >>>>>>> ?The arg_length_max may not be enough for handling filename. >>>>>> >>>>>> I have filed a new bug for this: >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8219896 >>>>>> >>>>>> though it seems very related to this issue. >>>>>> >>>>>>> So I suggest we keep the first problem discussed in this thread, and >>>>>>> create separate thread for the others. Do you agree? >>>>>> >>>>>> There is some overlap but yes this can be broken down somewhat - >>>>>> though dealing with the variable length "packet" is going to have to >>>>>> consider that what is received is in fact much larger than the >>>>>> purported maximum packet size if these long paths are expected and >>>> accepted. >>>>>> >>>>>> FWIW I don't see crashes or anything drastic if the arguments are too >>>>>> long - the operations just fail (in somewhat obscure ways sometimes). >>>>>> >>>>>>> >>>>>>> For me, I will refine my patch to use timeout as a fix for the first >>>>>>> one, and update it in this thread. And I will try to fix the second >>>>>>> one for Windows, and create a separate thread for discussing. And if >>>>>>> possible, I can help to fix the third one. >>>>>>> >>>>>>> What?s your opinion? >>>>>> >>>>>> That sounds fine ... >>>>>> >>>>>> Or you could choose to rollback JDK-8215622 and see how to solve that >>>>>> without increasing the arg count. Given this usage: >>>>>> >>>>>> jmap -histo:live,file=foo.txt >>>>>> >>>>>> I'm not sure why this is sent to the VM as multiple args rather than >>>>>> as a single composite arg that can then be parsed again by the actual >>>>>> "jmap" logic. There would be some double-up perhaps if the front-end >>>>>> tool wants to perform the command-line validation, but it would be >>>>>> easy enough I think to do that checking then send the original composite >>>> arg. >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> ----- >>>>>> >>>>>> >>>>>>> BRs, >>>>>>> >>>>>>> Lin >>>>>>> >>>>>>> *From:*Yasumasa Suenaga >>>>>>> *Sent:* Thursday, February 28, 2019 8:39 AM >>>>>>> *To:* ?? >>>>>>> *Cc:* David Holmes ; >>>>>>> serviceability-dev at openjdk.java.net >>>>>>> serviceability-dev at openjdk.java.net >>>>>>> >>>>>>> *Subject:* Re: Protocol version of Attach API >>>>>>> >>>>>>> 2019?2?28?(?) 0:04 ?? >>> >: >>>>>>> >>>>>>> Dear Suenaga, >>>>>>> Thanks for your reviewing. I will try to refine the patch. >>>>>>> For the argument length you mentioned, do you mean the >>>>>>> "arg_length_max" should be large enough to accept the max filename >>>>>>> length? >>>>>>> >>>>>>> Yes, but it is not enough. >>>>>>> >>>>>>> For example, jcmd VM.log might pass 2 or more paths to define logs. >>>>>>> >>>>>>> IMHO, all the handling of the argument length is at receiver >>>>>>> side in the attachListener, such as >>>>>>> >>>>>>> >>>> http://hg.openjdk.java.net/jdk/jdk/file/df3d253aaf81/src/hotspot/os/linux/att >>>> achListener_linux.cpp#l322, >>>>>>> for me it means that the VM side limits the argments length less >>>>>>> than arg_length_max, otherwise it will return NULL, which cause the >>>>>>> sender side (tools like jcmd and jmap) exits with error message. so >>>>>>> I think there may be no need to limit the argument size in tool side. >>>>>>> >>>>>>> IMHO all programs which use filesystem should support any locations >>>>>>> on it. >>>>>>> >>>>>>> So I think we should use dynamic memory (or GrowableArray) for it if >>>>>>> we do not change client side for compatibility. >>>>>>> >>>>>>> And from my experiment with jmap, the arguments send to >>>>>>> sockets are not arg0 only. as you can see in >>>>>>> >>>>>>> http://hg.openjdk.java.net/jdk/jdk/file/df3d253aaf81/src/jdk.jcmd/sh >>>>>>> are/classes/sun/tools/jmap/JMap.java#l193 >>>>>>> >>>>>>> and >>>>>>> >>>>>>> http://hg.openjdk.java.net/jdk/jdk/file/df3d253aaf81/src/jdk.jcmd/sh >>>>>>> are/classes/sun/tools/jmap/JMap.java#l133, >>>>>>> >>>>>>> jmap can pass arg0 as "filename", and arg1 as "-live", and both of >>>>>>> them can be NULL. so 00000 can be >>>>>>> 0000, and file can be null. so 00 may not >>>>>>> indicate it reach the end. >>>>>>> >>>>>>> We should consider for other tools - jstack and jinfo. >>>>>>> >>>>>>> (jstack is ok because it will not have long arguments) >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> BRs, >>>>>>> Lin >>>>>>> ________________________________________ >>>>>>> From: Yasumasa Suenaga >>>>>> > >>>>>>> Sent: Wednesday, February 27, 2019 8:10:14 PM >>>>>>> To: ?? >>>>>>> Cc: David Holmes; serviceability-dev at openjdk.java.net >>>>>>> >>>>>>> Subject: Re: Protocol version of Attach API >>>>>>> >>>>>>> Hi Lin, >>>>>>> >>>>>>> I think we need to research more about this. >>>>>>> IMHO we need to match length of arguments between >>>>>>> server (AttachListener) and client (serviceability tools) at least. >>>>>>> (please see previous email from me). >>>>>>> >>>>>>> I have some comments for your change: >>>>>>> >>>>>>> On 2019/02/27 18:22, ?? wrote: >>>>>>> > Dear All, >>>>>>> > Here I have figured out one solution based on timeout. would >>>>>>> you like help to see whether this is OK? >>>>>>> > --- a/src/hotspot/os/linux/attachListener_linux.cpp Tue Feb >>>>>>> 26 14:57:23 2019 +0530 >>>>>>> > +++ b/src/hotspot/os/linux/attachListener_linux.cpp Wed Feb >>>>>>> 27 17:21:48 2019 +0800 >>>>>>> > @@ -263,9 +263,29 @@ >>>>>>> > int off = 0; >>>>>>> > int left = max_len; >>>>>>> > >>>>>>> > + memset(buf, 0, max_len); >>>>>>> > + // set timeout for read >>>>>>> > + struct timeval timeout; >>>>>>> > + timeout.tv_sec = 3; >>>>>>> > + timeout.tv_usec = 0; >>>>>>> >>>>>>> I think timeout value should be configurable. >>>>>>> For example, we can introduce new flag in globals.hpp . >>>>>>> >>>>>>> >>>>>>> > + if(setsockopt(s, SOL_SOCKET, SO_RCVTIMEO, (struct >>>>>>> timeval*)&timeout, sizeof(timeout))) { >>>>>>> > + log_debug(attach)("Failed to set socket option SO_RCVTIMEO: >>>>>>> %s\n", strerror(errorno)); >>>>>>> > + printf("Warning: Failed to set socket option SO_RCVTIMEO: >>>>>>> %s!!!\n", strerror(errno)); >>>>>>> >>>>>>> You should not use printf(), and do you need to pass '\n' to >>>>>>> log_debug()? >>>>>>> >>>>>>> >>>>>>> > + } >>>>>>> > do { >>>>>>> > int n; >>>>>>> > - RESTARTABLE(read(s, buf+off, left), n); >>>>>>> > + log_debug(attach)("start reading data from socket....\n"); >>>>>>> > + n = read(s, buf+off, left); >>>>>>> >>>>>>> You should use RESTARTABLE macro. >>>>>>> read(2) might be interrupted by signal. >>>>>>> >>>>>>> >>>>>>> > + if (n < 0) { >>>>>>> > + if (errno == EWOULDBLOCK) { >>>>>>> >>>>>>> According to man page, read(2) sets EWOULDBLOCK or EAGAIN. >>>>>>> So you should check both errno. >>>>>>> >>>>>>> >>>>>>> > + for (int i = str_count; i < expected_str_count; i++) { >>>>>>> > + //timeout, fill reminded arguments with \0; >>>>>>> > + buf[off+i] = '\0'; >>>>>>> > + str_count++; >>>>>>> > + } >>>>>>> >>>>>>> You set zero to buf[] in above. >>>>>>> So you can remove this loop, and set str_count to expected_str_count >>>>>>> without manipulating buf[]. >>>>>>> >>>>>>> In addition, I prefer to add log_debug() at this >>>>>>> to record NULL arguments are added. >>>>>>> >>>>>>> >>>>>>> > + break;; >>>>>>> > + } >>>>>>> > + } >>>>>>> > assert(n <= left, "buffer was too small, impossible!"); >>>>>>> > buf[max_len - 1] = '\0'; >>>>>>> > if (n == -1) { >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> > Thanks. >>>>>>> > Lin >>>>>>> > >>>>>>> > ________________________________________ >>>>>>> > From: Yasumasa Suenaga >>>>>> > >>>>>>> > Sent: Wednesday, February 27, 2019 15:15 >>>>>>> > To: David Holmes; ?? >>>>>>> > Cc: serviceability-dev at openjdk.java.net >>>>>>> >>>>>>> > Subject: Re: Protocol version of Attach API >>>>>>> > >>>>>>> > On 2019/02/27 15:59, David Holmes wrote: >>>>>>> >> On 27/02/2019 4:10 pm, Yasumasa Suenaga wrote: >>>>>>> >>> Hi, >>>>>>> >>> >>>>>>> >>> Buffer size for receiving packets from client is >>>>>>> determined at [1]. >>>>>>> >> >>>>>>> >> Maximum buffer size, yes. >>>>>>> >> >>>>>>> >>> It defines length of command name and of argument. >>>>>>> >>> It is passed via Unix domain, so we fill '\0' to remaining >>>>>>> bytes and >>>>>>> >>> might be able to assume all arguments are passed with >>>>>>> empty string. >>>>>>> >> >>>>>>> >> Not sure what you mean. >>>>>>> >> >>>>>>> >> // The buffer is expected to be formatted as follows: >>>>>>> >> // 00000 >>>>>>> >> >>>>>>> >> so we can expect to read at least two things - the ver and cmd. >>>>>>> If we encounter 00 we can infer we reached the end. But we may not >>>>>>> have read the full data into the buffer, so can't tell if another >>>>>>> call to read() is needed yet - you only know after you've read >>>>>>> the 00. >>>>>>> >> >>>>>>> >>> BTW length of arguments is defined to 1024 in [2]. >>>>>>> >>> jcmd and jmap might pas file path - it might be JVM_MAXPATHLEN >>>>>>> (4097 bytes). >>>>>>> >>> Buffer size which is used in AttachListener seems not to >>>>>>> be enough. >>>>>>> >> >>>>>>> >> One has to assume/hope that the code sending the data is working >>>>>>> to the same protocol rules as the receiver! Otherwise this is just >>>>>>> completely broken. >>>>>>> > >>>>>>> > On Linux, client (VirtualMachineImpl) seems not to check length >>>>>>> of arguments: >>>>>>> > >>>>>>> >>>>>>> http://hg.openjdk.java.net/jdk/jdk/file/df3d253aaf81/src/jdk.attach/ >>>>>>> linux/classes/sun/tools/attach/VirtualMachineImpl.java#l168 >>>>>>> >>>>>>> > >>>>>>> > In case of jcmd, all options are passed to arg #1. It seems not >>>>>>> to check the length: >>>>>>> > >>>>>>> >>>>>>> http://hg.openjdk.java.net/jdk/jdk/file/df3d253aaf81/src/jdk.jcmd/sh >>>>>>> are/classes/sun/tools/jcmd/JCmd.java#l111 >>>>>>> >>>>>>> > >>>>>>> > >>>>>>> > I guess other tools (jstack, jmap, etc) which connect to >>>>>>> AttachListener are same. >>>>>>> > So we can fix both Attach API and AttachListener (it will be big >>>>>>> change!), >>>>>>> > but I concern we can keep protocol version... >>>>>>> > >>>>>>> > >>>>>>> > Thanks, >>>>>>> > >>>>>>> > Yasumasa >>>>>>> > >>>>>>> > >>>>>>> >> David >>>>>>> >> ----- >>>>>>> >> >>>>>>> >>> We might have to change more. >>>>>>> >>> >>>>>>> >>> >>>>>>> >>> Thanks, >>>>>>> >>> >>>>>>> >>> Yasumasa >>>>>>> >>> >>>>>>> >>> >>>>>>> >>> [1] >>>>>>> >>>>>>> http://hg.openjdk.java.net/jdk/jdk/file/df3d253aaf81/src/hotspot/os/ >>>>>>> linux/attachListener_linux.cpp#l254 >>>>>>> >>>>>>> >>> [2] >>>>>>> >>>>>>> http://hg.openjdk.java.net/jdk/jdk/file/df3d253aaf81/src/hotspot/sha >>>>>>> re/services/attachListener.hpp#l106 >>>>>>> >>>>>>> >>> >>>>>>> >>> >>>>>>> >>> On 2019/02/27 15:00, ?? wrote: >>>>>>> >>>> Another solution I can figure out is try to add timeout for >>>>>>> socket read. I will also investigate whether is works. >>>>>>> >>>> >>>>>>> >>>> Cheers, >>>>>>> >>>> Lin >>>>>>> >>>> >>>>>>> >>>> >>>>>>> >>>>> -----Original Message----- >>>>>>> >>>>> From: ?? >>>>>>> >>>>> Sent: Wednesday, February 27, 2019 1:52 PM >>>>>>> >>>>> To: 'David Holmes' >>>>>> >; Yasumasa Suenaga >>>>>>> >>>>> > >>>>>>> >>>>> Cc: serviceability-dev at openjdk.java.net >>>>>>> >>>>>>> >>>>> Subject: RE: Protocol version of Attach API >>>>>>> >>>>> >>>>>>> >>>>> Dear David, Yasumasa, >>>>>>> >>>>> I think it is hard to know how long the buffer is >>>>>>> passed from socket >>>>>>> >>>>> without changing the info of the message from the >>>>>>> receiver side. >>>>>>> >>>>> So how about when str_count reach to 3, we test it >>>>>>> with non_blocking >>>>>>> >>>>> read? >>>>>>> >>>>> >>>>>>> >>>>> >>>>>>> >>>>> Cheers, >>>>>>> >>>>> Lin >>>>>>> >>>>>> -----Original Message----- >>>>>>> >>>>>> From: David Holmes >>>>>> > >>>>>>> >>>>>> Sent: Wednesday, February 27, 2019 1:44 PM >>>>>>> >>>>>> To: Yasumasa Suenaga >>>>>> >; ?? >>>>>> > >>>>>>> >>>>>> Cc: serviceability-dev at openjdk.java.net >>>>>>> >>>>>>> >>>>>> Subject: Re: Protocol version of Attach API >>>>>>> >>>>>> >>>>>>> >>>>>> Hi Yasumasa, >>>>>>> >>>>>> >>>>>>> >>>>>> On 27/02/2019 1:05 pm, Yasumasa Suenaga wrote: >>>>>>> >>>>>>> Hi Lin, >>>>>>> >>>>>>> >>>>>>> >>>>>>> My proposal is a just idea, so you need to tweak it. >>>>>>> >>>>>>> >>>>>>> >>>>>>> AttachListener receives raw command. >>>>>>> >>>>>>> For example, jcmd is `jcmd\0`. Please see >>>>>>> >>>>>>> HotSpotVirtualMachine and extended classes. >>>>>>> >>>>>>> >>>>>>> >>>>>>> In case of jcmd, I guess AttachListener will receive message >>>>>>> >>>>>>> `\0jcmd\0\0\0\0` (I did not check it well). >>>>>>> >>>>>>> So I guess we can add '\0' when `str_count` does not reach >>>>>>> to maximum. >>>>>>> >>>>>> >>>>>>> >>>>>> I don't see how this approach can work. You have to know >>>> how >>>>>>> many >>>>>>> >>>>>> arguments are coming in the "packet", but that information >>>>>>> is not >>>>>>> >>>>>> available in the current Linux implementation.Without it you >>>>>>> can't >>>>>>> >>>>>> know when to stop calling read(). >>>>>>> >>>>>> >>>>>>> >>>>>> The protocol would need to be changed to send the "packet" >>>>>>> size, but >>>>>>> >>>>>> that's not compatible with older JDKs. >>>>>>> >>>>>> >>>>>>> >>>>>> Otherwise I think we have no choice but to use a >>>>>>> non-blocking read ... >>>>>>> >>>>>> though I'm still unsure if you can know for certain >>>>>>> that you've >>>>>>> >>>>>> reached the end of the "packet" ... >>>>>>> >>>>>> >>>>>>> >>>>>> Thanks, >>>>>>> >>>>>> David >>>>>>> >>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 2019/02/27 11:50, zanglin5 at jd.com >>>>>>> wrote: >>>>>>> >>>>>>>> Dear Yasumasa, >>>>>>> >>>>>>>> The fix you mentioned seems not work as expected. >>>>>>> >>>>>>>> I have done an experiment that use jdk1.8's "jcmd >>>>>>> help" to >>>>>>> >>>>>>>> attach to latest jdk. >>>>>>> >>>>>>>> it seem the whole "jcmd help" buffer is not >>>>>>> >>>>>>>> read completely at one time. in my case it is "jcmd", >>>>>>> "", >>>>>>> >>>>>>>> "help", and then block at while(). >>>>>>> >>>>>>>> After applied your change, it seems only the "jcmd" is >>>>>>> >>>>>>>> processed, the "", "help" is replaced by '\0'. >>>>>>> >>>>>>>> >>>>>>> >>>>>>>> BRs, >>>>>>> >>>>>>>> Lin >>>>>>> >>>>>>>> >>>>>>> From dmitry.chuyko at bell-sw.com Fri Mar 1 07:12:51 2019 From: dmitry.chuyko at bell-sw.com (Dmitry Chuyko) Date: Fri, 1 Mar 2019 10:12:51 +0300 Subject: RFR(S): 8214854: JDWP: Unforseen output truncation in logging In-Reply-To: References: <6b080c3a-27c9-fff5-147c-c5a3baa44c5c@bell-sw.com> <831d28ed-1fd8-90d3-1090-b434e5a6eeb2@oracle.com> Message-ID: <031b21df-8ff7-b08b-8c81-a2825f428bdf@bell-sw.com> On 3/1/19 6:36 AM, Chris Plummer wrote: > On 2/28/19 5:07 PM, David Holmes wrote: >> Hi Dmitry, >> >> On 1/03/2019 6:15 am, Dmitry Chuyko wrote: >>> Hello, >>> >>> Please review a small fix for GCC 8.x warning in log_messages.c. >>> Buffer sizes for parts of timestamp string can be adjusted to not >>> exceed maximum buffer size when combined, while being able to hold >>> necessary information. >>> >>> webrev: http://cr.openjdk.java.net/~dchuyko/8214854/webrev.00/ >>> bug: https://bugs.openjdk.java.net/browse/JDK-8214854 >>> testing: after the fix GCC 8 compiles working OpenJDK on Linux. >>> Dev-submit job started. >> >> This looks good. >> >> One query: >> >> ?51 typedef char timestamp_buffer[TIMESTAMP_SIZE]; >> >> Is this needed so that the size of the buffer is known by gcc inside >> get_time_stamp? (Generally I prefer to see explicitly when an array >> is being stack allocated, not have it hidden behind a typedef.) > This also caught my attention and then got me thinking on what basis > is gcc producing the warning in the first place. The warning is: > > ?src/jdk.jdwp.agent/share/native/libjdwp/log_messages.c:75:24: error: > '%.3d' directive output may be truncated writing between 3 and 11 > bytes into a region of size between 0 and 80 [-Werror=format-truncation=] > ??????????????????? "%s.%.3d %s", timestamp_prefix, > ??????????????????????? ^~~~ > > I think it deduced this by seeing that the call to get_time_stamp() is > passing in a char[80] for the tbuf argument. It then also sees that > the argument for the first %s is also a char[80] array, so is pointing > out that this first %s might fill (or nearly fill) tbuf, not leaving > room for the %.3d argument. If this is the case, there is no need for > the timestamp_buffer typedef. The real fix was making > timestamp_date_time and timestamp_timezone smaller so gcc doesn't > think they can overflow tbuf. > > Chris Current signature of helper function is get_time_stamp(char *tbuf, size_t ltbuf). If we drop typedef change, this function either logically needs ltbuf parameter or it internally assumes it to be somewhat (TIMESTAMP_SIZE+1). In first case we in general should care about arbitrary ltbuf size. Like 5 or 2. In second case there are no static checks/hints that we don't pass tbuf of diferent size. So I preferred to have this information in the function signature and to use known lengths of parts: get_time_stamp(timestamp_buffer tbuf). -Dmitry > > >> >> Thanks, >> David >> >> >>> -Dmitry >>> > > From zanglin5 at jd.com Fri Mar 1 09:22:47 2019 From: zanglin5 at jd.com (=?ks_c_5601-1987?B?7fXX+w==?=) Date: Fri, 1 Mar 2019 09:22:47 +0000 Subject: Protocol version of Attach API In-Reply-To: References: <04aadf26-d2c9-6f65-9815-06ad36c53219@gmail.com>

<4a5f251d-67ef-a81a-ef73-48f9aa9c853e@gmail.com> <0e7c23fec044419d845c5608b0586fcb@jd.com> <3dab08619bbd4254943361d2cdc1298b@jd.com>

Message-ID: Dear All, I have upload a webrev at http://cr.openjdk.java.net/~xiaofeya/8219721/webrev.00/ (just found there is a jank file "abc", I will omit it in next webrev, but let's review other changes first for time saving.) And here are my comments and questions: * With this patch , I have tested with jdk8/jdk11's jcmd/jmap, all works as expected * And passed tier1 test on my linux box * Besides change from 4 to 3 , I also found one compatibility issue of using old "jmap" like "jmap -histo:live", caused by the sequence of arguments for inspectheap. And I have include the fix in the webrev. What I am concerned is that is this trivial enough to avoid revert and redo. IMO, if you think it shoud be revert 8215622 with this change, please let me know. Thanks for all of your help and suggestions. BRs, Lin > -----Original Message----- > From: David Holmes > Sent: Friday, March 1, 2019 12:22 PM > To: Yasumasa Suenaga ; ?? > Cc: serviceability-dev at openjdk.java.net serviceability-dev at openjdk.java.net > > Subject: Re: Protocol version of Attach API > > On 1/03/2019 1:54 pm, Yasumasa Suenaga wrote: > > Hi, > > > > I agree with David. I think we should backout 8215622. > > Note that I conceded that if Lin's proposal for changing the 4 back to 3 fixed > everything, then I'm okay with making that fix rather than a full backout re- > do. > > I'm not at all convinced we need change the number of args at the protocol > level, regardless of the number of apparent "args" the command has. > > Cheers, > David > > > We should re-do 8215622 completely for all platforms (includes > > Windows) and support receiving requests from earlier serviceability > > tools. > > Problems about arguments of AttachListener should be worked as another > issues. > > > > > > Thanks, > > > > Yasumasa > > > > > > 2019?3?1?(?) 12:39 ?? : > >> > >> Hi David, > >> My understanding about those "arg" is that they are trying to make > limitation of the overall message length and make it convince to communicate > with sockets. > >> I will do more test try to see whether changing back to 3 makes > everything fine, and then prepare a webrev. > >> Thanks. > >> > >> Lin > >> ________________________________________ > >> From: David Holmes > >> Sent: Thursday, February 28, 2019 7:55:01 PM > >> To: ??; Yasumasa Suenaga > >> Cc: serviceability-dev at openjdk.java.net > >> serviceability-dev at openjdk.java.net > >> Subject: Re: Protocol version of Attach API > >> > >> Hi Lin, > >> > >> On 28/02/2019 7:30 pm, ?? wrote: > >>> Hi David, > >>> I am a little confused, do you think it is proper to made the patch as a > fix of https://bugs.openjdk.java.net/browse/JDK-8219721 so that we don't > need to backout and REDO? > >> > >> Generally I'd prefer to do the backout and then apply the revised fix > >> as it will make the changes easier to track. > >> > >> However, if you are saying that everything works fine just by > >> changing the 4 back to 3 everywhere, then that does seem a very > >> simple fix to apply directly. > >> > >> I admit that if that does work then I really don't understand what > >> these "arg" values actually means. :( Though it would explain why > >> windows appears to work fine even though it was left at 3. > >> > >> Thanks, > >> David > >> > >>> Thanks, > >>> LIn > >>> ________________________________________ > >>> From: ?? > >>> Sent: Thursday, February 28, 2019 4:50:12 PM > >>> To: David Holmes; Yasumasa Suenaga > >>> Cc: serviceability-dev at openjdk.java.net > >>> serviceability-dev at openjdk.java.net > >>> Subject: Re: Protocol version of Attach API > >>> > >>> Dear All, > >>> I have tried simply recover the max argument count makes jmap - > histo works and also makes the compatibility works fine. > >>> Following are the changes I made: > >>> > >>> diff -r 07dd34f487d4 src/hotspot/share/services/attachListener.hpp > >>> --- a/src/hotspot/share/services/attachListener.hpp Thu Feb 28 > 02:47:39 2019 +0100 > >>> +++ b/src/hotspot/share/services/attachListener.hpp Thu Feb 28 > 16:48:19 2019 +0800 > >>> @@ -106,7 +106,7 @@ > >>> enum { > >>> name_length_max = 16, // maximum length of name > >>> arg_length_max = 1024, // maximum length of argument > >>> - arg_count_max = 4 // maximum number of arguments > >>> + arg_count_max = 3 // maximum number of arguments > >>> }; > >>> > >>> // name of special operation that can be enqueued when all diff > >>> -r 07dd34f487d4 > src/jdk.attach/aix/classes/sun/tools/attach/VirtualMachineImpl.java > >>> --- a/src/jdk.attach/aix/classes/sun/tools/attach/VirtualMachineImpl.java > Thu Feb 28 02:47:39 2019 +0100 > >>> +++ > b/src/jdk.attach/aix/classes/sun/tools/attach/VirtualMachineImpl.java Thu > Feb 28 16:48:19 2019 +0800 > >>> @@ -138,7 +138,7 @@ > >>> * Execute the given command in the target VM. > >>> */ > >>> InputStream execute(String cmd, Object ... args) throws > AgentLoadException, IOException { > >>> - assert args.length <= 4; // includes null > >>> + assert args.length <= 3; // includes null > >>> > >>> // did we detach? > >>> synchronized (this) { > >>> @@ -166,7 +166,7 @@ > >>> writeString(s, PROTOCOL_VERSION); > >>> writeString(s, cmd); > >>> > >>> - for (int i = 0; i < 4; i++) { > >>> + for (int i = 0; i < 3; i++) { > >>> if (i < args.length && args[i] != null) { > >>> writeString(s, (String)args[i]); > >>> } else { > >>> diff -r 07dd34f487d4 > src/jdk.attach/linux/classes/sun/tools/attach/VirtualMachineImpl.java > >>> --- > a/src/jdk.attach/linux/classes/sun/tools/attach/VirtualMachineImpl.java > Thu Feb 28 02:47:39 2019 +0100 > >>> +++ > b/src/jdk.attach/linux/classes/sun/tools/attach/VirtualMachineImpl.java > Thu Feb 28 16:48:19 2019 +0800 > >>> @@ -143,7 +143,7 @@ > >>> * Execute the given command in the target VM. > >>> */ > >>> InputStream execute(String cmd, Object ... args) throws > AgentLoadException, IOException { > >>> - assert args.length <= 4; // includes null > >>> + assert args.length <= 3; // includes null > >>> > >>> // did we detach? > >>> synchronized (this) { > >>> diff -r 07dd34f487d4 > src/jdk.attach/macosx/classes/sun/tools/attach/VirtualMachineImpl.java > >>> --- > a/src/jdk.attach/macosx/classes/sun/tools/attach/VirtualMachineImpl.java > Thu Feb 28 02:47:39 2019 +0100 > >>> +++ > b/src/jdk.attach/macosx/classes/sun/tools/attach/VirtualMachineImpl.java > Thu Feb 28 16:48:19 2019 +0800 > >>> @@ -139,7 +139,7 @@ > >>> * Execute the given command in the target VM. > >>> */ > >>> InputStream execute(String cmd, Object ... args) throws > AgentLoadException, IOException { > >>> - assert args.length <= 4; // includes null > >>> + assert args.length <= 3; // includes null > >>> > >>> // did we detach? > >>> synchronized (this) { > >>> diff -r 07dd34f487d4 > src/jdk.attach/solaris/classes/sun/tools/attach/VirtualMachineImpl.java > >>> --- > a/src/jdk.attach/solaris/classes/sun/tools/attach/VirtualMachineImpl.java > Thu Feb 28 02:47:39 2019 +0100 > >>> +++ > b/src/jdk.attach/solaris/classes/sun/tools/attach/VirtualMachineImpl.java > Thu Feb 28 16:48:19 2019 +0800 > >>> @@ -126,7 +126,7 @@ > >>> * Execute the given command in the target VM. > >>> */ > >>> InputStream execute(String cmd, Object ... args) throws > AgentLoadException, IOException { > >>> - assert args.length <= 4; // includes null > >>> + assert args.length <= 3; // includes null > >>> > >>> // first check that we are still attached > >>> int door; > >>> diff -r 07dd34f487d4 > src/jdk.attach/windows/classes/sun/tools/attach/VirtualMachineImpl.java > >>> --- > a/src/jdk.attach/windows/classes/sun/tools/attach/VirtualMachineImpl.java > Thu Feb 28 02:47:39 2019 +0100 > >>> +++ > b/src/jdk.attach/windows/classes/sun/tools/attach/VirtualMachineImpl.java > Thu Feb 28 16:48:19 2019 +0800 > >>> @@ -77,7 +77,7 @@ > >>> InputStream execute(String cmd, Object ... args) > >>> throws AgentLoadException, IOException > >>> { > >>> - assert args.length <= 4; // includes null > >>> + assert args.length <= 3; // includes null > >>> > >>> // create a pipe using a random name > >>> Random rnd = new Random(); > >>> > >>> > >>> > >>> Thanks, > >>> Lin > >>> ________________________________________ > >>> From: ?? > >>> Sent: Thursday, February 28, 2019 3:24:52 PM > >>> To: David Holmes; Yasumasa Suenaga > >>> Cc: serviceability-dev at openjdk.java.net > >>> serviceability-dev at openjdk.java.net > >>> Subject: RE: Protocol version of Attach API > >>> > >>> Hi David, > >>> Since I don't have the access to JBS, may I ask your help to ceate sub- > task? Thanks. > >>> > >>> BRs, > >>> Lin > >>> > >>>> -----Original Message----- > >>>> From: David Holmes > >>>> Sent: Thursday, February 28, 2019 3:16 PM > >>>> To: ?? ; Yasumasa Suenaga > > >>>> Cc: serviceability-dev at openjdk.java.net > >>>> serviceability-dev at openjdk.java.net > >>>> > >>>> Subject: Re: Protocol version of Attach API > >>>> > >>>> Hi Lin, > >>>> > >>>> On 28/02/2019 4:49 pm, ?? wrote: > >>>>> Hi David, > >>>>> Your are right and thanks for pointing it out. when I worte > >>>>> that patch, I > >>>> was considering implement -filename and -incremental together. and > >>>> I must be too stupid to forget recover it when I divided the patch into > two. > >>>>> And it seems a good solution is to refine the original > >>>>> patch of jmap histo, > >>>> and try to composite all args as one when passing it to socket and > >>>> let attachlistener to handle the analyze. > >>>>> I will try that. > >>>>> One more, do I need to consider changing the jmap -dump also? > >>>> > >>>> I'm assuming -dump already works fine, so I'm just expecting to see > >>>> -histo handle the file in a similar manner. > >>>> > >>>> If you find this works I suggest creating a sub-task of 8215622 to > >>>> first backout the original changes (hg backout), and a second > >>>> sub-task to REDO with the new implementation. Each will need > >>>> reviewing separately in their own RFR thread. > >>>> > >>>> Thanks, > >>>> David > >>>> > >>>>> BRs, > >>>>> Lin > >>>>> ________________________________________ > >>>>> From: David Holmes > >>>>> Sent: Thursday, February 28, 2019 12:59:28 PM > >>>>> To: ??; Yasumasa Suenaga > >>>>> Cc: serviceability-dev at openjdk.java.net > >>>>> serviceability-dev at openjdk.java.net > >>>>> Subject: Re: Protocol version of Attach API > >>>>> > >>>>> Sorry I'm going to pick up on the rollback and re-do option here > >>>>> as I just had a closer look at jmap. Given jmap -dump already has > >>>>> more options than -histo does, why was any change to the "maximum > >>>>> number of args" needed in the first place ??? > >>>>> > >>>>> David > >>>>> > >>>>> On 28/02/2019 2:43 pm, David Holmes wrote: > >>>>>> Hi everyone, > >>>>>> > >>>>>> I'm not sure we're converging on a suitable solution here, but to > >>>>>> address the issues flagged by Lin below ... > >>>>>> > >>>>>> On 28/02/2019 12:39 pm, ?? wrote: > >>>>>>> Hi Suenaga, > >>>>>>> > >>>>>>> Thanks for your expaination about the arg_length_max, > >>>>>>> I generally agree with you that it is better to consider using > >>>>>>> dynamic memory, and that would be handled carefully to aviod > >>>>>>> introduce compatibility issue, plus it would be a big change. So > >>>>>>> let?s see what others suggest. > >>>>>>> > >>>>>>> Hi All, > >>>>>>> > >>>>>>> It seems for me that there are basically three problems forked > >>>>>>> by this > >>>>>>> thread? > >>>>>>> > >>>>>>> ?Compatibility issue with old jcmd alike tools with > >>>>>>> attachListener?s change. > >>>>>> > >>>>>> This is issue: > >>>>>> > >>>>>> https://bugs.openjdk.java.net/browse/JDK-8219721 > >>>>>> > >>>>>>> ?Only 3 arguments limited to passed by socket to attachListener > >>>>>>> for Windows, which cause 8215622 work abnormally on Windows. > >>>>>> > >>>>>> I have filed a new bug for this: > >>>>>> > >>>>>> https://bugs.openjdk.java.net/browse/JDK-8219895 > >>>>>> > >>>>>>> ?The arg_length_max may not be enough for handling filename. > >>>>>> > >>>>>> I have filed a new bug for this: > >>>>>> > >>>>>> https://bugs.openjdk.java.net/browse/JDK-8219896 > >>>>>> > >>>>>> though it seems very related to this issue. > >>>>>> > >>>>>>> So I suggest we keep the first problem discussed in this thread, > >>>>>>> and create separate thread for the others. Do you agree? > >>>>>> > >>>>>> There is some overlap but yes this can be broken down somewhat - > >>>>>> though dealing with the variable length "packet" is going to have > >>>>>> to consider that what is received is in fact much larger than the > >>>>>> purported maximum packet size if these long paths are expected > >>>>>> and > >>>> accepted. > >>>>>> > >>>>>> FWIW I don't see crashes or anything drastic if the arguments are > >>>>>> too long - the operations just fail (in somewhat obscure ways > sometimes). > >>>>>> > >>>>>>> > >>>>>>> For me, I will refine my patch to use timeout as a fix for the > >>>>>>> first one, and update it in this thread. And I will try to fix > >>>>>>> the second one for Windows, and create a separate thread for > >>>>>>> discussing. And if possible, I can help to fix the third one. > >>>>>>> > >>>>>>> What?s your opinion? > >>>>>> > >>>>>> That sounds fine ... > >>>>>> > >>>>>> Or you could choose to rollback JDK-8215622 and see how to solve > >>>>>> that without increasing the arg count. Given this usage: > >>>>>> > >>>>>> jmap -histo:live,file=foo.txt > >>>>>> > >>>>>> I'm not sure why this is sent to the VM as multiple args rather > >>>>>> than as a single composite arg that can then be parsed again by > >>>>>> the actual "jmap" logic. There would be some double-up perhaps if > >>>>>> the front-end tool wants to perform the command-line validation, > >>>>>> but it would be easy enough I think to do that checking then send > >>>>>> the original composite > >>>> arg. > >>>>>> > >>>>>> Thanks, > >>>>>> David > >>>>>> ----- > >>>>>> > >>>>>> > >>>>>>> BRs, > >>>>>>> > >>>>>>> Lin > >>>>>>> > >>>>>>> *From:*Yasumasa Suenaga > >>>>>>> *Sent:* Thursday, February 28, 2019 8:39 AM > >>>>>>> *To:* ?? > >>>>>>> *Cc:* David Holmes ; > >>>>>>> serviceability-dev at openjdk.java.net > >>>>>>> serviceability-dev at openjdk.java.net > >>>>>>> > >>>>>>> *Subject:* Re: Protocol version of Attach API > >>>>>>> > >>>>>>> 2019?2?28?(?) 0:04 ?? >>>> >: > >>>>>>> > >>>>>>> Dear Suenaga, > >>>>>>> Thanks for your reviewing. I will try to refine the patch. > >>>>>>> For the argument length you mentioned, do you mean the > >>>>>>> "arg_length_max" should be large enough to accept the max > filename > >>>>>>> length? > >>>>>>> > >>>>>>> Yes, but it is not enough. > >>>>>>> > >>>>>>> For example, jcmd VM.log might pass 2 or more paths to define logs. > >>>>>>> > >>>>>>> IMHO, all the handling of the argument length is at receiver > >>>>>>> side in the attachListener, such as > >>>>>>> > >>>>>>> > >>>> http://hg.openjdk.java.net/jdk/jdk/file/df3d253aaf81/src/hotspot/os > >>>> /linux/att > >>>> achListener_linux.cpp#l322, > >>>>>>> for me it means that the VM side limits the argments length less > >>>>>>> than arg_length_max, otherwise it will return NULL, which cause > the > >>>>>>> sender side (tools like jcmd and jmap) exits with error message. > so > >>>>>>> I think there may be no need to limit the argument size in tool > side. > >>>>>>> > >>>>>>> IMHO all programs which use filesystem should support any > >>>>>>> locations on it. > >>>>>>> > >>>>>>> So I think we should use dynamic memory (or GrowableArray) for > >>>>>>> it if we do not change client side for compatibility. > >>>>>>> > >>>>>>> And from my experiment with jmap, the arguments send to > >>>>>>> sockets are not arg0 only. as you can see in > >>>>>>> > >>>>>>> http://hg.openjdk.java.net/jdk/jdk/file/df3d253aaf81/src/jdk.jcm > >>>>>>> d/sh > >>>>>>> are/classes/sun/tools/jmap/JMap.java#l193 > >>>>>>> > >>>>>>> and > >>>>>>> > >>>>>>> http://hg.openjdk.java.net/jdk/jdk/file/df3d253aaf81/src/jdk.jcm > >>>>>>> d/sh are/classes/sun/tools/jmap/JMap.java#l133, > >>>>>>> > >>>>>>> jmap can pass arg0 as "filename", and arg1 as "-live", and both > of > >>>>>>> them can be NULL. so 00000 can be > >>>>>>> 0000, and file can be null. so 00 > may not > >>>>>>> indicate it reach the end. > >>>>>>> > >>>>>>> We should consider for other tools - jstack and jinfo. > >>>>>>> > >>>>>>> (jstack is ok because it will not have long arguments) > >>>>>>> > >>>>>>> Thanks, > >>>>>>> > >>>>>>> Yasumasa > >>>>>>> > >>>>>>> > >>>>>>> BRs, > >>>>>>> Lin > >>>>>>> ________________________________________ > >>>>>>> From: Yasumasa Suenaga >>>>>>> > > >>>>>>> Sent: Wednesday, February 27, 2019 8:10:14 PM > >>>>>>> To: ?? > >>>>>>> Cc: David Holmes; serviceability-dev at openjdk.java.net > >>>>>>> > >>>>>>> Subject: Re: Protocol version of Attach API > >>>>>>> > >>>>>>> Hi Lin, > >>>>>>> > >>>>>>> I think we need to research more about this. > >>>>>>> IMHO we need to match length of arguments between > >>>>>>> server (AttachListener) and client (serviceability tools) at least. > >>>>>>> (please see previous email from me). > >>>>>>> > >>>>>>> I have some comments for your change: > >>>>>>> > >>>>>>> On 2019/02/27 18:22, ?? wrote: > >>>>>>> > Dear All, > >>>>>>> > Here I have figured out one solution based on timeout. > would > >>>>>>> you like help to see whether this is OK? > >>>>>>> > --- a/src/hotspot/os/linux/attachListener_linux.cpp Tue Feb > >>>>>>> 26 14:57:23 2019 +0530 > >>>>>>> > +++ b/src/hotspot/os/linux/attachListener_linux.cpp Wed > Feb > >>>>>>> 27 17:21:48 2019 +0800 > >>>>>>> > @@ -263,9 +263,29 @@ > >>>>>>> > int off = 0; > >>>>>>> > int left = max_len; > >>>>>>> > > >>>>>>> > + memset(buf, 0, max_len); > >>>>>>> > + // set timeout for read > >>>>>>> > + struct timeval timeout; > >>>>>>> > + timeout.tv_sec = 3; > >>>>>>> > + timeout.tv_usec = 0; > >>>>>>> > >>>>>>> I think timeout value should be configurable. > >>>>>>> For example, we can introduce new flag in globals.hpp . > >>>>>>> > >>>>>>> > >>>>>>> > + if(setsockopt(s, SOL_SOCKET, SO_RCVTIMEO, (struct > >>>>>>> timeval*)&timeout, sizeof(timeout))) { > >>>>>>> > + log_debug(attach)("Failed to set socket option > SO_RCVTIMEO: > >>>>>>> %s\n", strerror(errorno)); > >>>>>>> > + printf("Warning: Failed to set socket option SO_RCVTIMEO: > >>>>>>> %s!!!\n", strerror(errno)); > >>>>>>> > >>>>>>> You should not use printf(), and do you need to pass '\n' to > >>>>>>> log_debug()? > >>>>>>> > >>>>>>> > >>>>>>> > + } > >>>>>>> > do { > >>>>>>> > int n; > >>>>>>> > - RESTARTABLE(read(s, buf+off, left), n); > >>>>>>> > + log_debug(attach)("start reading data from socket....\n"); > >>>>>>> > + n = read(s, buf+off, left); > >>>>>>> > >>>>>>> You should use RESTARTABLE macro. > >>>>>>> read(2) might be interrupted by signal. > >>>>>>> > >>>>>>> > >>>>>>> > + if (n < 0) { > >>>>>>> > + if (errno == EWOULDBLOCK) { > >>>>>>> > >>>>>>> According to man page, read(2) sets EWOULDBLOCK or EAGAIN. > >>>>>>> So you should check both errno. > >>>>>>> > >>>>>>> > >>>>>>> > + for (int i = str_count; i < expected_str_count; i++) { > >>>>>>> > + //timeout, fill reminded arguments with \0; > >>>>>>> > + buf[off+i] = '\0'; > >>>>>>> > + str_count++; > >>>>>>> > + } > >>>>>>> > >>>>>>> You set zero to buf[] in above. > >>>>>>> So you can remove this loop, and set str_count to > expected_str_count > >>>>>>> without manipulating buf[]. > >>>>>>> > >>>>>>> In addition, I prefer to add log_debug() at this > >>>>>>> to record NULL arguments are added. > >>>>>>> > >>>>>>> > >>>>>>> > + break;; > >>>>>>> > + } > >>>>>>> > + } > >>>>>>> > assert(n <= left, "buffer was too small, impossible!"); > >>>>>>> > buf[max_len - 1] = '\0'; > >>>>>>> > if (n == -1) { > >>>>>>> > >>>>>>> > >>>>>>> Thanks, > >>>>>>> > >>>>>>> Yasumasa > >>>>>>> > >>>>>>> > >>>>>>> > Thanks. > >>>>>>> > Lin > >>>>>>> > > >>>>>>> > ________________________________________ > >>>>>>> > From: Yasumasa Suenaga >>>>>>> > > >>>>>>> > Sent: Wednesday, February 27, 2019 15:15 > >>>>>>> > To: David Holmes; ?? > >>>>>>> > Cc: serviceability-dev at openjdk.java.net > >>>>>>> > >>>>>>> > Subject: Re: Protocol version of Attach API > >>>>>>> > > >>>>>>> > On 2019/02/27 15:59, David Holmes wrote: > >>>>>>> >> On 27/02/2019 4:10 pm, Yasumasa Suenaga wrote: > >>>>>>> >>> Hi, > >>>>>>> >>> > >>>>>>> >>> Buffer size for receiving packets from client is > >>>>>>> determined at [1]. > >>>>>>> >> > >>>>>>> >> Maximum buffer size, yes. > >>>>>>> >> > >>>>>>> >>> It defines length of command name and of argument. > >>>>>>> >>> It is passed via Unix domain, so we fill '\0' to remaining > >>>>>>> bytes and > >>>>>>> >>> might be able to assume all arguments are passed > >>>>>>> with empty string. > >>>>>>> >> > >>>>>>> >> Not sure what you mean. > >>>>>>> >> > >>>>>>> >> // The buffer is expected to be formatted as follows: > >>>>>>> >> // 00000 > >>>>>>> >> > >>>>>>> >> so we can expect to read at least two things - the ver and > cmd. > >>>>>>> If we encounter 00 we can infer we reached the end. But we may > not > >>>>>>> have read the full data into the buffer, so can't tell if another > >>>>>>> call to read() is needed yet - you only know after you've > >>>>>>> read the 00. > >>>>>>> >> > >>>>>>> >>> BTW length of arguments is defined to 1024 in [2]. > >>>>>>> >>> jcmd and jmap might pas file path - it might be > JVM_MAXPATHLEN > >>>>>>> (4097 bytes). > >>>>>>> >>> Buffer size which is used in AttachListener seems > >>>>>>> not to be enough. > >>>>>>> >> > >>>>>>> >> One has to assume/hope that the code sending the data is > working > >>>>>>> to the same protocol rules as the receiver! Otherwise this is just > >>>>>>> completely broken. > >>>>>>> > > >>>>>>> > On Linux, client (VirtualMachineImpl) seems not to check > length > >>>>>>> of arguments: > >>>>>>> > > >>>>>>> > >>>>>>> http://hg.openjdk.java.net/jdk/jdk/file/df3d253aaf81/src/jdk.att > >>>>>>> ach/ > >>>>>>> linux/classes/sun/tools/attach/VirtualMachineImpl.java#l168 > >>>>>>> > >>>>>>> > > >>>>>>> > In case of jcmd, all options are passed to arg #1. It seems not > >>>>>>> to check the length: > >>>>>>> > > >>>>>>> > >>>>>>> http://hg.openjdk.java.net/jdk/jdk/file/df3d253aaf81/src/jdk.jcm > >>>>>>> d/sh > >>>>>>> are/classes/sun/tools/jcmd/JCmd.java#l111 > >>>>>>> > >>>>>>> > > >>>>>>> > > >>>>>>> > I guess other tools (jstack, jmap, etc) which connect to > >>>>>>> AttachListener are same. > >>>>>>> > So we can fix both Attach API and AttachListener (it will be big > >>>>>>> change!), > >>>>>>> > but I concern we can keep protocol version... > >>>>>>> > > >>>>>>> > > >>>>>>> > Thanks, > >>>>>>> > > >>>>>>> > Yasumasa > >>>>>>> > > >>>>>>> > > >>>>>>> >> David > >>>>>>> >> ----- > >>>>>>> >> > >>>>>>> >>> We might have to change more. > >>>>>>> >>> > >>>>>>> >>> > >>>>>>> >>> Thanks, > >>>>>>> >>> > >>>>>>> >>> Yasumasa > >>>>>>> >>> > >>>>>>> >>> > >>>>>>> >>> [1] > >>>>>>> > >>>>>>> http://hg.openjdk.java.net/jdk/jdk/file/df3d253aaf81/src/hotspot > >>>>>>> /os/ > >>>>>>> linux/attachListener_linux.cpp#l254 > >>>>>>> > >>>>>>> >>> [2] > >>>>>>> > >>>>>>> http://hg.openjdk.java.net/jdk/jdk/file/df3d253aaf81/src/hotspot > >>>>>>> /sha > >>>>>>> re/services/attachListener.hpp#l106 > >>>>>>> > >>>>>>> >>> > >>>>>>> >>> > >>>>>>> >>> On 2019/02/27 15:00, ?? wrote: > >>>>>>> >>>> Another solution I can figure out is try to add timeout for > >>>>>>> socket read. I will also investigate whether is works. > >>>>>>> >>>> > >>>>>>> >>>> Cheers, > >>>>>>> >>>> Lin > >>>>>>> >>>> > >>>>>>> >>>> > >>>>>>> >>>>> -----Original Message----- > >>>>>>> >>>>> From: ?? > >>>>>>> >>>>> Sent: Wednesday, February 27, 2019 1:52 PM > >>>>>>> >>>>> To: 'David Holmes' >>>>>>> >; Yasumasa Suenaga > >>>>>>> >>>>> > > >>>>>>> >>>>> Cc: serviceability-dev at openjdk.java.net > >>>>>>> > >>>>>>> >>>>> Subject: RE: Protocol version of Attach API > >>>>>>> >>>>> > >>>>>>> >>>>> Dear David, Yasumasa, > >>>>>>> >>>>> I think it is hard to know how long the buffer is > >>>>>>> passed from socket > >>>>>>> >>>>> without changing the info of the message from the > >>>>>>> receiver side. > >>>>>>> >>>>> So how about when str_count reach to 3, we test it > >>>>>>> with non_blocking > >>>>>>> >>>>> read? > >>>>>>> >>>>> > >>>>>>> >>>>> > >>>>>>> >>>>> Cheers, > >>>>>>> >>>>> Lin > >>>>>>> >>>>>> -----Original Message----- > >>>>>>> >>>>>> From: David Holmes >>>>>>> > > >>>>>>> >>>>>> Sent: Wednesday, February 27, 2019 1:44 PM > >>>>>>> >>>>>> To: Yasumasa Suenaga >>>>>>> >; ?? >>>>>>> > > >>>>>>> >>>>>> Cc: serviceability-dev at openjdk.java.net > >>>>>>> > >>>>>>> >>>>>> Subject: Re: Protocol version of Attach API > >>>>>>> >>>>>> > >>>>>>> >>>>>> Hi Yasumasa, > >>>>>>> >>>>>> > >>>>>>> >>>>>> On 27/02/2019 1:05 pm, Yasumasa Suenaga wrote: > >>>>>>> >>>>>>> Hi Lin, > >>>>>>> >>>>>>> > >>>>>>> >>>>>>> My proposal is a just idea, so you need to tweak it. > >>>>>>> >>>>>>> > >>>>>>> >>>>>>> AttachListener receives raw command. > >>>>>>> >>>>>>> For example, jcmd is `jcmd\0`. Please see > >>>>>>> >>>>>>> HotSpotVirtualMachine and extended classes. > >>>>>>> >>>>>>> > >>>>>>> >>>>>>> In case of jcmd, I guess AttachListener will receive > message > >>>>>>> >>>>>>> `\0jcmd\0\0\0\0` (I did not check it > well). > >>>>>>> >>>>>>> So I guess we can add '\0' when `str_count` does not > reach > >>>>>>> to maximum. > >>>>>>> >>>>>> > >>>>>>> >>>>>> I don't see how this approach can work. You have > >>>>>>> to know > >>>> how > >>>>>>> many > >>>>>>> >>>>>> arguments are coming in the "packet", but that > information > >>>>>>> is not > >>>>>>> >>>>>> available in the current Linux implementation.Without it > you > >>>>>>> can't > >>>>>>> >>>>>> know when to stop calling read(). > >>>>>>> >>>>>> > >>>>>>> >>>>>> The protocol would need to be changed to send the > "packet" > >>>>>>> size, but > >>>>>>> >>>>>> that's not compatible with older JDKs. > >>>>>>> >>>>>> > >>>>>>> >>>>>> Otherwise I think we have no choice but to use a > >>>>>>> non-blocking read ... > >>>>>>> >>>>>> though I'm still unsure if you can know for > >>>>>>> certain that you've > >>>>>>> >>>>>> reached the end of the "packet" ... > >>>>>>> >>>>>> > >>>>>>> >>>>>> Thanks, > >>>>>>> >>>>>> David > >>>>>>> >>>>>> > >>>>>>> >>>>>>> > >>>>>>> >>>>>>> Thanks, > >>>>>>> >>>>>>> > >>>>>>> >>>>>>> Yasumasa > >>>>>>> >>>>>>> > >>>>>>> >>>>>>> > >>>>>>> >>>>>>> On 2019/02/27 11:50, zanglin5 at jd.com > >>>>>>> wrote: > >>>>>>> >>>>>>>> Dear Yasumasa, > >>>>>>> >>>>>>>> The fix you mentioned seems not work as > expected. > >>>>>>> >>>>>>>> I have done an experiment that use jdk1.8's "jcmd > >>>>>>> help" to > >>>>>>> >>>>>>>> attach to latest jdk. > >>>>>>> >>>>>>>> it seem the whole "jcmd help" buffer is not > >>>>>>> >>>>>>>> read completely at one time. in my case it is "jcmd", > >>>>>>> "", > >>>>>>> >>>>>>>> "help", and then block at while(). > >>>>>>> >>>>>>>> After applied your change, it seems only the "jcmd" > is > >>>>>>> >>>>>>>> processed, the "", "help" is replaced by '\0'. > >>>>>>> >>>>>>>> > >>>>>>> >>>>>>>> BRs, > >>>>>>> >>>>>>>> Lin > >>>>>>> >>>>>>>> > >>>>>>> From gary.adams at oracle.com Fri Mar 1 12:15:51 2019 From: gary.adams at oracle.com (Gary Adams) Date: Fri, 01 Mar 2019 07:15:51 -0500 Subject: RFR: JDK-4903717: nsk/jdi/ThreadReference/isSuspended/issuspended002 failing In-Reply-To: <4956c393-fe9e-1d76-5b03-e0a69cc00750@oracle.com> References: <5C6D78F6.6070409@oracle.com> <41cf1b33-1e43-0ee1-e1d9-a07d03a815cb@oracle.com> <693f8837-0498-6a01-5a56-1f2e0bb4e7c3@oracle.com> <89c99e78-cc56-845e-fe23-86b4c927b0d2@oracle.com> <5C7052ED.7020706@oracle.com> <5a1da25a-c4e4-f61b-a483-bfc2be2c3c4c@oracle.com> <41c12dcc-1c1b-5d59-6cc3-9869de85f357@oracle.com> <4956c393-fe9e-1d76-5b03-e0a69cc00750@oracle.com> Message-ID: <5C792276.3070502@oracle.com> I will move the logging statement up. Leaving the "docontinue" in the synchronized block will guarantee the debugger suspends the debuggee main thread without any chance of interference with logging commands. I'm sure this test was left with verbose logging enabled in the hopes of catching more details about the failing sequence of events. In this case it was the excessive logging that was causing the test to hang. Now that the test can run reliably, I think we should turn off the verbose logging. On 2/28/19, 4:44 PM, Alex Menkov wrote: > Hi Gary, > > Now "mainThread is out of: synchronized (lockingObject) {" record is > logged at the end of the scenario and that makes it useless. > > I think it would be better to call > pipe.println("docontinue") after logging of this message: > > log1("mainThread is out of: synchronized (lockingObject) {"); > + pipe.println("docontinue"); > + // wait until the scenario is completed to avoid deadlocks on logging > + synchronized (Threadissuspended002a.waitnotifyObj) { > + Threadissuspended002a.waitnotifyObj.wait(); > + } > > --alex > > On 02/27/2019 02:29, gary.adams at oracle.com wrote: >> Revised webrev : http://cr.openjdk.java.net/~gadams/4903717/webrev.02/ >> >> To prevent the main thread from holding the logging lock when >> the debugger suspends the debuggee, an explicit wait/notify is used >> so the test thread can safely issue log messages while the main thread >> remains suspended. >> >> Testing is in progress. >> >> On 2/26/19 3:49 PM, gary.adams at oracle.com wrote: >>> New fix will be coming tomorrow... >>> >>> I dumped the main and thread2 stacks at the >>> point of the timeout and found they were >>> both contending for the java.io.PrintStream >>> writeln() synchronization block. >>> >>> While writing log messages main thread held the >>> lock and was suspended and thread2 was blocked >>> trying to report it's progress. So thread2 never reaches the >>> breakpoint. >>> >>> java.nio.Buffer:292 in thread instance of >>> java.lang.Thread(name='main', id=1) >>> java.nio.CharBuffer:1260 in thread instance of >>> java.lang.Thread(name='main', id=1) >>> java.nio.CharBuffer:265 in thread instance of >>> java.lang.Thread(name='main', id=1) >>> java.nio.Buffer:223 in thread instance of >>> java.lang.Thread(name='main', id=1) >>> java.nio.CharBuffer:284 in thread instance of >>> java.lang.Thread(name='main', id=1) >>> java.nio.HeapCharBuffer:77 in thread instance of >>> java.lang.Thread(name='main', id=1) >>> java.nio.CharBuffer:396 in thread instance of >>> java.lang.Thread(name='main', id=1) >>> sun.nio.cs.StreamEncoder:280 in thread instance of >>> java.lang.Thread(name='main', id=1) >>> sun.nio.cs.StreamEncoder:125 in thread instance of >>> java.lang.Thread(name='main', id=1) >>> java.io.OutputStreamWriter:211 in thread instance of >>> java.lang.Thread(name='main', id=1) >>> java.io.BufferedWriter:120 in thread instance of >>> java.lang.Thread(name='main', id=1) >>> *java.io.PrintStream:653 in thread instance of >>> java.lang.Thread(name='main', id=1)* >>> java.io.PrintStream:958 in thread instance of >>> java.lang.Thread(name='main', id=1) >>> nsk.jdi.ThreadReference.isSuspended.issuspended002a:49 in thread >>> instance of java.lang.Thread(name='main', id=1) >>> nsk.jdi.ThreadReference.isSuspended.issuspended002a:124 in thread >>> instance of java.lang.Thread(name='main', id=1) >>> >>> >>> *java.io.PrintStream:650 in thread instance of >>> nsk.jdi.ThreadReference.isSuspended.Threadissuspended002a(name='testedThread', >>> id=589)* >>> java.io.PrintStream:958 in thread instance of >>> nsk.jdi.ThreadReference.isSuspended.Threadissuspended002a(name='testedThread', >>> id=589) >>> nsk.jdi.ThreadReference.isSuspended.issuspended002a:53 in thread >>> instance of >>> nsk.jdi.ThreadReference.isSuspended.Threadissuspended002a(name='testedThread', >>> id=589) >>> nsk.jdi.ThreadReference.isSuspended.Threadissuspended002a:203 in >>> thread instance of >>> nsk.jdi.ThreadReference.isSuspended.Threadissuspended002a(name='testedThread', >>> id=589) >>> >>> On 2/22/19 11:21 PM, serguei.spitsyn at oracle.com wrote: >>>> Hi Gary, >>>> >>>> Thank you for cosmetic changes. >>>> >>>> One thing to notice there is no guard against spurious wakeup for >>>> this: >>>> 108 Threadissuspended002a.waitnotifyObj.wait(); >>>> >>>> But I doubt it causes these timeouts as they are very well >>>> reproducible. >>>> >>>> This time I looked more to the debugger side. >>>> The code is poorly structured, it is hard to understand what it is >>>> doing. >>>> >>>> > - resumed main thread after breakpoint enabled and thread2 >>>> double resume >>>> >>>> Could you, please, explain more how does this help and why timeouts >>>> are intermittent? >>>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>> On 2/22/19 11:52 AM, Gary Adams wrote: >>>>> After 1000 successful testruns, it seems like a reasonable >>>>> solution is >>>>> to resume the main thread to resolve any locks holding back the >>>>> thread2 reaching the desired breakpoint. >>>>> >>>>> Updated webrev: >>>>> http://cr.openjdk.java.net/~gadams/4903717/webrev.01/ >>>>> >>>>> Changes in this iteration: >>>>> - cosmetic changes >>>>> - spaces around operators >>>>> - removed extra spaces around parens >>>>> - removed extra blank lines >>>>> - fixed typo in ERRROR >>>>> - fixed curl braces indenting >>>>> - resumed main thread after breakpoint enabled and thread2 >>>>> double resume >>>>> >>>>> >>>>> On 2/22/19, 6:19 AM, gary.adams at oracle.com wrote: >>>>>> I'll make a pass over the code for cosmetic fixes after the test >>>>>> is working correctly. >>>>>> >>>>>> I think the purpose of the lockingObject is to ensure the debugger >>>>>> has time to enable the breakpoint and perform the double resume >>>>>> before debuggee thread2 is released to run to the expected >>>>>> breakpoint. >>>>>> >>>>>> I'm assuming the timeout and break point location are correct, >>>>>> because >>>>>> the test passes most of the time. I'll dump the values to make sure. >>>>>> >>>>>> I'll also try to capture a thread dump at the time of the timeout, >>>>>> to see if the debuggee is hung somewhere unexpected. >>>>>> >>>>>> The test leaves the main thread suspended while thread2 is >>>>>> is resumed. I'll see if resuming the main thread unblocks thread2. >>>>>> >>>>>> ... >>>>>> >>>>>> On 2/21/19 10:18 PM, serguei.spitsyn at oracle.com wrote: >>>>>>> Hi Gary, >>>>>>> >>>>>>> I do not see any cause of this timeout yet. >>>>>>> However, the synchronization on the lockingObject looks >>>>>>> completely wasteful. >>>>>>> Moreover, it adds unreasonable complexity and makes the test >>>>>>> harder to understand. >>>>>>> >>>>>>> So, I'd suggested to do a small cleanup that includes: >>>>>>> - removing the lockingObject field and sync around it >>>>>>> - fix wrong indents in the fragment: >>>>>>> 161 synchronized (waitnotifyObj) { >>>>>>> 162 log("entered into block: synchronized >>>>>>> (waitnotifyObj)"); >>>>>>> 163 waitnotifyObj.notify(); } >>>>>>> >>>>>>> Thanks, >>>>>>> Serguei >>>>>>> >>>>>>> >>>>>>> On 2/21/19 10:57, gary.adams at oracle.com wrote: >>>>>>>> The commented out "docontinue" was done to match the style >>>>>>>> used in the other tests. I'll remove it for this test. >>>>>>>> >>>>>>>> In any event, the test still is failing 8 / 500 testruns on >>>>>>>> macosx and windows debug builds. >>>>>>>> >>>>>>>> The change is not sufficient. >>>>>>>> >>>>>>>> Here's a recent failure. You can see in the log that the >>>>>>>> debugger did >>>>>>>> wait until the new "docontinue" arrives outside the lockingObject >>>>>>>> synchronized block. >>>>>>>> >>>>>>>> It appears with the main thread suspended and the thread2 >>>>>>>> double resumed, >>>>>>>> it still does not continue executing up to the breakpoint in >>>>>>>> thread2. >>>>>>>> After the eventQueue.remove(timeout), when the vm is resumed >>>>>>>> you can see the debuggee messages expected leading up to the >>>>>>>> breakpoint. >>>>>>>> >>>>>>>> ... >>>>>>>> >>>>>>>> --> debugger: suspending VM with vm.suspend(); >>>>>>>> *debugee.stderr> **> mainThread: mainThread is out of: >>>>>>>> synchronized (lockingObject) {* >>>>>>>> --> debugger: thread2.suspendCount() == 2 >>>>>>>> --> debugger: : thread2.isSuspended() >>>>>>>> --> debugger: mainThread.suspendCount() == 1 >>>>>>>> --> debugger: : mainThread.isSuspended() >>>>>>>> --> debugger: double resuming thread2 with thread2.resume(); >>>>>>>> --> debugger: thread2.suspendCount() == 1 >>>>>>>> --> debugger: : thread2.isSuspended() >>>>>>>> --> debugger: enabling breakpRequest1 >>>>>>>> --> debugger: thread2.suspendCount() == 0 >>>>>>>> --> debugger: : !thread2.isAtBreakpoint() before call to >>>>>>>> breakpoint() >>>>>>>> --> debugger: waiting for BreakpointEvent >>>>>>>> *--> debugger: new: eventSet = eventQueue.remove();* >>>>>>>> # ERROR: ##> debugger: ERROR: timeout for waiting for a >>>>>>>> BreakpintEvent >>>>>>>> The following stacktrace is for failure analysis. >>>>>>>> nsk.share.TestFailure: ##> debugger: ERROR: timeout for >>>>>>>> waiting for a BreakpintEvent >>>>>>>> at nsk.share.Log.logExceptionForFailureAnalysis(Log.java:428) >>>>>>>> at nsk.share.Log.complain(Log.java:399) >>>>>>>> at >>>>>>>> nsk.jdi.ThreadReference.isSuspended.issuspended002.log3(issuspended002.java:104) >>>>>>>> at >>>>>>>> nsk.jdi.ThreadReference.isSuspended.issuspended002.breakpoint(issuspended002.java:555) >>>>>>>> at >>>>>>>> nsk.jdi.ThreadReference.isSuspended.issuspended002.runThis(issuspended002.java:389) >>>>>>>> at >>>>>>>> nsk.jdi.ThreadReference.isSuspended.issuspended002.run(issuspended002.java:90) >>>>>>>> at >>>>>>>> nsk.jdi.ThreadReference.isSuspended.issuspended002.main(issuspended002.java:85) >>>>>>>> at >>>>>>>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native >>>>>>>> Method) >>>>>>>> at >>>>>>>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >>>>>>>> at >>>>>>>> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>>>>>>> at java.base/java.lang.reflect.Method.invoke(Method.java:567) >>>>>>>> at >>>>>>>> PropertyResolvingWrapper.main(PropertyResolvingWrapper.java:104) >>>>>>>> at >>>>>>>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native >>>>>>>> Method) >>>>>>>> at >>>>>>>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >>>>>>>> at >>>>>>>> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>>>>>>> at java.base/java.lang.reflect.Method.invoke(Method.java:567) >>>>>>>> at >>>>>>>> com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127) >>>>>>>> at java.base/java.lang.Thread.run(Thread.java:835) >>>>>>>> *--> debugger: resuming both second and main thread >>>>>>>> debugee.stderr> **> mainThread: waiting for an instruction from >>>>>>>> the debugger ... --> debugger: the end of testing >>>>>>>> debugee.stderr> **> thread2: entered into block: synchronized >>>>>>>> (lockingObject) debugee.stderr> **> thread2: exited from block: >>>>>>>> synchronized (lockingObject) debugee.stderr> **> thread2: call >>>>>>>> to the method 'runt1' debugee.stderr> **> thread2: method >>>>>>>> 'runt1' enter* >>>>>>>> --> debugger: : returned string is 'checkend' >>>>>>>> debugee.stderr> **> mainThread: waiting for an instruction from >>>>>>>> the debugger ... >>>>>>>> >>>>>>>> >>>>>>>> On 2/20/19 5:15 PM, Alex Menkov wrote: >>>>>>>>> +1 >>>>>>>>> >>>>>>>>> But could you please remove old "pipe.println("docontinue")" >>>>>>>>> statement instead of commenting it out (no new webrev is >>>>>>>>> required) >>>>>>>>> >>>>>>>>> --alex >>>>>>>>> >>>>>>>>> On 02/20/2019 09:58, Chris Plummer wrote: >>>>>>>>>> Looks good. >>>>>>>>>> >>>>>>>>>> Chris >>>>>>>>>> >>>>>>>>>> On 2/20/19 7:57 AM, Gary Adams wrote: >>>>>>>>>>> The issuspended002 has been on the ProblemList since the >>>>>>>>>>> tests were moved to >>>>>>>>>>> the open repos. The proposed changeset applies the same fix >>>>>>>>>>> that was used in issuspended001. >>>>>>>>>>> >>>>>>>>>>> The current test fails when the debuggee replies with the >>>>>>>>>>> "docontinue" response, >>>>>>>>>>> while it is still holding a lock in a syncrhonized block. >>>>>>>>>>> When the debugger then >>>>>>>>>>> suspends the vm the debuggee test thread fails to proceed to >>>>>>>>>>> the expected breakpoint. >>>>>>>>>>> Testing in progress. >>>>>>>>>>> >>>>>>>>>>> Issue: https://bugs.openjdk.java.net/browse/JDK-4903717 >>>>>>>>>>> Webrev: http://cr.openjdk.java.net/~gadams/4903717/webrev.00/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> From daniel.fuchs at oracle.com Fri Mar 1 15:08:44 2019 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Fri, 1 Mar 2019 15:08:44 +0000 Subject: RFR: 8219585: [TESTBUG] sun/management/jmxremote/bootstrap/JMXInterfaceBindingTest.java passes trivially when it shouldn't In-Reply-To: <109f57d79590148149c0b9d3578ed68e716f99b3.camel@redhat.com> References: <388d8fbb5ba2110cbb17e1ca599f537f3ac6ffa1.camel@redhat.com>

<83e12309-70dd-37e7-a955-4b172af65460@oracle.com> <109f57d79590148149c0b9d3578ed68e716f99b3.camel@redhat.com> Message-ID: <726d3b2f-99eb-85a9-911b-867f1d0f02d7@oracle.com> Hi Severin, On 28/02/2019 15:47, Severin Gehwolf wrote: > Yes, thanks for looking at this Daniel. That was my determination as > well. However, even if we do all of the above, and add a test config > with /etc/hosts mapping a non-loopback address to "localhost" it would > break other tests. E.g. this one: > java/net/InetAddress/GetLocalHostWithSM.java > > So I'd have to explore whether your suggestion with > InetAddress.getLocalHost() is viable. It seems promising. > > I'll keep you posted. Thanks for keeping the investigation going! For what it's worth this is what I have been experimenting with: http://cr.openjdk.java.net/~dfuchs/experiment-8219585/experiment.00/ It's only a POC and obviously need more cleaning. Maybe some of the arbitrary timeouts in the test could be scaled in accordance to timeout-factor (I think there's an adjustTimeout(long) function somewhere in the test libs that does that). I ran it 50 times in our test system - and it passed on all platforms, so there's yet hope :-) hope this helps, -- daniel From sgehwolf at redhat.com Fri Mar 1 15:49:52 2019 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Fri, 01 Mar 2019 16:49:52 +0100 Subject: RFR: 8219585: [TESTBUG] sun/management/jmxremote/bootstrap/JMXInterfaceBindingTest.java passes trivially when it shouldn't In-Reply-To: <726d3b2f-99eb-85a9-911b-867f1d0f02d7@oracle.com> References: <388d8fbb5ba2110cbb17e1ca599f537f3ac6ffa1.camel@redhat.com>

<83e12309-70dd-37e7-a955-4b172af65460@oracle.com> <109f57d79590148149c0b9d3578ed68e716f99b3.camel@redhat.com> <726d3b2f-99eb-85a9-911b-867f1d0f02d7@oracle.com> Message-ID: <8b47d43d02cb02f89d4af83148b2d54a1d33a2da.camel@redhat.com> On Fri, 2019-03-01 at 15:08 +0000, Daniel Fuchs wrote: > I ran it 50 times in our test system - and it passed on all platforms, > so there's yet hope :-) Thanks for this! I take it, it actually runs the test on some of those test systems (no trivial pass)? Cheers, Severin From daniel.fuchs at oracle.com Fri Mar 1 16:13:18 2019 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Fri, 1 Mar 2019 16:13:18 +0000 Subject: RFR: 8219585: [TESTBUG] sun/management/jmxremote/bootstrap/JMXInterfaceBindingTest.java passes trivially when it shouldn't In-Reply-To: <8b47d43d02cb02f89d4af83148b2d54a1d33a2da.camel@redhat.com> References: <388d8fbb5ba2110cbb17e1ca599f537f3ac6ffa1.camel@redhat.com>

<83e12309-70dd-37e7-a955-4b172af65460@oracle.com> <109f57d79590148149c0b9d3578ed68e716f99b3.camel@redhat.com> <726d3b2f-99eb-85a9-911b-867f1d0f02d7@oracle.com> <8b47d43d02cb02f89d4af83148b2d54a1d33a2da.camel@redhat.com> Message-ID: <3403f08d-56a8-fbdf-b116-689972dd28e5@oracle.com> Hi Severin, On 01/03/2019 15:49, Severin Gehwolf wrote: > On Fri, 2019-03-01 at 15:08 +0000, Daniel Fuchs wrote: >> I ran it 50 times in our test system - and it passed on all platforms, >> so there's yet hope :-) > > Thanks for this! I take it, it actually runs the test on some of those > test systems (no trivial pass)? Yes - AFAICT it ran on all systems - no trivial pass. Used to fail on windows before I changed the parent/child communication channel to use plain stdin instead of relying on signals. Note that I had to tweak the various timeouts to make it pass in a seemingly reliable way - so there's still the risk that it might start failing again in some esoteric configs... best regards, -- daniel > > Cheers, > Severin > From chris.plummer at oracle.com Fri Mar 1 19:07:02 2019 From: chris.plummer at oracle.com (Chris Plummer) Date: Fri, 1 Mar 2019 11:07:02 -0800 Subject: RFR(S): 8214854: JDWP: Unforseen output truncation in logging In-Reply-To: <031b21df-8ff7-b08b-8c81-a2825f428bdf@bell-sw.com> References: <6b080c3a-27c9-fff5-147c-c5a3baa44c5c@bell-sw.com> <831d28ed-1fd8-90d3-1090-b434e5a6eeb2@oracle.com> <031b21df-8ff7-b08b-8c81-a2825f428bdf@bell-sw.com> Message-ID: On 2/28/19 11:12 PM, Dmitry Chuyko wrote: > On 3/1/19 6:36 AM, Chris Plummer wrote: >> On 2/28/19 5:07 PM, David Holmes wrote: >>> Hi Dmitry, >>> >>> On 1/03/2019 6:15 am, Dmitry Chuyko wrote: >>>> Hello, >>>> >>>> Please review a small fix for GCC 8.x warning in log_messages.c. >>>> Buffer sizes for parts of timestamp string can be adjusted to not >>>> exceed maximum buffer size when combined, while being able to hold >>>> necessary information. >>>> >>>> webrev: http://cr.openjdk.java.net/~dchuyko/8214854/webrev.00/ >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8214854 >>>> testing: after the fix GCC 8 compiles working OpenJDK on Linux. >>>> Dev-submit job started. >>> >>> This looks good. >>> >>> One query: >>> >>> ?51 typedef char timestamp_buffer[TIMESTAMP_SIZE]; >>> >>> Is this needed so that the size of the buffer is known by gcc inside >>> get_time_stamp? (Generally I prefer to see explicitly when an array >>> is being stack allocated, not have it hidden behind a typedef.) >> This also caught my attention and then got me thinking on what basis >> is gcc producing the warning in the first place. The warning is: >> >> ?src/jdk.jdwp.agent/share/native/libjdwp/log_messages.c:75:24: error: >> '%.3d' directive output may be truncated writing between 3 and 11 >> bytes into a region of size between 0 and 80 >> [-Werror=format-truncation=] >> ??????????????????? "%s.%.3d %s", timestamp_prefix, >> ??????????????????????? ^~~~ >> >> I think it deduced this by seeing that the call to get_time_stamp() >> is passing in a char[80] for the tbuf argument. It then also sees >> that the argument for the first %s is also a char[80] array, so is >> pointing out that this first %s might fill (or nearly fill) tbuf, not >> leaving room for the %.3d argument. If this is the case, there is no >> need for the timestamp_buffer typedef. The real fix was making >> timestamp_date_time and timestamp_timezone smaller so gcc doesn't >> think they can overflow tbuf. >> >> Chris > > Current signature of helper function is get_time_stamp(char *tbuf, > size_t ltbuf). If we drop typedef change, this function either > logically needs ltbuf parameter or it internally assumes it to be > somewhat (TIMESTAMP_SIZE+1). Hi Dmitry, Ok, but that's unrelated to the compiler warning and not needed for the fix. Using a typedef hides the fact that we are dealing with a char* so makes the code less readable. I actually prefer the way it was with a char* and size_t arguments. My next preference would be: ???? get_time_stamp(char tbuf[MAXLEN_TIMESTAMP+1]) Although I think it's also a bit odd. I don't recall seeing array sizes in function prototypes before. thanks, Chris > In first case we in general should care about arbitrary ltbuf size. > Like 5 or 2. In second case there are no static checks/hints that we > don't pass tbuf of diferent size. So I preferred to have this > information in the function signature and to use known lengths of > parts: get_time_stamp(timestamp_buffer tbuf). > > -Dmitry > >> >> >>> >>> Thanks, >>> David >>> >>> >>>> -Dmitry >>>> >> >> From dmitry.chuyko at bell-sw.com Fri Mar 1 21:46:53 2019 From: dmitry.chuyko at bell-sw.com (Dmitry Chuyko) Date: Sat, 2 Mar 2019 00:46:53 +0300 Subject: RFR(S): 8214854: JDWP: Unforseen output truncation in logging In-Reply-To: References: <6b080c3a-27c9-fff5-147c-c5a3baa44c5c@bell-sw.com> <831d28ed-1fd8-90d3-1090-b434e5a6eeb2@oracle.com> <031b21df-8ff7-b08b-8c81-a2825f428bdf@bell-sw.com> Message-ID: Ok. typedef part removed according to review comments (not related to fix). Updated webrev: http://cr.openjdk.java.net/~dchuyko/8214854/webrev.01/ -Dmitry On 3/1/19 10:07 PM, Chris Plummer wrote: > On 2/28/19 11:12 PM, Dmitry Chuyko wrote: >> On 3/1/19 6:36 AM, Chris Plummer wrote: >>> On 2/28/19 5:07 PM, David Holmes wrote: >>>> Hi Dmitry, >>>> >>>> On 1/03/2019 6:15 am, Dmitry Chuyko wrote: >>>>> Hello, >>>>> >>>>> Please review a small fix for GCC 8.x warning in log_messages.c. >>>>> Buffer sizes for parts of timestamp string can be adjusted to not >>>>> exceed maximum buffer size when combined, while being able to hold >>>>> necessary information. >>>>> >>>>> webrev: http://cr.openjdk.java.net/~dchuyko/8214854/webrev.00/ >>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8214854 >>>>> testing: after the fix GCC 8 compiles working OpenJDK on Linux. >>>>> Dev-submit job started. >>>> >>>> This looks good. >>>> >>>> One query: >>>> >>>> ?51 typedef char timestamp_buffer[TIMESTAMP_SIZE]; >>>> >>>> Is this needed so that the size of the buffer is known by gcc >>>> inside get_time_stamp? (Generally I prefer to see explicitly when >>>> an array is being stack allocated, not have it hidden behind a >>>> typedef.) >>> This also caught my attention and then got me thinking on what basis >>> is gcc producing the warning in the first place. The warning is: >>> >>> ?src/jdk.jdwp.agent/share/native/libjdwp/log_messages.c:75:24: >>> error: '%.3d' directive output may be truncated writing between 3 >>> and 11 bytes into a region of size between 0 and 80 >>> [-Werror=format-truncation=] >>> ??????????????????? "%s.%.3d %s", timestamp_prefix, >>> ??????????????????????? ^~~~ >>> >>> I think it deduced this by seeing that the call to get_time_stamp() >>> is passing in a char[80] for the tbuf argument. It then also sees >>> that the argument for the first %s is also a char[80] array, so is >>> pointing out that this first %s might fill (or nearly fill) tbuf, >>> not leaving room for the %.3d argument. If this is the case, there >>> is no need for the timestamp_buffer typedef. The real fix was making >>> timestamp_date_time and timestamp_timezone smaller so gcc doesn't >>> think they can overflow tbuf. >>> >>> Chris >> >> Current signature of helper function is get_time_stamp(char *tbuf, >> size_t ltbuf). If we drop typedef change, this function either >> logically needs ltbuf parameter or it internally assumes it to be >> somewhat (TIMESTAMP_SIZE+1). > Hi Dmitry, > > Ok, but that's unrelated to the compiler warning and not needed for > the fix. Using a typedef hides the fact that we are dealing with a > char* so makes the code less readable. I actually prefer the way it > was with a char* and size_t arguments. My next preference would be: > > ???? get_time_stamp(char tbuf[MAXLEN_TIMESTAMP+1]) > > Although I think it's also a bit odd. I don't recall seeing array > sizes in function prototypes before. > > thanks, > > Chris > >> In first case we in general should care about arbitrary ltbuf size. >> Like 5 or 2. In second case there are no static checks/hints that we >> don't pass tbuf of diferent size. So I preferred to have this >> information in the function signature and to use known lengths of >> parts: get_time_stamp(timestamp_buffer tbuf). >> >> -Dmitry >> >>> >>> >>>> >>>> Thanks, >>>> David >>>> >>>> >>>>> -Dmitry >>>>> >>> >>> > > From david.holmes at oracle.com Fri Mar 1 22:56:00 2019 From: david.holmes at oracle.com (David Holmes) Date: Sat, 2 Mar 2019 08:56:00 +1000 Subject: RFR(S): 8214854: JDWP: Unforseen output truncation in logging In-Reply-To: References: <6b080c3a-27c9-fff5-147c-c5a3baa44c5c@bell-sw.com> <831d28ed-1fd8-90d3-1090-b434e5a6eeb2@oracle.com> <031b21df-8ff7-b08b-8c81-a2825f428bdf@bell-sw.com>

Message-ID: Looks good. Thanks, David On 2/03/2019 7:46 am, Dmitry Chuyko wrote: > Ok. typedef part removed according to review comments (not related to fix). > > Updated webrev: http://cr.openjdk.java.net/~dchuyko/8214854/webrev.01/ > > -Dmitry > > On 3/1/19 10:07 PM, Chris Plummer wrote: >> On 2/28/19 11:12 PM, Dmitry Chuyko wrote: >>> On 3/1/19 6:36 AM, Chris Plummer wrote: >>>> On 2/28/19 5:07 PM, David Holmes wrote: >>>>> Hi Dmitry, >>>>> >>>>> On 1/03/2019 6:15 am, Dmitry Chuyko wrote: >>>>>> Hello, >>>>>> >>>>>> Please review a small fix for GCC 8.x warning in log_messages.c. >>>>>> Buffer sizes for parts of timestamp string can be adjusted to not >>>>>> exceed maximum buffer size when combined, while being able to hold >>>>>> necessary information. >>>>>> >>>>>> webrev: http://cr.openjdk.java.net/~dchuyko/8214854/webrev.00/ >>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8214854 >>>>>> testing: after the fix GCC 8 compiles working OpenJDK on Linux. >>>>>> Dev-submit job started. >>>>> >>>>> This looks good. >>>>> >>>>> One query: >>>>> >>>>> ?51 typedef char timestamp_buffer[TIMESTAMP_SIZE]; >>>>> >>>>> Is this needed so that the size of the buffer is known by gcc >>>>> inside get_time_stamp? (Generally I prefer to see explicitly when >>>>> an array is being stack allocated, not have it hidden behind a >>>>> typedef.) >>>> This also caught my attention and then got me thinking on what basis >>>> is gcc producing the warning in the first place. The warning is: >>>> >>>> ?src/jdk.jdwp.agent/share/native/libjdwp/log_messages.c:75:24: >>>> error: '%.3d' directive output may be truncated writing between 3 >>>> and 11 bytes into a region of size between 0 and 80 >>>> [-Werror=format-truncation=] >>>> ??????????????????? "%s.%.3d %s", timestamp_prefix, >>>> ??????????????????????? ^~~~ >>>> >>>> I think it deduced this by seeing that the call to get_time_stamp() >>>> is passing in a char[80] for the tbuf argument. It then also sees >>>> that the argument for the first %s is also a char[80] array, so is >>>> pointing out that this first %s might fill (or nearly fill) tbuf, >>>> not leaving room for the %.3d argument. If this is the case, there >>>> is no need for the timestamp_buffer typedef. The real fix was making >>>> timestamp_date_time and timestamp_timezone smaller so gcc doesn't >>>> think they can overflow tbuf. >>>> >>>> Chris >>> >>> Current signature of helper function is get_time_stamp(char *tbuf, >>> size_t ltbuf). If we drop typedef change, this function either >>> logically needs ltbuf parameter or it internally assumes it to be >>> somewhat (TIMESTAMP_SIZE+1). >> Hi Dmitry, >> >> Ok, but that's unrelated to the compiler warning and not needed for >> the fix. Using a typedef hides the fact that we are dealing with a >> char* so makes the code less readable. I actually prefer the way it >> was with a char* and size_t arguments. My next preference would be: >> >> ???? get_time_stamp(char tbuf[MAXLEN_TIMESTAMP+1]) >> >> Although I think it's also a bit odd. I don't recall seeing array >> sizes in function prototypes before. >> >> thanks, >> >> Chris >> >>> In first case we in general should care about arbitrary ltbuf size. >>> Like 5 or 2. In second case there are no static checks/hints that we >>> don't pass tbuf of diferent size. So I preferred to have this >>> information in the function signature and to use known lengths of >>> parts: get_time_stamp(timestamp_buffer tbuf). >>> >>> -Dmitry >>> >>>> >>>> >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>> >>>>>> -Dmitry >>>>>> >>>> >>>> >> >> From alexey.menkov at oracle.com Fri Mar 1 22:56:47 2019 From: alexey.menkov at oracle.com (Alex Menkov) Date: Fri, 1 Mar 2019 14:56:47 -0800 Subject: RFR: JDK-4903717: nsk/jdi/ThreadReference/isSuspended/issuspended002 failing In-Reply-To: <5C792276.3070502@oracle.com> References: <5C6D78F6.6070409@oracle.com> <41cf1b33-1e43-0ee1-e1d9-a07d03a815cb@oracle.com> <693f8837-0498-6a01-5a56-1f2e0bb4e7c3@oracle.com> <89c99e78-cc56-845e-fe23-86b4c927b0d2@oracle.com> <5C7052ED.7020706@oracle.com> <5a1da25a-c4e4-f61b-a483-bfc2be2c3c4c@oracle.com> <41c12dcc-1c1b-5d59-6cc3-9869de85f357@oracle.com> <4956c393-fe9e-1d76-5b03-e0a69cc00750@oracle.com> <5C792276.3070502@oracle.com> Message-ID: <559fb2df-a02e-b143-d9dd-647b87a07534@oracle.com> On 03/01/2019 04:15, Gary Adams wrote: > I will move the logging statement up. > > Leaving the "docontinue" in the synchronized block > will guarantee the debugger suspends the debuggee > main thread without any chance of interference > with logging commands. > > I'm sure this test was left with verbose logging enabled in the > hopes of catching more details about the failing sequence of > events. In this case it was the excessive logging that was causing > the test to hang. > > Now that the test can run reliably, I think we should turn off > the verbose logging. I think it would be better to keep the logging verbose to simplify analysis if the test fails in the future. --alex > > On 2/28/19, 4:44 PM, Alex Menkov wrote: >> Hi Gary, >> >> Now "mainThread is out of: synchronized (lockingObject) {" record is >> logged at the end of the scenario and that makes it useless. >> >> I think it would be better to call >> pipe.println("docontinue") after logging of this message: >> >> ? log1("mainThread is out of: synchronized (lockingObject) {"); >> +? pipe.println("docontinue"); >> +? // wait until the scenario is completed to avoid deadlocks on logging >> +? synchronized (Threadissuspended002a.waitnotifyObj) { >> +??? Threadissuspended002a.waitnotifyObj.wait(); >> +? } >> >> --alex >> >> On 02/27/2019 02:29, gary.adams at oracle.com wrote: >>> Revised webrev : http://cr.openjdk.java.net/~gadams/4903717/webrev.02/ >>> >>> To prevent the main thread from holding the logging lock when >>> the debugger suspends the debuggee, an explicit wait/notify is used >>> so the test thread can safely issue log messages while the main thread >>> remains suspended. >>> >>> Testing is in progress. >>> >>> On 2/26/19 3:49 PM, gary.adams at oracle.com wrote: >>>> New fix will be coming tomorrow... >>>> >>>> I dumped the main and thread2 stacks at the >>>> point of the timeout and found they were >>>> both contending for the java.io.PrintStream >>>> writeln() synchronization block. >>>> >>>> While writing log messages main thread held the >>>> lock and was suspended and thread2 was blocked >>>> trying to report it's progress. So thread2 never reaches the >>>> breakpoint. >>>> >>>> java.nio.Buffer:292 in thread instance of >>>> java.lang.Thread(name='main', id=1) >>>> java.nio.CharBuffer:1260 in thread instance of >>>> java.lang.Thread(name='main', id=1) >>>> java.nio.CharBuffer:265 in thread instance of >>>> java.lang.Thread(name='main', id=1) >>>> java.nio.Buffer:223 in thread instance of >>>> java.lang.Thread(name='main', id=1) >>>> java.nio.CharBuffer:284 in thread instance of >>>> java.lang.Thread(name='main', id=1) >>>> java.nio.HeapCharBuffer:77 in thread instance of >>>> java.lang.Thread(name='main', id=1) >>>> java.nio.CharBuffer:396 in thread instance of >>>> java.lang.Thread(name='main', id=1) >>>> sun.nio.cs.StreamEncoder:280 in thread instance of >>>> java.lang.Thread(name='main', id=1) >>>> sun.nio.cs.StreamEncoder:125 in thread instance of >>>> java.lang.Thread(name='main', id=1) >>>> java.io.OutputStreamWriter:211 in thread instance of >>>> java.lang.Thread(name='main', id=1) >>>> java.io.BufferedWriter:120 in thread instance of >>>> java.lang.Thread(name='main', id=1) >>>> *java.io.PrintStream:653 in thread instance of >>>> java.lang.Thread(name='main', id=1)* >>>> java.io.PrintStream:958 in thread instance of >>>> java.lang.Thread(name='main', id=1) >>>> nsk.jdi.ThreadReference.isSuspended.issuspended002a:49 in thread >>>> instance of java.lang.Thread(name='main', id=1) >>>> nsk.jdi.ThreadReference.isSuspended.issuspended002a:124 in thread >>>> instance of java.lang.Thread(name='main', id=1) >>>> >>>> >>>> *java.io.PrintStream:650 in thread instance of >>>> nsk.jdi.ThreadReference.isSuspended.Threadissuspended002a(name='testedThread', >>>> id=589)* >>>> java.io.PrintStream:958 in thread instance of >>>> nsk.jdi.ThreadReference.isSuspended.Threadissuspended002a(name='testedThread', >>>> id=589) >>>> nsk.jdi.ThreadReference.isSuspended.issuspended002a:53 in thread >>>> instance of >>>> nsk.jdi.ThreadReference.isSuspended.Threadissuspended002a(name='testedThread', >>>> id=589) >>>> nsk.jdi.ThreadReference.isSuspended.Threadissuspended002a:203 in >>>> thread instance of >>>> nsk.jdi.ThreadReference.isSuspended.Threadissuspended002a(name='testedThread', >>>> id=589) >>>> >>>> On 2/22/19 11:21 PM, serguei.spitsyn at oracle.com wrote: >>>>> Hi Gary, >>>>> >>>>> Thank you for cosmetic changes. >>>>> >>>>> One thing to notice there is no guard against spurious wakeup for >>>>> this: >>>>> ?? 108 Threadissuspended002a.waitnotifyObj.wait(); >>>>> >>>>> But I doubt it causes these timeouts as they are very well >>>>> reproducible. >>>>> >>>>> This time I looked more to the debugger side. >>>>> The code is poorly structured, it is hard to understand what it is >>>>> doing. >>>>> >>>>> >?? - resumed main thread after breakpoint enabled and thread2 >>>>> ???? double resume >>>>> >>>>> Could you, please, explain more how does this help and why timeouts >>>>> are intermittent? >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>> >>>>> On 2/22/19 11:52 AM, Gary Adams wrote: >>>>>> After 1000 successful testruns, it seems like a reasonable >>>>>> solution is >>>>>> to resume the main thread to resolve any locks holding back the >>>>>> thread2 reaching the desired breakpoint. >>>>>> >>>>>> ? Updated webrev: >>>>>> http://cr.openjdk.java.net/~gadams/4903717/webrev.01/ >>>>>> >>>>>> Changes in this iteration: >>>>>> ? - cosmetic changes >>>>>> ???? - spaces around operators >>>>>> ???? - removed extra spaces around parens >>>>>> ???? - removed extra blank lines >>>>>> ???? - fixed typo in ERRROR >>>>>> ???? - fixed curl braces indenting >>>>>> ? - resumed main thread after breakpoint enabled and thread2 >>>>>> ???? double resume >>>>>> >>>>>> >>>>>> On 2/22/19, 6:19 AM, gary.adams at oracle.com wrote: >>>>>>> I'll make a pass over the code for cosmetic fixes after the test >>>>>>> is working correctly. >>>>>>> >>>>>>> I think the purpose of the lockingObject is to ensure the debugger >>>>>>> has time to enable the breakpoint and perform the double resume >>>>>>> before debuggee thread2 is released to run to the expected >>>>>>> breakpoint. >>>>>>> >>>>>>> I'm assuming the timeout and break point location are correct, >>>>>>> because >>>>>>> the test passes most of the time. I'll dump the values to make sure. >>>>>>> >>>>>>> I'll also try to capture a thread dump at the time of the timeout, >>>>>>> to see if the debuggee is hung somewhere unexpected. >>>>>>> >>>>>>> The test leaves the main thread suspended while thread2 is >>>>>>> is resumed. I'll see if resuming the main thread unblocks thread2. >>>>>>> >>>>>>> ... >>>>>>> >>>>>>> On 2/21/19 10:18 PM, serguei.spitsyn at oracle.com wrote: >>>>>>>> Hi Gary, >>>>>>>> >>>>>>>> I do not see any cause of this timeout yet. >>>>>>>> However, the synchronization on the lockingObject looks >>>>>>>> completely wasteful. >>>>>>>> Moreover, it adds unreasonable complexity and makes the test >>>>>>>> harder to understand. >>>>>>>> >>>>>>>> So, I'd suggested to do a small cleanup that includes: >>>>>>>> ? - removing the lockingObject field and sync around it >>>>>>>> ? - fix wrong indents in the fragment: >>>>>>>> ? 161???????? synchronized (waitnotifyObj)??????????????????? { >>>>>>>> ? 162???????????? log("entered into block:? synchronized >>>>>>>> (waitnotifyObj)"); >>>>>>>> ? 163???????????? waitnotifyObj.notify();???????????????????? } >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Serguei >>>>>>>> >>>>>>>> >>>>>>>> On 2/21/19 10:57, gary.adams at oracle.com wrote: >>>>>>>>> The commented out "docontinue" was done to match the style >>>>>>>>> used in the other tests. I'll remove it for this test. >>>>>>>>> >>>>>>>>> In any event, the test still is failing 8 / 500 testruns on >>>>>>>>> macosx and windows debug builds. >>>>>>>>> >>>>>>>>> The change is not sufficient. >>>>>>>>> >>>>>>>>> Here's a recent failure. You can see in the log that the >>>>>>>>> debugger did >>>>>>>>> wait until the new "docontinue" arrives outside the lockingObject >>>>>>>>> synchronized block. >>>>>>>>> >>>>>>>>> It appears with the main thread suspended and the thread2 >>>>>>>>> double resumed, >>>>>>>>> it still does not continue executing up to the breakpoint in >>>>>>>>> thread2. >>>>>>>>> After the eventQueue.remove(timeout), when the vm is resumed >>>>>>>>> you can see the debuggee messages expected leading up to the >>>>>>>>> breakpoint. >>>>>>>>> >>>>>>>>> ... >>>>>>>>> >>>>>>>>> --> debugger:?????? suspending VM with vm.suspend(); >>>>>>>>> *debugee.stderr> **> mainThread: mainThread is out of: >>>>>>>>> synchronized (lockingObject) {* >>>>>>>>> --> debugger:????????? thread2.suspendCount() == 2 >>>>>>>>> --> debugger:????? :?? thread2.isSuspended() >>>>>>>>> --> debugger:????????? mainThread.suspendCount() == 1 >>>>>>>>> --> debugger:????? :?? mainThread.isSuspended() >>>>>>>>> --> debugger:?????? double resuming thread2 with thread2.resume(); >>>>>>>>> --> debugger:????????? thread2.suspendCount() == 1 >>>>>>>>> --> debugger:????? :?? thread2.isSuspended() >>>>>>>>> --> debugger:????? enabling breakpRequest1 >>>>>>>>> --> debugger:????????? thread2.suspendCount() == 0 >>>>>>>>> --> debugger:????? :? !thread2.isAtBreakpoint() before call to >>>>>>>>> breakpoint() >>>>>>>>> --> debugger:??????? waiting for BreakpointEvent >>>>>>>>> *--> debugger: new: eventSet = eventQueue.remove();* >>>>>>>>> # ERROR: ##> debugger: ERROR:? timeout for waiting for a >>>>>>>>> BreakpintEvent >>>>>>>>> The following stacktrace is for failure analysis. >>>>>>>>> nsk.share.TestFailure: ##> debugger: ERROR:? timeout for >>>>>>>>> waiting for a BreakpintEvent >>>>>>>>> ??? at nsk.share.Log.logExceptionForFailureAnalysis(Log.java:428) >>>>>>>>> ??? at nsk.share.Log.complain(Log.java:399) >>>>>>>>> ??? at >>>>>>>>> nsk.jdi.ThreadReference.isSuspended.issuspended002.log3(issuspended002.java:104) >>>>>>>>> >>>>>>>>> ??? at >>>>>>>>> nsk.jdi.ThreadReference.isSuspended.issuspended002.breakpoint(issuspended002.java:555) >>>>>>>>> >>>>>>>>> ??? at >>>>>>>>> nsk.jdi.ThreadReference.isSuspended.issuspended002.runThis(issuspended002.java:389) >>>>>>>>> >>>>>>>>> ??? at >>>>>>>>> nsk.jdi.ThreadReference.isSuspended.issuspended002.run(issuspended002.java:90) >>>>>>>>> >>>>>>>>> ??? at >>>>>>>>> nsk.jdi.ThreadReference.isSuspended.issuspended002.main(issuspended002.java:85) >>>>>>>>> >>>>>>>>> ??? at >>>>>>>>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native >>>>>>>>> Method) >>>>>>>>> ??? at >>>>>>>>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >>>>>>>>> >>>>>>>>> ??? at >>>>>>>>> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>>>>>>>> >>>>>>>>> ??? at java.base/java.lang.reflect.Method.invoke(Method.java:567) >>>>>>>>> ??? at >>>>>>>>> PropertyResolvingWrapper.main(PropertyResolvingWrapper.java:104) >>>>>>>>> ??? at >>>>>>>>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native >>>>>>>>> Method) >>>>>>>>> ??? at >>>>>>>>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >>>>>>>>> >>>>>>>>> ??? at >>>>>>>>> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>>>>>>>> >>>>>>>>> ??? at java.base/java.lang.reflect.Method.invoke(Method.java:567) >>>>>>>>> ??? at >>>>>>>>> com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127) >>>>>>>>> >>>>>>>>> ??? at java.base/java.lang.Thread.run(Thread.java:835) >>>>>>>>> *--> debugger: resuming both second and main thread >>>>>>>>> debugee.stderr> **> mainThread: waiting for an instruction from >>>>>>>>> the debugger ... --> debugger: the end of testing >>>>>>>>> debugee.stderr> **> thread2: entered into block: synchronized >>>>>>>>> (lockingObject) debugee.stderr> **> thread2: exited from block: >>>>>>>>> synchronized (lockingObject) debugee.stderr> **> thread2: call >>>>>>>>> to the method 'runt1' debugee.stderr> **> thread2: method >>>>>>>>> 'runt1' enter* >>>>>>>>> --> debugger:????? : returned string is 'checkend' >>>>>>>>> debugee.stderr> **> mainThread: waiting for an instruction from >>>>>>>>> the debugger ... >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2/20/19 5:15 PM, Alex Menkov wrote: >>>>>>>>>> +1 >>>>>>>>>> >>>>>>>>>> But could you please remove old "pipe.println("docontinue")" >>>>>>>>>> statement instead of commenting it out (no new webrev is >>>>>>>>>> required) >>>>>>>>>> >>>>>>>>>> --alex >>>>>>>>>> >>>>>>>>>> On 02/20/2019 09:58, Chris Plummer wrote: >>>>>>>>>>> Looks good. >>>>>>>>>>> >>>>>>>>>>> Chris >>>>>>>>>>> >>>>>>>>>>> On 2/20/19 7:57 AM, Gary Adams wrote: >>>>>>>>>>>> The issuspended002 has been on the ProblemList since the >>>>>>>>>>>> tests were moved to >>>>>>>>>>>> the open repos. The proposed changeset applies the same fix >>>>>>>>>>>> that was used in issuspended001. >>>>>>>>>>>> >>>>>>>>>>>> The current test fails when the debuggee replies with the >>>>>>>>>>>> "docontinue" response, >>>>>>>>>>>> while it is still holding a lock in a syncrhonized block. >>>>>>>>>>>> When the debugger then >>>>>>>>>>>> suspends the vm the debuggee test thread fails to proceed to >>>>>>>>>>>> the expected breakpoint. >>>>>>>>>>>> Testing in progress. >>>>>>>>>>>> >>>>>>>>>>>> ? Issue: https://bugs.openjdk.java.net/browse/JDK-4903717 >>>>>>>>>>>> ? Webrev: http://cr.openjdk.java.net/~gadams/4903717/webrev.00/ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> > From chris.plummer at oracle.com Fri Mar 1 23:28:08 2019 From: chris.plummer at oracle.com (Chris Plummer) Date: Fri, 1 Mar 2019 15:28:08 -0800 Subject: RFR(S): 8214854: JDWP: Unforseen output truncation in logging In-Reply-To: References: <6b080c3a-27c9-fff5-147c-c5a3baa44c5c@bell-sw.com> <831d28ed-1fd8-90d3-1090-b434e5a6eeb2@oracle.com> <031b21df-8ff7-b08b-8c81-a2825f428bdf@bell-sw.com>

Message-ID: <16703b12-cd1c-d497-ea80-1af47dd93805@oracle.com> +1 thanks, Chris On 3/1/19 2:56 PM, David Holmes wrote: > Looks good. > > Thanks, > David > > On 2/03/2019 7:46 am, Dmitry Chuyko wrote: >> Ok. typedef part removed according to review comments (not related to >> fix). >> >> Updated webrev: http://cr.openjdk.java.net/~dchuyko/8214854/webrev.01/ >> >> -Dmitry >> >> On 3/1/19 10:07 PM, Chris Plummer wrote: >>> On 2/28/19 11:12 PM, Dmitry Chuyko wrote: >>>> On 3/1/19 6:36 AM, Chris Plummer wrote: >>>>> On 2/28/19 5:07 PM, David Holmes wrote: >>>>>> Hi Dmitry, >>>>>> >>>>>> On 1/03/2019 6:15 am, Dmitry Chuyko wrote: >>>>>>> Hello, >>>>>>> >>>>>>> Please review a small fix for GCC 8.x warning in log_messages.c. >>>>>>> Buffer sizes for parts of timestamp string can be adjusted to >>>>>>> not exceed maximum buffer size when combined, while being able >>>>>>> to hold necessary information. >>>>>>> >>>>>>> webrev: http://cr.openjdk.java.net/~dchuyko/8214854/webrev.00/ >>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8214854 >>>>>>> testing: after the fix GCC 8 compiles working OpenJDK on Linux. >>>>>>> Dev-submit job started. >>>>>> >>>>>> This looks good. >>>>>> >>>>>> One query: >>>>>> >>>>>> ?51 typedef char timestamp_buffer[TIMESTAMP_SIZE]; >>>>>> >>>>>> Is this needed so that the size of the buffer is known by gcc >>>>>> inside get_time_stamp? (Generally I prefer to see explicitly when >>>>>> an array is being stack allocated, not have it hidden behind a >>>>>> typedef.) >>>>> This also caught my attention and then got me thinking on what >>>>> basis is gcc producing the warning in the first place. The warning >>>>> is: >>>>> >>>>> ?src/jdk.jdwp.agent/share/native/libjdwp/log_messages.c:75:24: >>>>> error: '%.3d' directive output may be truncated writing between 3 >>>>> and 11 bytes into a region of size between 0 and 80 >>>>> [-Werror=format-truncation=] >>>>> ??????????????????? "%s.%.3d %s", timestamp_prefix, >>>>> ??????????????????????? ^~~~ >>>>> >>>>> I think it deduced this by seeing that the call to >>>>> get_time_stamp() is passing in a char[80] for the tbuf argument. >>>>> It then also sees that the argument for the first %s is also a >>>>> char[80] array, so is pointing out that this first %s might fill >>>>> (or nearly fill) tbuf, not leaving room for the %.3d argument. If >>>>> this is the case, there is no need for the timestamp_buffer >>>>> typedef. The real fix was making timestamp_date_time and >>>>> timestamp_timezone smaller so gcc doesn't think they can overflow >>>>> tbuf. >>>>> >>>>> Chris >>>> >>>> Current signature of helper function is get_time_stamp(char *tbuf, >>>> size_t ltbuf). If we drop typedef change, this function either >>>> logically needs ltbuf parameter or it internally assumes it to be >>>> somewhat (TIMESTAMP_SIZE+1). >>> Hi Dmitry, >>> >>> Ok, but that's unrelated to the compiler warning and not needed for >>> the fix. Using a typedef hides the fact that we are dealing with a >>> char* so makes the code less readable. I actually prefer the way it >>> was with a char* and size_t arguments. My next preference would be: >>> >>> ???? get_time_stamp(char tbuf[MAXLEN_TIMESTAMP+1]) >>> >>> Although I think it's also a bit odd. I don't recall seeing array >>> sizes in function prototypes before. >>> >>> thanks, >>> >>> Chris >>> >>>> In first case we in general should care about arbitrary ltbuf size. >>>> Like 5 or 2. In second case there are no static checks/hints that >>>> we don't pass tbuf of diferent size. So I preferred to have this >>>> information in the function signature and to use known lengths of >>>> parts: get_time_stamp(timestamp_buffer tbuf). >>>> >>>> -Dmitry >>>> >>>>> >>>>> >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> >>>>>> >>>>>>> -Dmitry >>>>>>> >>>>> >>>>> >>> >>> From daniil.x.titov at oracle.com Sat Mar 2 04:33:07 2019 From: daniil.x.titov at oracle.com (Daniil Titov) Date: Fri, 01 Mar 2019 20:33:07 -0800 Subject: RFR 8218167: nsk/jvmti/scenarios/sampling/SP02/sp02t003 fails Message-ID: <3312FD66-27D9-416C-B3D9-14B0A7B78461@oracle.com> Please review the change that fix intermittent failure for test nsk/jvmti/scenarios/sampling/SP02/sp02t003 when running with Graal. The problem with the test here is that method checkThread() looks for the test method in the top "commonDepth" frames where "commonDepth" is a minimum of "frameCount" (returned by jvmti->GetFrameCount) and "frameStackSize"( returned by jvmti->GetStackTrace). If a compilation is triggered between these 2 calls then there are cases when "frameCount" is 2, "frameStackSize" is 4, and the frame stack is as the following: [0] adjustCompilationLevel [1] adjustCompilationLevel [2] testedMethod [3] run In this case the test looks for the test method only in 2 top frames and fails. The fix ensures that the test iterates over all frames in the frame stack when looking for the test method. Webrev: http://cr.openjdk.java.net/~dtitov/8218167/webrev.01 Bug: https://bugs.openjdk.java.net/browse/JDK-8218167 Thanks! --Daniil From dean.long at oracle.com Sat Mar 2 05:13:51 2019 From: dean.long at oracle.com (dean.long at oracle.com) Date: Fri, 1 Mar 2019 21:13:51 -0800 Subject: RFR 8218167: nsk/jvmti/scenarios/sampling/SP02/sp02t003 fails In-Reply-To: <3312FD66-27D9-416C-B3D9-14B0A7B78461@oracle.com> References: <3312FD66-27D9-416C-B3D9-14B0A7B78461@oracle.com> Message-ID: Looks good, but what about sp06t003?? Doesn't it have the same problem?? Are there any other tests using similar logic? dl On 3/1/19 8:33 PM, Daniil Titov wrote: > Please review the change that fix intermittent failure for test nsk/jvmti/scenarios/sampling/SP02/sp02t003 when running with Graal. > > The problem with the test here is that method checkThread() looks for the test method in the top "commonDepth" frames where "commonDepth" is a minimum of "frameCount" (returned by jvmti->GetFrameCount) and "frameStackSize"( returned by jvmti->GetStackTrace). > > If a compilation is triggered between these 2 calls then there are cases when "frameCount" is 2, "frameStackSize" is 4, and the frame stack is as the following: > > [0] adjustCompilationLevel > [1] adjustCompilationLevel > [2] testedMethod > [3] run > > In this case the test looks for the test method only in 2 top frames and fails. > > The fix ensures that the test iterates over all frames in the frame stack when looking for the test method. > > Webrev: http://cr.openjdk.java.net/~dtitov/8218167/webrev.01 > Bug: https://bugs.openjdk.java.net/browse/JDK-8218167 > > Thanks! > --Daniil > > From dmitry.chuyko at bell-sw.com Sat Mar 2 08:51:55 2019 From: dmitry.chuyko at bell-sw.com (Dmitry Chuyko) Date: Sat, 2 Mar 2019 11:51:55 +0300 Subject: RFR(S): 8214854: JDWP: Unforseen output truncation in logging In-Reply-To: <16703b12-cd1c-d497-ea80-1af47dd93805@oracle.com> References: <6b080c3a-27c9-fff5-147c-c5a3baa44c5c@bell-sw.com> <831d28ed-1fd8-90d3-1090-b434e5a6eeb2@oracle.com> <031b21df-8ff7-b08b-8c81-a2825f428bdf@bell-sw.com>

<16703b12-cd1c-d497-ea80-1af47dd93805@oracle.com> Message-ID: Thank you for reviews. Pushed. -Dmitry On 3/2/19 2:28 AM, Chris Plummer wrote: > +1 > > thanks, > > Chris > > On 3/1/19 2:56 PM, David Holmes wrote: >> Looks good. >> >> Thanks, >> David >> >> On 2/03/2019 7:46 am, Dmitry Chuyko wrote: >>> Ok. typedef part removed according to review comments (not related >>> to fix). >>> >>> Updated webrev: http://cr.openjdk.java.net/~dchuyko/8214854/webrev.01/ >>> >>> -Dmitry >>> >>> On 3/1/19 10:07 PM, Chris Plummer wrote: >>>> On 2/28/19 11:12 PM, Dmitry Chuyko wrote: >>>>> On 3/1/19 6:36 AM, Chris Plummer wrote: >>>>>> On 2/28/19 5:07 PM, David Holmes wrote: >>>>>>> Hi Dmitry, >>>>>>> >>>>>>> On 1/03/2019 6:15 am, Dmitry Chuyko wrote: >>>>>>>> Hello, >>>>>>>> >>>>>>>> Please review a small fix for GCC 8.x warning in >>>>>>>> log_messages.c. Buffer sizes for parts of timestamp string can >>>>>>>> be adjusted to not exceed maximum buffer size when combined, >>>>>>>> while being able to hold necessary information. >>>>>>>> >>>>>>>> webrev: http://cr.openjdk.java.net/~dchuyko/8214854/webrev.00/ >>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8214854 >>>>>>>> testing: after the fix GCC 8 compiles working OpenJDK on Linux. >>>>>>>> Dev-submit job started. >>>>>>> >>>>>>> This looks good. >>>>>>> >>>>>>> One query: >>>>>>> >>>>>>> ?51 typedef char timestamp_buffer[TIMESTAMP_SIZE]; >>>>>>> >>>>>>> Is this needed so that the size of the buffer is known by gcc >>>>>>> inside get_time_stamp? (Generally I prefer to see explicitly >>>>>>> when an array is being stack allocated, not have it hidden >>>>>>> behind a typedef.) >>>>>> This also caught my attention and then got me thinking on what >>>>>> basis is gcc producing the warning in the first place. The >>>>>> warning is: >>>>>> >>>>>> ?src/jdk.jdwp.agent/share/native/libjdwp/log_messages.c:75:24: >>>>>> error: '%.3d' directive output may be truncated writing between 3 >>>>>> and 11 bytes into a region of size between 0 and 80 >>>>>> [-Werror=format-truncation=] >>>>>> ??????????????????? "%s.%.3d %s", timestamp_prefix, >>>>>> ??????????????????????? ^~~~ >>>>>> >>>>>> I think it deduced this by seeing that the call to >>>>>> get_time_stamp() is passing in a char[80] for the tbuf argument. >>>>>> It then also sees that the argument for the first %s is also a >>>>>> char[80] array, so is pointing out that this first %s might fill >>>>>> (or nearly fill) tbuf, not leaving room for the %.3d argument. If >>>>>> this is the case, there is no need for the timestamp_buffer >>>>>> typedef. The real fix was making timestamp_date_time and >>>>>> timestamp_timezone smaller so gcc doesn't think they can overflow >>>>>> tbuf. >>>>>> >>>>>> Chris >>>>> >>>>> Current signature of helper function is get_time_stamp(char *tbuf, >>>>> size_t ltbuf). If we drop typedef change, this function either >>>>> logically needs ltbuf parameter or it internally assumes it to be >>>>> somewhat (TIMESTAMP_SIZE+1). >>>> Hi Dmitry, >>>> >>>> Ok, but that's unrelated to the compiler warning and not needed for >>>> the fix. Using a typedef hides the fact that we are dealing with a >>>> char* so makes the code less readable. I actually prefer the way it >>>> was with a char* and size_t arguments. My next preference would be: >>>> >>>> ???? get_time_stamp(char tbuf[MAXLEN_TIMESTAMP+1]) >>>> >>>> Although I think it's also a bit odd. I don't recall seeing array >>>> sizes in function prototypes before. >>>> >>>> thanks, >>>> >>>> Chris >>>> >>>>> In first case we in general should care about arbitrary ltbuf >>>>> size. Like 5 or 2. In second case there are no static checks/hints >>>>> that we don't pass tbuf of diferent size. So I preferred to have >>>>> this information in the function signature and to use known >>>>> lengths of parts: get_time_stamp(timestamp_buffer tbuf). >>>>> >>>>> -Dmitry >>>>> >>>>>> >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> >>>>>>> >>>>>>>> -Dmitry >>>>>>>> >>>>>> >>>>>> >>>> >>>> > > From david.holmes at oracle.com Mon Mar 4 07:22:34 2019 From: david.holmes at oracle.com (David Holmes) Date: Mon, 4 Mar 2019 17:22:34 +1000 Subject: Protocol version of Attach API In-Reply-To: References:

<4a5f251d-67ef-a81a-ef73-48f9aa9c853e@gmail.com> <0e7c23fec044419d845c5608b0586fcb@jd.com> <3dab08619bbd4254943361d2cdc1298b@jd.com>

Message-ID: Hi Lin, I think this is fine to address the problem that was introduced. There's more to be done in this area as there is obviously a misunderstanding about the "args" expected in the "packet" versus the 'args' for any particular command. With this change we can close JDK-8219895 as "Not an issue". Thanks, David On 1/03/2019 7:22 pm, ?? wrote: > Dear All, > I have upload a webrev at http://cr.openjdk.java.net/~xiaofeya/8219721/webrev.00/ (just found there is a jank file "abc", I will omit it in next webrev, but let's review other changes first for time saving.) > And here are my comments and questions: > * With this patch , I have tested with jdk8/jdk11's jcmd/jmap, all works as expected > * And passed tier1 test on my linux box > * Besides change from 4 to 3 , I also found one compatibility issue of using old "jmap" like "jmap -histo:live", caused by the sequence of arguments for inspectheap. And I have include the fix in the webrev. What I am concerned is that is this trivial enough to avoid revert and redo. IMO, if you think it shoud be revert 8215622 with this change, please let me know. > > Thanks for all of your help and suggestions. > > BRs, > Lin > >> -----Original Message----- >> From: David Holmes >> Sent: Friday, March 1, 2019 12:22 PM >> To: Yasumasa Suenaga ; ?? >> Cc: serviceability-dev at openjdk.java.net serviceability-dev at openjdk.java.net >> >> Subject: Re: Protocol version of Attach API >> >> On 1/03/2019 1:54 pm, Yasumasa Suenaga wrote: >>> Hi, >>> >>> I agree with David. I think we should backout 8215622. >> >> Note that I conceded that if Lin's proposal for changing the 4 back to 3 fixed >> everything, then I'm okay with making that fix rather than a full backout re- >> do. >> >> I'm not at all convinced we need change the number of args at the protocol >> level, regardless of the number of apparent "args" the command has. >> >> Cheers, >> David >> >>> We should re-do 8215622 completely for all platforms (includes >>> Windows) and support receiving requests from earlier serviceability >>> tools. >>> Problems about arguments of AttachListener should be worked as another >> issues. >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> 2019?3?1?(?) 12:39 ?? : >>>> >>>> Hi David, >>>> My understanding about those "arg" is that they are trying to make >> limitation of the overall message length and make it convince to communicate >> with sockets. >>>> I will do more test try to see whether changing back to 3 makes >> everything fine, and then prepare a webrev. >>>> Thanks. >>>> >>>> Lin >>>> ________________________________________ >>>> From: David Holmes >>>> Sent: Thursday, February 28, 2019 7:55:01 PM >>>> To: ??; Yasumasa Suenaga >>>> Cc: serviceability-dev at openjdk.java.net >>>> serviceability-dev at openjdk.java.net >>>> Subject: Re: Protocol version of Attach API >>>> >>>> Hi Lin, >>>> >>>> On 28/02/2019 7:30 pm, ?? wrote: >>>>> Hi David, >>>>> I am a little confused, do you think it is proper to made the patch as a >> fix of https://bugs.openjdk.java.net/browse/JDK-8219721 so that we don't >> need to backout and REDO? >>>> >>>> Generally I'd prefer to do the backout and then apply the revised fix >>>> as it will make the changes easier to track. >>>> >>>> However, if you are saying that everything works fine just by >>>> changing the 4 back to 3 everywhere, then that does seem a very >>>> simple fix to apply directly. >>>> >>>> I admit that if that does work then I really don't understand what >>>> these "arg" values actually means. :( Though it would explain why >>>> windows appears to work fine even though it was left at 3. >>>> >>>> Thanks, >>>> David >>>> >>>>> Thanks, >>>>> LIn >>>>> ________________________________________ >>>>> From: ?? >>>>> Sent: Thursday, February 28, 2019 4:50:12 PM >>>>> To: David Holmes; Yasumasa Suenaga >>>>> Cc: serviceability-dev at openjdk.java.net >>>>> serviceability-dev at openjdk.java.net >>>>> Subject: Re: Protocol version of Attach API >>>>> >>>>> Dear All, >>>>> I have tried simply recover the max argument count makes jmap - >> histo works and also makes the compatibility works fine. >>>>> Following are the changes I made: >>>>> >>>>> diff -r 07dd34f487d4 src/hotspot/share/services/attachListener.hpp >>>>> --- a/src/hotspot/share/services/attachListener.hpp Thu Feb 28 >> 02:47:39 2019 +0100 >>>>> +++ b/src/hotspot/share/services/attachListener.hpp Thu Feb 28 >> 16:48:19 2019 +0800 >>>>> @@ -106,7 +106,7 @@ >>>>> enum { >>>>> name_length_max = 16, // maximum length of name >>>>> arg_length_max = 1024, // maximum length of argument >>>>> - arg_count_max = 4 // maximum number of arguments >>>>> + arg_count_max = 3 // maximum number of arguments >>>>> }; >>>>> >>>>> // name of special operation that can be enqueued when all diff >>>>> -r 07dd34f487d4 >> src/jdk.attach/aix/classes/sun/tools/attach/VirtualMachineImpl.java >>>>> --- a/src/jdk.attach/aix/classes/sun/tools/attach/VirtualMachineImpl.java >> Thu Feb 28 02:47:39 2019 +0100 >>>>> +++ >> b/src/jdk.attach/aix/classes/sun/tools/attach/VirtualMachineImpl.java Thu >> Feb 28 16:48:19 2019 +0800 >>>>> @@ -138,7 +138,7 @@ >>>>> * Execute the given command in the target VM. >>>>> */ >>>>> InputStream execute(String cmd, Object ... args) throws >> AgentLoadException, IOException { >>>>> - assert args.length <= 4; // includes null >>>>> + assert args.length <= 3; // includes null >>>>> >>>>> // did we detach? >>>>> synchronized (this) { >>>>> @@ -166,7 +166,7 @@ >>>>> writeString(s, PROTOCOL_VERSION); >>>>> writeString(s, cmd); >>>>> >>>>> - for (int i = 0; i < 4; i++) { >>>>> + for (int i = 0; i < 3; i++) { >>>>> if (i < args.length && args[i] != null) { >>>>> writeString(s, (String)args[i]); >>>>> } else { >>>>> diff -r 07dd34f487d4 >> src/jdk.attach/linux/classes/sun/tools/attach/VirtualMachineImpl.java >>>>> --- >> a/src/jdk.attach/linux/classes/sun/tools/attach/VirtualMachineImpl.java >> Thu Feb 28 02:47:39 2019 +0100 >>>>> +++ >> b/src/jdk.attach/linux/classes/sun/tools/attach/VirtualMachineImpl.java >> Thu Feb 28 16:48:19 2019 +0800 >>>>> @@ -143,7 +143,7 @@ >>>>> * Execute the given command in the target VM. >>>>> */ >>>>> InputStream execute(String cmd, Object ... args) throws >> AgentLoadException, IOException { >>>>> - assert args.length <= 4; // includes null >>>>> + assert args.length <= 3; // includes null >>>>> >>>>> // did we detach? >>>>> synchronized (this) { >>>>> diff -r 07dd34f487d4 >> src/jdk.attach/macosx/classes/sun/tools/attach/VirtualMachineImpl.java >>>>> --- >> a/src/jdk.attach/macosx/classes/sun/tools/attach/VirtualMachineImpl.java >> Thu Feb 28 02:47:39 2019 +0100 >>>>> +++ >> b/src/jdk.attach/macosx/classes/sun/tools/attach/VirtualMachineImpl.java >> Thu Feb 28 16:48:19 2019 +0800 >>>>> @@ -139,7 +139,7 @@ >>>>> * Execute the given command in the target VM. >>>>> */ >>>>> InputStream execute(String cmd, Object ... args) throws >> AgentLoadException, IOException { >>>>> - assert args.length <= 4; // includes null >>>>> + assert args.length <= 3; // includes null >>>>> >>>>> // did we detach? >>>>> synchronized (this) { >>>>> diff -r 07dd34f487d4 >> src/jdk.attach/solaris/classes/sun/tools/attach/VirtualMachineImpl.java >>>>> --- >> a/src/jdk.attach/solaris/classes/sun/tools/attach/VirtualMachineImpl.java >> Thu Feb 28 02:47:39 2019 +0100 >>>>> +++ >> b/src/jdk.attach/solaris/classes/sun/tools/attach/VirtualMachineImpl.java >> Thu Feb 28 16:48:19 2019 +0800 >>>>> @@ -126,7 +126,7 @@ >>>>> * Execute the given command in the target VM. >>>>> */ >>>>> InputStream execute(String cmd, Object ... args) throws >> AgentLoadException, IOException { >>>>> - assert args.length <= 4; // includes null >>>>> + assert args.length <= 3; // includes null >>>>> >>>>> // first check that we are still attached >>>>> int door; >>>>> diff -r 07dd34f487d4 >> src/jdk.attach/windows/classes/sun/tools/attach/VirtualMachineImpl.java >>>>> --- >> a/src/jdk.attach/windows/classes/sun/tools/attach/VirtualMachineImpl.java >> Thu Feb 28 02:47:39 2019 +0100 >>>>> +++ >> b/src/jdk.attach/windows/classes/sun/tools/attach/VirtualMachineImpl.java >> Thu Feb 28 16:48:19 2019 +0800 >>>>> @@ -77,7 +77,7 @@ >>>>> InputStream execute(String cmd, Object ... args) >>>>> throws AgentLoadException, IOException >>>>> { >>>>> - assert args.length <= 4; // includes null >>>>> + assert args.length <= 3; // includes null >>>>> >>>>> // create a pipe using a random name >>>>> Random rnd = new Random(); >>>>> >>>>> >>>>> >>>>> Thanks, >>>>> Lin >>>>> ________________________________________ >>>>> From: ?? >>>>> Sent: Thursday, February 28, 2019 3:24:52 PM >>>>> To: David Holmes; Yasumasa Suenaga >>>>> Cc: serviceability-dev at openjdk.java.net >>>>> serviceability-dev at openjdk.java.net >>>>> Subject: RE: Protocol version of Attach API >>>>> >>>>> Hi David, >>>>> Since I don't have the access to JBS, may I ask your help to ceate sub- >> task? Thanks. >>>>> >>>>> BRs, >>>>> Lin >>>>> >>>>>> -----Original Message----- >>>>>> From: David Holmes >>>>>> Sent: Thursday, February 28, 2019 3:16 PM >>>>>> To: ?? ; Yasumasa Suenaga >> >>>>>> Cc: serviceability-dev at openjdk.java.net >>>>>> serviceability-dev at openjdk.java.net >>>>>> >>>>>> Subject: Re: Protocol version of Attach API >>>>>> >>>>>> Hi Lin, >>>>>> >>>>>> On 28/02/2019 4:49 pm, ?? wrote: >>>>>>> Hi David, >>>>>>> Your are right and thanks for pointing it out. when I worte >>>>>>> that patch, I >>>>>> was considering implement -filename and -incremental together. and >>>>>> I must be too stupid to forget recover it when I divided the patch into >> two. >>>>>>> And it seems a good solution is to refine the original >>>>>>> patch of jmap histo, >>>>>> and try to composite all args as one when passing it to socket and >>>>>> let attachlistener to handle the analyze. >>>>>>> I will try that. >>>>>>> One more, do I need to consider changing the jmap -dump also? >>>>>> >>>>>> I'm assuming -dump already works fine, so I'm just expecting to see >>>>>> -histo handle the file in a similar manner. >>>>>> >>>>>> If you find this works I suggest creating a sub-task of 8215622 to >>>>>> first backout the original changes (hg backout), and a second >>>>>> sub-task to REDO with the new implementation. Each will need >>>>>> reviewing separately in their own RFR thread. >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> >>>>>>> BRs, >>>>>>> Lin >>>>>>> ________________________________________ >>>>>>> From: David Holmes >>>>>>> Sent: Thursday, February 28, 2019 12:59:28 PM >>>>>>> To: ??; Yasumasa Suenaga >>>>>>> Cc: serviceability-dev at openjdk.java.net >>>>>>> serviceability-dev at openjdk.java.net >>>>>>> Subject: Re: Protocol version of Attach API >>>>>>> >>>>>>> Sorry I'm going to pick up on the rollback and re-do option here >>>>>>> as I just had a closer look at jmap. Given jmap -dump already has >>>>>>> more options than -histo does, why was any change to the "maximum >>>>>>> number of args" needed in the first place ??? >>>>>>> >>>>>>> David >>>>>>> >>>>>>> On 28/02/2019 2:43 pm, David Holmes wrote: >>>>>>>> Hi everyone, >>>>>>>> >>>>>>>> I'm not sure we're converging on a suitable solution here, but to >>>>>>>> address the issues flagged by Lin below ... >>>>>>>> >>>>>>>> On 28/02/2019 12:39 pm, ?? wrote: >>>>>>>>> Hi Suenaga, >>>>>>>>> >>>>>>>>> Thanks for your expaination about the arg_length_max, >>>>>>>>> I generally agree with you that it is better to consider using >>>>>>>>> dynamic memory, and that would be handled carefully to aviod >>>>>>>>> introduce compatibility issue, plus it would be a big change. So >>>>>>>>> let?s see what others suggest. >>>>>>>>> >>>>>>>>> Hi All, >>>>>>>>> >>>>>>>>> It seems for me that there are basically three problems forked >>>>>>>>> by this >>>>>>>>> thread? >>>>>>>>> >>>>>>>>> ?Compatibility issue with old jcmd alike tools with >>>>>>>>> attachListener?s change. >>>>>>>> >>>>>>>> This is issue: >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8219721 >>>>>>>> >>>>>>>>> ?Only 3 arguments limited to passed by socket to attachListener >>>>>>>>> for Windows, which cause 8215622 work abnormally on Windows. >>>>>>>> >>>>>>>> I have filed a new bug for this: >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8219895 >>>>>>>> >>>>>>>>> ?The arg_length_max may not be enough for handling filename. >>>>>>>> >>>>>>>> I have filed a new bug for this: >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8219896 >>>>>>>> >>>>>>>> though it seems very related to this issue. >>>>>>>> >>>>>>>>> So I suggest we keep the first problem discussed in this thread, >>>>>>>>> and create separate thread for the others. Do you agree? >>>>>>>> >>>>>>>> There is some overlap but yes this can be broken down somewhat - >>>>>>>> though dealing with the variable length "packet" is going to have >>>>>>>> to consider that what is received is in fact much larger than the >>>>>>>> purported maximum packet size if these long paths are expected >>>>>>>> and >>>>>> accepted. >>>>>>>> >>>>>>>> FWIW I don't see crashes or anything drastic if the arguments are >>>>>>>> too long - the operations just fail (in somewhat obscure ways >> sometimes). >>>>>>>> >>>>>>>>> >>>>>>>>> For me, I will refine my patch to use timeout as a fix for the >>>>>>>>> first one, and update it in this thread. And I will try to fix >>>>>>>>> the second one for Windows, and create a separate thread for >>>>>>>>> discussing. And if possible, I can help to fix the third one. >>>>>>>>> >>>>>>>>> What?s your opinion? >>>>>>>> >>>>>>>> That sounds fine ... >>>>>>>> >>>>>>>> Or you could choose to rollback JDK-8215622 and see how to solve >>>>>>>> that without increasing the arg count. Given this usage: >>>>>>>> >>>>>>>> jmap -histo:live,file=foo.txt >>>>>>>> >>>>>>>> I'm not sure why this is sent to the VM as multiple args rather >>>>>>>> than as a single composite arg that can then be parsed again by >>>>>>>> the actual "jmap" logic. There would be some double-up perhaps if >>>>>>>> the front-end tool wants to perform the command-line validation, >>>>>>>> but it would be easy enough I think to do that checking then send >>>>>>>> the original composite >>>>>> arg. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> David >>>>>>>> ----- >>>>>>>> >>>>>>>> >>>>>>>>> BRs, >>>>>>>>> >>>>>>>>> Lin >>>>>>>>> >>>>>>>>> *From:*Yasumasa Suenaga >>>>>>>>> *Sent:* Thursday, February 28, 2019 8:39 AM >>>>>>>>> *To:* ?? >>>>>>>>> *Cc:* David Holmes ; >>>>>>>>> serviceability-dev at openjdk.java.net >>>>>>>>> serviceability-dev at openjdk.java.net >>>>>>>>> >>>>>>>>> *Subject:* Re: Protocol version of Attach API >>>>>>>>> >>>>>>>>> 2019?2?28?(?) 0:04 ?? >>>>> >: >>>>>>>>> >>>>>>>>> Dear Suenaga, >>>>>>>>> Thanks for your reviewing. I will try to refine the patch. >>>>>>>>> For the argument length you mentioned, do you mean the >>>>>>>>> "arg_length_max" should be large enough to accept the max >> filename >>>>>>>>> length? >>>>>>>>> >>>>>>>>> Yes, but it is not enough. >>>>>>>>> >>>>>>>>> For example, jcmd VM.log might pass 2 or more paths to define logs. >>>>>>>>> >>>>>>>>> IMHO, all the handling of the argument length is at receiver >>>>>>>>> side in the attachListener, such as >>>>>>>>> >>>>>>>>> >>>>>> http://hg.openjdk.java.net/jdk/jdk/file/df3d253aaf81/src/hotspot/os >>>>>> /linux/att >>>>>> achListener_linux.cpp#l322, >>>>>>>>> for me it means that the VM side limits the argments length less >>>>>>>>> than arg_length_max, otherwise it will return NULL, which cause >> the >>>>>>>>> sender side (tools like jcmd and jmap) exits with error message. >> so >>>>>>>>> I think there may be no need to limit the argument size in tool >> side. >>>>>>>>> >>>>>>>>> IMHO all programs which use filesystem should support any >>>>>>>>> locations on it. >>>>>>>>> >>>>>>>>> So I think we should use dynamic memory (or GrowableArray) for >>>>>>>>> it if we do not change client side for compatibility. >>>>>>>>> >>>>>>>>> And from my experiment with jmap, the arguments send to >>>>>>>>> sockets are not arg0 only. as you can see in >>>>>>>>> >>>>>>>>> http://hg.openjdk.java.net/jdk/jdk/file/df3d253aaf81/src/jdk.jcm >>>>>>>>> d/sh >>>>>>>>> are/classes/sun/tools/jmap/JMap.java#l193 >>>>>>>>> >>>>>>>>> and >>>>>>>>> >>>>>>>>> http://hg.openjdk.java.net/jdk/jdk/file/df3d253aaf81/src/jdk.jcm >>>>>>>>> d/sh are/classes/sun/tools/jmap/JMap.java#l133, >>>>>>>>> >>>>>>>>> jmap can pass arg0 as "filename", and arg1 as "-live", and both >> of >>>>>>>>> them can be NULL. so 00000 can be >>>>>>>>> 0000, and file can be null. so 00 >> may not >>>>>>>>> indicate it reach the end. >>>>>>>>> >>>>>>>>> We should consider for other tools - jstack and jinfo. >>>>>>>>> >>>>>>>>> (jstack is ok because it will not have long arguments) >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> BRs, >>>>>>>>> Lin >>>>>>>>> ________________________________________ >>>>>>>>> From: Yasumasa Suenaga >>>>>>>> > >>>>>>>>> Sent: Wednesday, February 27, 2019 8:10:14 PM >>>>>>>>> To: ?? >>>>>>>>> Cc: David Holmes; serviceability-dev at openjdk.java.net >>>>>>>>> >>>>>>>>> Subject: Re: Protocol version of Attach API >>>>>>>>> >>>>>>>>> Hi Lin, >>>>>>>>> >>>>>>>>> I think we need to research more about this. >>>>>>>>> IMHO we need to match length of arguments between >>>>>>>>> server (AttachListener) and client (serviceability tools) at least. >>>>>>>>> (please see previous email from me). >>>>>>>>> >>>>>>>>> I have some comments for your change: >>>>>>>>> >>>>>>>>> On 2019/02/27 18:22, ?? wrote: >>>>>>>>> > Dear All, >>>>>>>>> > Here I have figured out one solution based on timeout. >> would >>>>>>>>> you like help to see whether this is OK? >>>>>>>>> > --- a/src/hotspot/os/linux/attachListener_linux.cpp Tue Feb >>>>>>>>> 26 14:57:23 2019 +0530 >>>>>>>>> > +++ b/src/hotspot/os/linux/attachListener_linux.cpp Wed >> Feb >>>>>>>>> 27 17:21:48 2019 +0800 >>>>>>>>> > @@ -263,9 +263,29 @@ >>>>>>>>> > int off = 0; >>>>>>>>> > int left = max_len; >>>>>>>>> > >>>>>>>>> > + memset(buf, 0, max_len); >>>>>>>>> > + // set timeout for read >>>>>>>>> > + struct timeval timeout; >>>>>>>>> > + timeout.tv_sec = 3; >>>>>>>>> > + timeout.tv_usec = 0; >>>>>>>>> >>>>>>>>> I think timeout value should be configurable. >>>>>>>>> For example, we can introduce new flag in globals.hpp . >>>>>>>>> >>>>>>>>> >>>>>>>>> > + if(setsockopt(s, SOL_SOCKET, SO_RCVTIMEO, (struct >>>>>>>>> timeval*)&timeout, sizeof(timeout))) { >>>>>>>>> > + log_debug(attach)("Failed to set socket option >> SO_RCVTIMEO: >>>>>>>>> %s\n", strerror(errorno)); >>>>>>>>> > + printf("Warning: Failed to set socket option SO_RCVTIMEO: >>>>>>>>> %s!!!\n", strerror(errno)); >>>>>>>>> >>>>>>>>> You should not use printf(), and do you need to pass '\n' to >>>>>>>>> log_debug()? >>>>>>>>> >>>>>>>>> >>>>>>>>> > + } >>>>>>>>> > do { >>>>>>>>> > int n; >>>>>>>>> > - RESTARTABLE(read(s, buf+off, left), n); >>>>>>>>> > + log_debug(attach)("start reading data from socket....\n"); >>>>>>>>> > + n = read(s, buf+off, left); >>>>>>>>> >>>>>>>>> You should use RESTARTABLE macro. >>>>>>>>> read(2) might be interrupted by signal. >>>>>>>>> >>>>>>>>> >>>>>>>>> > + if (n < 0) { >>>>>>>>> > + if (errno == EWOULDBLOCK) { >>>>>>>>> >>>>>>>>> According to man page, read(2) sets EWOULDBLOCK or EAGAIN. >>>>>>>>> So you should check both errno. >>>>>>>>> >>>>>>>>> >>>>>>>>> > + for (int i = str_count; i < expected_str_count; i++) { >>>>>>>>> > + //timeout, fill reminded arguments with \0; >>>>>>>>> > + buf[off+i] = '\0'; >>>>>>>>> > + str_count++; >>>>>>>>> > + } >>>>>>>>> >>>>>>>>> You set zero to buf[] in above. >>>>>>>>> So you can remove this loop, and set str_count to >> expected_str_count >>>>>>>>> without manipulating buf[]. >>>>>>>>> >>>>>>>>> In addition, I prefer to add log_debug() at this >>>>>>>>> to record NULL arguments are added. >>>>>>>>> >>>>>>>>> >>>>>>>>> > + break;; >>>>>>>>> > + } >>>>>>>>> > + } >>>>>>>>> > assert(n <= left, "buffer was too small, impossible!"); >>>>>>>>> > buf[max_len - 1] = '\0'; >>>>>>>>> > if (n == -1) { >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> > Thanks. >>>>>>>>> > Lin >>>>>>>>> > >>>>>>>>> > ________________________________________ >>>>>>>>> > From: Yasumasa Suenaga >>>>>>>> > >>>>>>>>> > Sent: Wednesday, February 27, 2019 15:15 >>>>>>>>> > To: David Holmes; ?? >>>>>>>>> > Cc: serviceability-dev at openjdk.java.net >>>>>>>>> >>>>>>>>> > Subject: Re: Protocol version of Attach API >>>>>>>>> > >>>>>>>>> > On 2019/02/27 15:59, David Holmes wrote: >>>>>>>>> >> On 27/02/2019 4:10 pm, Yasumasa Suenaga wrote: >>>>>>>>> >>> Hi, >>>>>>>>> >>> >>>>>>>>> >>> Buffer size for receiving packets from client is >>>>>>>>> determined at [1]. >>>>>>>>> >> >>>>>>>>> >> Maximum buffer size, yes. >>>>>>>>> >> >>>>>>>>> >>> It defines length of command name and of argument. >>>>>>>>> >>> It is passed via Unix domain, so we fill '\0' to remaining >>>>>>>>> bytes and >>>>>>>>> >>> might be able to assume all arguments are passed >>>>>>>>> with empty string. >>>>>>>>> >> >>>>>>>>> >> Not sure what you mean. >>>>>>>>> >> >>>>>>>>> >> // The buffer is expected to be formatted as follows: >>>>>>>>> >> // 00000 >>>>>>>>> >> >>>>>>>>> >> so we can expect to read at least two things - the ver and >> cmd. >>>>>>>>> If we encounter 00 we can infer we reached the end. But we may >> not >>>>>>>>> have read the full data into the buffer, so can't tell if another >>>>>>>>> call to read() is needed yet - you only know after you've >>>>>>>>> read the 00. >>>>>>>>> >> >>>>>>>>> >>> BTW length of arguments is defined to 1024 in [2]. >>>>>>>>> >>> jcmd and jmap might pas file path - it might be >> JVM_MAXPATHLEN >>>>>>>>> (4097 bytes). >>>>>>>>> >>> Buffer size which is used in AttachListener seems >>>>>>>>> not to be enough. >>>>>>>>> >> >>>>>>>>> >> One has to assume/hope that the code sending the data is >> working >>>>>>>>> to the same protocol rules as the receiver! Otherwise this is just >>>>>>>>> completely broken. >>>>>>>>> > >>>>>>>>> > On Linux, client (VirtualMachineImpl) seems not to check >> length >>>>>>>>> of arguments: >>>>>>>>> > >>>>>>>>> >>>>>>>>> http://hg.openjdk.java.net/jdk/jdk/file/df3d253aaf81/src/jdk.att >>>>>>>>> ach/ >>>>>>>>> linux/classes/sun/tools/attach/VirtualMachineImpl.java#l168 >>>>>>>>> >>>>>>>>> > >>>>>>>>> > In case of jcmd, all options are passed to arg #1. It seems not >>>>>>>>> to check the length: >>>>>>>>> > >>>>>>>>> >>>>>>>>> http://hg.openjdk.java.net/jdk/jdk/file/df3d253aaf81/src/jdk.jcm >>>>>>>>> d/sh >>>>>>>>> are/classes/sun/tools/jcmd/JCmd.java#l111 >>>>>>>>> >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > I guess other tools (jstack, jmap, etc) which connect to >>>>>>>>> AttachListener are same. >>>>>>>>> > So we can fix both Attach API and AttachListener (it will be big >>>>>>>>> change!), >>>>>>>>> > but I concern we can keep protocol version... >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > Thanks, >>>>>>>>> > >>>>>>>>> > Yasumasa >>>>>>>>> > >>>>>>>>> > >>>>>>>>> >> David >>>>>>>>> >> ----- >>>>>>>>> >> >>>>>>>>> >>> We might have to change more. >>>>>>>>> >>> >>>>>>>>> >>> >>>>>>>>> >>> Thanks, >>>>>>>>> >>> >>>>>>>>> >>> Yasumasa >>>>>>>>> >>> >>>>>>>>> >>> >>>>>>>>> >>> [1] >>>>>>>>> >>>>>>>>> http://hg.openjdk.java.net/jdk/jdk/file/df3d253aaf81/src/hotspot >>>>>>>>> /os/ >>>>>>>>> linux/attachListener_linux.cpp#l254 >>>>>>>>> >>>>>>>>> >>> [2] >>>>>>>>> >>>>>>>>> http://hg.openjdk.java.net/jdk/jdk/file/df3d253aaf81/src/hotspot >>>>>>>>> /sha >>>>>>>>> re/services/attachListener.hpp#l106 >>>>>>>>> >>>>>>>>> >>> >>>>>>>>> >>> >>>>>>>>> >>> On 2019/02/27 15:00, ?? wrote: >>>>>>>>> >>>> Another solution I can figure out is try to add timeout for >>>>>>>>> socket read. I will also investigate whether is works. >>>>>>>>> >>>> >>>>>>>>> >>>> Cheers, >>>>>>>>> >>>> Lin >>>>>>>>> >>>> >>>>>>>>> >>>> >>>>>>>>> >>>>> -----Original Message----- >>>>>>>>> >>>>> From: ?? >>>>>>>>> >>>>> Sent: Wednesday, February 27, 2019 1:52 PM >>>>>>>>> >>>>> To: 'David Holmes' >>>>>>>> >; Yasumasa Suenaga >>>>>>>>> >>>>> > >>>>>>>>> >>>>> Cc: serviceability-dev at openjdk.java.net >>>>>>>>> >>>>>>>>> >>>>> Subject: RE: Protocol version of Attach API >>>>>>>>> >>>>> >>>>>>>>> >>>>> Dear David, Yasumasa, >>>>>>>>> >>>>> I think it is hard to know how long the buffer is >>>>>>>>> passed from socket >>>>>>>>> >>>>> without changing the info of the message from the >>>>>>>>> receiver side. >>>>>>>>> >>>>> So how about when str_count reach to 3, we test it >>>>>>>>> with non_blocking >>>>>>>>> >>>>> read? >>>>>>>>> >>>>> >>>>>>>>> >>>>> >>>>>>>>> >>>>> Cheers, >>>>>>>>> >>>>> Lin >>>>>>>>> >>>>>> -----Original Message----- >>>>>>>>> >>>>>> From: David Holmes >>>>>>>> > >>>>>>>>> >>>>>> Sent: Wednesday, February 27, 2019 1:44 PM >>>>>>>>> >>>>>> To: Yasumasa Suenaga >>>>>>>> >; ?? >>>>>>>> > >>>>>>>>> >>>>>> Cc: serviceability-dev at openjdk.java.net >>>>>>>>> >>>>>>>>> >>>>>> Subject: Re: Protocol version of Attach API >>>>>>>>> >>>>>> >>>>>>>>> >>>>>> Hi Yasumasa, >>>>>>>>> >>>>>> >>>>>>>>> >>>>>> On 27/02/2019 1:05 pm, Yasumasa Suenaga wrote: >>>>>>>>> >>>>>>> Hi Lin, >>>>>>>>> >>>>>>> >>>>>>>>> >>>>>>> My proposal is a just idea, so you need to tweak it. >>>>>>>>> >>>>>>> >>>>>>>>> >>>>>>> AttachListener receives raw command. >>>>>>>>> >>>>>>> For example, jcmd is `jcmd\0`. Please see >>>>>>>>> >>>>>>> HotSpotVirtualMachine and extended classes. >>>>>>>>> >>>>>>> >>>>>>>>> >>>>>>> In case of jcmd, I guess AttachListener will receive >> message >>>>>>>>> >>>>>>> `\0jcmd\0\0\0\0` (I did not check it >> well). >>>>>>>>> >>>>>>> So I guess we can add '\0' when `str_count` does not >> reach >>>>>>>>> to maximum. >>>>>>>>> >>>>>> >>>>>>>>> >>>>>> I don't see how this approach can work. You have >>>>>>>>> to know >>>>>> how >>>>>>>>> many >>>>>>>>> >>>>>> arguments are coming in the "packet", but that >> information >>>>>>>>> is not >>>>>>>>> >>>>>> available in the current Linux implementation.Without it >> you >>>>>>>>> can't >>>>>>>>> >>>>>> know when to stop calling read(). >>>>>>>>> >>>>>> >>>>>>>>> >>>>>> The protocol would need to be changed to send the >> "packet" >>>>>>>>> size, but >>>>>>>>> >>>>>> that's not compatible with older JDKs. >>>>>>>>> >>>>>> >>>>>>>>> >>>>>> Otherwise I think we have no choice but to use a >>>>>>>>> non-blocking read ... >>>>>>>>> >>>>>> though I'm still unsure if you can know for >>>>>>>>> certain that you've >>>>>>>>> >>>>>> reached the end of the "packet" ... >>>>>>>>> >>>>>> >>>>>>>>> >>>>>> Thanks, >>>>>>>>> >>>>>> David >>>>>>>>> >>>>>> >>>>>>>>> >>>>>>> >>>>>>>>> >>>>>>> Thanks, >>>>>>>>> >>>>>>> >>>>>>>>> >>>>>>> Yasumasa >>>>>>>>> >>>>>>> >>>>>>>>> >>>>>>> >>>>>>>>> >>>>>>> On 2019/02/27 11:50, zanglin5 at jd.com >>>>>>>>> wrote: >>>>>>>>> >>>>>>>> Dear Yasumasa, >>>>>>>>> >>>>>>>> The fix you mentioned seems not work as >> expected. >>>>>>>>> >>>>>>>> I have done an experiment that use jdk1.8's "jcmd >>>>>>>>> help" to >>>>>>>>> >>>>>>>> attach to latest jdk. >>>>>>>>> >>>>>>>> it seem the whole "jcmd help" buffer is not >>>>>>>>> >>>>>>>> read completely at one time. in my case it is "jcmd", >>>>>>>>> "", >>>>>>>>> >>>>>>>> "help", and then block at while(). >>>>>>>>> >>>>>>>> After applied your change, it seems only the "jcmd" >> is >>>>>>>>> >>>>>>>> processed, the "", "help" is replaced by '\0'. >>>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>> BRs, >>>>>>>>> >>>>>>>> Lin >>>>>>>>> >>>>>>>> >>>>>>>>> From zanglin5 at jd.com Mon Mar 4 07:30:47 2019 From: zanglin5 at jd.com (=?euc-kr?B?7fXX+w==?=) Date: Mon, 4 Mar 2019 07:30:47 +0000 Subject: Protocol version of Attach API In-Reply-To: References:

<4a5f251d-67ef-a81a-ef73-48f9aa9c853e@gmail.com> <0e7c23fec044419d845c5608b0586fcb@jd.com> <3dab08619bbd4254943361d2cdc1298b@jd.com>

, Message-ID: Thanks David, I will update the webrev for the fix by EOD today. Then we can review it. I agree with you for more work to do for the args, and that seems the precondition for me to move forward for other enhancement of the jmap/jcmd alike tools. I will try to investigate it. BRs, Lin ________________________________________ From: David Holmes Sent: Monday, March 4, 2019 3:22:34 PM To: ??; Yasumasa Suenaga; Hohensee, Paul Cc: serviceability-dev at openjdk.java.net serviceability-dev at openjdk.java.net Subject: Re: Protocol version of Attach API Hi Lin, I think this is fine to address the problem that was introduced. There's more to be done in this area as there is obviously a misunderstanding about the "args" expected in the "packet" versus the 'args' for any particular command. With this change we can close JDK-8219895 as "Not an issue". Thanks, David On 1/03/2019 7:22 pm, ?? wrote: > Dear All, > I have upload a webrev at http://cr.openjdk.java.net/~xiaofeya/8219721/webrev.00/ (just found there is a jank file "abc", I will omit it in next webrev, but let's review other changes first for time saving.) > And here are my comments and questions: > * With this patch , I have tested with jdk8/jdk11's jcmd/jmap, all works as expected > * And passed tier1 test on my linux box > * Besides change from 4 to 3 , I also found one compatibility issue of using old "jmap" like "jmap -histo:live", caused by the sequence of arguments for inspectheap. And I have include the fix in the webrev. What I am concerned is that is this trivial enough to avoid revert and redo. IMO, if you think it shoud be revert 8215622 with this change, please let me know. > > Thanks for all of your help and suggestions. > > BRs, > Lin > >> -----Original Message----- >> From: David Holmes >> Sent: Friday, March 1, 2019 12:22 PM >> To: Yasumasa Suenaga ; ?? >> Cc: serviceability-dev at openjdk.java.net serviceability-dev at openjdk.java.net >> >> Subject: Re: Protocol version of Attach API >> >> On 1/03/2019 1:54 pm, Yasumasa Suenaga wrote: >>> Hi, >>> >>> I agree with David. I think we should backout 8215622. >> >> Note that I conceded that if Lin's proposal for changing the 4 back to 3 fixed >> everything, then I'm okay with making that fix rather than a full backout re- >> do. >> >> I'm not at all convinced we need change the number of args at the protocol >> level, regardless of the number of apparent "args" the command has. >> >> Cheers, >> David >> >>> We should re-do 8215622 completely for all platforms (includes >>> Windows) and support receiving requests from earlier serviceability >>> tools. >>> Problems about arguments of AttachListener should be worked as another >> issues. >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> 2019?3?1?(?) 12:39 ?? : >>>> >>>> Hi David, >>>> My understanding about those "arg" is that they are trying to make >> limitation of the overall message length and make it convince to communicate >> with sockets. >>>> I will do more test try to see whether changing back to 3 makes >> everything fine, and then prepare a webrev. >>>> Thanks. >>>> >>>> Lin >>>> ________________________________________ >>>> From: David Holmes >>>> Sent: Thursday, February 28, 2019 7:55:01 PM >>>> To: ??; Yasumasa Suenaga >>>> Cc: serviceability-dev at openjdk.java.net >>>> serviceability-dev at openjdk.java.net >>>> Subject: Re: Protocol version of Attach API >>>> >>>> Hi Lin, >>>> >>>> On 28/02/2019 7:30 pm, ?? wrote: >>>>> Hi David, >>>>> I am a little confused, do you think it is proper to made the patch as a >> fix of https://bugs.openjdk.java.net/browse/JDK-8219721 so that we don't >> need to backout and REDO? >>>> >>>> Generally I'd prefer to do the backout and then apply the revised fix >>>> as it will make the changes easier to track. >>>> >>>> However, if you are saying that everything works fine just by >>>> changing the 4 back to 3 everywhere, then that does seem a very >>>> simple fix to apply directly. >>>> >>>> I admit that if that does work then I really don't understand what >>>> these "arg" values actually means. :( Though it would explain why >>>> windows appears to work fine even though it was left at 3. >>>> >>>> Thanks, >>>> David >>>> >>>>> Thanks, >>>>> LIn >>>>> ________________________________________ >>>>> From: ?? >>>>> Sent: Thursday, February 28, 2019 4:50:12 PM >>>>> To: David Holmes; Yasumasa Suenaga >>>>> Cc: serviceability-dev at openjdk.java.net >>>>> serviceability-dev at openjdk.java.net >>>>> Subject: Re: Protocol version of Attach API >>>>> >>>>> Dear All, >>>>> I have tried simply recover the max argument count makes jmap - >> histo works and also makes the compatibility works fine. >>>>> Following are the changes I made: >>>>> >>>>> diff -r 07dd34f487d4 src/hotspot/share/services/attachListener.hpp >>>>> --- a/src/hotspot/share/services/attachListener.hpp Thu Feb 28 >> 02:47:39 2019 +0100 >>>>> +++ b/src/hotspot/share/services/attachListener.hpp Thu Feb 28 >> 16:48:19 2019 +0800 >>>>> @@ -106,7 +106,7 @@ >>>>> enum { >>>>> name_length_max = 16, // maximum length of name >>>>> arg_length_max = 1024, // maximum length of argument >>>>> - arg_count_max = 4 // maximum number of arguments >>>>> + arg_count_max = 3 // maximum number of arguments >>>>> }; >>>>> >>>>> // name of special operation that can be enqueued when all diff >>>>> -r 07dd34f487d4 >> src/jdk.attach/aix/classes/sun/tools/attach/VirtualMachineImpl.java >>>>> --- a/src/jdk.attach/aix/classes/sun/tools/attach/VirtualMachineImpl.java >> Thu Feb 28 02:47:39 2019 +0100 >>>>> +++ >> b/src/jdk.attach/aix/classes/sun/tools/attach/VirtualMachineImpl.java Thu >> Feb 28 16:48:19 2019 +0800 >>>>> @@ -138,7 +138,7 @@ >>>>> * Execute the given command in the target VM. >>>>> */ >>>>> InputStream execute(String cmd, Object ... args) throws >> AgentLoadException, IOException { >>>>> - assert args.length <= 4; // includes null >>>>> + assert args.length <= 3; // includes null >>>>> >>>>> // did we detach? >>>>> synchronized (this) { >>>>> @@ -166,7 +166,7 @@ >>>>> writeString(s, PROTOCOL_VERSION); >>>>> writeString(s, cmd); >>>>> >>>>> - for (int i = 0; i < 4; i++) { >>>>> + for (int i = 0; i < 3; i++) { >>>>> if (i < args.length && args[i] != null) { >>>>> writeString(s, (String)args[i]); >>>>> } else { >>>>> diff -r 07dd34f487d4 >> src/jdk.attach/linux/classes/sun/tools/attach/VirtualMachineImpl.java >>>>> --- >> a/src/jdk.attach/linux/classes/sun/tools/attach/VirtualMachineImpl.java >> Thu Feb 28 02:47:39 2019 +0100 >>>>> +++ >> b/src/jdk.attach/linux/classes/sun/tools/attach/VirtualMachineImpl.java >> Thu Feb 28 16:48:19 2019 +0800 >>>>> @@ -143,7 +143,7 @@ >>>>> * Execute the given command in the target VM. >>>>> */ >>>>> InputStream execute(String cmd, Object ... args) throws >> AgentLoadException, IOException { >>>>> - assert args.length <= 4; // includes null >>>>> + assert args.length <= 3; // includes null >>>>> >>>>> // did we detach? >>>>> synchronized (this) { >>>>> diff -r 07dd34f487d4 >> src/jdk.attach/macosx/classes/sun/tools/attach/VirtualMachineImpl.java >>>>> --- >> a/src/jdk.attach/macosx/classes/sun/tools/attach/VirtualMachineImpl.java >> Thu Feb 28 02:47:39 2019 +0100 >>>>> +++ >> b/src/jdk.attach/macosx/classes/sun/tools/attach/VirtualMachineImpl.java >> Thu Feb 28 16:48:19 2019 +0800 >>>>> @@ -139,7 +139,7 @@ >>>>> * Execute the given command in the target VM. >>>>> */ >>>>> InputStream execute(String cmd, Object ... args) throws >> AgentLoadException, IOException { >>>>> - assert args.length <= 4; // includes null >>>>> + assert args.length <= 3; // includes null >>>>> >>>>> // did we detach? >>>>> synchronized (this) { >>>>> diff -r 07dd34f487d4 >> src/jdk.attach/solaris/classes/sun/tools/attach/VirtualMachineImpl.java >>>>> --- >> a/src/jdk.attach/solaris/classes/sun/tools/attach/VirtualMachineImpl.java >> Thu Feb 28 02:47:39 2019 +0100 >>>>> +++ >> b/src/jdk.attach/solaris/classes/sun/tools/attach/VirtualMachineImpl.java >> Thu Feb 28 16:48:19 2019 +0800 >>>>> @@ -126,7 +126,7 @@ >>>>> * Execute the given command in the target VM. >>>>> */ >>>>> InputStream execute(String cmd, Object ... args) throws >> AgentLoadException, IOException { >>>>> - assert args.length <= 4; // includes null >>>>> + assert args.length <= 3; // includes null >>>>> >>>>> // first check that we are still attached >>>>> int door; >>>>> diff -r 07dd34f487d4 >> src/jdk.attach/windows/classes/sun/tools/attach/VirtualMachineImpl.java >>>>> --- >> a/src/jdk.attach/windows/classes/sun/tools/attach/VirtualMachineImpl.java >> Thu Feb 28 02:47:39 2019 +0100 >>>>> +++ >> b/src/jdk.attach/windows/classes/sun/tools/attach/VirtualMachineImpl.java >> Thu Feb 28 16:48:19 2019 +0800 >>>>> @@ -77,7 +77,7 @@ >>>>> InputStream execute(String cmd, Object ... args) >>>>> throws AgentLoadException, IOException >>>>> { >>>>> - assert args.length <= 4; // includes null >>>>> + assert args.length <= 3; // includes null >>>>> >>>>> // create a pipe using a random name >>>>> Random rnd = new Random(); >>>>> >>>>> >>>>> >>>>> Thanks, >>>>> Lin >>>>> ________________________________________ >>>>> From: ?? >>>>> Sent: Thursday, February 28, 2019 3:24:52 PM >>>>> To: David Holmes; Yasumasa Suenaga >>>>> Cc: serviceability-dev at openjdk.java.net >>>>> serviceability-dev at openjdk.java.net >>>>> Subject: RE: Protocol version of Attach API >>>>> >>>>> Hi David, >>>>> Since I don't have the access to JBS, may I ask your help to ceate sub- >> task? Thanks. >>>>> >>>>> BRs, >>>>> Lin >>>>> >>>>>> -----Original Message----- >>>>>> From: David Holmes

Starting an Agent from the Command-Line Interface

Starting an Agent from the Command-Line Interface

Starting an Agent After VM Startup

Starting an Agent After VM Startup

Including an Agent in an Executable JAR file

Including an Agent in an Executable JAR file

Loading agent classes and the modules/classes available to the agent - * class

Loading agent classes and the modules/classes available to the agent + * class

Manifest Attributes

Manifest Attributes

Instrumenting code in modules

Instrumenting code in modules

Starting an Agent from the Command-Line Interface

Starting an Agent from the Command-Line Interface

Starting an Agent After VM Startup

Starting an Agent After VM Startup

Including an Agent in an Executable JAR file

Including an Agent in an Executable JAR file

Loading agent classes and the modules/classes available to the > agent > - * class

Loading agent classes and the modules/classes available to the > agent > + * class

Manifest Attributes

Manifest Attributes

Instrumenting code in modules

Instrumenting code in modules