RFR: 8039173 Propagate errors from Diagnostic Commands as exceptions in the attach framework

[Staffan Larsen]

All,

In the attach framework, the result is sent back to the attaching process as a stream. The first value in the stream is an integer that is interpreted as the completion status. For successful operations, this value is 0. In that case the attaching process continues to read output from the stream and displays that as the result of the operation.

When the completion status signals an error, the rest of the stream output is ignored. This means that there is currently no way for an attach operation to signal an error and provide an error message.

I would like to change the protocol slightly so that the client continues to read from the stream even if the completion status signalled an error. The data that is read would be considered the error message and would be included as the message in the IOException thrown by the attachers code when an operation fails.
 
To do this, I propose a new subclass of IOException, AttachOperationFailedException, which will be thrown in these cases from the implementation of VirtualMachine. The client can catch this exception and display the error message as well as signal that the operation failed.

I have limited the error message to 200 bytes, since I did not want a runaway Diagnostic Command to enormous amounts of output in the client.

When I was testing out this change, I also discovered buffer errors in the native implementations for HotSpotVirtualMachine. Fixes for these are included in this patch.

The jcmd executable has been modified to 1) have an exit code of 0 if the command succeeded and 1 if it failed, 2) print out a generic message if the diagnostic command did not output anything.

Since output from jcmd was modified, the JMXStartStopTest needed a slight modification as well.

At a later stage, I am planning to change the hotspot code so that exceptions in Diagnostic Commands actually set the completion status to an error. Today, the completation status is set to success, just because we want the error message to propagate.

bug: https://bugs.openjdk.java.net/browse/JDK-8039173
webrev: http://cr.openjdk.java.net/~sla/8039173/webrev.00/

Thanks,
/Staffan