Integrated: 8252657: JVMTI agent is not unloaded when Agent_OnAttach is failed

[Yasumasa Suenaga]

On Sat, 5 Sep 2020 14:26:17 GMT, Yasumasa Suenaga <ysuenaga at openjdk.org> wrote:

> If `Agent_OnAttach()` in JVMTI agent which is attempted to load via JVMTI.agent_load dcmd is failed, it would not be unloaded.  
> We've [discussed it on serviceability-dev](https://mail.openjdk.java.net/pipermail/serviceability-dev/2020-September/032839.html). This PR is a continuation of that.
> 
> This PR also includes to call `Agent_OnUnload()` when `Agent_OnAttach()` failed.
> 
> How to reproduce:
> 
> 1. Build JVMTI agent for test
> $ git clone https://github.com/YaSuenag/jvmti-examples.git
> $ cd jvmti-examples/helloworld/out/build
> $ cmake ../..
> 
> 2. Run JShell
> 
> 3. Load JVMTI agent via `jcmd <PID> JVMTI.agent_load` with "error" ("error" means `Agent_OnAttach()` returns JNI_ERR)
> $ jcmd
> 89456 jdk.jshell.execution.RemoteExecutionControl 45651
> 89547 sun.tools.jcmd.JCmd
> 89436 jdk.jshell/jdk.internal.jshell.tool.JShellToolProvider
> $ jcmd 89436 JVMTI.agent_load `pwd`/libhelloworld.so error
> 89436:
> return code: -1
> 
> 4. Check loaded libraries via `jcmd <PID> VM.dynlibs`
> $ jcmd 89436 VM.dynlibs | grep libhelloworld
> 7f2f8b06b000-7f2f8b06c000 r--p 00000000 fd:00 11818202 /home/ysuenaga/github/jvmti-examples/helloworld/out/build/libhelloworld.so
> 7f2f8b06c000-7f2f8b06d000 r-xp 00001000 fd:00 11818202 /home/ysuenaga/github/jvmti-examples/helloworld/out/build/libhelloworld.so
> 7f2f8b06d000-7f2f8b06e000 r--p 00002000 fd:00 11818202 /home/ysuenaga/github/jvmti-examples/helloworld/out/build/libhelloworld.so
> 7f2f8b06e000-7f2f8b06f000 r--p 00002000 fd:00 11818202 /home/ysuenaga/github/jvmti-examples/helloworld/out/build/libhelloworld.so
> 7f2f8b06f000-7f2f8b070000 rw-p 00003000 fd:00 11818202 /home/ysuenaga/github/jvmti-examples/helloworld/out/build/libhelloworld.so

This pull request has now been integrated.

Changeset: 90960c5f
Author:    Yasumasa Suenaga <ysuenaga at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/90960c5f
Stats:     15 lines in 3 files changed: 10 ins; 0 del; 5 mod

8252657: JVMTI agent is not unloaded when Agent_OnAttach is failed

Reviewed-by: dholmes, sspitsyn

-------------

PR: https://git.openjdk.java.net/jdk/pull/19