RFR(S): 8227122: [TESTBUG] Create Docker sidecar test cases

[Severin Gehwolf]

Hi Misha,

On Tue, 2019-07-02 at 15:24 -0700, mikhailo.seledtsov at oracle.com wrote:
> Please review this new test that uses a Docker sidecar pattern to 
> manage/monitor JVM running in the main payload container.
> 
> Sidecar is a common pattern used in the cloud environments for 
> monitoring among other uses. In side car pattern the main 
> application/service container that runs the payload is paired with a 
> sidecar container. It is achieved by sharing certain namespace
> aspects 
> between the two containers such as PID namespace, specific 
> sub-directories, IPC and more.
> 
> This test implements the following cases:
>    - "jcmd -l" to list java processes running in "main" container
> from 
> the "sidecar" container
>    - "jhsdb jinfo" in the sidecar configuration
>    - jcmd <some-command>
> 
> This change also builds a basis for more test cases in the future.
> 
> Minor changes were done to DockerTestUtils:
>    - changing access to DOCKER_COMMAND constant to public
>    - minor spelling and terminology corrections
> 
> 
>      JBS: https://bugs.openjdk.java.net/browse/JDK-8227122
>      Webrev: http://cr.openjdk.java.net/~mseledtsov/8227122.00/
>      Testing:
>          1. ran Docker tests on Linux-x64 - PASS
>          2. Running Docker tests in test cluster - in progress
> 


// JCMD does not work in sidecar configuration, except for "jcmd -l".
// Including this test case to assist in reproduction of the problem.
// t.assertIsAlive();
// testCase03(mainProcPid);

FWIW, "jcmd -l" doesn't work in this case either. It only sees itself
as far as I can tell. It should see the JVM of the host container too.
That issue can be fixed by creating a shared /tmp filesystem and
mounting into both containers.

What's more, this seems to be a case of AttachListener::is_init_trigger[1] and
VirtualMachineImpl.createAttachFile[2] disagreeing. The former looks in
$(pwd)/.attach_pid<pid> or /tmp/.attach_pid<pid> and the latter creates
it in /proc/<pid>/root/tmp/.attach_pid<ns_pid>.

There seems to be more issues involved. As attaching to a JVM inside a
container doesn't seem to work from outside which is supposed to be
fixed with JDK-8179498. That alone seems to warrant a bug.


    private static DockerThread startMainContainer() throws Exception {
        // start "main" container (the observee)
        DockerRunOptions opts = commonDockerOpts("EventGeneratorLoop");
        opts.addDockerOpts("--cap-add=SYS_PTRACE", "--ipc=shareable")

Is '--ipc=shareable' really needed? It's not a supported option for my
docker here :-(

Thanks,
Severin

[1] http://hg.openjdk.java.net/jdk/jdk/file/ba72dac556c3/src/hotspot/os/linux/attachListener_linux.cpp#l500
[2] http://hg.openjdk.java.net/jdk/jdk/file/ba72dac556c3/src/jdk.attach/linux/classes/sun/tools/attach/VirtualMachineImpl.java#l295