Call for Discussion: New Project: Hamburg

Wed Jun 24 11:31:35 UTC 2020

Thanks Martijn!

We will also need to find a sponsoring home for this project, so any
idea is welcomed ;)

Cheers,
Mario

On Wed, Jun 24, 2020 at 12:30 PM Martijn Verburg
<martijnverburg at gmail.com> wrote:
>
> Hi Mario,
>
> Thanks for posting this.  Our group (Microsoft Java Engineering) is going to take a look at all the links and info and revert our thoughts back here.  It's the end of the quarter for us, so it might be sometime next week in case you're wondering why the silence :-)
>
> Cheers,
> Martijn
>
>
> On Tue, 23 Jun 2020 at 16:09, Mario Torre <neugens at redhat.com> wrote:
>>
>> Hello OpenJDK developers.
>>
>> [I apologise, this is a long email, but I thought sharing more details
>> would be useful for the discussion.]
>>
>> I would like to start a discussion for a new Project, Project Hamburg
>> [1], whose primary goal is to address the issue of exposing and
>> organising JFR recordings within a container setup, where direct
>> access to the target JVM is not possible.
>>
>> The recent discussion about the tentative backport of the Streaming
>> API for JFR to 11u and 8u has highlighted the desire to have a
>> standardised way to easily obtain the JFR recordings not only for
>> (quasi) real time monitoring but also as a mean of accessing event
>> metrics from OpenJDKs within remote installations, like those in
>> servers deployments and specifically from within containers - for
>> example Kubernetes/OpenShift-managed cloud.
>>
>> In the last few months, some of us have been working on a project that
>> aims to make it easier to use JFR and JDK Mission Control in those
>> contexts, and I would like to start a discussion about potentially
>> promoting our code as an official OpenJDK Project, where it may be
>> useful for a larger community of developers.
>>
>> At the current state, Project Hamburg lives in a github repository
>> under the name “Container-JFR”, and has several components designed to
>> work together at various [optional] levels of integration, the
>> proposed Project will maintain overall the same layout.
>>
>> [Container JFR - https://github.com/rh-jmc-team/container-jfr]
>>
>> A containerized web service that communicates with JVMs in the same
>> Kubernetes namespace. This communication involves invoking JFR
>> operations over remote JMX, which is made available by the VMs'
>> corresponding Kubernetes Services, and does not need to be exposed to
>> the external world. Instead, clients can interact with Container JFR
>> via a WebSocket command-based interface. These commands include
>> operations like starting and stopping recordings with given options,
>> listing in-memory recordings, listing event types, and saving a
>> recording to persistent storage, etc...
>>
>> This web service also serves raw JFR files for download and analysis
>> with JMC, and generates HTML reports identical to JMC's automated
>> analysis.
>>
>> [Container JFR Core - https://github.com/rh-jmc-team/container-jfr-core]
>>
>> This project is a dependency of Container JFR. It contains interfaces
>> and utilities for calling JMC code to establish remote JMX connections
>> and communicate with JFR. The JMC dependencies are abstracted so we
>> can replace this module without requiring access to the JMC API
>> (although since JMC is also an OpenJDK Project we think this is not a
>> problem).
>>
>> [Container JFR Web - https://github.com/rh-jmc-team/container-jfr-web]
>>
>> A front-end to graphically communicate with Container JFR via web
>> browser. It allows the user to connect to a JVM from a drop down list
>> of available JVMs. Once connected, Container JFR Web displays a table
>> of available event types and their options. Users can then use a
>> wizard to create a recording, and see a list of recordings present in
>> that JVM. For each recording, the user can see the automated analysis
>> results provided by JMC code, save the recording to persistent
>> in-cluster storage, download the recording to local disk, or view
>> compatible metrics in a Grafana dashboard. This mimics some of the
>> functionality of JMC to provide an overview of the target JVM and
>> offers the option to download the JFR file further analysis in JMC.
>>
>> [JFR Datasource - https://github.com/rh-jmc-team/jfr-datasource]
>>
>> This project parses a provided JFR file and serves it as a Grafana
>> SimpleJson datasource in order to populate a Grafana dashboard with
>> metrics from the JFR file.
>>
>> [Container JFR Operator - https://github.com/rh-jmc-team/container-jfr-operator]
>>
>> A Kubernetes operator that deploys and manages all of the above
>> projects with a Kubernetes/OpenShift cluster. At this stage, this
>> project is built using Operator SDK.
>>
>> In addition to the standard deployment of all Project Hamburg
>> components along with Grafana, the operator also supports a "minimal"
>> option. This deploys only Container JFR, and no web front-end or
>> Grafana components.
>>
>> The operator also provides a Kubernetes API for JFR using Custom
>> Resource Definitions. This API consists of FlightRecorder objects
>> created by the operator for each compatible Kubernetes Service in its
>> namespace. These objects contain information like JFR event types for
>> the target JVM. A consumer of the API can create Recording objects for
>> a FlightRecorder object and view the status of them via kubectl/oc.
>>
>> The operator reconciles these objects with Container JFR to retrieve
>> information and manage JFR recordings.
>>
>> Furthermore, we plan to incorporate over time API to deploy and
>> control the JMC Agent [2], to allow existing applications to be
>> instrumented without the need to specifically write Events in the
>> application code.
>>
>> Not all the components need to be used together, for example one can
>> easily change the Web part for direct integration into their own
>> framework management consoles or skip the Operator completely, but my
>> hope is that this Project will help in two key aspects: provide a
>> stable and standardised API to access Flight Recordings without
>> exposing the underlying JMX connections and offer a cross platform
>> functionality similar to that of the JFR Streaming that works cross
>> platform also for older releases.
>>
>> The Project will live as a separate entity from OpenJDK, and we don't
>> expect changes to the OpenJDK code base to be necessary, although the
>> experience gathered may of course lead to fixes, additional APIs and
>> JFR Events to support a container aware OpenJDK over time; if this
>> happens to be the case, such changes will be discussed as usual on the
>> proper development mailing list and, when necessary, be addressed via
>> JEPs.
>>
>> I propose to lead this Project with me and Elliott Baron as the
>> initial reviewers.
>>
>> Any comments and feedback is welcomed!
>>
>> Cheers,
>> Mario
>>
>> [1] The project is formerly known as container-jfr, and is of course
>> meant to bridge the worlds of containers with the world of JFR.
>> Hamburg is a bridge city, it’s probably the city with the highest
>> number of bridges in the World - or certainly in Europe. It’s also one
>> of the homes for the largest container ship in the World, the HMM
>> Algeciras, so I thought this name would be perfectly fitting!
>>
>> [2] https://wiki.openjdk.java.net/display/jmc/The+JMC+Agent
>>
>> --
>> Mario Torre
>> Associate Manager, Software Engineering
>> Red Hat GmbH <https://www.redhat.com>
>> 9704 A60C B4BE A8B8 0F30  9205 5D7E 4952 3F65 7898
>>

-- 
Mario Torre
Associate Manager, Software Engineering
Red Hat GmbH <https://www.redhat.com>
9704 A60C B4BE A8B8 0F30  9205 5D7E 4952 3F65 7898