Proposal: Always-on Statistical History

Thu Nov 15 21:37:40 UTC 2018

This was true up until Oracle open sourced it (JDK 11).

If JFR is the framework around which we decide to get these types of metrics from the JVM (in addition to JMX),  then I think that we (the community) should continue to build out JFR adding in those metrics that are not already captured.

Kind regards,
Kirk Pepperdine

> On Nov 15, 2018, at 9:07 AM, Simon Roberts <simon at dancingcloudservices.com> wrote:
> 
> I don't begin to claim to know the politics, legalities, boundaries of JFR license conditionsm and so forth" but:
> 
> Java Flight Recorder requires a commercial license for use in production."
> 
> Whereas, this as I understand is the *open* jdk list. So, I for one would feel hard done by if your view prevailed and only the paying clients got access to a valuable feature.
> 
> 
> On Thu, Nov 15, 2018 at 9:40 AM Roger Riggs <Roger.Riggs at oracle.com <mailto:Roger.Riggs at oracle.com>> wrote:
> Hi,
> 
> This looks like it has significant overlap with JFR.
> I don't think we want to start building in multiple mechanisms to keep 
> tabs on a running VM.
> 
> $.02, Roger
> 
> 
> On 11/14/2018 04:27 PM, Thomas Stüfe wrote:
> > Hi Bernd,
> >
> > On Wed, Nov 14, 2018 at 10:07 PM Bernd Eckenfels <ecki at zusammenkunft.net <mailto:ecki at zusammenkunft.net>> wrote:
> >> Looks good Thomas,
> > thanks!
> >
> >> what would be the typical memory usage with the Default Settings?
> > ~ 80 Kb. Its very small.
> >
> >> Does the downsampling support min/max style rollups?
> > Not sure what you mean. Do you mean does it preserve peaks? Not yet,
> > such a feature would have to be added.
> >
> > Right now, downsampling is very primitive for performance reasons. For
> > snapshot values like heap size etc we just throw away the samples, so
> > you loose temporary peaks. For counter-like values-over-time (e.g.
> > number of pages swapped in etc), they just refer then to a larger time
> > span.
> >
> > Best Regards, Thomas
> >
> >>
> >>
> >> --
> >> http://bernd.eckenfels.net <http://bernd.eckenfels.net/>
> >>
> >>
> >>
> >> Von: Thomas Stüfe
> >> Gesendet: Mittwoch, 14. November 2018 16:29
> >> An: serviceability-dev at openjdk.java.net <mailto:serviceability-dev at openjdk.java.net> serviceability-dev at openjdk.java.net <mailto:serviceability-dev at openjdk.java.net>
> >> Betreff: Proposal: Always-on Statistical History
> >>
> >>
> >>
> >> Hi all,
> >>
> >>
> >>
> >> We have that feature in our port which we would like to contribute,
> >>
> >> and I would like to gauge opinions.
> >>
> >>
> >>
> >> First off, I am not sure which list is correct. This is more of a
> >>
> >> serviceability issue, but implementation wise it fit hs-runtime
> >>
> >> better. I'll start with serviceability, but feel free crosspost if
> >>
> >> needed.
> >>
> >>
> >>
> >> Second, I am aware that this may require a JEP. If necessary and the
> >>
> >> feedback is positive, I will draft one.
> >>
> >>
> >>
> >> ----
> >>
> >>
> >>
> >> In our port we have something called "Statistics History". Basically
> >>
> >> this is a rolling history, spanning up to 10 days, of a number of key
> >>
> >> values. Key values range from JVM specifics like heap size, metaspace
> >>
> >> size, number of threads etc, to platform specifics like memory
> >>
> >> footprint, cpu load, io- and swapping activity etc.
> >>
> >>
> >>
> >> A periodic tasks collects those values, in - by default - 15 second
> >>
> >> intervals. They are then fed into a FIFO. FIFO spans 10 days. To save
> >>
> >> memory that FIFO is downsampled in two steps, so we have the last n
> >>
> >> hours in high resolution and the last n days in low resolution (of
> >>
> >> course all these parameters are configurable).
> >>
> >>
> >>
> >> The history report can be triggered via jcmd, and also could get
> >>
> >> printed in the hs.err file (open for debate).
> >>
> >>
> >>
> >> ---
> >>
> >>
> >>
> >> Here some examples of how the whole thing looks like:
> >>
> >>
> >>
> >> http://cr.openjdk.java.net/~stuefe/webrevs/stathist/examples/stathist-volker.txt <http://cr.openjdk.java.net/~stuefe/webrevs/stathist/examples/stathist-volker.txt>
> >>
> >>
> >>
> >> http://cr.openjdk.java.net/~stuefe/webrevs/stathist/examples/stathist-s390x.txt <http://cr.openjdk.java.net/~stuefe/webrevs/stathist/examples/stathist-s390x.txt>
> >>
> >>
> >>
> >> ---
> >>
> >>
> >>
> >> This feature has been really popular with our support folk over the
> >>
> >> years. Be it that the VM is starved for resources by the OS, that we
> >>
> >> have some slow- or fast developing leak situation etc: these values
> >>
> >> are a first and easy way to get a first stab at a situation, before we
> >>
> >> start more expensive analysis.
> >>
> >>
> >>
> >> The explicit design goal of this history was to be very cheap - cheap
> >>
> >> enough to be *always on* and getting forgotten. It is, in our port,
> >>
> >> enabled by default. That way, if a problem occurs at a customer site,
> >>
> >> we immediately see developments spanning the last 10 days, without
> >>
> >> having to reproduce the issue.
> >>
> >>
> >>
> >> It is also robust enough to be usable during error reporting without
> >>
> >> endangering the error reporting process or falsifying the picture.
> >>
> >>
> >>
> >> I am aware that this crosses over into JFR territory. But this feature
> >>
> >> does not attempt to replace JFR, it is intended instead a cheap always
> >>
> >> on first stop historical overview.
> >>
> >>
> >>
> >> --
> >>
> >>
> >>
> >> I have a patch which can be applied atop of jdk12:
> >>
> >>
> >>
> >> http://cr.openjdk.java.net/~stuefe/webrevs/stathist/stathist.patch <http://cr.openjdk.java.net/~stuefe/webrevs/stathist/stathist.patch>
> >>
> >>
> >>
> >> It works, passes our nightlies and no regressions are shown in dapapo
> >>
> >> benchmarks.
> >>
> >>
> >>
> >> Please tell me what you think. Given enough interest, I will attempt
> >>
> >> to contribute (drafting a JEP if necessary.)
> >>
> >>
> >>
> >> Thanks and Kind Regards,
> >>
> >>
> >>
> >> Thomas
> >>
> >>
> 
> 
> 
> -- 
> Simon Roberts
> (303) 249 3613
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20181115/c73b0fc2/attachment-0001.html>