Log Visualization Tools

[Ramki Ramakrishna]

Historically, the non-standard ("extra fancy") flag output has not been 
governed
by any rules. The basic logging format however has basically not changed 
since
1.4.2, as far as i recall. Of course, each collector, typically released 
in a major new release,
has had its little quirks even within the basic format. Unfortunately, 
the situation
has not been governed by any strict rules -- although we have never 
knowingly introduced
changes to formatting in minor releases -- or even major releases -- for 
fear of breaking
existing log-parsing scripts, I am sure some have slipped through. In 
the absence
of a spec for the format, QA has never written tests to protect against 
inadvertent
regressions. (Sorry for talking like that; i know it sounds rather like 
a lawyer or politician
talking when i read that email back! :-) I am hoping things will get 
better, more standardized,
going forward, so people can use tools without fear of them breaking 
with a new release.

-- ramki

On 8/5/2011 2:06 PM, Matt Fowles wrote:
> Ramki~
>
> What rules govern the upgrade of logging formats?  Can the format only
> be changed on major releases or can we just add a flag 'new format' to
> minor releases?
>
> Matt
>
> On Fri, Aug 5, 2011 at 4:33 PM, Ramki Ramakrishna
> <y.s.ramakrishna at oracle.com>  wrote:
>> Sorry for the noise: my response was sent to the wrong list in error;
>> corrected herewith.
>>
>> -- ramki
>>
>> On 8/5/2011 1:28 PM, Ramki Ramakrishna wrote:
>>> Same here -- i sometimes use an internal homegrown awk script to
>>> extract the
>>> metrics, so they can be massaged into a data file amenable to plotting
>>> with gnuplot.
>>> That script does not take well to the extra output either, so we
>>> usually strip out the
>>> extra output and deal with only the more fundamental metrics only. The
>>> extra output
>>> from the more fancy flags has thus far been consumed only by humans or
>>> extracted on an ad-hoc basis into
>>> spreadsheets and such. This is clearly not a nice state of affairs. I
>>> believe there is
>>> work (or plans?) underway for some kind of logging framework into
>>> which the JVM will feed
>>> its metrics, and hopefully the tooling that consumes those logs will
>>> be able to
>>> deal with all these issues in a more uniform fashion once and for
>>> all.... Unfortunately,
>>> I have no real details of that work, though...
>>>
>>> Then there is gchisto which is GC-specific (but which also cannot
>>> consume the output
>>> from the more fancy flags), but that has been placed on the backseat
>>> as other issues
>>> have intervened.
>>> In general, until GC logging formats are standardized, tools that
>>> consume textual
>>> output from the JVM/GC will tend to break unless changes to these text
>>> formats are
>>> carefully controlled. There has been some talk on and off about trying to
>>> standardize those formats, but I am not sure about the status of that.
>>> May be the
>>> logging framework mentioned earlier will provide a superstructure from
>>> which such
>>> textual standardization will result naturally.
>>>
>>> -- ramki
>>>
>>> On 8/5/2011 12:40 PM, Eric Caspole wrote:
>>>> Sometimes I use HPjmeter for plain Xloggc, but I don't think it can
>>>> do the fancy extra flags either.
>>>>
>>>> On Aug 5, 2011, at 1:51 PM, Matt Fowles wrote:
>>>>
>>>>> All~
>>>>>
>>>>> What tools do people know of or have for parsing gc logs and
>>>>> visualizing the results?
>>>>>
>>>>> The only thing I can find, GCViewer, (from
>>>>> http://www.tagtraum.com/gcviewer.html) seems like it has not been
>>>>> updated for a while and does not parse a lot of more complicated logs
>>>>> (-XX:+PrintTenuringDistribution or -XX:PrintCMSStatistics=1).
>>>>>
>>>>> Are there more tools out there?  Are there in house tools that people
>>>>> are willing to share?
>>>>>
>>>>> Matt
>>>>> _______________________________________________
>>>>> hotspot-gc-use mailing list
>>>>> hotspot-gc-use at openjdk.java.net
>>>>> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
>>>>>
>>>> _______________________________________________
>>>> hotspot-gc-use mailing list
>>>> hotspot-gc-use at openjdk.java.net
>>>> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
>> _______________________________________________
>> hotspot-gc-use mailing list
>> hotspot-gc-use at openjdk.java.net
>> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
>>
_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use at openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use