<html><head><meta http-equiv="Content-Type" content="text/html charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div>Hi,</div><div><br></div><div>I would agree that there are great merits to tag-based systems instead of hierarchical systems. However, the JEP does not have a hierarchical system in mind. Components, as the JEP refers to them, are not hierarchical, but they are disjunct. Disjunct in the sense that a particular log output in the source code can only belong to one component - this would be a difference from a purely tag based system. For each component in the proposal, there is the possibility of logging messages of different severities; these are the 'levels' (maybe a bad and confusing terminology?). </div><div><br></div><div>I think I do understand your use-case and I would be happy if we could find a workable system that solves this use-case. I gave it a shot in [1] - my suggestion #3. This suggested implementation had some serious problems in the way you could select log messages. I would love to see a better proposal for how this could be implemented.</div><div><br></div><div>Logging information in binary certainly reduces bulk and can also make it much easier to parse and visualize the data. The JRockit Flight Recorder technology has a binary format for storing information (it is also a self-describing format which includes meta-data to facilitate parsing) which is optimized for storage speed and size. As you probably know, we are working on a port of JFR to HotSpot. </div><div><br></div><div>Thanks,</div><div>/Staffan</div><div><br></div><div>[1] <a href="http://mail.openjdk.java.net/pipermail/serviceability-dev/2012-June/006369.html">http://mail.openjdk.java.net/pipermail/serviceability-dev/2012-June/006369.html</a></div><br><div><div>On 15 aug 2012, at 10:44, Kirk Pepperdine <<a href="mailto:kirk@kodewerk.com">kirk@kodewerk.com</a>> wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><meta http-equiv="Content-Type" content="text/html charset=us-ascii"><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Hi Dmitry,<div><br></div><div>Lets start with this.</div><div><br></div><div><ul style="margin-top: 1ex; margin-bottom: 1ex; margin-right: 2em; padding-left: 2em; list-style-type: square; font-family: 'Bitstream Vera Sans', 'Luxi Sans', Verdana, Arial, Helvetica; font-size: 13.600000381469727px; line-height: 12.800000190734863px; "><li style="margin-top: 0pt; margin-bottom: 0pt; ">Common logging command-line options for all components</li><li style="margin-top: 0pt; margin-bottom: 0pt; ">Logging is performed at different levels: error, warning, info, debug, trace</li><li style="margin-top: 0pt; margin-bottom: 0pt; ">Logging messages are in human-readable plain text</li></ul><div><br></div><div><div>IMHO, these current set of goals are based on the assumptions that all component is hierarchical in nature and that they will log the same way without the possibility of binary dumps. I would argue these assumptions over-reach and preclude certain types of logging that take place in the JVM today.</div><div><br></div><div>The current system of command-line (experimental) options follow a format that is quite consistent. The options provide semantic meaning as to what will be logged. This semantic meaning is lost when we reduce everything to the level of error, warning, info, debug and trace. It's sets up arbitrary categories in which developers will be left to decide what level a particular activity should be logged at. To make this real I'll use the -XX:+PrintTenuringDistribution flag as an example. When I set that flag it's very clear what's to be logged. My question is, should this be logged at the info level? How about debug? Maybe trace? And if I set those levels, what else comes along for the ride? IOWs, if I attempt to keep the same model of GC logging, we'd need different levels. If we need different levels then we're about to violate the first goal.</div><div><br></div><div>What this example suggests is that gc logging would be better served by using categories (or subjects if we can take something from the pub/sub metaphor) or tags or labels or what ever you want to call them. Using tags, I can create a system of levels using error, warning, info, debug, and trace if that fits how I'm setting up logging. But from levels it's difficult to implement things using categories. This is a case where the one of the non-goals should be, we shouldn't prevent people from structuring their logs as they see best fits the component they are instrumenting.</div><div><br></div><div>The non-goals that should be reconsidered is to force human-readability. I would argue that this is again over-reaching in that properly structured log files should almost *never* be read by a human. In the cases why force an unnecessary requirement. I say this because I've run into a number of applications where logging was the primary bottleneck. In those cases there were two primary problems, one was bulk and the other was threading (lock contention which I shan't address here). Now while trying to solve the problem of bulkiness and lock contention might be considered a premature optimization at this stage and I certainty wouldn't disagree, I would not like to see designs or specifications that would naturally preclude some very obvious solutions. One of the solutions to bulk is to use a more compact format be some form of binary. That option has been taken away for what I would call an ideological requirement. There is no technical reason why I shouldn't be able to log in binary other than connivence. In addition, you don't know what else is happening on the system (disk??) you're logging to so it behoves you to be as light as possible.</div><div><br></div><div>I'll leave it at this for now.</div><div><br></div><div>-- Kirk</div><div><br></div><div>On 2012-08-14, at 12:17 PM, Dmitry Samersoff <<a href="mailto:Dmitry.Samersoff@oracle.com">Dmitry.Samersoff@oracle.com</a>> wrote:</div></div><div><br class="Apple-interchange-newline"><blockquote type="cite">Kirk,<br><br><blockquote type="cite">However I do have very serious concerns about this JEP in that it<br>doesn't fix the problems that exist in current logging frameworks,<br>it only mimics them.<br></blockquote><br><blockquote type="cite"><a href="http://openjdk.java.net/jeps/158">http://openjdk.java.net/jeps/158</a><br></blockquote><br>Any comments is much appreciated.<br><br>Personally, I think that log rotation is out of scope and responsibility<br>of JVM.<br><br>-Dmitry<br><br><br>On 2012-08-14 13:26, Kirk Pepperdine wrote:<br><blockquote type="cite">Hi Yasumasa,<br><br>I'm not sure that log file rotation is a part of this JEP. <br>However I do have very serious concerns about this JEP in that it doesn't fix the problems that exist in current logging frameworks, it only mimics them.<br><br>Regards,<br>Kirk<br><br>On 2012-08-14, at 11:20 AM, Yasumasa Suenaga <<a href="mailto:suenaga.yasumasa@lab.ntt.co.jp">suenaga.yasumasa@lab.ntt.co.jp</a>> wrote:<br><br><blockquote type="cite">Hi Staffan,<br><br>May I ask you 2 questions about this JEP?<br><br>1. One of goals of this JEP is defined as following:<br> "File rotation of log files by size (similar to what is available for GC logs today)"<br> My patch realizes log rotation by external trigger.<br> 7090324 is included in this JEP?<br> (Is 7090324 included in "Logging can be configured dynamically at runtime via jcmd or MBeans" ? )<br><br>2. I've posted a patch for "6950794: Make the GC log file name parameterized" .<br> <a href="http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2012-July/004758.html">http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2012-July/004758.html</a><br> Could you include this RFE in this JEP?<br> If we use log rotation, I think that name of logs is very important for log management.<br><br><br>Thanks,<br><br>Yasumasa<br><br><br>On 2012/08/14 16:50, Staffan Larsen wrote:<br><blockquote type="cite">Hi Yasumasa,<br><br>This work should be done in the context of the Unified Logging JEP [1]. Unfortunately, that work has not yet started due to resource constraints. Comments on the proposal are welcome.<br><br>[1] <a href="http://openjdk.java.net/jeps/158">http://openjdk.java.net/jeps/158</a><br><br>/Staffan<br><br>On 31 jul 2012, at 12:45, Yasumasa Suenaga <<a href="mailto:suenaga.yasumasa@lab.ntt.co.jp">suenaga.yasumasa@lab.ntt.co.jp</a> <<a href="mailto:suenaga.yasumasa@lab.ntt.co.jp">mailto:suenaga.yasumasa@lab.ntt.co.jp</a>>> wrote:<br><br><blockquote type="cite">Hi,<br><br>I've posted a patch for gc log rotation via jinfo tool last year.<br>Then I received an email that essence of my patch will be implemented as<br>"subcommands" of jcmd.<br><br>Now, jcmd is provided by OpenJDK. However, current jcmd does not seem to<br>implement function of gclog rotation.<br><br>Will this RFE be implemented?<br><br>I need to rotate gclog which is synchronized with signal, cron, etc... on Linux.<br>So, if this RFE is not implemented for a while, I would like to contribute<br>patch for jcmd.<br><br><br>Regards,<br><br>Yasumasa<br><br><br>On 2011/09/29 21:15, James Melvin wrote:<br><blockquote type="cite">Hi Yasumasa,<br><br>Thanks very much for your OpenJDK contribution! As part of the effort to<br>port JRockit features to HotSpot, we plan to introduce a consolidated<br>commandline serviceability tool (jcmd) to potentially replace many of<br>the existing tools in the bin directory, such as jmap, jstack, jinfo and<br>others. Over the next few update releases, we plan to add several jcmd<br>*subcommands* instead to accomplish specific tasks or affect the running<br>JVM instance. We'd like to use the essence of your contribution in one<br>of the jcmd subcommands (instead of extending jinfo) to ask the JVM to<br>begin rotating GC logs. We hope you find this attractive and hope you<br>will help review and perfect the change.<br><br>Thanks,<br><br>Jim Melvin<br>Sr. Engineering Manager<br>Oracle<br><br><br><br>On 9/26/11 6:38 AM, Yasumasa Suenaga wrote:<br><blockquote type="cite">(I've changed subject of this email to new RFE.)<br><br>This RFE is enhancement of current gclog implementation.<br>So, I'd like to discuss about rotating gclog.<br><br>My customers need gclog rotation which is invoked by external trigger.<br>So I've requested this RFE and made a patch.<br><br><br>In many case on Linux, logfile is rotated by signal (e.g. SIGHUP) .<br>The aim of this RFE is to synchronize gclog and the other logs.<br><br><br>Thanks,<br><br>Yasumasa<br><br>(2011/09/22 20:55), Rainer Jung wrote:<br><blockquote type="cite">On 22.09.2011 13:20, Dmitry Samersoff wrote:<br><blockquote type="cite"><br>Yasumasa,<br><br>On 2011-09-22 04:47, Yasumasa Suenaga wrote:<br><blockquote type="cite">If we can think Java on Linux and Solaris only, syslog is best<br>solution.<br>However, Windows usually doesn't have syslog.<br><br>So, I think that gclog is needed for logging GC stats with platform<br>independent in realtime.<br></blockquote><br>Windows has it's own logging API as reach as syslog is or ever better<br>as well as numerous syslog implementations.<br><br>Native windows logging API was completely redesigned for Windows 2008<br>server and now it allows for developers to send a structured events from<br>theirs application.<br></blockquote><br>AFAIK log rotation for loggc is already implemented though not<br>necessarily yet released. The change discussed here is about supporting<br>an externally generated rotation trigger, e.g. via a signal, instead of<br>only rotating by size or time via a startup parameter.<br><br>If you want support for syslog or Windows APIs included, it would be<br>best to start a new discussion.<br><br>A GC log for an application under load might easily produce a block of<br>about 1.5 KB size every few seconds. I seriously doubt, that the need<br>for log file rotation can be argued away by referring to syslog or<br>Windows log API as the correct solution.<br><br>The messages are not really line formatted, the format can vary a lot<br>(depending on the excat XX switches), the traffic can be quite high and<br>AFAIK the JVM writes it synchronously, so there must be absolutely no<br>risk in writing it out with very little latency. In addition, for<br>analysis, you wouldn't want to look at each event individually, but<br>instead process the whole file through a script or tool, which should<br>not depend on the output specifics of a platform specific log apparatus.<br><br>Regards,<br><br>Rainer<br><br></blockquote><br></blockquote></blockquote><br></blockquote><br></blockquote><br></blockquote><br></blockquote><br><br>-- <br>Dmitry Samersoff<br>Java Hotspot development team, SPB04<br>* There will come soft rains ...<br><br><br></blockquote></div><br></div></div></blockquote></div><br></body></html>