On Dec 3, 2007 1:16 PM, Ciciora, Paul <Ciciora@cboe.com> wrote:
The idea is you'd include the time stamp as a tagged value. Then you could have the format you wanted such as seconds since process startup (my favorite) and each time format would have a separate tag.
But those stamps are only included in GC logs, right? Thanks, Neo
-----Original Message----- From: Neo Jia [mailto:neojia@gmail.com] Sent: Monday, December 03, 2007 3:13 PM To: Ciciora, Paul Cc: hotspot-gc-dev@openjdk.java.net
Subject: Re: PrintGCStats
I like the idea about tag/value stuffs.
But I do worry about sending to a separate file unless we put a timestamp for each log entry, otherwise it would be hard to debug.
Thanks, Neo
On Dec 3, 2007 12:25 PM, Ciciora, Paul <Ciciora@cboe.com> wrote:
Since GC formats are constantly changing and different collectors log stats differently I'd love to so you adopt a harvestable output format. The idea would be to send stats to a separate file and use a name/value style format. That way you could send whatever you wanted there and all the harvester script or program would have to know is the tag name. The name could be terse to save space. You could register the tag names so open source changes would conform. If you did that then format changes to the human readable output could be separate. You wouldn't have to be as sensitive to breaking scripts.
Just a thought. I'm probably not the first one to suggest this.
-----Original Message----- From: hotspot-gc-dev-bounces@openjdk.java.net [mailto:hotspot-gc-dev-bounces@openjdk.java.net] On Behalf Of hotspot-gc-dev-request@openjdk.java.net Sent: Monday, December 03, 2007 2:00 PM To: hotspot-gc-dev@openjdk.java.net Subject: hotspot-gc-dev Digest, Vol 6, Issue 1
Send hotspot-gc-dev mailing list submissions to hotspot-gc-dev@openjdk.java.net
To subscribe or unsubscribe via the World Wide Web, visit http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-dev or, via email, send a message with subject or body 'help' to hotspot-gc-dev-request@openjdk.java.net
You can reach the person managing the list at hotspot-gc-dev-owner@openjdk.java.net
When replying, please edit your Subject line so it is more specific than "Re: Contents of hotspot-gc-dev digest..."
Today's Topics:
1. PrintGCStats (Naas, Dave) 2. Re: PrintGCStats (Peter B. Kessler)
----------------------------------------------------------------------
Message: 1 Date: Mon, 3 Dec 2007 10:57:30 -0600 From: "Naas, Dave" <naasd@cboe.com> Subject: PrintGCStats To: <hotspot-gc-dev@openjdk.java.net> Message-ID:
<4BD174404CF4A34C98322DC926CF862B2289F2@MSMAIL.cboent.cboe.com>
Content-Type: text/plain; charset="iso-8859-1"
Regarding the posting from Oct 26, 2007:
http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2007-October/00006
4.html where Mr. Kessler states: quote > > I'll see if we can get an updated version of PrintGCStats out to replace the ancient one available at > > <http://java.sun.com/developer/technicalArticles/Programming/turbo/> > > That should give you the kinds of statistics you are asking for. > > ... peter unquote
I was wondering if a new version of PrintGCStats and PrintGCFixup will indeed be made available or if we should proceed with our own updates to handle new jdk releases, etc.??
A specific, recent example is jdk 1.6 with the -XX:PrintFLSStatistics flag. We have found that this setting prevents the current PrintGCStats from properly parsing ParNew data and thus doesn't generate the Promotion stats (Promo.csv file). The following snippet shows how the ParNew line is now split (shown in bold), thanks to the -XX:PrintFLSStatistics:
========================================================================
========================================================================
====== {Heap before GC invocations=0 (full 0): par new generation total 305088K, used 277376K [0x2f800000, 0x43d00000, 0x43d00000) eden space 277376K, 100% used [0x2f800000, 0x406e0000, 0x406e0000) from space 27712K, 0% used [0x406e0000, 0x406e0000, 0x421f0000) to space 27712K, 0% used [0x421f0000, 0x421f0000, 0x43d00000) concurrent mark-sweep generation total 2739200K, used 0K [0x43d00000, 0xeb000000, 0xeb000000) concurrent-mark-sweep perm gen total 131072K, used 14913K [0xeb000000, 0xf3000000, 0xfb000000) 8.746: [GC Before GC: Statistics for BinaryTreeDictionary: ------------------------------------ Total Free Space: 701235200 Max Chunk Size: 701235200 Number of Blocks: 1 Av. Block Size: 701235200 Tree Height: 1 Before GC: Statistics for BinaryTreeDictionary: ------------------------------------ Total Free Space: 0 Max Chunk Size: 0 Number of Blocks: 0 Tree Height: 0 8.746: [ParNew Desired survivor size 14188544 bytes, new threshold 1 (max 4) - age 1: 15763640 bytes, 15763640 total - age 2: 56 bytes, 15763696 total : 277376K->15474K(305088K), 0.5657873 secs] 277376K->15474K(3044288K)After GC: Statistics for BinaryTreeDictionary: ------------------------------------ Total Free Space: 701218816 Max Chunk Size: 701218816 Number of Blocks: 1 Av. Block Size: 701218816 Tree Height: 1 After GC: Statistics for BinaryTreeDictionary: ------------------------------------ Total Free Space: 0 Max Chunk Size: 0 Number of Blocks: 0 Tree Height: 0 , 0.5668866 secs] [Times: user=0.34 sys=1.07, real=0.57 secs]
========================================================================
========================================================================
======
Thank you, Dave
David Naas Chicago Board Options Exchange 312-786-7222
------------------------------
Message: 2 Date: Mon, 03 Dec 2007 10:19:41 -0800 From: "Peter B. Kessler" <Peter.Kessler@Sun.COM> Subject: Re: PrintGCStats To: "Naas, Dave" <naasd@cboe.com> Cc: hotspot-gc-dev@openjdk.java.net Message-ID: <475448BD.3030401@Sun.COM> Content-Type: text/plain; format=flowed; charset=ISO-8859-1
It looks like the most recent PrintGCFixup (and PrintGCStats and CompareGCStats) is "posted" in
http://permalink.gmane.org/gmane.comp.java.openjdk.hotspot.gc.devel/51
You have to join all the lines that got wrapped (sigh). Sometimes you have to run PrintGCFixup more than once to get it to remove things so that it can see other things to remove. Though, in your case, running it twice doesn't help remove the output of PrintFLSStatistics. It does look like if you add a cleaner for PrintFLSStatistics, you'll have to run PrintGCFixup at least twice, since that output gets in the way of the removal of the PrintTenuringDistribution flag.
PrintGCFixup and friends look like a prime candidate for putting in hotspot/tools/ and letting people make improvements. If you wanted to contribute your addition for removing PrintFLSStatistics, I'm sure we could find someone here to sponsor it.
... peter
Naas, Dave wrote:
Regarding the posting from Oct 26, 2007:
http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2007-October/00006
4.html
where Mr. Kessler states: quote > > I'll see if we can get an updated version of PrintGCStats out to replace the ancient one available at > > <http://java.sun.com/developer/technicalArticles/Programming/turbo/> > > That should give you the kinds of statistics you are asking for. > > ... peter unquote
I was wondering if a new version of PrintGCStats and PrintGCFixup will indeed be made available or if we should proceed with our own updates to handle new jdk releases, etc.??
A specific, recent example is jdk 1.6 with the -XX:PrintFLSStatistics flag. We have found that this setting prevents the current PrintGCStats from properly parsing ParNew data and thus doesn't generate the Promotion stats (Promo.csv file). The following snippet shows how the ParNew line is now split (shown in bold), thanks to the -XX:PrintFLSStatistics:
========================================================================
========================================================================
======
{Heap before GC invocations=0 (full 0): par new generation total 305088K, used 277376K [0x2f800000, 0x43d00000, 0x43d00000) eden space 277376K, 100% used [0x2f800000, 0x406e0000, 0x406e0000) from space 27712K, 0% used [0x406e0000, 0x406e0000, 0x421f0000) to space 27712K, 0% used [0x421f0000, 0x421f0000, 0x43d00000) concurrent mark-sweep generation total 2739200K, used 0K [0x43d00000, 0xeb000000, 0xeb000000) concurrent-mark-sweep perm gen total 131072K, used 14913K [0xeb000000, 0xf3000000, 0xfb000000) 8.746: [GC Before GC: Statistics for BinaryTreeDictionary: ------------------------------------ Total Free Space: 701235200 Max Chunk Size: 701235200 Number of Blocks: 1 Av. Block Size: 701235200 Tree Height: 1 Before GC: Statistics for BinaryTreeDictionary: ------------------------------------ Total Free Space: 0 Max Chunk Size: 0 Number of Blocks: 0 Tree Height: 0 8.746: [ParNew Desired survivor size 14188544 bytes, new threshold 1 (max 4) - age 1: 15763640 bytes, 15763640 total - age 2: 56 bytes, 15763696 total : 277376K->15474K(305088K), 0.5657873 secs] 277376K->15474K(3044288K)After GC: Statistics for BinaryTreeDictionary: ------------------------------------ Total Free Space: 701218816 Max Chunk Size: 701218816 Number of Blocks: 1 Av. Block Size: 701218816 Tree Height: 1 After GC: Statistics for BinaryTreeDictionary: ------------------------------------ Total Free Space: 0 Max Chunk Size: 0 Number of Blocks: 0 Tree Height: 0 , 0.5668866 secs] [Times: user=0.34 sys=1.07, real=0.57 secs]
========================================================================
========================================================================
======
Thank you, Dave
David Naas Chicago Board Options Exchange 312-786-7222
End of hotspot-gc-dev Digest, Vol 6, Issue 1 ********************************************
-- I would remember that if researchers were not ambitious probably today we haven't the technology we are using!
-- I would remember that if researchers were not ambitious probably today we haven't the technology we are using!