<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
p.m-5496290937553094592msolistparagraph, li.m-5496290937553094592msolistparagraph, div.m-5496290937553094592msolistparagraph
{mso-style-name:m_-5496290937553094592msolistparagraph;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:1882788047;
mso-list-template-ids:772590448;}
@list l0:level1
{mso-level-tab-stop:.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level2
{mso-level-tab-stop:1.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level3
{mso-level-tab-stop:1.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level4
{mso-level-tab-stop:2.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level5
{mso-level-tab-stop:2.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level6
{mso-level-tab-stop:3.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level7
{mso-level-tab-stop:3.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level8
{mso-level-tab-stop:4.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level9
{mso-level-tab-stop:4.5in;
mso-level-number-position:left;
text-indent:-.25in;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal">Thanks for your review, JC. New webrev: <a href="http://cr.openjdk.java.net/~phh/8196989/webrev.03/">
http://cr.openjdk.java.net/~phh/8196989/webrev.03/</a><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I updated the copyright date in memoryService.hpp because I forgot to do so in the patch for
<a href="https://bugs.openjdk.java.net/browse/JDK-8195115">https://bugs.openjdk.java.net/browse/JDK-8195115</a>. Thomas asked me to fix in it a separate CR, so I’ve reverted it. Ditto the #include fixes in g1FullGCOopClosures.inline.hpp and g1HeapVerifier.cpp.
At one point during development, clang complained about the latter, but no longer does.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">The ‘break’ on the same line as the ‘}’ was in the original version, but I’ve moved it. :)<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">The comment is indeed a bit opaque. I changed it to:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> // Only check heap pools that support a usage threshold.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> // This is typically only the old generation space<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> // since the other spaces are expected to get filled up.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> if (p.getType() == MemoryType.HEAP &&<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> p.isUsageThresholdSupported()) {<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> // In all collectors except G1, only the old generation supports a<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> // usage threshold. The G1 legacy mode "G1 Old Gen" also does. In<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> // G1 default mode, both the old space ("G1 Old Space": it's not<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> // really a generation in the non-G1 collector sense) and the<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> // humongous space ("G1 Humongous Space"), support a usage threshold.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> // So, the following condition is true for all non-G1 old generations,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> // for the G1 legacy old gen, and for the G1 default humongous space.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> // It is not true for the G1 default old gen.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> //<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> // We're allocating humongous objects in this test, so the G1 default<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> // mode "G1 Old Space" occupancy doesn't change, because humongous<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> // objects are allocated in the "G1 Humongous Space". If we allowed<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> // the G1 default mode "G1 Old Space", notification would never<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> // happen because no objects are allocated there.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> if (!p.getName().equals("G1 Old Space")) {<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Finally, the G1MonitoringScope constructor now does a better job of selecting a memory manager.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Paul<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">JC Beyler <jcbeyler@google.com><br>
<b>Date: </b>Wednesday, October 17, 2018 at 4:47 PM<br>
<b>To: </b>"Hohensee, Paul" <hohensee@amazon.com><br>
<b>Cc: </b>"hotspot-gc-dev@openjdk.java.net" <hotspot-gc-dev@openjdk.java.net>, "serviceability-dev@openjdk.java.net" <serviceability-dev@openjdk.java.net><br>
<b>Subject: </b>Re: 8196989: Revamp G1 JMX MemoryPool and GarbageCollector MXBean definitions<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal">Hi Paul, <o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I looked at this but it took time for me to "digest" it and I haven't entirely gone through the real GC changes :)<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">My few remarks on the webrev itself are:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> - <a href="http://cr.openjdk.java.net/~phh/8196989/webrev.02/src/hotspot/share/services/memoryService.hpp.udiff.html">http://cr.openjdk.java.net/~phh/8196989/webrev.02/src/hotspot/share/services/memoryService.hpp.udiff.html</a><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> - There is no need to change the copyright, right?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"> - <a href="http://cr.openjdk.java.net/~phh/8196989/webrev.02/test/hotspot/jtreg/gc/TestMemoryMXBeansAndPoolsPresence.java.udiff.html">http://cr.openjdk.java.net/~phh/8196989/webrev.02/test/hotspot/jtreg/gc/TestMemoryMXBeansAndPoolsPresence.java.udiff.html</a><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> - the break seems to be on the wrong line, no?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">+ } break;<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"> - <a href="http://cr.openjdk.java.net/~phh/8196989/webrev.02/test/jdk/java/lang/management/MemoryMXBean/LowMemoryTest.java.udiff.html">
http://cr.openjdk.java.net/~phh/8196989/webrev.02/test/jdk/java/lang/management/MemoryMXBean/LowMemoryTest.java.udiff.html</a><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> and<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <a href="http://cr.openjdk.java.net/~phh/8196989/webrev.02/test/jdk/java/lang/management/MemoryMXBean/MemoryManagement.java.udiff.html">
http://cr.openjdk.java.net/~phh/8196989/webrev.02/test/jdk/java/lang/management/MemoryMXBean/MemoryManagement.java.udiff.html</a><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<div>
<p class="MsoNormal">+ // In G1, humongous objects are tracked in the old space only in<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">+ // legacy monitoring mode. In default mode, G1 tracks humongous<o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">+ // objects in the humongous space, which latter also supports a<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">+ // usage threshold. Since we're allocating humongous objects in<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">+ // this test, in default mode the old space doesn't change. For<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">+ // this test, we use the old space in legacy mode (it's called<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">+ // "G1 Old Gen" and the humongous space in default mode. If we<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">+ // used "G1 Old Space" in default mode, notification would never<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">+ // happen.<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">-> latter seems ot be the wrong word or something is missing in that sentence<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">-> the parenthesis is never closed (it's called.... is missing a ) somewhere<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Thanks,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Jc<o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Wed, Oct 17, 2018 at 1:18 PM Hohensee, Paul <<a href="mailto:hohensee@amazon.com">hohensee@amazon.com</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Ping.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b><span style="color:black">From:
</span></b><span style="color:black">serviceability-dev <<a href="mailto:serviceability-dev-bounces@openjdk.java.net" target="_blank">serviceability-dev-bounces@openjdk.java.net</a>> on behalf of "Hohensee, Paul" <<a href="mailto:hohensee@amazon.com" target="_blank">hohensee@amazon.com</a>><br>
<b>Date: </b>Thursday, October 11, 2018 at 6:46 PM<br>
<b>To: </b>"<a href="mailto:hotspot-gc-dev@openjdk.java.net" target="_blank">hotspot-gc-dev@openjdk.java.net</a>" <<a href="mailto:hotspot-gc-dev@openjdk.java.net" target="_blank">hotspot-gc-dev@openjdk.java.net</a>>, "<a href="mailto:serviceability-dev@openjdk.java.net" target="_blank">serviceability-dev@openjdk.java.net</a>"
<<a href="mailto:serviceability-dev@openjdk.java.net" target="_blank">serviceability-dev@openjdk.java.net</a>><br>
<b>Subject: </b>Re: 8196989: Revamp G1 JMX MemoryPool and GarbageCollector MXBean definitions</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Any takers? :)<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b><span style="color:black">From:
</span></b><span style="color:black">serviceability-dev <<a href="mailto:serviceability-dev-bounces@openjdk.java.net" target="_blank">serviceability-dev-bounces@openjdk.java.net</a>> on behalf of "Hohensee, Paul" <<a href="mailto:hohensee@amazon.com" target="_blank">hohensee@amazon.com</a>><br>
<b>Date: </b>Monday, October 8, 2018 at 7:50 PM<br>
<b>To: </b>"<a href="mailto:hotspot-gc-dev@openjdk.java.net" target="_blank">hotspot-gc-dev@openjdk.java.net</a>" <<a href="mailto:hotspot-gc-dev@openjdk.java.net" target="_blank">hotspot-gc-dev@openjdk.java.net</a>>, "<a href="mailto:serviceability-dev@openjdk.java.net" target="_blank">serviceability-dev@openjdk.java.net</a>"
<<a href="mailto:serviceability-dev@openjdk.java.net" target="_blank">serviceability-dev@openjdk.java.net</a>><br>
<b>Subject: </b>RFR: 8196989: Revamp G1 JMX MemoryPool and GarbageCollector MXBean definitions</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Bug:
<a href="https://bugs.openjdk.java.net/browse/JDK-8196989" target="_blank">https://bugs.openjdk.java.net/browse/JDK-8196989</a><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">CSR:
<a href="https://bugs.openjdk.java.net/browse/JDK-8196991" target="_blank">https://bugs.openjdk.java.net/browse/JDK-8196991</a><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Webrev:
<a href="http://cr.openjdk.java.net/~phh/8196989/webrev.02/" target="_blank">http://cr.openjdk.java.net/~phh/8196989/webrev.02/</a><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">As requested, I split the jstat counter update off from the MXBean update. This is the MXBean update. The jstat counter RFE is
<a href="https://bugs.openjdk.java.net/browse/JDK-8210965" target="_blank">https://bugs.openjdk.java.net/browse/JDK-8210965</a> and its CSR is
<a href="https://bugs.openjdk.java.net/browse/JDK-8210966" target="_blank">https://bugs.openjdk.java.net/browse/JDK-8210966</a>.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">The MXBean CSR is in draft state, I’d greatly appreciate review and sign-off.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">It’s been suggested that we add another pool to represent the free region set, but doing so would be incompatible with existing MXBean use invariants for all GCs. These are:<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<ol start="1" type="1">
<li class="m-5496290937553094592msolistparagraph" style="mso-list:l0 level1 lfo1">
The sum of the pools’ MemoryUsage.max properties is the total reserved heap size.<o:p></o:p></li><li class="m-5496290937553094592msolistparagraph" style="mso-list:l0 level1 lfo1">
The sum of the pools’ MemoryUsage.committed properties is the total committed size.<o:p></o:p></li><li class="m-5496290937553094592msolistparagraph" style="mso-list:l0 level1 lfo1">
The sum of the pools’ MemoryUsage.used properties is the total size of the memory containing objects, live and dead-and-yet-to-be-collected, as the case might be, plus intentional gaps between them.<o:p></o:p></li><li class="m-5496290937553094592msolistparagraph" style="mso-list:l0 level1 lfo1">
The total free space is (sum of the max properties – sum of the used properties).<o:p></o:p></li><li class="m-5496290937553094592msolistparagraph" style="mso-list:l0 level1 lfo1">
The total uncommitted space is (sum of the max properties – sum of the committed properties).<o:p></o:p></li><li class="m-5496290937553094592msolistparagraph" style="mso-list:l0 level1 lfo1">
The total committed free space is (2) – (3).<o:p></o:p></li></ol>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">To keep invariants 1, 2 and 3, the free region pool’s “max” property should be “undefined” (i.e., -1). The intuitive, to me, “used” property value would be the total free space,
but that would violate invariant 4 above. Defining the “committed” property as the total committed free space would violate invariants 2 and 6.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">The patch passes the submit repo, hotspot tier1, and, separately, the serviceability, jfr, and gc jtreg tests. I’m uncertain how to construct a test that checks for valid MXBean
content: the existing tests don’t. Any such test will be fragile due to possible future Hotspot changes that affect the values, and to run-to-run variability. I’ve done by-hand comparisons between the old and new MXBean content using the SwingSet2 demo, including
using App CDS, and the numbers look reasonable.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">The guts of the change are in G1MonitoringSupport::recalculate_sizes, initialize_serviceability, memory_managers, memory_pools, and G1MonitoringScope. I also defined TraceConcMemoryManagerStats
to track the concurrent cycle in a way analogous to TraceCMSMemoryManagerStats. The changes to the includes in g1FullGCOopClosures.inline.hpp and g1HeapVerifier.cpp are to satisfy compiler complaints. I changed the 3<sup>rd</sup> argument to the G1MonitoringScope
constructor to a mixed_gc flag, and use accessor methods instead of direct field accesses when accessor methods exist. I believe I’ve minimized the latter. I updated the copyright date to 2018 in memoryService.hpp because I neglected to do so in my previous
G1 MXBean patch.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Thanks,<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Paul<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><br clear="all">
<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal">-- <o:p></o:p></p>
<div>
<div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal">Thanks, <o:p></o:p></p>
<div>
<p class="MsoNormal">Jc<o:p></o:p></p>
</div>
</div>
</div>
</div>
</body>
</html>