<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
I am sorry, you should probably talk with Java support -- there are
too many<br>
entangled vectors here, and it wouldn't be appropriate for us to use
this<br>
list for your support problems. I hope you understand.<br>
<br>
Having said that, to find out whether your JVM has a certain option
on or off by default,<br>
you can do -XX:+PrintFlagsFinal and grep for the flag of yr choice,
to find the answer to your question:-<br>
<br>
% $java -XX:+PrintFlagsFinal -version | grep -i escape<br>
bool DoEscapeAnalysis =
true {C2 product}<br>
bool EstimateArgEscape =
true {product}<br>
java version "1.6.0_26"<br>
Java(TM) SE Runtime Environment (build 1.6.0_26-b03)<br>
Java HotSpot(TM) Server VM (build 20.1-b02, mixed mode)<br>
<br>
-- ramki<br>
<br>
On 9/30/2011 8:57 AM, Srini Padman wrote:
<blockquote
cite="mid:1317398266.52316.YahooMailClassic@web162020.mail.bf1.yahoo.com"
type="cite">
<pre wrap="">A quick follow up on this:
Testing with JRE 1.6.0_26 (which we were hoping to upgrade to), our QE team has reported a JVM crash (zipped crash dump file attached). This seems almost exactly the same as the one reported here (supposedly introduced in update 25 and fixed in update 27, which would put us right in the crosshairs):
<a class="moz-txt-link-freetext" href="http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7042582">http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7042582</a>
<a class="moz-txt-link-freetext" href="http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7013538">http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7013538</a>
Before I assume that: the suggested workaround is -XX:-DoEscapeAnalysis - does this mean that the default is +DoEscapeAnalysis? In other words, is that bug something that affects everybody or only those who explicitly use -XX:+DoEscapeAnalysis? We don't use +DoEscapeAnalysis explicitly, so there is no reason for it to affect us if this is not an option enabled by default.
Of course, we are confirming that this is not merely a matter of memory constraints (seems unlikely, since the tests are the same as were run on previous JREs).
Regards,
Srini.
--- On Fri, 9/30/11, Srini Padman <a class="moz-txt-link-rfc2396E" href="mailto:srini_was@yahoo.com"><srini_was@yahoo.com></a> wrote:
</pre>
<blockquote type="cite">
<pre wrap="">From: Srini Padman <a class="moz-txt-link-rfc2396E" href="mailto:srini_was@yahoo.com"><srini_was@yahoo.com></a>
Subject: Re: Intermittent issue with long concurrent marking phase
To: <a class="moz-txt-link-abbreviated" href="mailto:Y.S.Ramakrishna@oracle.com">Y.S.Ramakrishna@oracle.com</a>
Cc: <a class="moz-txt-link-abbreviated" href="mailto:hotspot-gc-use@openjdk.java.net">hotspot-gc-use@openjdk.java.net</a>
Date: Friday, September 30, 2011, 9:05 AM
Hi Ramki,
Apologies, both for mis-reading your original response (re:
long initial-mark phases) and for choosing the wrong list.
Thank you very much for redirecting it to gc-use.
I just want to clarify a couple of points from your last
response, for the record.
To answer the question about the long stop-the-world
initial marking phase: this is the longest I know of, but we
have seen other instances where it lasted 3-4 seconds. In
those cases as well, the "user/sys" times were much smaller
than the "real" time so things clearly seem to be completely
stalled. Also as a matter of background - the reason we
moved to using the CMS collectors was that, prior to this,
we were occasionally seeing extremely long (sometimes
lasting more than a minute) full GCs. It is quite possible
that the same factors that caused such long full GCs in the
past are causing somewhat shorter (but still not _short_)
initial mark with the CMS collector. In any case, I didn't
view this as being related to 6692906 to begin with, and am
glad to get confirmation that you don't think it is either.
Regarding your point "A definite symptom of 6692906 can be
diagnosed if the JVM completely stalls into a livelock (a
few threads in the JVM intermittently active, but your
application will forever stop forward progress from that
point on). It doesn't look like you have observed that
latter symptom however?" - my understanding of the symptom
(based on the summary at <a class="moz-txt-link-freetext" href="http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2010-February/001549.html">http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2010-February/001549.html</a>,
for example) was that the wait/stall resolves itself after
tens of seconds for reasons unknown. Our symptom is closer
to this, in that the application does not stop _forever_
from that point onwards. [It is entirely possible that there
were further findings beyond what is in the posting that I
am not aware of.]
We will initiate efforts to reach out to JVM support - of
course, in the meanwhile, any feedback or help on this forum
is very welcome!
Regards,
Srini.
--- On Thu, 9/29/11, Y. S. Ramakrishna <a class="moz-txt-link-rfc2396E" href="mailto:y.s.ramakrishna@oracle.com"><y.s.ramakrishna@oracle.com></a>
wrote:
From: Y. S. Ramakrishna <a class="moz-txt-link-rfc2396E" href="mailto:y.s.ramakrishna@oracle.com"><y.s.ramakrishna@oracle.com></a>
Subject: Re: Intermittent issue with long concurrent
marking phase
To: "Srini Padman" <a class="moz-txt-link-rfc2396E" href="mailto:srini_was@yahoo.com"><srini_was@yahoo.com></a>
Cc: <a class="moz-txt-link-abbreviated" href="mailto:hotspot-gc-use@openjdk.java.net">hotspot-gc-use@openjdk.java.net</a>
Date: Thursday, September 29, 2011, 8:54 PM
Hi Srini -- As I indicated, if you cannot upgrade easily to
test
if the issue is fixed, you should probably engage JVM
support to
get to a proper diagnosis of the issue affecting your
production systems.
more inline below ...
On 09/29/11 07:44, Srini Padman wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Hi Ramki,
Thank you very much for your reply.
It is not *always* that the concurrent marking phase
</pre>
</blockquote>
<pre wrap="">takes this long, although it happens often enough. For
example, in the full GC log corresponding to the snippet I
pasted in my posting (attached, zipped) there is only that
one instance.
I see that there is one instance of the long _initial-mark_
pause
and as i stated the whole process seems stalled at that
time.
It's definitely not the stall/livelock issue of CR
6692906.
[In other words, you may be dealing with several different
issues
here and you will need to disentangle them.]
</pre>
<blockquote type="cite">
<pre wrap=""> I think I know why you are asking - based on my
</pre>
</blockquote>
<pre wrap="">understanding of Bug # 6692906 (more accurately, based on
discussions around it on this list), I was under the
impression that such long CM phases will happen all the time
(if they happen at all). Does the fact that it is
intermittent raise the possibility that this is a different
issue? I realize that you might not be able to answer this
based on the bits of information you have, but perhaps the
full GC log will tell you something that you don't already
know.
Again, my question about once/many was only about the long
initial mark pause
of which there is exactly one in your log.
The stall of mutators during concurrent marking, which you
are
conjecturing is 6692906, is a different issue: for that you
should
either upgrade and test, or seek JVM support help. It is
definitely
possible that the symptoms of 6692906 will happen
infrequently or
intermittently. A definite symptom of 6692906 can be
diagnosed if
the JVM completely stalls into a livelock (a few threads in
the
JVM intermittently active, but your application will
forever
stop forward progress from that point on). It doesn't look
like
you have observed that latter symptom however?
Sorry I can't really help more at this time, but perhaps
someone
from the community may be able to. But I really suggest
either
an upgrade or seeking JVM support help. (This is not a
professional
support alias.)
In general, the GC-use alias is better suited to questions
such as this.
GC-dev should be used for GC development questions
involving
the main development trunk. Issues of uses in the field of
older versions should be addressed to <a class="moz-txt-link-abbreviated" href="mailto:hotspot-gc-use@o.j.n">hotspot-gc-use@o.j.n</a>
So I've taken the liberty to send this to <a class="moz-txt-link-abbreviated" href="mailto:hotspot-gc-use@o.j.n">hotspot-gc-use@o.j.n</a>
All the best!
-- ramki
</pre>
<blockquote type="cite">
<pre wrap=""> Regards,
Srini.
--- On *Thu, 9/29/11, Ramki Ramakrishna /<a class="moz-txt-link-rfc2396E" href="mailto:y.s.ramakrishna@oracle.com"><y.s.ramakrishna@oracle.com></a>/*
</pre>
</blockquote>
<pre wrap="">wrote:
</pre>
<blockquote type="cite">
<pre wrap="">
From: Ramki Ramakrishna <a class="moz-txt-link-rfc2396E" href="mailto:y.s.ramakrishna@oracle.com"><y.s.ramakrishna@oracle.com></a>
Subject: Re: Intermittent issue with long
</pre>
</blockquote>
<pre wrap="">concurrent marking phase
</pre>
<blockquote type="cite">
<pre wrap=""> To: "Srini Padman" <a class="moz-txt-link-rfc2396E" href="mailto:srini_was@yahoo.com"><srini_was@yahoo.com></a>
Cc: <a class="moz-txt-link-abbreviated" href="mailto:hotspot-gc-dev@openjdk.java.net">hotspot-gc-dev@openjdk.java.net</a>
Date: Thursday, September 29, 2011, 4:24 AM
Hi Srini -- (inline below)
On 9/28/2011 4:50 AM, Srini Padman wrote:
</pre>
<blockquote type="cite">
<pre wrap="">
Questions:
1\ is it clear based on the
</pre>
</blockquote>
</blockquote>
<pre wrap="">description above that the issue is
</pre>
<blockquote type="cite">
<blockquote type="cite">
<pre wrap=""> identical to 6692906 (<a class="moz-txt-link-freetext" href="http://bugs.sun.com/view_bug.do?bug_id=6692906">http://bugs.sun.com/view_bug.do?bug_id=6692906</a>)?
</pre>
</blockquote>
<pre wrap="">
Very likely the same bug.
</pre>
<blockquote type="cite">
<pre wrap=""> 2\ will we benefit by upgrading to a more
</pre>
</blockquote>
</blockquote>
<pre wrap="">recent JRE [1.6.0_26
</pre>
<blockquote type="cite">
<blockquote type="cite">
<pre wrap=""> being the one under consideration]?
</pre>
</blockquote>
<pre wrap="">
Definitely worth trying.
</pre>
<blockquote type="cite">
<pre wrap=""> 3\ I have seen recommendations to use
"-XX:-CMSConcurrentMTEnabled" on some web
</pre>
</blockquote>
</blockquote>
<pre wrap="">forums - but I have
</pre>
<blockquote type="cite">
<blockquote type="cite">
<pre wrap=""> concerns about this; if we don't allow for
</pre>
</blockquote>
</blockquote>
<pre wrap="">concurrent marking to
</pre>
<blockquote type="cite">
<blockquote type="cite">
<pre wrap=""> use multiple threads, then isnt there a
</pre>
</blockquote>
</blockquote>
<pre wrap="">danger of marking
</pre>
<blockquote type="cite">
<blockquote type="cite">
<pre wrap=""> proceeding so slowly that we might end up
</pre>
</blockquote>
</blockquote>
<pre wrap="">running out of memory
</pre>
<blockquote type="cite">
<blockquote type="cite">
<pre wrap=""> i.e., garbage created much faster than it
</pre>
</blockquote>
</blockquote>
<pre wrap="">is collected]?
</pre>
<blockquote type="cite">
<blockquote type="cite">
<pre wrap="">
</pre>
</blockquote>
<pre wrap="">
Your concerns are very legitimate (especially
</pre>
</blockquote>
<pre wrap="">given the length of
</pre>
<blockquote type="cite">
<pre wrap=""> the concurrent mark phase) and the number of
</pre>
</blockquote>
<pre wrap="">cores you have.
</pre>
<blockquote type="cite">
<pre wrap="">
</pre>
<blockquote type="cite">
<pre wrap=""> Any help is greatly appreciated.
</pre>
</blockquote>
</blockquote>
<pre wrap="">Please let me know if any
</pre>
<blockquote type="cite">
<blockquote type="cite">
<pre wrap=""> additional information is needed at all. I
</pre>
</blockquote>
</blockquote>
<pre wrap="">haven't attached the
</pre>
<blockquote type="cite">
<blockquote type="cite">
<pre wrap=""> full GC log (it caused problems with
</pre>
</blockquote>
</blockquote>
<pre wrap="">posting) but will gladly send
</pre>
<blockquote type="cite">
<blockquote type="cite">
<pre wrap=""> it directly to anybody who would like.
</pre>
</blockquote>
<pre wrap="">
The long initial mark pause is definitely
</pre>
</blockquote>
<pre wrap="">concerning -- Does it show
</pre>
<blockquote type="cite">
<pre wrap=""> up regularly
in the GC logs or is the snippet above an
</pre>
</blockquote>
<pre wrap="">anomaly? Curisously, as
</pre>
<blockquote type="cite">
<pre wrap=""> the process time
shows, the user and system time are both low
</pre>
</blockquote>
<pre wrap="">but the elapsed time is
</pre>
<blockquote type="cite">
<pre wrap=""> very large.
That looks like a total stall of the process,
</pre>
</blockquote>
<pre wrap="">and I have no conjectures
</pre>
<blockquote type="cite">
<pre wrap=""> based on available data.
I suggest talking with your Java support folk
</pre>
</blockquote>
<pre wrap="">if you reproduce this
</pre>
<blockquote type="cite">
<pre wrap=""> after upgrading to
6u28 (or whatever).
best regards.
-- ramki
</pre>
<blockquote type="cite">
<pre wrap="">
Regards,
Srini.
</pre>
</blockquote>
</blockquote>
<pre wrap="">
</pre>
</blockquote>
<pre wrap="">></pre>
<pre wrap="">
<fieldset class="mimeAttachmentHeader"></fieldset>
_______________________________________________
hotspot-gc-use mailing list
<a class="moz-txt-link-abbreviated" href="mailto:hotspot-gc-use@openjdk.java.net">hotspot-gc-use@openjdk.java.net</a>
<a class="moz-txt-link-freetext" href="http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use">http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use</a>
</pre>
</blockquote>
</body>
</html>