Proposal: MaxTenuringThreshold and age field size changes

Wed Jun 4 20:17:03 UTC 2008

Hi,

we are one of the customers Tony was referring to that are "missing" the 
lost age bit. During the last days, I ran some tests in different 
configurations (that's why I reply only now). Tony asked me to post the 
results of my analysis, so here it is! Probably some of it is 
interesting for you...

We use ParNew/CMS as garbage collectors. Our goal is to collect all 
medium-lived objects for most scenarios before they tenure into the old 
gen. Therefore, we've been using a MaxTenuringThreshold (MTT) of 24, 
which proved to be a good trade-off between collecting all medium-lived 
objects in the young heap for most scenarios, while at the other hand 
not copy objects of some other scenarios (that live a little longer, 
i.e. more than 24 cycles) too often.

Unfortunately, with the lost age bit, configuring MTT=24 currently 
activates the NeverTenure policy (which we didn't notice when updating 
to the latest Java 5 releases).

I now ran test for different scenarios (using the latest Java 6 build) 
to figure out how to best configure MTT for our application.

It turned out, that MTT=24 ("never tenure") is a bad idea for us. For 
scenarios with medium-lived objects that survive more than 24 gc cycles, 
this increases the copying overhead significantly, leading to much 
longer young gc times (it's better to tenure them into old after a while 
instead of copying them too often). Also (and this was most surprising 
to me!), even for scenarios where all medium-lived objects die after 3-4 
gc cycles, these objects are sometimes *not* collected, piling up at age 
15. Both lead to the situation that the entire survivor space fills up 
to 100%. Sometimes, 90% of this is garbage, that is still being copied 
at each gc cycle. Also, since the survivor spaces are full, other 
objects that even may be younger than those in the survivor spaces, need
to tenure into old because there's no space for them in the survivor 
spaces. While MTT=24 used to be "good" for JVMs with 5 age bits, now 
with only 4 bits, this setting turned out to be rather bad. It not only 
increased young gc times significantly, but also caused objects to 
prematurely tenure into old (causing unecessary CMS cycles) and also 
increased the CPU usage (due to more gc work and probably also because 
of L1/L2 cache effects when copying more objects than necessary).

With MTT=15, we had much shorter young gc times, and we never filled up 
the survivor spaces. However, MTT=15 is too short for many scenarios: 
Now we have for many scenarios objects tenuring into old which didn't 
tenure before with MTT=24 and 5 age bits.

Doubling the eded size solved this problem. Since the gc frequency this 
way decreased by factor 2, 15 gc cycles now last as many seconds as 
(with half the eden) 30 cycles would have lasted before. Since the gc 
time (almost) only depends on the number of surviving objects and not 
the eden size, the effect from doubling the eden size on the gc times 
was negligible: For scenarios where no objects were tenuring into old 
with the small eden, there was no change in gc times at all (after all, 
the same amount of objects were surviving a gc). For scenarios where 
with the small eden, some objects were tenuring which now, with
the large eden, don't tenure any more, there were more objects surviving 
a gc in the young heap, so the young gc times increased a little. 
However, since we avoid CMS cycles with the larger eden by keeping those 
objects in the young gen, this pays off again.

Generally, young gc times with MTT=24 ("never tenure") are around factor 
2-4 of those with MTT=15, depending on the scenario (130ms vs 25ms, 
180ms vs 80ms). This is mostly because much more objects survive a gc 
cycle (including many dead objects) with MTT=24.

So, as a conclusion: Yes, we are missing the 5th age bit. NeverTenure 
works very bad for us. MTT=15 with our original eden size is too small, 
but increasing the eden size allows us to get similar behavior with 
MTT=15 as with the original configuration (MTT=24 and 5 age bits).

I think, Tony's suggestion to limit the configurable MTT to 2^n-1 (with 
n being the age bits) is a good solution. At least according to my 
tests, this is much better then activating the "never tenure" policy 
when the user is not aware of this.

I hope some of this may have been helpful for you. I will send some more
detailed results and logfiles directly to Tony.

Nick.

Tony Printezis schrieb:
> Ramki,
> 
> Y Srinivas Ramakrishna wrote:
>> In the 64 bit JVM, I would rather vote for reserving the available 
>> bits for
>> possible future use rather than use them for age. 
> I think, in the 64-bit JVM, we have 26 bits available on the mark word. 
> I think we should be able to use at least one of them to expand the age 
> field. In fact, I was going to vote that we use two and expand the age 
> field to 6 bits for a max value of 63.
>> It would seem that 15
>> would almost always be sufficient, and if not there is always 
>> NeverTenure :-)
>> [But I guess your customers are evidence that 15 is not enough in some
>> cases;
> I know of at least two customers who see better GC behavior with a MTT > 
> 15.
>>  where do we draw the line? I guess that we do not really know, and that
>> you are suggesting we give the users as many age bits as we 
>> "reasonably can" for
>> a specific platform/implementation.]
>>
>> On the other hand, if people feel more age bits are needed for the 64-bit
>> case, I would like us to agree that the actual number available in a
>> given release would be bounded below by 4, but would otherwise be
>> free to change from one release to another, along with the semantics
>> for ceiling the MTT as suggested in your note below. [For example: "Hey,
>> where did my age bit go? I was specifying MTT=256 in JDK 19
>> and was happy with the performance, and now the JVM's ceiling the MTT
>> at 128 since JDK 20."]
>>   
> I think this is definitely reasonable. And, in fact, it makes using a 
> 2n-1 ceiling for MTT, as we discussed, even more important. :-)
>> Just out of curiosity, and to do a very rough survey, should one do a 
>> quick
>> poll of hotspot-gc-use to see how many people on the
>> list missed the age-bit that vanished across the 5u5->5u6 transition
>> and wished the bit were back?
>>   
> I can do that.
>> Anyway, ... my $0.02 :-)
>>   
> Tony
>> ----- Original Message -----
>> From: Tony Printezis <tony.printezis at sun.com>
>> Date: Wednesday, May 28, 2008 3:15 pm
>> Subject: Proposal: MaxTenuringThreshold and age field size changes
>> To: "hotspot-gc-dev at openjdk.java.net" <hotspot-gc-dev at openjdk.java.net>
>>
>>
>>  
>>> Hi all,
>>>
>>> We are considering making some changes to the way the 
>>> MaxTenuringThreshold parameter is handled and to the size of the age 
>>> field. We'd appreciate your feedback.
>>>
>>> *Background.* The -XX:MaxTenuringThreshold=n (or MTT for short) 
>>> parameter dictates how many times objects will be copied within the 
>>> young generation (i.e., between the survivor spaces) before they are 
>>> promoted (tenured) to the old generation. Each object has an "age" 
>>> field in its header which is incremented every time an object is 
>>> copied within the young generation. When the age field reaches the 
>>> value of MTT, the
>>> object is promoted to the old generation (I've left out some detail 
>>> here...). The parameter -XX:+NeverTenure tells the GC never to tenure
>>> objects willingly (they will be promoted only when the target survivor
>>> space is full).
>>>
>>> (out of curiosity: does anyone actually use -XX:+NeverTenure?)
>>>
>>> Originally, in the HotSpot JVM, we had 5 bits per object for the age 
>>> field (for a max value of 31, so values of MTT would make sense if 
>>> they were <= 31). A couple of years ago (since 5u6 IIRC), the age 
>>> field "lost" one bit and it now only has 4 (for a max value of 15).
>>>
>>> *The Problem.* We have got reports from customers that, when they use 
>>> a MTT > 15 in the latest JVMs (say, when they use their older settings),
>>> the tenuring policy behaves strangely and piles up objects on age 15 
>>> (i.e., objects reach age 15 and are not then promoted, but are kept at
>>> age 15). This happens because if MTT is set to a value higher than the
>>> max age (which is currently 15, as I said earlier), the policy 
>>> essentially behaves the same as -XX:+NeverTenure. We have got at least
>>> two reports that this has a detrimental effect on performance since 
>>> (a) objects are either retained too long in the young gen, imposing 
>>> copying overhead, or (b) objects are promoted too early, since 
>>> objects much older than them, and which they should have got promoted 
>>> some time ago, filled up the survivor space, forcing younger objects 
>>> to be promoted instead. We also got some reports from customers that 
>>> the decrease in
>>> age field size has actually created performance issues for them (they
>>> get better GC behavior if they keep objects in the young gen longer, 
>>> so that they can reclaim more of them in the young gen and promote 
>>> less).
>>>
>>> *Change Proposal.* We are thinking of applying the following changes:
>>>
>>> - If MTT is set by the user to a value > max age, we will actually set
>>> it to max age. So, a high MTT value will not be equivalent to setting
>>> -XX:+NeverTenure (if users really want they never tenure policy, 
>>> they'd get it with the -XX:+NeverTenure parameter). We believe that 
>>> this is the more natural interpretation of the MTT parameter (and it 
>>> will be less
>>> confusing to users with "old" settings), even though it will behave 
>>> differently to the way it currently behaves.
>>>
>>> - Even though we cannot add an extra bit to the age field in the 
>>> 32-bit JVM (the space in an object's mark word is at a premium in the 
>>> 32-bit
>>> JVM), we think we can increase the age field size, and as a result the
>>> max value of MTT, in the 64-bit JVM (to even higher than 5 bits too). 
>>> We feel this is a worthwhile change, given that 64-bit JVM users will be
>>> able to use MTT values > 15, despite the confusion that it might cause
>>> (the max MTT value will be different in the 32-bit and 64-bit JVMs; 
>>> incidentally, the "compressed oops" JVM will also have the larger age
>>> fields too).
>>>
>>> Let us know what you think,
>>>
>>> Tony, HS GC Group
>>>
>>> -- 
>>> ----------------------------------------------------------------------
>>> | Tony Printezis, Staff Engineer    | Sun Microsystems Inc.          |
>>> |                                   | MS BUR02-311                   |
>>> | e-mail: tony.printezis at sun.com    | 35 Network Drive               |
>>> | office: +1 781 442 0998 (x20998)  | Burlington, MA01803-0902, USA  |
>>> ----------------------------------------------------------------------
>>> e-mail client: Thunderbird (Solaris)
>>>
>>>
>>>     
> 