RFR (S): JEP-142: Reduce Cache Contention on Specified Fields

[David Holmes]

Hi Kirk,

I hear what you are saying.

But I don't think expecting hotspot to be able to determine this is 
feasible - certainly not in the short to medium term. What you are 
suggesting requires dynamic adjustment to the object layout as hotspot 
determines that it is sub-optimal, and that level of dynamism simply 
isn't part of hotspot's current design. And I don't know whether 
state-of-the-art runtime performance measuring is at a point where you 
would know what data to gather, or how to analyse it, to make a decision 
like this. And even then that decision would be a local one.

So we either do something or do nothing. This proposed API, like a 
number of advanced concurrency API's is certainly targeted at a very 
select group of expert developers (I don't even count myself in that 
group in this case). And even those experts are going to be hampered by 
local decisions (eg in a JDK core library) impacting global 
(application) performance.

But without this we have to resort to manually padding fields to try and 
get the desired affects. And that makes numerous assumptions regarding 
normal field layout etc, that are VM dependent.

So which is worse:
a) providing an advanced API that the average programmer won't know how 
to use and could easily misuse (if silly enough to mess with things they 
don't understand); or
b) having library/application writers introduce Java-level field padding 
that makes various assumptions about the underlying VM

?

I think (b) is worse.

Cheers,
David


On 26/11/2012 7:22 AM, Kirk Pepperdine wrote:
> Hi David,
>
> I'm completely in agreement that there are some applications where this optimization is necessary. That said, I've more fundamental concerns with respect to this API. Traditionally the JVM has steered away from having developers hint at optimization as these optimizations have typically been ill thought out. My mantra has always been, measure, don't guess. WIthout denigrating any group of developers I would say that very very few know how to get the measures to even start to understand how to apply this "optimization". I would suggest that it would be better to leave the application of this optimization to HotSpot and the JIT compiler. I would also suggest that one could do this is the JVM were to use more of the diagnostics (MSRs) available to it to know when and how to apply this particular optimization. If this all sounds expensive... well, turning on CPU event counters is cheap but evaluating the data maybe a cycle sync... but then again maybe not. But more to the poin!
 t, getti
ng this type of profiling and optimization into HotSpot seems like a far better solution than to give developers access to how variables get laid out.
>
> Sorry Aleksey, I know you get it.. but you're not in the same class of developer as those that would be using this feature.
>
> Regards,
> Kirk
>
> On 2012-11-25, at 9:40 PM, David Holmes<david.holmes at oracle.com>  wrote:
>
>> Hi Aleksey,
>>
>> This seems good for local decisions about placement/layout, but I'm concerned about the global impact on footprint and performance. What if individually these kinds of changes yield better microbenchmark performance, but collectively they yield worse application performance? At present this is an all-or-nothing API - though after some thought I'm not sure it could be anything else.
>>
>> David
>>
>> On 23/11/2012 7:33 AM, Aleksey Shipilev wrote:
>>> Hi,
>>>
>>> After some internal discussions with Doug Lea, Dave Dice and others, I
>>> would like to solicit the initial feedback on the implementation of
>>> JEP-142, aka @Contended [1]:
>>>    http://openjdk.java.net/jeps/142
>>>
>>> The webrev for the initial version is here:
>>>    http://shipilev.net/pub/jdk/hotspot/contended/webrev-2/
>>>
>>> Implementation overview. Hotspot code is currently laying out the fields
>>> to optimize the memory footprint, rearranging fields freely to both
>>> satisfy alignment requirements for fields and making the less gaps
>>> possible. We leverage the same infrastructure to exempt specific fields
>>> from the packing, and pushing them outside the dense packed block at
>>> sparse offsets, naturally making up the appropriate padding.
>>>
>>> In order to demarcate the specific classes and/or fields eligible for
>>> such the padding, we use new @Contended annotation. Runtime discovery of
>>> annotations reuses the code John (?) laid out for some of
>>> JSR292-specific annotations.
>>>
>>> The behavior of this annotation is as follows:
>>>
>>> A. Marking the class as contended:
>>>
>>>      @Contended
>>>      public static class ContendedTest2 {
>>>          private Object plainField1;
>>>          private Object plainField2;
>>>          private Object plainField3;
>>>          private Object plainField4;
>>>      }
>>>
>>> ...makes the entire field block to be padded from the both sides:
>>> (below is the output of new tracing -XX:+PrintFieldLayout)
>>>
>>>    TestContended$ContendedTest2: field layout
>>>      Entire class is marked contended
>>>       @140 --- instance fields start ---
>>>       @140 "plainField1" Ljava.lang.Object;
>>>       @144 "plainField2" Ljava.lang.Object;
>>>       @148 "plainField3" Ljava.lang.Object;
>>>       @152 "plainField4" Ljava.lang.Object;
>>>       @288 --- instance fields end ---
>>>       @288 --- instance ends ---
>>>
>>> Note that we use 128 bytes, twice the cache line size on most hardware
>>> to adjust for adjacent sector prefetchers extending the false sharing
>>> collisions to two cache lines.
>>>
>>> B. Marking the field as contended:
>>>
>>>      public static class ContendedTest1 {
>>>          @Contended
>>>          private Object contendedField1;
>>>          private Object plainField1;
>>>          private Object plainField2;
>>>          private Object plainField3;
>>>          private Object plainField4;
>>>      }
>>>
>>> ...pushes the field out of dense block and effectively applies padding:
>>>
>>>     TestContended$ContendedTest1: field layout
>>>       @ 12 --- instance fields start ---
>>>       @ 12 "plainField1" Ljava.lang.Object;
>>>       @ 16 "plainField2" Ljava.lang.Object;
>>>       @ 20 "plainField3" Ljava.lang.Object;
>>>       @ 24 "plainField4" Ljava.lang.Object;
>>>       @156 "contendedField1" Ljava.lang.Object; (contended, group = 0)
>>>       @288 --- instance fields end ---
>>>       @288 --- instance ends ---
>>>
>>> C. Marking multiple fields makes each field padded:
>>>
>>>      public static class ContendedTest4 {
>>>          @Contended
>>>          private Object contendedField1;
>>>
>>>          @Contended
>>>          private Object contendedField2;
>>>
>>>          private Object plainField3;
>>>          private Object plainField4;
>>>      }
>>>
>>> ...pushes both fields with individual padding for each:
>>>
>>>     TestContended$ContendedTest4: field layout
>>>       @ 12 --- instance fields start ---
>>>       @ 12 "plainField3" Ljava.lang.Object;
>>>       @ 16 "plainField4" Ljava.lang.Object;
>>>       @148 "contendedField1" Ljava.lang.Object; (contended, group = 0)
>>>       @280 "contendedField2" Ljava.lang.Object; (contended, group = 0)
>>>       @416 --- instance fields end ---
>>>       @416 --- instance ends ---
>>>
>>> *** IV. Contention groups
>>>
>>> There are cases where you want to separate the *group* of fields that
>>> are experiencing contention with everything else but not pairwise. This
>>> is the usual thing for some of the code updating two fields at once.
>>> While marking both with @Contended would be sufficient, we can optimize
>>> the memory footprint by not applying padding between them. In order to
>>> demarcate these groups, we have the parameter in the annotation
>>> describing the equivalence class for contention group.
>>>
>>> So that:
>>>
>>>      public static class ContendedTest5 {
>>>          @Contended("updater1")
>>>          private Object contendedField1;
>>>
>>>          @Contended("updater1")
>>>          private Object contendedField2;
>>>
>>>          @Contended("updater2")
>>>          private Object contendedField3;
>>>
>>>          private Object plainField5;
>>>          private Object plainField6;
>>>      }
>>>
>>> ...is laid out as:
>>>
>>>     TestContended$ContendedTest5: field layout
>>>       @ 12 --- instance fields start ---
>>>       @ 12 "plainField5" Ljava.lang.Object;
>>>       @ 16 "plainField6" Ljava.lang.Object;
>>>       @148 "contendedField1" Ljava.lang.Object; (contended, group = 12)
>>>       @152 "contendedField2" Ljava.lang.Object; (contended, group = 12)
>>>       @284 "contendedField3" Ljava.lang.Object; (contended, group = 15)
>>>       @416 --- instance fields end ---
>>>       @416 --- instance ends ---
>>>
>>> Note $contendedField1 and $contendedField2 are padded from everything
>>> else, but still densely packed with each other.
>>>
>>> The code is known to work at least on Linux x86-64, tested with a few
>>> microtests. The layout of fields without @Contended is not affected, so
>>> this is presumably a safe change. I will try to run more tests against
>>> this implementation with JPRT, but will appreciate the design, API, and
>>> draft implementation review meanwhile...
>>>
>>> Thanks,
>>> Aleksey.
>