RFR (M) CR 8050147: StoreLoad barrier interferes with stack usages

[Aleksey Shipilev]

On 07/23/2014 11:38 AM, Aleksey Shipilev wrote:
> On 07/23/2014 01:38 AM, Aleksey Shipilev wrote:
>>  * 1x4x1 Intel Core (Haswell-DT) benefits from offsetting %esp as well.
>> There is an interesting hump on lower backoffs with addl (%esp-8), which
>> seems to be a second-order microarchitectural effect. Unfortunately, we
>> don't have large Haswells available at the moment to dive into this:
>>   http://cr.openjdk.java.net/~shade/8050147/micros/Intel-Haswell.data.png
> 
> Finally managed to get the profile on this Haswell. The hottest loops
> are identical, although the second variant is consistently better.
> 
> This produces 1200.318 +- 18.520 ns/op:
> 
>   2.65%             0x00007fdcb87190b0: mov    (%rsp),%r10
>                     0x00007fdcb87190b4: mov    0xc(%r10),%r10d
>   0.06%             0x00007fdcb87190b8: mov    %r10d,0xc(%rbp)
>   2.66%             0x00007fdcb87190bc: lock addl $0x0,-0x8(%rsp)
>  80.08%   95.59%    0x00007fdcb87190c2: mov    %rbp,%rsi
>                     0x00007fdcb87190c5: xchg   %ax,%ax
>                     0x00007fdcb87190c7: callq  0x00007fdcb85d7fe0
>                     0x00007fdcb87190cc: test   %eax,%eax
>                     0x00007fdcb87190ce: jne    0x00007fdcb87190b0
> 
> This produces 891.427 +- 15.277 ns/op:
> 
>   3.52%    0.02%    0x00007ffa3918f030: mov    (%rsp),%r10
>                     0x00007ffa3918f034: mov    0xc(%r10),%r10d
>   0.07%             0x00007ffa3918f038: mov    %r10d,0xc(%rbp)
>   3.68%             0x00007ffa3918f03c: lock addl $0x0,-0x48(%rsp)
>  76.54%   97.13%    0x00007ffa3918f042: mov    %rbp,%rsi
>                     0x00007ffa3918f045: xchg   %ax,%ax
>                     0x00007ffa3918f047: callq  0x00007ffa39045fe0
>                     0x00007ffa3918f04c: test   %eax,%eax
>                     0x00007ffa3918f04e: jne    0x00007ffa3918f030
> 
> Full logs here:
>  http://cr.openjdk.java.net/~shade/8050147/haswell-barrier2.perfasm
>  http://cr.openjdk.java.net/~shade/8050147/haswell-barrier3.perfasm
> 
> It is puzzling to me why do we have the difference here. In the logs
> there, you may see the second-hottest method, looping(), which is called
> in this busy loop. It does:
> 
>  2.60%             0x00007fdcb871b200: sub    $0x18,%rsp
>                    0x00007fdcb871b207: mov    %rbp,0x10(%rsp)
> 
> ...which seems to callee-save %rbp to (%rsp-8)? Any pointers how we
> manage the stack in callees? It would seem we need to step back more
> than -8 to dodge callee-saves, but how much?
> 
> It seems odd this affects Haswell so much. I've checked on my
> SandyBridge laptop, and we have the same code, but performance is
> consistent. Barring that, it would seem like some the second-order
> microarchitectural effect on Haswell. ...which makes me say this is the
> mode we should switch to:
> 
>> ...or "lock addl (%esp-CL-8), 0), pessimistically padding away from
>> stack users:
>>   http://cr.openjdk.java.net/~shade/8050147/webrev.02/

Ping. Anyone?

-Aleksey.