<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
tt
{mso-style-priority:99;
font-family:"Courier New";}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle21
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 2.0cm 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="DE" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">Thanks for the contribution, and thanks everybody for discussing and reviewing.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">I’ve pushed it.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">Best regards,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">Martin<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span lang="EN-US"> Michihiro Horie [mailto:HORIE@jp.ibm.com]
<br>
<b>Sent:</b> Dienstag, 5. Juni 2018 00:42<br>
<b>To:</b> Kim Barrett <kim.barrett@oracle.com><br>
<b>Cc:</b> Andrew Haley (aph@redhat.com) <aph@redhat.com>; david.holmes@oracle.com; Erik Österlund <erik.osterlund@oracle.com>; hotspot-gc-dev@openjdk.java.net; Doerr, Martin <martin.doerr@sap.com>; ppc-aix-port-dev@openjdk.java.net<br>
<b>Subject:</b> Re: RFR(M): 8154736: enhancement of cmpxchg and copy_to_survivor for ppc64<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p><tt><span style="font-size:10.0pt">>> On Jun 1, 2018, at 11:08 AM, Michihiro Horie <<a href="mailto:HORIE@jp.ibm.com">HORIE@jp.ibm.com</a>> wrote:</span></tt><span style="font-size:10.0pt;font-family:"Courier New""><br>
<tt>>> </tt><br>
<tt>>> Hi Kim, Erik, and Martin,</tt><br>
<tt>>> </tt><br>
<tt>>> Thank you very much for reminding me that an acquire barrier in the else-statement for “!test_mark->is_marked()” is necessary under the criteria of not relying on the consume.
</tt><br>
<tt>>> </tt><br>
<tt>>> I uploaded a new webrev : <a href="http://cr.openjdk.java.net/~mhorie/8154736/webrev.13/">
http://cr.openjdk.java.net/~mhorie/8154736/webrev.13/</a></tt><br>
<tt>>> This change uses forwardee_acquire(), which would generate better code on ARM.
</tt><br>
<tt>>> </tt><br>
<tt>>> Necessary barriers are located in all the paths in copy_to_survivor_space, and the returned new_obj can be safely handled in the caller sites.</tt><br>
<tt>>> </tt><br>
<tt>>> I measured SPECjbb2015 with the latest webrev. Critical-jOPS improved by 5%. Since my previous measurement with implicit consume showed 6% improvement, adding acquire barriers degraded the performance a little, but 5% is still good enough.</tt><br>
<tt>></tt><br>
<tt>>Looks good.</tt></span><br>
<br>
<span style="font-size:10.0pt">Thanks a lot, Kim!</span><br>
<br>
<br>
<span style="font-size:10.0pt">Best regards,</span><br>
<span style="font-size:10.0pt">--</span><br>
<span style="font-size:10.0pt">Michihiro,</span><br>
<span style="font-size:10.0pt">IBM Research - Tokyo</span><br>
<br>
<img border="0" width="16" height="16" style="width:.1666in;height:.1666in" id="_x0000_i1025" src="cid:image001.gif@01D3FCB1.92FB25A0" alt="Inactive hide details for Kim Barrett ---2018/06/05 05:08:48---> On Jun 1, 2018, at 11:08 AM, Michihiro Horie <HORIE@jp.ibm.com"><span style="font-size:10.0pt;color:#424282">Kim
Barrett ---2018/06/05 05:08:48---> On Jun 1, 2018, at 11:08 AM, Michihiro Horie <<a href="mailto:HORIE@jp.ibm.com">HORIE@jp.ibm.com</a>> wrote: ></span><br>
<br>
<span style="font-size:10.0pt;color:#5F5F5F">From: </span><span style="font-size:10.0pt">Kim Barrett <<a href="mailto:kim.barrett@oracle.com">kim.barrett@oracle.com</a>></span><br>
<span style="font-size:10.0pt;color:#5F5F5F">To: </span><span style="font-size:10.0pt">Michihiro Horie <<a href="mailto:HORIE@jp.ibm.com">HORIE@jp.ibm.com</a>></span><br>
<span style="font-size:10.0pt;color:#5F5F5F">Cc: </span><span style="font-size:10.0pt">"Doerr, Martin" <<a href="mailto:martin.doerr@sap.com">martin.doerr@sap.com</a>>, "Andrew Haley (<a href="mailto:aph@redhat.com">aph@redhat.com</a>)" <<a href="mailto:aph@redhat.com">aph@redhat.com</a>>,
"<a href="mailto:david.holmes@oracle.com">david.holmes@oracle.com</a>" <<a href="mailto:david.holmes@oracle.com">david.holmes@oracle.com</a>>, "Erik Österlund" <<a href="mailto:erik.osterlund@oracle.com">erik.osterlund@oracle.com</a>>, "<a href="mailto:hotspot-gc-dev@openjdk.java.net">hotspot-gc-dev@openjdk.java.net</a>"
<<a href="mailto:hotspot-gc-dev@openjdk.java.net">hotspot-gc-dev@openjdk.java.net</a>>, "<a href="mailto:ppc-aix-port-dev@openjdk.java.net">ppc-aix-port-dev@openjdk.java.net</a>" <<a href="mailto:ppc-aix-port-dev@openjdk.java.net">ppc-aix-port-dev@openjdk.java.net</a>></span><br>
<span style="font-size:10.0pt;color:#5F5F5F">Date: </span><span style="font-size:10.0pt">2018/06/05 05:08</span><br>
<span style="font-size:10.0pt;color:#5F5F5F">Subject: </span><span style="font-size:10.0pt">Re: RFR(M): 8154736: enhancement of cmpxchg and copy_to_survivor for ppc64</span><o:p></o:p></p>
<div class="MsoNormal">
<hr size="2" width="100%" noshade="" style="color:#8091A5" align="left">
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
<br>
<br>
<tt><span style="font-size:10.0pt">> On Jun 1, 2018, at 11:08 AM, Michihiro Horie <<a href="mailto:HORIE@jp.ibm.com">HORIE@jp.ibm.com</a>> wrote:</span></tt><span style="font-size:10.0pt;font-family:"Courier New""><br>
<tt>> </tt><br>
<tt>> Hi Kim, Erik, and Martin,</tt><br>
<tt>> </tt><br>
<tt>> Thank you very much for reminding me that an acquire barrier in the else-statement for “!test_mark->is_marked()” is necessary under the criteria of not relying on the consume.
</tt><br>
<tt>> </tt><br>
<tt>> I uploaded a new webrev : <a href="http://cr.openjdk.java.net/~mhorie/8154736/webrev.13/">
http://cr.openjdk.java.net/~mhorie/8154736/webrev.13/</a></tt><br>
<tt>> This change uses forwardee_acquire(), which would generate better code on ARM.
</tt><br>
<tt>> </tt><br>
<tt>> Necessary barriers are located in all the paths in copy_to_survivor_space, and the returned new_obj can be safely handled in the caller sites.</tt><br>
<tt>> </tt><br>
<tt>> I measured SPECjbb2015 with the latest webrev. Critical-jOPS improved by 5%. Since my previous measurement with implicit consume showed 6% improvement, adding acquire barriers degraded the performance a little, but 5% is still good enough.</tt><br>
<br>
<tt>Looks good.</tt><br>
<br>
<tt>> </tt><br>
<tt>> </tt><br>
<tt>> Best regards,</tt><br>
<tt>> --</tt><br>
<tt>> Michihiro,</tt><br>
<tt>> IBM Research - Tokyo</tt><br>
<tt>> </tt><br>
<tt>> "Doerr, Martin" ---2018/05/30 16:18:09---Hi Erik, the current implementation works on PPC because of "MP+sync+addr".</tt><br>
<tt>> </tt><br>
<tt>> From: "Doerr, Martin" <<a href="mailto:martin.doerr@sap.com">martin.doerr@sap.com</a>></tt><br>
<tt>> To: "Erik Österlund" <<a href="mailto:erik.osterlund@oracle.com">erik.osterlund@oracle.com</a>>, Kim Barrett <<a href="mailto:kim.barrett@oracle.com">kim.barrett@oracle.com</a>>, Michihiro Horie <<a href="mailto:HORIE@jp.ibm.com">HORIE@jp.ibm.com</a>>,
"Andrew Haley (<a href="mailto:aph@redhat.com">aph@redhat.com</a>)" <<a href="mailto:aph@redhat.com">aph@redhat.com</a>></tt><br>
<tt>> Cc: "<a href="mailto:david.holmes@oracle.com">david.holmes@oracle.com</a>" <<a href="mailto:david.holmes@oracle.com">david.holmes@oracle.com</a>>, "<a href="mailto:hotspot-gc-dev@openjdk.java.net">hotspot-gc-dev@openjdk.java.net</a>" <<a href="mailto:hotspot-gc-dev@openjdk.java.net">hotspot-gc-dev@openjdk.java.net</a>>,
"<a href="mailto:ppc-aix-port-dev@openjdk.java.net">ppc-aix-port-dev@openjdk.java.net</a>" <<a href="mailto:ppc-aix-port-dev@openjdk.java.net">ppc-aix-port-dev@openjdk.java.net</a>></tt><br>
<tt>> Date: 2018/05/30 16:18</tt><br>
<tt>> Subject: RE: RFR(M): 8154736: enhancement of cmpxchg and copy_to_survivor for ppc64</tt><br>
<tt>> </tt><br>
<tt>> </tt><br>
<tt>> </tt><br>
<tt>> </tt><br>
<tt>> Hi Erik,</tt><br>
<tt>> </tt><br>
<tt>> the current implementation works on PPC because of "MP+sync+addr".</tt><br>
<tt>> So we already rely on ordering of "load volatile field" + "implicit consume" on the reader's side. We have never seen any issues related to this with the compilers we have been using during the ~10 years the PPC implementation exists.</tt><br>
<tt>> </tt><br>
<tt>> PPC supports "MP+lwsync+addr" the same way, so Michihiro's proposal doesn't make it unreliable for PPC.</tt><br>
<tt>> </tt><br>
<tt>> But I'm ok with evaluating acquire barriers although they are not required by the PPC/ARM memory models.</tt><br>
<tt>> ARM/aarch64 will also be affected when the o->forwardee uses load_acquire. So somebody should check the impact. If it is not acceptable we may need to introduce explicit consume.</tt><br>
<tt>> </tt><br>
<tt>> Implicit consume is also bad in shared code because somebody may want to run it on DEC Alpha.</tt><br>
<tt>> </tt><br>
<tt>> Thanks and best regards,</tt><br>
<tt>> Martin</tt><br>
<tt>> </tt><br>
<tt>> </tt><br>
<tt>> -----Original Message-----</tt><br>
<tt>> From: Erik Österlund [<a href="mailto:erik.osterlund@oracle.com">mailto:erik.osterlund@oracle.com</a>]
</tt><br>
<tt>> Sent: Dienstag, 29. Mai 2018 14:01</tt><br>
<tt>> To: Doerr, Martin <<a href="mailto:martin.doerr@sap.com">martin.doerr@sap.com</a>>; Kim Barrett <<a href="mailto:kim.barrett@oracle.com">kim.barrett@oracle.com</a>>; Michihiro Horie <<a href="mailto:HORIE@jp.ibm.com">HORIE@jp.ibm.com</a>></tt><br>
<tt>> Cc: <a href="mailto:david.holmes@oracle.com">david.holmes@oracle.com</a>; Gustavo Bueno Romero <<a href="mailto:gromero@br.ibm.com">gromero@br.ibm.com</a>>;
<a href="mailto:hotspot-dev@openjdk.java.net">hotspot-dev@openjdk.java.net</a>; <a href="mailto:hotspot-gc-dev@openjdk.java.net">
hotspot-gc-dev@openjdk.java.net</a>; <a href="mailto:ppc-aix-port-dev@openjdk.java.net">
ppc-aix-port-dev@openjdk.java.net</a></tt><br>
<tt>> Subject: Re: RFR(M): 8154736: enhancement of cmpxchg and copy_to_survivor for ppc64</tt><br>
<tt>> </tt><br>
<tt>> Hi Martin and Michihiro,</tt><br>
<tt>> </tt><br>
<tt>> On 2018-05-29 12:30, Doerr, Martin wrote:</tt><br>
<tt>> > Hi Kim,</tt><br>
<tt>> ></tt><br>
<tt>> > I'm trying to understand how this is related to Michihiro's change. The else path of the initial test is not affected by it AFAICS.</tt><br>
<tt>> > So it sounds like a request to fix the current implementation in addition to what his original intend was.</tt><br>
<tt>> </tt><br>
<tt>> I think we are just trying to nail down the correct fencing and just go </tt>
<br>
<tt>> for that. And yes, this is arguably a pre-existing problem, but in a </tt><br>
<tt>> race involving the very same accesses that we are changing the fencing </tt>
<br>
<tt>> for. So it is not completely unrelated I suppose.</tt><br>
<tt>> </tt><br>
<tt>> In particular, hotspot has code that assumes that if you on the writer </tt>
<br>
<tt>> side issue a full fence before publishing a pointer to newly initialized </tt>
<br>
<tt>> data, then the initializing stores and their side effects should be </tt><br>
<tt>> globally "visible" across the system before the pointer to it is </tt><br>
<tt>> published, and hence elide the need for acquire on the loading side, </tt><br>
<tt>> without relying on retained data dependencies on the loader side. I </tt><br>
<tt>> believe this code falls under that category. It is assumed that the </tt><br>
<tt>> leading fence of the CAS publishing the forwarding pointer makes the </tt><br>
<tt>> initializing stores globally observable before publishing a pointer to </tt>
<br>
<tt>> the initialized data, hence assuming that any loads able to observe the </tt>
<br>
<tt>> new pointer would not rely on acquire or data dependent loads to </tt><br>
<tt>> correctly read the initialized data.</tt><br>
<tt>> </tt><br>
<tt>> Unfortunately, this is not reliable in the IRIW case, as per the litmus </tt>
<br>
<tt>> test "MP+sync+ctrl" as described in "Understanding POWER </tt><br>
<tt>> multiprocessors" (<a href="https://dl.acm.org/citation.cfm?id=1993520">https://dl.acm.org/citation.cfm?id=1993520</a>), as
</tt><br>
<tt>> opposed to "MP+sync+addr" that gets away with it because of the data </tt><br>
<tt>> dependency (not IRIW). Similarly, an isync does the job too on the </tt><br>
<tt>> reader side as shown in MP+sync+ctrlisync. So while what I believe was </tt>
<br>
<tt>> the previous reasoning that the leading sync of the CAS would elide the </tt>
<br>
<tt>> necessity for acquire on the reader side without relying on data </tt><br>
<tt>> dependent loads (implicit consume), I think that assumption was wrong in </tt>
<br>
<tt>> the first place and that we do indeed need explicit acquire (even with </tt>
<br>
<tt>> the precious conservative CAS fencing) in this context to not rely on </tt>
<br>
<tt>> implicit consume semantics generating the required data dependent loads </tt>
<br>
<tt>> on the reader side. In practice though, the leading sync of the CAS has </tt>
<br>
<tt>> been enough to generate the correct machine code. Now, with the leading </tt>
<br>
<tt>> sync removed, we are increasing the possible holes in the generated </tt><br>
<tt>> machine code due to this flawed reasoning. So it would be nice to do </tt><br>
<tt>> something more sound instead that does not make such assumptions.</tt><br>
<tt>> </tt><br>
<tt>> > Anyway, I agree with that implicit consume is not good. And I think it would be good to treat both o->forwardee() the same way.</tt><br>
<tt>> > What about keeping memory_order_release for the CAS and using acquire for both o->forwardee()?</tt><br>
<tt>> > The case in which the CAS succeeds is safe because the current thread has created new_obj so it doesn't need memory barriers to access it.</tt><br>
<tt>> </tt><br>
<tt>> Sure, that sounds good to me.</tt><br>
<tt>> </tt><br>
<tt>> Thanks,</tt><br>
<tt>> /Erik</tt><br>
<tt>> </tt><br>
<tt>> > Thanks and best regards,</tt><br>
<tt>> > Martin</tt><br>
<tt>> ></tt><br>
<tt>> ></tt><br>
<tt>> > -----Original Message-----</tt><br>
<tt>> > From: Kim Barrett [<a href="mailto:kim.barrett@oracle.com">mailto:kim.barrett@oracle.com</a>]</tt><br>
<tt>> > Sent: Dienstag, 29. Mai 2018 01:54</tt><br>
<tt>> > To: Michihiro Horie <<a href="mailto:HORIE@jp.ibm.com">HORIE@jp.ibm.com</a>></tt><br>
<tt>> > Cc: Erik Osterlund <<a href="mailto:erik.osterlund@oracle.com">erik.osterlund@oracle.com</a>>;
<a href="mailto:david.holmes@oracle.com">david.holmes@oracle.com</a>; Gustavo Bueno Romero <<a href="mailto:gromero@br.ibm.com">gromero@br.ibm.com</a>>;
<a href="mailto:hotspot-dev@openjdk.java.net">hotspot-dev@openjdk.java.net</a>; <a href="mailto:hotspot-gc-dev@openjdk.java.net">
hotspot-gc-dev@openjdk.java.net</a>; <a href="mailto:ppc-aix-port-dev@openjdk.java.net">
ppc-aix-port-dev@openjdk.java.net</a>; Doerr, Martin <<a href="mailto:martin.doerr@sap.com">martin.doerr@sap.com</a>></tt><br>
<tt>> > Subject: Re: RFR(M): 8154736: enhancement of cmpxchg and copy_to_survivor for ppc64</tt><br>
<tt>> ></tt><br>
<tt>> >> On May 28, 2018, at 4:12 AM, Michihiro Horie <<a href="mailto:HORIE@jp.ibm.com">HORIE@jp.ibm.com</a>> wrote:</tt><br>
<tt>> >></tt><br>
<tt>> >> Hi Erik,</tt><br>
<tt>> >></tt><br>
<tt>> >> Thank you very much for your review.</tt><br>
<tt>> >></tt><br>
<tt>> >> I understood that implicit consume should not be used in the shared code. Also, I believe performance degradation would be negligible even if we use acquire.</tt><br>
<tt>> >></tt><br>
<tt>> >> New webrev uses memory_order_acq_rel: <a href="http://cr.openjdk.java.net/~mhorie/8154736/webrev.10">
http://cr.openjdk.java.net/~mhorie/8154736/webrev.10</a></tt><br>
<tt>> > This is missing the acquire barrier on the else branch for the initial test, so fails to meet</tt><br>
<tt>> > the previously described minimal requirements for even possibly being sufficient. Any</tt><br>
<tt>> > analysis of weakening the CAS barriers must consider that test and successor code.</tt><br>
<tt>> ></tt><br>
<tt>> > In the analysis, it’s not just the lexically nearby debugging / logging code that needs to be</tt><br>
<tt>> > considered; the forwardee is being returned to caller(s) that will presumably do something</tt><br>
<tt>> > with that object.</tt><br>
<tt>> ></tt><br>
<tt>> > Since the whole point of this discussion is performance, any proposed change should come</tt><br>
<tt>> > with performance information.</tt><br>
<tt>> ></tt><br>
<br>
<br>
<br>
</span><br>
<br>
<o:p></o:p></p>
</div>
</body>
</html>