<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:DengXian;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"\@DengXian";
panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:12.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:12.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-AU" link="#0563C1" vlink="#954F72" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt">Hi all,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">I’m working on a C2 write barrier. Interesting enough, the barrier code works fine on x86_64, and only fails with RISC-V. I’m wondering whether there’s anything in the RISC-V codegen that might cause the above
behaviour.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">I would really appreciate if people could point out the correct pattern for writing such barrier code, or any hint for further debugging.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Below are more details.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">The barrier in its simplest form transforms each oop store into an oop store and a runtime call. Here is an example for CAS.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New""> virtual Node* atomic_cmpxchg_val_at_resolved(C2AtomicParseAccess& access, Node* expected_val, Node* new_val, const Type* value_type) const {<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New""> Node* result = BarrierSetC2::atomic_cmpxchg_val_at_resolved(access, expected_val, new_val, value_type);<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New""> if (access.is_oop()) object_reference_write_post(access.kit(), access.base(), access.addr().node(), new_val);<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New""> return result;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New""> }<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New"">void MMTkObjectBarrierSetC2::object_reference_write_post(GraphKit* kit, Node* src, Node* slot, Node* val) const {<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New""> if (can_remove_barrier(kit, &kit->gvn(), src, slot, val, /* skip_const_null */ true)) return;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New""> MMTkIdealKit ideal(kit, true);<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New""> const TypeFunc* tf = __ func_type(src->bottom_type(), slot->bottom_type(), val->bottom_type());<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New""> Node* x = __ make_leaf_call(tf, FN_ADDR(MMTkBarrierSetRuntime::object_reference_write_post_call), "mmtk_barrier_call", src, slot, val);<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New""> kit->final_sync(ideal); // Final sync IdealKit and GraphKit.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New"">}<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Currently, I’m having an assertion error in ReduceInst (frame #0).
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"># Internal Error (/home/zixianc/mmtk-riscv/jdk-mmtk/src/hotspot/share/opto/matcher.cpp:1791), pid=1275, tid=1291<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"># assert(C->node_arena()->contains(s->_leaf) || !has_new_node(s->_leaf)) failed: duplicating node that's already been matched<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">I have been debugging in gdb and narrowed down the offending method to java.util.concurrent.ConcurrentHashMap::casTabAt. The s->_leaf in above is the AddP (o48) below.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">In gdb, the call stack of the C2 compiler starts with match_tree(CompareAndSwapP (o58)) (frame #3), which in turn calls ReduceInst (frame #2), ReduceInst_Interior (frame #1), and finally ReduceInst on AddP
(o48) (frame #0).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Dumping AddP (o48) gives the following.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New"">leaf->dump(3)
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New"">o28 ConI === o0 [[ o29 ]] #int:3<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New"">o27 ConvI2L === _ o11 [[ o29 ]] #long:minint..maxint<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New"">o0 Root === o0 o44 o74 [[ o0 o1 o3 o39 o36 o28 o32 o76 0 ]] inner<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New"">o29 LShiftL === _ o27 o28 [[ o49 ]]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New"">o3 Start === o3 o0 [[ o3 o5 o6 o7 o8 o9 o10 o11 o12 o13 ]] #{0:control, 1:abIO, 2:memory, 3:rawptr:BotPTR, 4:return_address, 5:java/util/concurrent/ConcurrentHashMap$Node *[int:>=0]
*, 6:int, 7:java/util/concurrent/ConcurrentHashMap$Node *, 8:java/util/co<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New"">ncurrent/ConcurrentHashMap$Node *}<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New"">o32 ConL === o0 [[ o48 ]] #long:24<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New"">o49 AddP === _ o10 o10 o29 [[ o48 ]]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New"">o10 Parm === o3 [[ o37 o61 o49 o49 o48 ]] Parm0: java/util/concurrent/ConcurrentHashMap$Node *[int:>=0] *<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New"">o48 AddP === _ o10 o49 o32 [[ o58 o61 ]]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New"">leaf->dump(-3)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New"">o48 AddP === _ o10 o49 o32 [[ o58 o61 ]]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New"">o58 CompareAndSwapP === o55 o56 o48 o75 |o45 [[ o59 o69 o74 9 11 34 ]]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New"">o61 CallLeaf === o55 o1 o56 o1 o1 (o10 o48 o13 ) [[ o62 o63 ]] # mmtk_barrier_call void ( java/util/concurrent/ConcurrentHashMap$Node *[int:>=0]:NotNull *, java/util/concurrent/ConcurrentHashMap$Node
*[int:>=0]:NotNull+any *, java/util/concurrent/ConcurrentH<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New"">ashMap$Node * )<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New"">o59 SCMemProj === o58 [[ o64 32 ]] Memory: @BotPTR *+bot, idx=Bot;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New"">o69 MemBarAcquire === o66 o1 o67 o1 o1 o58 [[ o70 o71 10 ]]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New"">o74 Return === o70 o6 o71 o8 o9 returns o58 [[ o0 1 ]]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New""> 9 Ret === o70 o6 o71 o8 o9 o58 [[ ]]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New"">11 membar_acquire === o66 o1 o67 o1 o1 o58 [[ ]] !jvms: ConcurrentHashMap::casTabAt @ bci:17 (line 765)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New"">34 SCMemProj === o58 [[ ]] Memory: @BotPTR *+bot, idx=Bot; !jvms: ConcurrentHashMap::casTabAt @ bci:17 (line 765)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New"">o62 Proj === o61 [[ o65 ]] #0<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New"">o63 Proj === o61 [[ o64 ]] #2 Memory: @rawptr:BotPTR, idx=Raw;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New"">o64 MergeMem === _ o1 o56 o63 o59 [[ o65 13 ]] { N63:rawptr:BotPTR N59:java/lang/Object *[int:>=0]+any * } Memory: @BotPTR *+bot, idx=Bot;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New"">32 MergeMem === _ 0 25 33 o59 [[ ]] { N33:rawptr:BotPTR N59:java/lang/Object *[int:>=0]+any * } Memory: @BotPTR *+bot, idx=Bot; !jvms: ConcurrentHashMap::casTabAt @ bci:17 (line
765)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New"">o70 Proj === o69 [[ o74 9 ]] #0<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New"">o71 Proj === o69 [[ o74 9 ]] #2 Memory: @BotPTR *+bot, idx=Bot;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New"">10 MachProj === o69 [[ ]] #0/unmatched !jvms: ConcurrentHashMap::casTabAt @ bci:17 (line 765)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New"">o0 Root === o0 o44 o74 [[ o0 o1 o3 o39 o36 o28 o32 o76 0 ]] inner<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New""> 1 Root === 1 2 o74 [[ 1 6 ]] inner<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New"">o65 MemBarCPUOrder === o62 o1 o64 o1 o1 [[ o66 o67 12 ]]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">My guess is that the AddP node (o48) has been reduced when generating machine code for CallLeaf (o61), so when visiting CompareAndSwapP (o58), AddP (o48) will be visited again, and thus failing the assertion.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">If I change<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New""> const TypeFunc* tf = __ func_type(src->bottom_type(), slot->bottom_type(), val->bottom_type());<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New""> Node* x = __ make_leaf_call(tf, FN_ADDR(MMTkBarrierSetRuntime::object_reference_write_post_call), "mmtk_barrier_call", src, slot, val);<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">into<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New""> const TypeFunc* tf = __ func_type(src->bottom_type());<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New""> Node* x = __ make_leaf_call(tf, FN_ADDR(MMTkBarrierSetRuntime::object_reference_write_post_call), "mmtk_barrier_call", src);<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">the assertion error will disappear.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Note that the slot is the AddP node (an offset from the src pointer).</span><span style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt">Sincerely,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt">Zixian</span><o:p></o:p></p>
</div>
</body>
</html>