<html><head><meta http-equiv="Content-Type" content="text/html charset=windows-1252"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Oh, re-read your letter, yup, there seems to be a problem. Have you observed that in practice?<br><div><div><br></div><div>igor</div><div><div><div><br><div><div>On Jun 26, 2013, at 9:27 PM, Igor Veresov <<a href="mailto:iggy.veresov@gmail.com">iggy.veresov@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><meta http-equiv="Content-Type" content="text/html charset=windows-1252"><base href="x-msg://368/"><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">The cards that are stored in the buffers are not available for concurrent processing right when they are enqueued. Instead they are passed to the processing threads when the buffer fills up. This passing of the buffer involves signaling of a condition (like pthread_cond_signal(), literally) that has a write barrier for sure, which would guarantee that the cards in the buffer, and contents of the card table, and the contents of the object are "in sync".<div><br></div><div>The only place in the generated code where there has to be a store-store barrier (for non-TSO architectures) is between the actual field store and the dirtying of the card.</div><div><br></div><div>Does this make sense?</div><div><br></div><div>igor</div><div><div><br></div><div><br><div><div>On May 23, 2013, at 6:12 AM, "Doerr, Martin" <<a href="mailto:martin.doerr@sap.com">martin.doerr@sap.com</a>> wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div bgcolor="white" lang="DE" link="blue" vlink="purple" style="font-family: Helvetica; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><div class="WordSection1" style="page: WordSection1; "><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="color: rgb(31, 73, 125); ">Hi John,<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="color: rgb(31, 73, 125); "> </span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="color: rgb(31, 73, 125); ">thank you very much for your comments. Your last line explains exactly what we are concerned about.<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="color: rgb(31, 73, 125); ">Does anybody plan to prevent this situation?<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="color: rgb(31, 73, 125); "><o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="color: rgb(31, 73, 125); ">I don’t want to propose adding StoreLoad barriers in all G1 post barriers because I’d expect undesired performance impact.<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="color: rgb(31, 73, 125); ">Would it be feasible to rescan all cards which have been dirtied (at least once) during the next stop-the-world phase?<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="color: rgb(31, 73, 125); ">Maybe anybody has a better idea.<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="color: rgb(31, 73, 125); "> </span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="color: rgb(31, 73, 125); ">Kind regards,<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="color: rgb(31, 73, 125); ">Martin<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US"> </span></div><div><div style="border-style: solid none none; border-top-width: 1pt; border-top-color: rgb(181, 196, 223); padding: 3pt 0cm 0cm; "><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><b><span lang="EN-US" style="font-size: 10pt; font-family: Tahoma, sans-serif; color: windowtext; ">From:</span></b><span lang="EN-US" style="font-size: 10pt; font-family: Tahoma, sans-serif; color: windowtext; "><span class="Apple-converted-space"> </span>John Cuthbertson [mailto:john.cuthbertson@<a href="http://oracle.com/">oracle.com</a>]<span class="Apple-converted-space"> </span><br><b>Sent:</b><span class="Apple-converted-space"> </span>Donnerstag, 23. Mai 2013 02:29<br><b>To:</b><span class="Apple-converted-space"> </span>Doerr, Martin<br><b>Cc:</b><span class="Apple-converted-space"> </span><a href="mailto:hotspot-gc-dev@openjdk.java.net">hotspot-gc-dev@openjdk.java.net</a>; Mikael Gerdin; Braun, Matthias<br><b>Subject:</b><span class="Apple-converted-space"> </span>Re: G1 question: concurrent cleaning of dirty cards<o:p></o:p></span></div></div></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><o:p> </o:p></div><p class="MsoNormal" style="margin: 0cm 0cm 12pt; font-size: 11pt; font-family: Calibri, sans-serif; ">Hi Martin,<br><br>An enqueued card let's the refinment threads know that the oops spanned by that card need to be walked but we're only interested in the latest contents of the fields in those oops. IOW the oop in (3') doesn't need to be the oop stored in (1). If there's a subsequent store (3) to the same location then we want the load at (3') to see the lastest contents. For example suppose we have:<br><br>x.f = a;<br>x.f = b;<br><br>If the application thread sees the card spanning x.f is dirty at the second store then we won't enqueue the card after the second store. As long as the refinement thread sees 'b' when the card is 'refined' then we're OK since we no longer need to add an entry into the RSet for the region containing a - we do need an entry in the RSet for the region containing b.<br><br>If the application thread sees the card as clean at the second store before the refinement thread loads x.f we have just needlessly enqueued the card again.<br><br><b>It is only if the application thread sees the card as dirty but the refinement thread reads 'a' then there could be a problem. We have a missing RSet entry for 'b'.</b><br><br>JohnC<o:p></o:p></p><div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; ">On 5/17/2013 1:29 AM, Doerr, Martin wrote:<o:p></o:p></div></div><blockquote style="margin-top: 5pt; margin-bottom: 5pt; "><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="font-family: 'Courier New'; ">Hi all,</span><o:p></o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="font-family: 'Courier New'; "> </span><o:p></o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="font-family: 'Courier New'; ">we have a question about the interaction between G1 post barriers and the refinement thread's concurrent dirty card cleaning.</span><o:p></o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="font-family: 'Courier New'; ">The case in which the G1 post barrier sees a clean card is obviously not problematic, because it will add an entry in a dirty card queue.</span><o:p></o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="font-family: 'Courier New'; ">However, in case in which the Java thread (mutator thread) sees the card already dirtied, it won’t enqueue the card again. Which is safe as long as its stored oop (1) is seen and processed (3’) by the parallel refinement after having cleaned the card (1’):</span><o:p></o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="font-family: 'Courier New'; "> </span><o:p></o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="font-family: 'Courier New'; ">Java Thread (mutator) Refinement Thread (G1RemSet::concurrentRefineOneCard_impl calls oops_on_card_seq_iterate_careful)</span><o:p></o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="font-family: 'Courier New'; "> </span><o:p></o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="font-family: 'Courier New'; ">(1) store(oop)</span><o:p></o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="font-family: 'Courier New'; ">( StoreLoad required here ?)</span><o:p></o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="font-family: 'Courier New'; ">(2) load(card==dirty)</span><o:p></o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="font-family: 'Courier New'; "> </span><o:p></o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="font-family: 'Courier New'; "> (1’) store(card==clean)</span><o:p></o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="font-family: 'Courier New'; "> (2’) StoreLoad barrier</span><o:p></o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="font-family: 'Courier New'; "> (3’) load(oop)</span><o:p></o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="font-family: 'Courier New'; "> </span><o:p></o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="font-family: 'Courier New'; ">So the refinement thread seems to rely on getting the oop which was written BEFORE the (2) load(card==dirty) was observed.</span><o:p></o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="font-family: 'Courier New'; ">We wonder how this ordering is guaranteed? There are no StoreLoad barriers in the Java Thread's path. (StoreLoad ordering needs explicit barriers even on TSO platforms.)</span><o:p></o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="font-family: 'Courier New'; "> </span><o:p></o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="font-family: 'Courier New'; ">Kind regards,</span><o:p></o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="font-family: 'Courier New'; ">Martin</span><o:p></o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "> <o:p></o:p></div></blockquote><p class="MsoNormal" style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="font-size: 12pt; font-family: 'Times New Roman', serif; "></span></p></div></div></blockquote></div><br></div></div></div></blockquote></div><br></div></div></div></div></body></html>