<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
Hi Andrew and Aleksey,
<div class=""><br class="">
</div>
<div class="">It is of course true that a hypothetical OS + hardware combination could /in theory/ be smart enough to not send the TLB purge message to certain CPUs and hence not flush latent stores there. But in practice none does that which I know of and
certainly none that we support. As I said in the original email where I proposed the solution, I already had a look at our architectures (and a few more) in the linux kernel and XNU/BSD - and it’s safe everywhere. And as I said earlier, the closest match I
found to out-smart the barrier is itanium that broadcasts the TLB purge with a special instruction rather than IPI: ptc.ga. It takes an address range and purges the corresponding TLB entries. However, according to Intel’s own documentation, even such fancy
solution still flushes all latent memory accesses on remote CPUs regardless.</div>
<div class=""><br class="">
</div>
<div class="">I don’t know what windows does because it’s open source but we only have x86 there and its hardware has no support for doing it any other way than with IPI messages which is all we need. And if we feel that scared, windows has a system call that
does exactly what we want and with the architecture I propose it’s trivial to specialize the barrier for windows do use this instead.</div>
<div class=""><br class="">
</div>
<div class="">If there was to suddenly pop up a magical fancy OS + hardware solution that is too clever for this optimization (seems unlikely to me) then there are other ways of issuing such a global fence. But I don’t see the point in doing that now when there
is no such problem in sight.</div>
<div class=""><br class="">
</div>
<div class="">Thanks,</div>
<div class="">/Erik</div>
<div class=""><br class="">
<div>
<blockquote type="cite" class="">
<div class="">On 12 May 2015, at 21:19, Andrew Haley <<a href="mailto:aph@redhat.com" class="">aph@redhat.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">On
05/12/2015 08:25 PM, Aleksey Shipilev wrote:</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<blockquote type="cite" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
On 12.05.2015 22:03, Andrew Haley wrote:<br class="">
<blockquote type="cite" class="">On 05/12/2015 07:54 PM, Aleksey Shipilev wrote:<br class="">
<blockquote type="cite" class="">On 12.05.2015 20:23, Erik Österlund wrote:<br class="">
<blockquote type="cite" class="">Out of curiosity I patched the thing and my fix can be found here:
<a href="http://cr.openjdk.java.net/~eosterlund/8079315/webrev.v1/" class="">http://cr.openjdk.java.net/~eosterlund/8079315/webrev.v1/</a> <<a href="http://cr.openjdk.java.net/%7Eeosterlund/8079315/webrev.v1/" class="">http://cr.openjdk.java.net/%7Eeosterlund/8079315/webrev.v1/</a>><br class="">
</blockquote>
<br class="">
Wait, how does it work? I presumed you need to poll the serialization page (and then handle the possible trap) in mutator, between storing the reference and reading the card mark. Just mprotect-ing a page does not smell like a serializing event, if you don't
actually access the page.<br class="">
</blockquote>
<br class="">
I think it is, because the kernel has to interrupt every CPU in that<br class="">
process to flush its write buffer and TLB. I can't think of any other<br class="">
way of making munmap() work.<br class="">
</blockquote>
<br class="">
If you have a platform with a software-filled TLB, can't you accurately<br class="">
track what CPUs should perform the TLB shootdowns? Or if there is some<br class="">
other way for a CPU to communicate the TLB contents back to OS.<br class="">
</blockquote>
<br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">There
isn't AFAIK: the TLB is always a CPU-local structure.</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<blockquote type="cite" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
If that is in place, then mprotect can only affect the cores that<br class="">
actually accessed the serialization page. I feel that relying on the<br class="">
premise that page mapping changes are globally synchronized is<br class="">
dangerous.<br class="">
</blockquote>
<br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">You're
probably right about that. I suppose it would be possible</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">in
theory for there to be a broadcast TLB invalidate event as part</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">of
the cache coherency protocol.</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">I
wonder if the kernel people would be receptive to the idea of a</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">"execute
a memory fence on every CPU" system call. It would not be</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">hard
to do and it would be very useful.</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">Andrew.</span></div>
</blockquote>
</div>
<br class="">
</div>
</body>
</html>