[jmm-dev] VarHandle.safepoint() methods

[Gil Tene]

Hit send too soon. The code would look sort of like this (put methods cleaned up to be void):

public class CloseableMappedByteBuffer {

  // Actually unmaps a byte buffer.
  private native void unmap(MappedByteBuffer b);

  private volatile MappedByteBuffer _buf;

  public CloseableMappedByteBuffer wrap(MappedByteBuffer buf) {
      return new CloseableMappedByteBuffer(buf);
  }

  public void unmap() {
      MappedByteBuffer buf = buf();
      vh.setOpaque(null);
      Runtime.checkpoint();
      // Now every thread sees the updated _buf, we can unmap it.
      unmap(buf);
  }

  public void put(byte val) {
     Runtime.runWithNoCheckpointBoundaries( (val) -> _buf.put(val) );
  }

  public void put(int index, byte val) {
     Runtime.runWithNoCheckpointBoundaries( (index, val) -> _buf.put(index, val) );
  }

  public byte get() {
      return Runtime.callWithNoCheckpointBoundaries( () -> _buf.get() );
  }

  public byte get(int index) {
      return Runtime.callWithNoCheckpointBoundaries( () -> _buf.get(index) );
  }

  ...etc, for all the ByteBuffer methods.
}

> On Jan 4, 2017, at 5:03 PM, Gil Tene <gil at azul.com> wrote:
> 
> To avoid confusion with (and ties to) the under-the-hood notion of a safepoint in JVMs, I'd suggest we use another term to describe this. "Checkpoint", "Line in the Sand", "Phase", are useful example terms to play with perhaps.
> 
> Epoch based an RCU based schemes share a lot of what you are looking to do, I think. But as I understand it, you are looking to avoid forcing participating threads into some specific behaviors (e.g. like periodically ticking as would be needing by RCU schemes, or tracking of phase numbers in Epoch schemes), and you are hoping that the semantic behavior chosen could be optimized by JVMs into a "virtually no cost" operation by leveraging the already-paid-for safepoint polling mechanisms that exist in all generated and runtime code. I like the notion of sinking the cost into that stuff...
> 
> As Sanjoy notes, if you are looking to have JITs fold this into the existing safepoint polling stuff, there is an issue with compound operations that may include implicit safepoint/checkpoint polling opportunities (e..g between each and every bytebode executed in the interpreter]. So the operation you would need to use in place of buf().get() is probably something more like "vh.followAsMappedByteBufferAndGet()" and "vh.followAsMappedByteBufferAndSet(byte vaiue)". And that gets messy…
> 
> However, here is a scheme that is abstracted away, but could be made to work IMO:
> 
> Using the term "checkpoint", we would describe this as:
> 
> The runtime is free to insert "checkpoint crossing boundaries" anywhere in code (between any two bytecodes), except where specifically prohibited by contract.
> 
> "Accessors" perform sets operations that [by contract[ cannot be reordered across checkpoint crossing boundaries, but are otherwise schedule-able like any other operation.
> 
> And:
> 
> "Modifiers" performs checkpoint operations that will initiate a global checkpoint transition, and will only return when all threads are assured to no longer be executing code that preceded a checkpoint crossing boundary that executes after the checkpoint operation has been initiated.
> 
> The specific set of APIs that could be used to support this can look like this:
> 
> Runtime.runWithNoCheckpointBoundaries(Runnable r) will run r, guaranteeing that no checkpoint crossing boundaries exist within the execution of r.
> 
> <T> T Runtime.callWithNoCheckpointBoundaries(Callable<T> c) will call c, guaranteeing that no checkpoint crossing boundaries exist within the execution of c.
> 
> Runtime.checkpoint() will return after all threads are assured to no longer be executing code that preceded a checkpoint crossing boundary that executes after the checkpoint() call has been initiated.
> 
> And the runtime is [otherwise] free to place checkpoint crossing boundaries anywhere it wants as long as those fall only *between* any two bytecodes (and not within one).
> 
> The code for CloseableMappedByteBuffer then looks something like this:
> 
> public class CloseableMappedByteBuffer {
> 
>   // Actually unmaps a byte buffer.
>   private native void unmap(MappedByteBuffer b);
> 
>   private volatile MappedByteBuffer _buf;
> 
>   public CloseableMappedByteBuffer wrap(MappedByteBuffer buf) {
>       return new CloseableMappedByteBuffer(buf);
>   }
> 
>   public void unmap() {
>       MappedByteBuffer buf = buf();
>       vh.setOpaque(null);
>       Runtime.checkpoint();
>       // Now every thread sees the updated _buf, we can unmap it.
>       unmap(buf);
>   }
> 
>   public byte put(byte val) {
>       return Runtime.runWithNoCheckpointBoundaries( (val) -> _buf.put(val) );
>   }
> 
>   public byte put(int index, byte val) {
>       return Runtime.runWithNoCheckpointBoundaries( (index, val) -> _buf.put(index, val) );
>   }
> 
>   public byte get() {
>       return Runtime.callWithNoCheckpointBoundaries( () -> _buf.get() );
>   }
> 
>   public byte get(int index) {
>       return Runtime.callWithNoCheckpointBoundaries( () -> _buf.get(index) );
>   }
> 
>   ...etc, for all the ByteBuffer methods.
> }
> 
> The JVM can then implement checkpoint crossing boundaries using safepoints if it wishes. Checkpointing a thread is such a safepoint-based scheme will consist of bringing it to a safepoint that falls outside of code that is prohibited from having checkpoint polling opportunities. Most safepoint polling opportunities will fall in code that can treat them as checkpoint crossing boundaries when triggered. But (e.g. when running interpreted) there can certainly exist safepoint polling opportunities inside code executed under runWithNoCheckpointBoundaries() or callWithNoCheckpointBoundaries(). When executing code within runWithNoCheckpointBoundaries() or callWithNoCheckpointBoundaries(), safepoint polling opportunities, even if triggered, will not consider the checkpoint as "crossed" and the JVM will await for the threads involved to trigger at other safepoint polling opportunities that fall outside of those regions.
> 
> — Gil.
> 
> 
>> On Jan 4, 2017, at 1:04 PM, Doug Lea <dl at cs.oswego.edu> wrote:
>> 
>> On 01/04/2017 01:04 PM, Andrew Haley wrote:
>>> This is a proposal for a new VarHandle method, but it also is an
>>> extension to the Java Memory Model, so I'm starting the discussion
>>> here.
>>> 
>>> My intention is to provide an efficient way to handle the case where a
>>> field has reads which outnumber writes by several orders of magnitude.
>>> Once a field has been successfully modified, no thread may observe a
>>> stale value.  Writes to a field are expected to be very rare, and can
>>> be expensive.
>>> 
>>> It provides a secure and fast way to do something like a volatile
>>> access but without any memory fences (or even any restrictions on
>>> memory reordering) on the reader side.
>>> 
>>> I propose two new VarHandle methods:
>>> 
>>> Object getSafepoint()
>>> 
>>> Returns the value of a variable, with memory semantics of reading as
>>> if the variable was declared non-volatile, except that this load
>>> shall not be reordered with a preceding safepoint.
>>> 
>> 
>> Just to make sure we agree on interpretation before further
>> contemplating this:
>> 
>> The intent is to act as plain get(), except that that upon
>> safepoint exit, it must act as a getAcquire().  And further that it
>> need not be treated as getAcquire after safepoint check that does
>> not result in a safepoint execution?
>> 
>> And the motivation for doing this is to allow compilers to split paths
>> within loops and other cases where caching in a register would be a big
>> win because the values are heavily used and safepoints rarely trigger.
>> 
>> Yes?
>> 
>> -Doug
>> 
>>> static void safepoint()
>>> 
>>> Wait for a safepoint.  When this method returns, every thread in the
>>> JVM shall have executed a safepoint.  Ensures that loads and stores
>>> before the safepoint will not be reordered with loads and stores
>>> after the safepoint.
>>> 
>>> [Note that VarHandle.safepoint() does not specify whether the VM
>>> safepoints immediately or the thread waits for a safepoint which would
>>> occur anyway.]
>>> 
>>> A mechanism like this is needed in order to implement
>>> MappedByteBuffer.unmap() securely and efficiently, but I think this
>>> mechanism is sufficiently useful that it should be exposed as an API.
>>> 
>>> Some background: MappedByteBuffer.unmap() does not exist.
>>> Implementing it in a sufficiently efficient way is an intractable
>>> problem that has been open since 2002.  It is hard to do because it is
>>> essential that no thread can see the underlying memory once a
>>> MappedByteBuffer has been unmapped, because a new MappedByteBuffer may
>>> have been allocated the same address.  In the current MappedByteBuffer
>>> implementation a buffer is unmapped once the GC has determined it is
>>> not reachable, but there can be a very long delay, and in practice
>>> systems run out of native memory before unmapping happens.
>>> 
>>> In order to get around this problem, some Java projects have been
>>> using kludges based on Unsafe to access private fields of
>>> MappedByteBuffer and forcibly unmap the buffer.
>>> 
>>> It is possible to use an indirection wrapper for all accesses to a
>>> hypothetical unmappable MappedByteBuffer, but such an indirection
>>> would need to use some kind of volatile memory read on every access in
>>> order to avoid a race condition where the buffer was closed while
>>> another thread was still accessing it.  ByteBuffers have to be very
>>> fast, and adding a volatile memory access to every MappedByteBuffer
>>> access would render them useless.
>>> 
>>>> From an implementation point of view, getSafepoint() is a plain read
>>> except that a JIT compiler cannot reorder it with a safepoint.
>>> getSafepoint() can be hoisted out of loops and doesn't inhibit
>>> vectorization, so the overhead of getSafepoint() can be made extremely
>>> low, and hopefully almost zero.
>>> 
>>> I realize that this definition is problematic in that "safepoints" are
>>> not defined anywhere in the JMM, and it might be tricky to formalize,
>>> but it's sufficiently useful that I believe it's worth the effort.
>>> 
>>> I also realize that we need much better names for these methods, ones
>>> that do not refer to any HotSpot-specific mechanism.  However, I can't
>>> think of any better names at the moment.
>>> 
>>> [There is a precedent for this mechanism, but it is internal to the
>>> VM: it is very similar to biased locking.  A biased lock does not
>>> require any memory fences in the case where the lock is biased towards
>>> the current thread, but if the lock is biased towards another thread
>>> the VM safepoints and the bias is removed.  After that, every thread
>>> sees the unbiased lock.]
>>> 
>>> Finally, a correct (but inefficient) way to implement these methods
>>> would be to use getVolatile() for getSafepoint() and fullFence() for
>>> safepoint().
>>> 
>>> 
>>> Example: A closeable MappedByteBuffer.
>>> 
>>> public class CloseableMappedByteBuffer {
>>> 
>>>   // Actually unmaps a byte buffer.
>>>   private native void unmap(MappedByteBuffer b);
>>> 
>>>   private volatile MappedByteBuffer _buf;
>>>   private VarHandle vh;
>>> 
>>>   private MappedByteBuffer buf() {
>>>       return (MappedByteBuffer) v.getSafepoint();
>>>   }
>>> 
>>>   public CloseableMappedByteBuffer wrap(MappedByteBuffer buf) {
>>>       return new CloseableMappedByteBuffer(buf);
>>>   }
>>> 
>>>   public void unmap() {
>>>       MappedByteBuffer buf = buf();
>>>       vh.setOpaque(null);
>>>       VarHandle.safepoint();
>>>       // Now every thread sees the updated _buf, we can unmap it.
>>>       unmap(buf);
>>>   }
>>> 
>>>   public byte get() {
>>>       return buf().get();
>>>   }
>>> 
>>>   public byte get(int index) {
>>>       return buf().get(index);
>>>   }
>>> 
>>>   ...etc, for all the ByteBuffer methods.
>>> }
>>> 
>>> Andrew.
>>> 
>>> 
>