Suggested fix for JDK-4724038 (Add unmap method to MappedByteBuffer)

[Paul Sandoz]

HI Mike,

This is fundamentally about *integrity* of the runtime. It follows there are security implications, but it’s still fundamentally an integrity issue and guarding an unsafe operation with a Security Manager is unfortunately an insufficient solution.

Paul.

On 7 Sep 2015, at 18:41, Mike Hearn <hearn at vinumeris.com> wrote:

> https://bugs.openjdk.java.net/browse/JDK-4724038
> 
> This bug (really: lack of feature) filed in 2002 can be worked around by
> using internal APIs. However, post jigsaw, that won't be so easy any more.
> I have a bit of code that uses the workaround, so, I'd rather it didn't
> break.
> 
> I don't have an OpenJDK bug tracker account so I'll comment here
> instead. Mark Reinhold said:
> 
> "We'd be thrilled if someone could come up with a workable solution, so
> we'll leave this bug open in the hope that it will attract attention from
> someone more clever than we are."
> 
> The underlying issue is that if you can unmap a file mapping, a racing
> thread could remap something else into the same place, confusing secure
> code that still has access to the MappedByteBuffer object. A simple if()
> check could avoid this, but at the cost of slowing down access.
> 
> I have two suggested fixes:
> 
> 1. Have an unmap method that throws if a SecurityManager is present.
> Non-secure code that owns its own JVM (i.e. nearly all of it) would then be
> able to unmap files and delete them on Windows. Sandboxed code would still
> suffer, but OK, so be it.
> 
> 2. Have an internal subclass or wrapper object around MappedByteBuffer that
> is only used when a SecurityManager is present, which does the check
> against the volatile field. The JVM should optimise out the virtual method
> call or inline the wrapper code, thus ensuring only sandboxed code pays the
> cost.
> 
> In both cases, an unmap method could be added in such a way that the race
> condition Mark describes should not be an issue.
> 
> Thoughts?