Alternatives for Unsafe field access

[Nathan Mittler]

Thanks, everyone!

Just to be clear: we'll have to support older versions of generated code so
relying on the generated message to export VarHandles wouldn't really be an
option.  And, we will have to access private fields of another (generated)
class from within our runtime.

I'm a bit concerned with a few potential penalties incurred when using
VarHandles vs a single contiguous buffer:

1) Allocation cost: Allocating a single contiguous buffer would be
relatively cheap compared to allocating an array of VarHandles. This sort
of thing could significantly impact the startup time for applications that
use many message types.

2) Storage size: There would likely be a lot more storage required per
message type, since we would have to maintain other metadata in a buffer,
while also maintaining an array of VarHandles instances. This could blow up
quickly for apps using lots of messages.

3) Runtime performance: I suspect that there will be a performance penalty
incurred when iterating over an array of VarHandles vs a single continuous
buffer due to cache behavior.

Also, do I need to be concerned of setAccessible throwing on certain
platforms? I've never seen it fail ... perhaps it won't be an issue?

Thanks,
Nathan


On Thu, Dec 8, 2016 at 1:13 PM, Uwe Schindler <uschindler at apache.org> wrote:

> Hi,
>
> Of course, private field access is allowed to your own class if you use a
> private lookup object. The setAccessible hack is only needed when
> reflection would also be needed for frameworks or other "superusers".
>
> Uwe
>
> Am 8. Dezember 2016 22:10:32 MEZ schrieb Uwe Schindler <
> uschindler at apache.org>:
> >Hi,
> >
> >You can first do standard reflection, then use setAccessible and
> >finally use the Lookup object to convert the now-accessible Field
> >instance to a MethodHandle or VarHandle. You just have to know that
> >way, existing since Java 7.
> >
> >Uwe
> >
> >Am 8. Dezember 2016 21:49:04 MEZ schrieb Nathan Mittler
> ><nathanmittler at google.com>:
> >>Hi Roger,
> >>If I read that correctly, lookups still have to go through access
> >>checks
> >>which would seem to imply that they can't be used for private field
> >>access.
> >>Or am I misunderstanding?
> >>
> >>On Thu, Dec 8, 2016 at 11:51 AM, Roger Riggs <Roger.Riggs at oracle.com>
> >>wrote:
> >>
> >>> Hi Nathan,
> >>>
> >>> Have you looked at VarHandles?  [1]
> >>> It is possible to use MethodsHandles.Lookup to get a VarHandle to an
> >>> unreflected field.
> >>>
> >>> Roger
> >>>
> >>> [1] http://download.java.net/java/jdk9/docs/api/java/lang/invoke
> >>> /MethodHandles.Lookup.html
> >>>
> >>>
> >>>
> >>> On 12/8/2016 1:03 PM, Nathan Mittler wrote:
> >>>
> >>>> Hi everyone,
> >>>>
> >>>> Apologies in advance if this isn't the correct forum for this
> >>question. My
> >>>> team is working on an experimental runtime for Google's protocol
> >>buffers
> >>>> which is currently relying on sun.misc.Unsafe to perform various
> >>tasks
> >>>> efficiently. I'm aware that the plan is to eventually remove Unsafe
> >>>> altogether, and I want to make sure that we still have a way to
> >move
> >>>> forward with future versions of Java.
> >>>>
> >>>> The code is up on a github branch (
> >>>> https://github.com/google/protobuf/tree/java_experimental).
> >>>>
> >>>> For an example of the sorts of things our runtime needs to do, you
> >>can
> >>>> look
> >>>> at the serialization code (
> >>>> https://github.com/google/protobuf/blob/java_experimental/
> >>>>
> >>java/core/src/main/java/com/google/protobuf/
> AbstractProto3Schema.java#L49
> >>>> ).
> >>>>
> >>>> The idea is that the runtime dynamically determines information
> >>about
> >>>> message fields (e.g. field types, offsets) and stores it into a
> >>single
> >>>> buffer.  Our main need is fast read/write access to the private
> >>fields of
> >>>> our generated message classes. Java reflection would be too slow
> >and
> >>would
> >>>> require data representation as a list of objects, rather than a
> >>continuous
> >>>> buffer (much less compact and cache-friendly). Security
> >restrictions
> >>would
> >>>> be a concern as well. At first glance at Java 9, I don't see any
> >>>> facilities
> >>>> that would help here.
> >>>>
> >>>> This would seem to be a fairly common use case a low-level
> >>serialization
> >>>> framework, such as protobuf. Are there any thoughts on how such a
> >>use case
> >>>> could be supported post-Unsafe?
> >>>>
> >>>> Thanks,
> >>>> Nathan
> >>>>
> >>>
> >>>
> >
> >--
> >Uwe Schindler
> >Achterdiek 19, 28357 Bremen
> >https://www.thetaphi.de
>