What's the status of / relation between "JEP 169: Value Objects" / "Value Types for Java" / "Object Layout"

Tue Feb 3 21:44:02 UTC 2015

>
> But even if #2 is solved somehow for some subset of instantiations that
> the VM can auto-magically determine can be converted to a "placement-new"
> sort of allocation / construction sequence, there are plenty of final field
> assignment cases where the final ref field refers to an object that is
> already referred to elsewhere (before the assignment). E.g. it is quite
> common for private final ref fields to be initialized from constructor
> parameters (e.g. HashMap.Entry). It is also common to gave final ref fields
> assigned to the return values of method calls. And for those final fields
> to refer to object that are common across multiple referring instances. All
> those cases are non-inlineable by definition.

I'm probably not making myself clear here.  The automagic optimization I'm
talking about right now is purely for the same situations where one would
use ObjectLayout.createXXX instead, except it's determined by the runtime.
Now, I'm handwaving over the "determined by the runtime" aspect, and for
sake of this discussion, considering only the "obvious" candidates (say, an
explicit "new" operator and the field is not set from constructor argument
at all).

> It's not nulls. It's a reference to something that isn't "inlineable".
> Like setting the final field to a reference to an already-allocated-at-assignement-time
> object.

> If the VM can determine ahead of time (ahead of instantiating any
> instances of a containing class) that ALL final field assignments performed
> in the future will be made from in-place instantiations that can be safely
> and auto-magically converted to placement-news, then there will be no
> wasted space. However, for every class where some late-discovered
> (discovered after an instance has been created) situation leads to a loss
> of the optimization, potential memory waste is also there.

> It's true that the memory waste would be contained to the cases where the
> reference is assigned to a non-"placement-new" result, and that in mixed
> situations the pre-allocated space in the containing object will still be
> used. It's just hard to guess how much of each case there is in real code.

Hmm, I wasn't even thinking of doing any dynamic optimizations here --
without giving it too much thought, it would be something determined at
class load time.  A "verification" pass would be done of the class that
would determine whether this class is suitable for this optimization
(again, let's assume very straightforward requirement -- final field set
with explicit "new" call).  This is somewhat analogous to using compressed
oops -- at heap init time, VM checks what the max heap size could be, and
then determines whether it can use compressed oops and what kind -- it
doesn't matter whether the heap actually gets to that size or not, this is
not toggled dynamically.  So, in this case, the only waste I could see is
if nulls are stored -- class layout would contain reserved space for the
embedded object, but it's actually null for most instances of the class,
leading to waste.

I think cache line co-placement makes sense for fields (e.g. an @Together
> thing to mirror the @Contended stuff), but I doubt that it makes much sense
> for objects, mostly due to size (not that many realistic objects will fit
> together in one cache line anyway). A natural "inline in declaration order"
> approach for intrinsic objects is probably just as good as anything else,
> and will require no special annotation.

Right, this does depend on size of the objects and you're probably right
here.  Given that object headers would still be present, 4 objects' worth
of headers is already a cacheline on most 64bit architectures.  But for
those cases where the cpu uses 2 lines as the coherence unit and/or
cacheline sizes grow, it could be useful.

On Tue, Feb 3, 2015 at 4:08 PM, Gil Tene <gil at azulsystems.com> wrote:

>
> > On Feb 3, 2015, at 12:20 PM, Vitaly Davidovich <vitalyd at gmail.com>
> wrote:
> >
> > 1) My assumption is that vast majority of classes will benefit from
> inline storage of their collaborators.  The waste would be in classes that
> have hot and cold members together, but I think that's the minority? But,
> for those cases, it would be beneficial to allow out-of-band layout (i.e.
> today's layout) via annotation or whatever.
>
> I *think* it's a small (and fragile) minority, rather than the vast
> majority. That is, I think that auto-magic (implicit optimization without a
> special factory based instantiation) is limited in potential scope and
> impact: Without special magic (see #2 below), 100% of current final field
> assignments are non-inlineable because they (by definition) assign the
> field to a reference value that refers to an
> already-allocated-at-assignment-time object.
>
> #2 is hard enough to do for trivial case (those that initialize the field
> to refer to an in-place instantiated object that does not escape ahead of
> the assignment, and whose all escaping post-assignemnt references can be
> safely hunted down and converted). And it is much harder for the plentiful
> non-trivial cases (the allocation/instantiation are done "far enough apart"
> such that interleaved operations complexify the analysis).
>
> But even if #2 is solved somehow for some subset of instantiations that
> the VM can auto-magically determine can be converted to a "placement-new"
> sort of allocation / construction sequence, there are plenty of final field
> assignment cases where the final ref field refers to an object that is
> already referred to elsewhere (before the assignment). E.g. it is quite
> common for private final ref fields to be initialized from constructor
> parameters (e.g. HashMap.Entry). It is also common to gave final ref fields
> assigned to the return values of method calls. And for those final fields
> to refer to object that are common across multiple referring instances. All
> those cases are non-inlineable by definition.
>
> > Why do you say this would likely lead to memory waste? If the object is
> allocated one way or another, it's going to take up space.  The only case i
> see being problematic is when null is stored in majority of instances.
>
> It's not nulls. It's a reference to something that isn't "inlineable".
> Like setting the final field to a reference to an
> already-allocated-at-assignement-time object.
>
> If the VM can determine ahead of time (ahead of instantiating any
> instances of a containing class) that ALL final field assignments performed
> in the future will be made from in-place instantiations that can be safely
> and auto-magically converted to placement-news, then there will be no
> wasted space. However, for every class where some late-discovered
> (discovered after an instance has been created) situation leads to a loss
> of the optimization, potential memory waste is also there.
>
> It's true that the memory waste would be contained to the cases where the
> reference is assigned to a non-"placement-new" result, and that in mixed
> situations the pre-allocated space in the containing object will still be
> used. It's just hard to guess how much of each case there is in real code.
>
> > I guess this boils down to whether one thinks it's more likely, by
> default, that objects embedded in others are hot/cold in terms of access; I
> think it's more common for them to be accessed together and so default
> should cater to that.
> >
> > When/if inlined layout is available, next logical thing one may request
> is specifying layout *order* to try and place commonly accessed data on
> same cacheline.  This is less important for streaming cases, but would be
> nice for random walks.
>
> I think cache line co-placement makes sense for fields (e.g. an @Together
> thing to mirror the @Contended stuff), but I doubt that it makes much sense
> for objects, mostly due to size (not that many realistic objects will fit
> together in one cache line anyway). A natural "inline in declaration order"
> approach for intrinsic objects is probably just as good as anything else,
> and will require no special annotation.
>
> >
> > 2) yes, this basically requires a placement-new like thing to be
> implemented in the VM.  No disagreement that it's intrusive.
> >
>
> It's more than intrusive,. It's incompatible with the semantics of all
> current object-instntiating code, making auto-magical optimization that
> coverts current instantiations to placement-new a "very hard" thing.
>
> Unfortunately, both #1 and #2 would need to be addressed for an
> optimization to apply, so it's enough for one to be "too hard" for non of
> it to fly.
>
> > sent from my phone
> >
> > On Feb 3, 2015 2:10 PM, "Gil Tene" <gil at azulsystems.com> wrote:
> >
> >> On Feb 3, 2015, at 9:13 AM, Vitaly Davidovich <vitalyd at gmail.com>
> wrote:
> >>
> >> Gil, not sure if you saw my reply to Volker, but I agree -- I was
> simply asking why request this optimization via explicit syntax and not do
> it automatically in the runtime (with all the same restrictions, caveats,
> fine print, etc).
> >
> > Well, there are a few reasons why this is "very hard":
> >
> > 1. It's "hard" to discern at runtime that an object assigned to a final
> reference field can/should be "inlined" into it's containing object:
> >
> > 1.1 Object-Inlining decisions are not instance-specific, they are
> class-global.
> >
> > 1.1.1 If a single containing instance inlines such an object, all
> instances of the same class must spend the space.
> >
> > 1.1.2 If a single "containing" instance does not inline the object
> referred to by the final reference field, NONE of the instances of the
> containing class can gain from the deference-avoiding (dead reckoning)
> optimization.
> >
> > 1.1.3 If we tried to inline all final reference declared objects without
> a specific declaration of intent, we'd likely end up with a lot  of wasted
> space.
> >
> > 2. Construction semantics: Even if you could auto-discern the need to
> "inline" the objects, you face a hard problem is dealing with how the
> "inlined" object (not the final ref pointing to it) is constructed and
> initialized:
> >
> > 2.1 Any factory-based instantiation of the referred to object is
> by-definition not inline-able (the factory based creation with a specific
> inlining intent and enough provided context, such as constructWithin(),
> being an exception).
> >
> > 2.2 Current non-factory based instantiation options in Java (new,
> reflection, methodHandles) all perform their own internal allocation of
> instance storage, and would not be able to use the "inlined" space without
> some serious surgery. Even if such surgery to all internal instantiation
> forms was done, getting the target instance location to the instantiation
> code is also "very hard", given that instantiation logically occurs before
> the assignment of the resulting reference to the final field, and many
> operations can happen in between. Approaches that attempt to override a
> sequence of operations (e.g. the simplest stuff like new; dup; push arg1;
> push arg2; invoke_special; putfield (into final ref field), or much more
> complicated ones...) such that the intermediate heap reference is never
> exposed and can be replaced with a reference to the already-allocated space
> only work in trivial situations, and tend to fail in all sorts of
> interesting common-case ways (e.g. when perfectly innocent instrumentation
> is involved).
> >
> > Now "very hard" probably does not mean impossible. But it has so much
> open ended stuff that makes it a huge thing to tackle safely. Given the
> fact that much of the context of org.ObjectLayout benefits will never be
> auto-discernible without using explicit statements to describe the expected
> semantics (using a StructuredArray to state optimization-enabling limiting
> semantics, for example), adding explicit intent declaration (as opposed to
> auto-optimization) for intrinsic objects seems natural there.
> >
> >>
> >> On Tue, Feb 3, 2015 at 11:58 AM, Gil Tene <gil at azulsystems.com> wrote:
> >> A couple of point here, specific to org.ObjectLayout (
> http://objectlayout.org):
> >>
> >> Declaration:
> >> The ObjectLayout @Intrinsic (http://objectlayout.org) annotation is
> used for declaring what you refer to as "inline" objects. It is
> specifically not intended to be a layout control directive, but an
> optimization hint. Whether or not a JVM "inlines" the intrinsic object
> within the containing one, and how/where that "inlining" happens becomes a
> JVM-specific implementation concern, and no a semantic one.
> >>
> >> Field initialization:
> >>
> >> Implicit, undeclared choices to "inline" all final referenced fields
> fail very quickly when attempted in practice. E.g. final fields can (and
> often will) be set to refer to pre-existing-at-construction-time objects,
> which are (by definition) impossible to "inline". In addition, there are
> many common uses final reference fields where "inlining" is no possible
> because the actual object size of the referred-to object is not a global
> constant (e.g. it will be set to a construction-time or parameter-based
> choice of subclass).
> >>
> >> We've given Intrinsic Object initialization a lot of thought in
> org.ObjectLayout. The dedicated initialization API is there to assure
> several things, including exact-type (the field's specific declared type)
> choice. It could turn into a less-verbose version in some future JDK if
> language support was added (e.g. to avoid mentioning the
> needed-only-to-conform-with-syntax things like "this", and the field name
> in the constructWithin() call), but I expect the semantics to need to be
> similar even if the syntax was made less verbose.
> >>
> >> -- Gil.
> >>
> >> > On Feb 3, 2015, at 8:40 AM, Vitaly Davidovich <vitalyd at gmail.com>
> wrote:
> >> >
> >> > Hi Volker,
> >> >
> >> > Sorry, I may have been unclear in my question.  As you say,
> ObjectLayout
> >> > requires that you annotate the fields that you'd like inlined and
> then also
> >> > use special API to construct those objects.  I'm wondering whether,
> >> > instead, all private final fields are automatically inlined, and only
> cases
> >> > where you'd like to layout the field out-of-band would require
> annotation.
> >> > This would be controlled via a cmdline flag, as you say, similar to
> perhaps
> >> > how compressed oops are enabled (or not).  Note that I'm talking about
> >> > purely layout of reference types, not value types.
> >> >
> >> > The "concern" with having to explicitly annotate and use dedicated
> APIs to
> >> > opt-in is that adoption will be fairly low, whereas I think most of
> the
> >> > time one would want inlined storage layout.
> >> >
> >> > Thanks
> >> >
> >> >
> >> > On Tue, Feb 3, 2015 at 11:29 AM, Volker Simonis <
> volker.simonis at gmail.com>
> >> > wrote:
> >> >
> >> >> Hi Vitaly,
> >> >>
> >> >> for PackedObjects/ObjectLayout you need to specially annotate the
> >> >> classes and/or fields which you want to allocate "inline". Once
> you've
> >> >> done that you have no choice with the PackedObjects approach.
> >> >> ObjectLayout is a little special here, because it can run with any
> >> >> Java VM in which case it will still use the default reference model.
> >> >> But it can potentially be optimized by some VM's to provide the flat
> >> >> object layout. I expect these optimizations to be controllable by a
> >> >> command line option.
> >> >>
> >> >> With the "Value Types for Java" [1] approach you'll have the
> >> >> possiblitly to express the behavior right in Java like in the
> >> >> following example from [1]:
> >> >>
> >> >> final __ByValue class Point {
> >> >> static Point origin = __MakeValue(0, 0);
> >> >>
> >> >> I think the default will always be "reference semantics" in Java but
> >> >> with various degrees of freedom to optionally choose value semantics.
> >> >>
> >> >> Regards,
> >> >> Volker
> >> >>
> >> >> [1] http://cr.openjdk.java.net/~jrose/values/values-0.html
> >> >>
> >> >> On Mon, Feb 2, 2015 at 9:19 PM, Vitaly Davidovich <vitalyd at gmail.com
> >
> >> >> wrote:
> >> >>> Volker (or anyone else for that matter),
> >> >>>
> >> >>> Just curious -- do you envision "inline" layout of objects as
> something
> >> >> one
> >> >>> would have to opt-in or as the default layout for all objects in a
> heap?
> >> >> It
> >> >>> seems like this should be the default (assuming zero to minimal
> overhead
> >> >> for
> >> >>> loading the references) as I think wanting "out of line"
> allocations is
> >> >> more
> >> >>> rare.
> >> >>>
> >> >>> Thanks
> >> >>>
> >> >>> On Mon, Feb 2, 2015 at 2:06 PM, Volker Simonis <
> volker.simonis at gmail.com
> >> >>>
> >> >>> wrote:
> >> >>>>
> >> >>>> Hi Brian,
> >> >>>>
> >> >>>> thanks a lot for your detailed answer and apologies for the late
> reply
> >> >>>> (I was a little distracted by FOSDEM :)
> >> >>>>
> >> >>>> All your comments have been clear and reasonable and are much
> >> >>>> appreciated. Please find my additional answers inline:
> >> >>>>
> >> >>>> On Thu, Jan 29, 2015 at 6:05 PM, Brian Goetz <
> brian.goetz at oracle.com>
> >> >>>> wrote:
> >> >>>>>> Question: is JEP 169 still under active development or has it
> been
> >> >>>>>> merged into the more general "Value types for Java" proposal
> below?
> >> >>>>>
> >> >>>>>
> >> >>>>> It has been merged into the more general Value Types for Java
> >> >> proposal.
> >> >>>>>
> >> >>>>
> >> >>>> Then maybe this JEP should be closed to avoid further confusion?
> >> >>>>
> >> >>>>>> The "Value types for Java" approach clearly seems to be the most
> >> >>>>>> general but also the most complex proposal.
> >> >>>>>
> >> >>>>>
> >> >>>>> For some meanings of "complex".  It is certainly the most
> intrusive
> >> >> and
> >> >>>>> large; new bytecodes, new type signatures.  But from a user-model
> >> >>>>> perspective, value types are actually fairly simple.
> >> >>>>>
> >> >>>>>> It's out of scope for Java
> >> >>>>>> 9 and still questionable for Java 10 and above. The
> "PackedObject"
> >> >> and
> >> >>>>>> "ObjectLayout" approaches are clearly simpler and more limited in
> >> >>>>>> scope as they only concentrate on better object layout.
> >> >>>>>
> >> >>>>>
> >> >>>>> To your list, I'd add: Project Panama, the sister project to
> Valhalla.
> >> >>>>> Panama focuses on interop with native code and data, including
> layout
> >> >>>>> specification.  A key goal of Packed was to be able to access
> off-heap
> >> >>>>> native data in its native format, rather than marshalling it
> across
> >> >> the
> >> >>>>> JNI
> >> >>>>> boundary.  Panama is focused on this problem as well, but aims to
> >> >> treat
> >> >>>>> it
> >> >>>>> as a separate problem from Java object layout, resulting in what
> we
> >> >>>>> believe
> >> >>>>> to be a cleaner decomposition of the two concerns.
> >> >>>>>
> >> >>>>
> >> >>>> Your right. I somehow missed to look at Panama more deeply because
> I
> >> >>>> always thought it is only about FFI. John Rose nicely explains the
> >> >>>> various parts of Panama in this mail
> >> >>>>
> >> >>
> http://mail.openjdk.java.net/pipermail/panama-dev/2014-October/000042.html
> >> >>>> where he also mentions the intention of Panama to create new
> flatter
> >> >>>> data layouts in the Heap and the relation of Panama to
> PackedObjects
> >> >>>> and ObjectLayout.
> >> >>>>
> >> >>>>> Packed is an interesting mix of memory density (object embedding
> and
> >> >>>>> packed
> >> >>>>> arrays) and native interop.  But mixing the two goals also has
> costs;
> >> >>>>> our
> >> >>>>> approach is to separate them into orthogonal concerns, and we
> think
> >> >> that
> >> >>>>> Valhalla and Panama do just that.  So in many ways, while a larger
> >> >>>>> project,
> >> >>>>> the combination of Valhalla+Panama addresses the problem that
> Packed
> >> >>>>> did, in
> >> >>>>> a cleaner way.
> >> >>>>>
> >> >>>>>> Question: is there a chance to get a some sort of Java-only but
> >> >>>>>> transparently optimizable structure package like "ObjectLayout"
> into
> >> >>>>>> Java early (i.e. Java 9)?
> >> >>>>>
> >> >>>>>
> >> >>>>> It would depend on a lot of things -- including the level of
> readiness
> >> >>>>> of
> >> >>>>> the design and implementation, and the overlap with anticipated
> future
> >> >>>>> features.  We've reviewed some of the early design of
> ObjectLayout and
> >> >>>>> provided feedback to the projects architects; currently, I think
> it's
> >> >> in
> >> >>>>> the
> >> >>>>> "promising exploration" stage, but I think multiple rounds of
> >> >>>>> simplification
> >> >>>>> are needed before it is ready to be considered for "everybody's
> Java."
> >> >>>>> But
> >> >>>>> if the choice is to push something that's not ready into 9, or to
> wait
> >> >>>>> longer -- there's not actually a choice to be made there.
> >> >>>>>
> >> >>>>> I appreciate the desire to "get something you can use now", but we
> >> >> have
> >> >>>>> to
> >> >>>>> be prepared to support whatever we push into Java for the next 20
> >> >> years,
> >> >>>>> and
> >> >>>>> deal with the additional constraints it generates -- which can be
> an
> >> >>>>> enormous cost.  (Even thought the direct cost is mostly borne by
> >> >> Oracle,
> >> >>>>> the
> >> >>>>> indirect cost is borne by everyone, in the form of slower
> progress on
> >> >>>>> everything else.)  So I am very wary of the motivation of "well,
> >> >>>>> something
> >> >>>>> better is coming, but this works now, so can we push it in?"  I'd
> >> >> prefer
> >> >>>>> to
> >> >>>>> focus on answering whether this is right thing for Java for the
> next
> >> >> 20
> >> >>>>> years.
> >> >>>>>
> >> >>>>>> In my eyes this wouldn't contradict with a more general solution
> like
> >> >>>>>> the one proposed in the "Value types for Java" approach while
> still
> >> >>>>>> offering quite significant performance improvements for quite a
> big
> >> >>>>>> range of problems.
> >> >>>>>
> >> >>>>>
> >> >>>>> The goals of the ObjectLayout effort has overlap with, but also
> >> >> differs
> >> >>>>> from, the goals of Valhalla.  And herein is the problem; neither
> >> >>>>> generalizes
> >> >>>>> the other, and I don't think we do the user base a great favor by
> >> >>>>> pursuing
> >> >>>>> two separate neither-coincident-nor-orthogonal approaches.  I
> suspect,
> >> >>>>> though, that after a few rounds of simplification, ObjectLayout
> could
> >> >>>>> morph
> >> >>>>> into something that fit either coincidently or orthogonally with
> the
> >> >>>>> Valhalla work -- which would be great.  But, as you know, our
> >> >> resources
> >> >>>>> are
> >> >>>>> limited, so we (Oracle) can't really afford to invest in both.
> And
> >> >> such
> >> >>>>> simplification takes time -- getting to that "aha" moment when you
> >> >>>>> realize
> >> >>>>> you can simplify something is generally an incompressible process.
> >> >>>>>
> >> >>>>>> Question: what would be the right place to propose something
> like the
> >> >>>>>> "ObjectLayout" library for Java 9/10? Would that fit within the
> >> >>>>>> umbrella of the Valhalla project or would it be done within its
> own
> >> >>>>>> project / under it's own JEP?
> >> >>>>>
> >> >>>>>
> >> >>>>> Suggesting a version number at this point would be putting the
> cart
> >> >>>>> before
> >> >>>>> the horse (you'll note that we've not even proposed a version
> number
> >> >> for
> >> >>>>> Valhalla; the closest we've gotten to that is "after 9".)
> >> >>>>>
> >> >>>>> OpenJDK Projects are a tool for building a community around a
> body of
> >> >>>>> work;
> >> >>>>> JEPs are a project-management tool for defining, scoping, and
> tracking
> >> >>>>> the
> >> >>>>> progress of a feature.  Given where OL is, it would be reasonable
> to
> >> >>>>> start a
> >> >>>>> Project, which would become the nexus of collaboration that could
> >> >>>>> eventually
> >> >>>>> produce a JEP.
> >> >>>>>
> >> >>>>
> >> >>>> That sounds reasonable. I'll speak with the ObjectLayout people to
> >> >>>> hear what they think about starting a new OpenJDK project.
> >> >>>>
> >> >>>> And after all you've said before, I also came to the conclusion
> that
> >> >>>> investigating ways of new in-heap object layouts seem to be better
> of
> >> >>>> in the Panama project. So I'll also ask there what they think about
> >> >>>> it. Maybe ObjectLayout could become part of Panama or maybe we
> could
> >> >>>> just start a new subproject of Panama with the same/similar goals.
> >> >>>>
> >> >>>>> Hope this helps,
> >> >>>>> -Brian
> >> >>>>
> >> >>>> It really did!
> >> >>>>
> >> >>>> Thanks again,
> >> >>>> Volker
> >> >>>> _______________________________________________
> >> >>>> mlvm-dev mailing list
> >> >>>> mlvm-dev at openjdk.java.net
> >> >>>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
> >> >>>
> >> >>>
> >> >>>
> >> >>> _______________________________________________
> >> >>> mlvm-dev mailing list
> >> >>> mlvm-dev at openjdk.java.net
> >> >>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
> >> >>>
> >> >> _______________________________________________
> >> >> mlvm-dev mailing list
> >> >> mlvm-dev at openjdk.java.net
> >> >> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
> >> >>
> >>
> >>
> >
>
>