From twhitmore.nz at gmail.com  Sun Feb  1 06:12:10 2015
From: twhitmore.nz at gmail.com (Thomas W)
Date: Sun, 1 Feb 2015 19:12:10 +1300
Subject: Idea how to implement VT/VO compatibility in JVM
Message-ID: <CAE6vqCk2UAZ0xJ-yshR4pNVzTbfSUZy=QhOQGuNZGz8K0qWO8g@mail.gmail.com>

Hi Peter,

> There could also be an abbreviation:
> void m3(Collection<any ?> c) { ... }
>
> ...equivalent to m2 above, but used
> only if you don't need T in the
> method

Yes, it would be extremely desirable to be able to generically deal with
collections -- specialized or not -- including iterating them via 'Object'.

I proposed previously that we provide reftype-compatible bridge methods,
and have specialized collections "assignment compatible" to Collection<?>
via this boxed API.

I don't see either why 'remove()' -- or, from a compatible other
collection, 'add(T item)' should not be possible.

T obviously cannot be concretely implemented, but we already have the
concept of erasure -- T essentially erases to a boxed object.

We should also be able to use equals(), hashCode() and toString() on an
'any T'.

Regards,
Thomas
On 29 Jan 2015 19:55, <valhalla-dev-request at openjdk.java.net> wrote:

> Send valhalla-dev mailing list submissions to
>         valhalla-dev at openjdk.java.net
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         http://mail.openjdk.java.net/mailman/listinfo/valhalla-dev
> or, via email, send a message with subject or body 'help' to
>         valhalla-dev-request at openjdk.java.net
>
> You can reach the person managing the list at
>         valhalla-dev-owner at openjdk.java.net
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of valhalla-dev digest..."
>
>
> Today's Topics:
>
>    1. What's the status of / relation between "JEP 169: Value
>       Objects" /        "Value Types for Java" / "Object Layout" (Volker
> Simonis)
>    2. Re: Idea how to implement VT/VO compatibility in JVM
>       (Peter Levart)
>    3. Re: Idea how to implement VT/VO compatibility in JVM
>       (Vitaly Davidovich)
>    4. Re: Idea how to implement VT/VO compatibility in JVM
>       (Peter Levart)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Wed, 28 Jan 2015 21:40:49 +0100
> From: Volker Simonis <volker.simonis at gmail.com>
> To: Da Vinci Machine Project <mlvm-dev at openjdk.java.net>,
>         valhalla-dev at openjdk.java.net
> Subject: What's the status of / relation between "JEP 169: Value
>         Objects" /      "Value Types for Java" / "Object Layout"
> Message-ID:
>         <
> CA+3eh122g3Ax+-PCL3EvX3V-cQkorva2j_c42_i4uRLOqw42Fg at mail.gmail.com>
> Content-Type: text/plain; charset=UTF-8
>
> Hi everybody,
>
> I've recently did some research on Java "value objects" / "value
> types" / "object layout" (I'll be actually giving a short talk on the
> topic at FOSDEM[0] this weekend). I just want to quickly summarize my
> current findings here and gently ask for feedback in case you think
> I've totally misunderstood something. Of course any comments and
> additional information is highly welcome as well.
>
> 1. JEP 169: Value Objects [1]
>  - Created by John Rose in 2012 (last update in Sep. 2014)
>  - Still in "Draft" state
>  - Proposes a new "lockPermanently()" operator which marks objects as
> immutable
>  - Seems to be only a little "helper functionality" to simplify
> automatic boxing/unboxing and escape analysis
>  - Referenced the mlwm mailing list and repository but the mlwm repo
> seems dead since about 15 month now
>
> Question: is JEP 169 still under active development or has it been
> merged into the more general "Value types for Java" proposal below?
>
> 2. "Value types for Java" / "State of the Values" [2]
>  - By J. Rose, B. Goetz and Guy Stele
>  - Based on earlier ideas from "Value Types in the VM" [3]
>  - Newest and most elaborate proposal
>  - Proposes general (i.e. function arguments, return values,
> variables, arrays), "immutable" value types
>  - Requires fundamental changes to the VM as well as to the Java language
>  - Related to the "State of the Specialization" proposal [4] about
> support for generics over primitive and value types by B Goetz.
>  - Discussed and developed in the OpenJDK "Valhalla" [5] project
>  - Still very early stage (i.e. no "code" available yet)
>
> 3. PackedObjects as provided by the IBM J9 [6,7]
>  - Flattens the memory layout of "@Packed" object fields and array
>  - Removes object headers of and references to "@Packed" objects
>  - Object headers can be generated on the fly (kind of "auto-boxing")
>  - Currently the most complete and mature solution
>  - Not Java-compatible (e.g. can not write to a nested "@Packed"
> fields). Must be enabled as an experimental extension.
>
> 4. ObjectLayout [8]
>  - A pure Java, layout-optimized data structure package
>  - Designed similar to "@ValueSafe"/"ValueType" in [3] and "Value-base
> classes" in Java 8 [9]
>  - Designed such that it can be tranparently optimized within the VM
>  - VM can transparently layout "@Intrinsic" objects within other objects
>  - All objects are still complete Java object with valid header
>  - The Java part of the library is mature, first native
> VM-optimizations on the way [10]
>
> The "Value types for Java" approach clearly seems to be the most
> general but also the most complex proposal. It's out of scope for Java
> 9 and still questionable for Java 10 and above. The "PackedObject" and
> "ObjectLayout" approaches are clearly simpler and more limited in
> scope as they only concentrate on better object layout. However the
> "ObjectLayout" proposal demonstrates that this is still possible
> within the current Java specification while the "PackedObjects"
> proposal demonstrated that an optimizing implementation is feasible.
> I've recently built a prototype which intrinsifies/optimizes some
> parts of the "ObjectLayout" proposal in the HotSpot [10].
>
> Question: is there a chance to get a some sort of Java-only but
> transparently optimizable structure package like "ObjectLayout" into
> Java early (i.e. Java 9)?
>
> In my eyes this wouldn't contradict with a more general solution like
> the one proposed in the "Value types for Java" approach while still
> offering quite significant performance improvements for quite a big
> range of problems. And if carefully designed, it could be easily
> retrofitted to use the new, general "Value Types" once they are
> available.
>
> Question: what would be the right place to propose something like the
> "ObjectLayout" library for Java 9/10? Would that fit within the
> umbrella of the Valhalla project or would it be done within its own
> project / under it's own JEP?
>
> Thanks for your patience,
> Volker
>
>
> [0] https://fosdem.org/2015/schedule/event/packed_objects/
> [1] http://openjdk.java.net/jeps/169
> [2] http://cr.openjdk.java.net/~jrose/values/values-0.html
> [3] https://blogs.oracle.com/jrose/entry/value_types_in_the_vm
> [4] http://cr.openjdk.java.net/~briangoetz/valhalla/specialization.html
> [5] http://openjdk.java.net/projects/valhalla
> [6]
> http://www.slideshare.net/rsciampacone/javaone-2013-introduction-to-packedobjects?related=1
> [7] http://medianetwork.oracle.com/video/player/2623645005001
> [8] http://objectlayout.org
> [9]
> http://docs.oracle.com/javase/8/docs/api/java/lang/doc-files/ValueBased.html
> [10]
> https://github.com/simonis/ObjectLayout/tree/hotspot_intrinsification/hotspot
>
>
> ------------------------------
>
> Message: 2
> Date: Wed, 28 Jan 2015 22:56:09 +0100
> From: Peter Levart <peter.levart at gmail.com>
> To: Maurizio Cimadamore <maurizio.cimadamore at oracle.com>,       Vitaly
>         Davidovich <vitalyd at gmail.com>, St?phane ?pardaud <
> stef at epardaud.fr>
> Cc: valhalla-dev <valhalla-dev at openjdk.java.net>
> Subject: Re: Idea how to implement VT/VO compatibility in JVM
> Message-ID: <54C95AF9.3090004 at gmail.com>
> Content-Type: text/plain; charset=utf-8; format=flowed
>
>
> On 01/22/2015 02:26 PM, Maurizio Cimadamore wrote:
> >
> > On 22/01/15 13:19, Vitaly Davidovich wrote:
> >> Can you expand a bit on the part where you say frameworks can't
> >> iterate a
> >> Collection<any T> without knowing the instantiation? Do you mean
> >> existing
> >> methods that take Collection<?> won't work without change or
> >> something else?
> > In the current prototype, such methods will work, but they will work
> > with all reference-parameterizations - example:
> >
> > void m(Collection<?> c) { ... }
> >
> > List<String> ls = ...
> > List<Object> lo = ...
> > List<int> li = ...
> > List<MyPointValue> lmpv = ...
> >
> > m(ls); //ok
> > m(lo); //ok
> > m(li); //fail, List<int> is not subtype of Collection<?>
> > m(lmpv); //fail, List<MyPointValue> is not subtype of Collection<?>
> >
> > In other words, Collection<?> means Collection<? extends Object>.
> > Therefore, primitives and values are outside the domain supported by
> > the unbounded wildcard.
> >
> > To fix that, you need to change 'm' as follows:
> >
> > <any T> void m2(Collection<T> c) { ... }
> >
> > m(ls); //ok
> > m(lo); //ok
> > m(li); //ok
> > m(lmpv); //ok
> >
> > Is this what you were asking?
> >
> > Maurizio
>
> There could also be an abbreviation:
>
> void m3(Collection<any ?> c) { ... }
>
> ...which would be equivalent to m2 above, but used only if you don't
> need T in the method (you only invoke methods on 'c' that are not
> mentioning T from Collection<any T> in their signature, such as size(),
> clear(), ...). Maybe you could also call methods, that mention T in
> return type, but you could not assign the result to anything (or maybe
> to Object in certain situations), for example:
>
> class Foo<any T> {
>
>      T getOne() { ... }
>
>      Optional<T> getMaybe() { ... }
>
> }
>
>
> void m4(Foo<any ?> anyFoo) {
>
>      anyFoo.getOne(); // can call, but not use the result
>
>      Object r = anyFoo.getMaybe(); // can assign to Object since
> Optional<?>, Optional<int>, etc. are all Objects
>
> }
>
> (I'm still considering Optional as reference type here)
>
> Peter
>
>
> >
> >>
> >> sent from my phone
> >> On Jan 22, 2015 8:12 AM, "St?phane ?pardaud" <stef at epardaud.fr> wrote:
> >>
> >>> On 01/22/2015 12:18 PM, Maurizio Cimadamore wrote:
> >>>
> >>>> I don't think there's an 'hole' as such in your proposal - but
> >>>> there are
> >>>> 'unknowns'.
> >>>>
> >>> Well, that's already good news. I prefer unknowns to known holes ;)
> >>>
> >>>   Performance-wise, I think the 'risk' is that if we keep the API
> >>> the way
> >>>> they are today (i.e. removeAll(Collection<?>) and friends), the
> >>>> boxing path
> >>>> will pretty much be the norm. Now, in a perfect world, as value
> >>>> types are
> >>>> non-polymorphic (or their polymorphism is very restricted) and
> >>>> immutable,
> >>>> that would suggest that VM should have enough of an hint to perform
> >>>> boxing
> >>>> elimination and such, so that the cost you end up paying is
> >>>> negligible. How
> >>>> much of this is reality? As you, I'm not a VM guru - but I think
> >>>> it's a
> >>>> question worth asking. It seems a likely scenario that the JVM will do
> >>>> great in most cases, and have some bad performance cliffs in others
> >>>> - is
> >>>> that something we are willing to sign up for?
> >>>>
> >>> OK, that's fair. By allowing people to use value types boxed we
> >>> allow them
> >>> to shoot themselves in the foot perf-wise (assuming the VM can't help),
> >>> because we make value types much more accessible. This is mostly due to
> >>> `any T` being opt-in rather than the new default for `T` where `T
> >>> extends
> >>> Object` is not specified. If we also changed `T` to mean `any T`
> >>> (source
> >>> only, so for newly compiled code) then all generics code would be
> >>> specializable by default and that means that users of generic code will
> >>> always be able to use the specialised code as long as they can
> >>> instantiate
> >>> the generic type argument at compile-time (as is already the
> >>> limitation).
> >>> If they can't, well they already have to use `Object` and boxing and
> >>> can't
> >>> traverse collections because they're not `List<Object>` (ATM).
> >>>
> >>> Making it the default has down sides, probably in larger class
> >>> files, but
> >>> as long as it's not the default, there will always be
> >>> incompatibilities in
> >>> libraries that will forget to opt-in, which will mean that either (ATM)
> >>> they can't be used with containers of value types, or that the
> >>> containers
> >>> have to be wrapped to box the value types (unless we fix that as I
> >>> suggest). IMO that's already going to cause compatibility issues and
> >>> there
> >>> may arise a "coding guideline" (remember them from C++ sore points?)
> >>> that
> >>> people will be strongly encourage to use `val T` everywhere just in
> >>> case
> >>> someone wants to use an unboxed value type.
> >>>
> >>> The rift between primitives and Object is already a famous sore
> >>> point in
> >>> Java (not criticising, this was a choice made a long time ago, for
> >>> valid
> >>> reasons, we just have to deal with it) which has been "fixed" in most
> >>> non-Java JVM-languages due to popular demand. I don't think the new
> >>> default
> >>> with value types should be that generics don't accept value types.
> >>>
> >>>   On the language-side unknowns, how much code out there is relying on
> >>>> being able to access fields on Foo<?> or raw Foo types? This is
> >>>> perhaps not
> >>>> common, but I think we need to gather data points on this i.e. by
> >>>> looking
> >>>> at existing open source projects (help welcome here!). Other
> >>>> possible weak
> >>>> points are that this doesn't necessarily address all the issue w.r.t.
> >>>> language uniformity - i.e. how is an ArrayList<int> supposed to
> >>>> answer to a
> >>>> question of the kind 'is instance of List<?>' ?
> >>>>
> >>> Fields that don't involve the `any T` are fine. Fields that do may
> >>> have to
> >>> revert to autoboxing _if_ we feel we _must_ accomodate that to
> >>> autobox not
> >>> just value types but their containers.
> >>>
> >>> `ArrayList<int>` is special, mostly due to the fact that I'm not
> >>> sure we
> >>> can retrofit `Integer` to be a value type, so existing primitives
> >>> may not
> >>> get the `List<int> === List<Integer>` that I suggest for value types
> >>> (though I'd be very interested in making this work).
> >>>
> >>> If we take an easier example assuming a value type named `Date`,
> >>> then yes
> >>> I expect that:
> >>>
> >>> - `List<val Date> instanceof List<?>` == `true`
> >>> - `List<__Boxed Date> instanceof List<?>` == `true`
> >>>
> >>>   I'm not denying there's something there worth exploring (as I have in
> >>>> fact already said), but it seems to me that, while you can go a
> >>>> long way
> >>>> with bridges, there are still questions that bridges alone simply
> >>>> do not
> >>>> have an answer for.
> >>>>
> >>> Thanks, and yes I agree there are still questions remaining, and more
> >>> importantly an implementation lacking, but hopefully if the proposal
> >>> sounds
> >>> decent I can help with the proto.
> >>>
> >>> Thanks for your answers BTW :)
> >>>
> >
>
>
>
> ------------------------------
>
> Message: 3
> Date: Wed, 28 Jan 2015 22:58:35 -0500
> From: Vitaly Davidovich <vitalyd at gmail.com>
> To: Peter Levart <peter.levart at gmail.com>
> Cc: valhalla-dev at openjdk.java.net
> Subject: Re: Idea how to implement VT/VO compatibility in JVM
> Message-ID:
>         <
> CAHjP37GND6b+LyC9uvK5fjfDRqmnbHi-d3mrGN-HUdgFckf1Yg at mail.gmail.com>
> Content-Type: text/plain; charset=UTF-8
>
> Sorry Peter, I'm having a hard time understanding what this buys us - could
> you explain what problem this solves exactly? It's certainly not for
> backcompat from the description, so unclear.
>
> sent from my phone
> On Jan 28, 2015 4:56 PM, "Peter Levart" <peter.levart at gmail.com> wrote:
>
> >
> > On 01/22/2015 02:26 PM, Maurizio Cimadamore wrote:
> >
> >
> > On 22/01/15 13:19, Vitaly Davidovich wrote:
> >
> > Can you expand a bit on the part where you say frameworks can't iterate a
> > Collection<any T> without knowing the instantiation? Do you mean existing
> > methods that take Collection<?> won't work without change or something
> > else?
> >
> > In the current prototype, such methods will work, but they will work with
> > all reference-parameterizations - example:
> >
> > void m(Collection<?> c) { ... }
> >
> > List<String> ls = ...
> > List<Object> lo = ...
> > List<int> li = ...
> > List<MyPointValue> lmpv = ...
> >
> > m(ls); //ok
> > m(lo); //ok
> > m(li); //fail, List<int> is not subtype of Collection<?>
> > m(lmpv); //fail, List<MyPointValue> is not subtype of Collection<?>
> >
> > In other words, Collection<?> means Collection<? extends Object>.
> > Therefore, primitives and values are outside the domain supported by the
> > unbounded wildcard.
> >
> > To fix that, you need to change 'm' as follows:
> >
> > <any T> void m2(Collection<T> c) { ... }
> >
> > m(ls); //ok
> > m(lo); //ok
> > m(li); //ok
> > m(lmpv); //ok
> >
> > Is this what you were asking?
> >
> > Maurizio
> >
> >
> > There could also be an abbreviation:
> >
> > void m3(Collection<any ?> c) { ... }
> >
> > ...which would be equivalent to m2 above, but used only if you don't need
> > T in the method (you only invoke methods on 'c' that are not mentioning T
> > from Collection<any T> in their signature, such as size(), clear(), ...).
> > Maybe you could also call methods, that mention T in return type, but you
> > could not assign the result to anything (or maybe to Object in certain
> > situations), for example:
> >
> > class Foo<any T> {
> >
> >     T getOne() { ... }
> >
> >     Optional<T> getMaybe() { ... }
> >
> > }
> >
> >
> > void m4(Foo<any ?> anyFoo) {
> >
> >     anyFoo.getOne(); // can call, but not use the result
> >
> >     Object r = anyFoo.getMaybe(); // can assign to Object since
> > Optional<?>, Optional<int>, etc. are all Objects
> >
> > }
> >
> > (I'm still considering Optional as reference type here)
> >
> > Peter
> >
> >
> >
> >
> > sent from my phone
> > On Jan 22, 2015 8:12 AM, "St?phane ?pardaud" <stef at epardaud.fr>
> > <stef at epardaud.fr> wrote:
> >
> > On 01/22/2015 12:18 PM, Maurizio Cimadamore wrote:
> >
> > I don't think there's an 'hole' as such in your proposal - but there are
> > 'unknowns'.
> >
> >  Well, that's already good news. I prefer unknowns to known holes ;)
> >
> >   Performance-wise, I think the 'risk' is that if we keep the API the way
> >
> > they are today (i.e. removeAll(Collection<?>) and friends), the boxing
> > path
> > will pretty much be the norm. Now, in a perfect world, as value types are
> > non-polymorphic (or their polymorphism is very restricted) and immutable,
> > that would suggest that VM should have enough of an hint to perform
> boxing
> > elimination and such, so that the cost you end up paying is negligible.
> > How
> > much of this is reality? As you, I'm not a VM guru - but I think it's a
> > question worth asking. It seems a likely scenario that the JVM will do
> > great in most cases, and have some bad performance cliffs in others - is
> > that something we are willing to sign up for?
> >
> >  OK, that's fair. By allowing people to use value types boxed we allow
> > them
> > to shoot themselves in the foot perf-wise (assuming the VM can't help),
> > because we make value types much more accessible. This is mostly due to
> > `any T` being opt-in rather than the new default for `T` where `T extends
> > Object` is not specified. If we also changed `T` to mean `any T` (source
> > only, so for newly compiled code) then all generics code would be
> > specializable by default and that means that users of generic code will
> > always be able to use the specialised code as long as they can
> instantiate
> > the generic type argument at compile-time (as is already the limitation).
> > If they can't, well they already have to use `Object` and boxing and
> can't
> > traverse collections because they're not `List<Object>` (ATM).
> >
> > Making it the default has down sides, probably in larger class files, but
> > as long as it's not the default, there will always be incompatibilities
> in
> > libraries that will forget to opt-in, which will mean that either (ATM)
> > they can't be used with containers of value types, or that the containers
> > have to be wrapped to box the value types (unless we fix that as I
> > suggest). IMO that's already going to cause compatibility issues and
> there
> > may arise a "coding guideline" (remember them from C++ sore points?) that
> > people will be strongly encourage to use `val T` everywhere just in case
> > someone wants to use an unboxed value type.
> >
> > The rift between primitives and Object is already a famous sore point in
> > Java (not criticising, this was a choice made a long time ago, for valid
> > reasons, we just have to deal with it) which has been "fixed" in most
> > non-Java JVM-languages due to popular demand. I don't think the new
> > default
> > with value types should be that generics don't accept value types.
> >
> >   On the language-side unknowns, how much code out there is relying on
> >
> > being able to access fields on Foo<?> or raw Foo types? This is perhaps
> > not
> > common, but I think we need to gather data points on this i.e. by looking
> > at existing open source projects (help welcome here!). Other possible
> weak
> > points are that this doesn't necessarily address all the issue w.r.t.
> > language uniformity - i.e. how is an ArrayList<int> supposed to answer to
> > a
> > question of the kind 'is instance of List<?>' ?
> >
> >  Fields that don't involve the `any T` are fine. Fields that do may have
> > to
> > revert to autoboxing _if_ we feel we _must_ accomodate that to autobox
> not
> > just value types but their containers.
> >
> > `ArrayList<int>` is special, mostly due to the fact that I'm not sure we
> > can retrofit `Integer` to be a value type, so existing primitives may not
> > get the `List<int> === List<Integer>` that I suggest for value types
> > (though I'd be very interested in making this work).
> >
> > If we take an easier example assuming a value type named `Date`, then yes
> > I expect that:
> >
> > - `List<val Date> instanceof List<?>` == `true`
> > - `List<__Boxed Date> instanceof List<?>` == `true`
> >
> >   I'm not denying there's something there worth exploring (as I have in
> >
> > fact already said), but it seems to me that, while you can go a long way
> > with bridges, there are still questions that bridges alone simply do not
> > have an answer for.
> >
> >  Thanks, and yes I agree there are still questions remaining, and more
> > importantly an implementation lacking, but hopefully if the proposal
> > sounds
> > decent I can help with the proto.
> >
> > Thanks for your answers BTW :)
> >
> >
> >
> >
>
>
> ------------------------------
>
> Message: 4
> Date: Thu, 29 Jan 2015 07:55:16 +0100
> From: Peter Levart <peter.levart at gmail.com>
> To: Vitaly Davidovich <vitalyd at gmail.com>
> Cc: valhalla-dev at openjdk.java.net
> Subject: Re: Idea how to implement VT/VO compatibility in JVM
> Message-ID: <54C9D954.9080209 at gmail.com>
> Content-Type: text/plain; charset=utf-8; format=flowed
>
> On 01/29/2015 04:58 AM, Vitaly Davidovich wrote:
> > Sorry Peter, I'm having a hard time understanding what this buys us -
> could
> > you explain what problem this solves exactly? It's certainly not for
> > backcompat from the description, so unclear.
>
> I'm just contemplating about the usefulness of a construct.
>
> Equivalent to relation of:
>
> <T> void m(Collection<T> c) { ... }
>
> vs.
>
> void m(Collection<?> c) { ... }
>
>
> There could also be:
>
> <any T> void m(Collection<T> c) { ... }
>
> vs.
>
> void m(Collection<any ?> c) { ... }
>
>
> Although the later (any ?) is less useful than the former (?), there is
> still some use to be found in the anyfied one. And the language features
> should be orthogonal if at all possible.
>
>
> Regards, Peter
>
> >
> > sent from my phone
> > On Jan 28, 2015 4:56 PM, "Peter Levart" <peter.levart at gmail.com> wrote:
> >
> >> On 01/22/2015 02:26 PM, Maurizio Cimadamore wrote:
> >>
> >>
> >> On 22/01/15 13:19, Vitaly Davidovich wrote:
> >>
> >> Can you expand a bit on the part where you say frameworks can't iterate
> a
> >> Collection<any T> without knowing the instantiation? Do you mean
> existing
> >> methods that take Collection<?> won't work without change or something
> >> else?
> >>
> >> In the current prototype, such methods will work, but they will work
> with
> >> all reference-parameterizations - example:
> >>
> >> void m(Collection<?> c) { ... }
> >>
> >> List<String> ls = ...
> >> List<Object> lo = ...
> >> List<int> li = ...
> >> List<MyPointValue> lmpv = ...
> >>
> >> m(ls); //ok
> >> m(lo); //ok
> >> m(li); //fail, List<int> is not subtype of Collection<?>
> >> m(lmpv); //fail, List<MyPointValue> is not subtype of Collection<?>
> >>
> >> In other words, Collection<?> means Collection<? extends Object>.
> >> Therefore, primitives and values are outside the domain supported by the
> >> unbounded wildcard.
> >>
> >> To fix that, you need to change 'm' as follows:
> >>
> >> <any T> void m2(Collection<T> c) { ... }
> >>
> >> m(ls); //ok
> >> m(lo); //ok
> >> m(li); //ok
> >> m(lmpv); //ok
> >>
> >> Is this what you were asking?
> >>
> >> Maurizio
> >>
> >>
> >> There could also be an abbreviation:
> >>
> >> void m3(Collection<any ?> c) { ... }
> >>
> >> ...which would be equivalent to m2 above, but used only if you don't
> need
> >> T in the method (you only invoke methods on 'c' that are not mentioning
> T
> >> from Collection<any T> in their signature, such as size(), clear(),
> ...).
> >> Maybe you could also call methods, that mention T in return type, but
> you
> >> could not assign the result to anything (or maybe to Object in certain
> >> situations), for example:
> >>
> >> class Foo<any T> {
> >>
> >>      T getOne() { ... }
> >>
> >>      Optional<T> getMaybe() { ... }
> >>
> >> }
> >>
> >>
> >> void m4(Foo<any ?> anyFoo) {
> >>
> >>      anyFoo.getOne(); // can call, but not use the result
> >>
> >>      Object r = anyFoo.getMaybe(); // can assign to Object since
> >> Optional<?>, Optional<int>, etc. are all Objects
> >>
> >> }
> >>
> >> (I'm still considering Optional as reference type here)
> >>
> >> Peter
> >>
> >>
> >>
> >>
> >> sent from my phone
> >> On Jan 22, 2015 8:12 AM, "St?phane ?pardaud" <stef at epardaud.fr>
> >> <stef at epardaud.fr> wrote:
> >>
> >> On 01/22/2015 12:18 PM, Maurizio Cimadamore wrote:
> >>
> >> I don't think there's an 'hole' as such in your proposal - but there are
> >> 'unknowns'.
> >>
> >>   Well, that's already good news. I prefer unknowns to known holes ;)
> >>
> >>    Performance-wise, I think the 'risk' is that if we keep the API the
> way
> >>
> >> they are today (i.e. removeAll(Collection<?>) and friends), the boxing
> >> path
> >> will pretty much be the norm. Now, in a perfect world, as value types
> are
> >> non-polymorphic (or their polymorphism is very restricted) and
> immutable,
> >> that would suggest that VM should have enough of an hint to perform
> boxing
> >> elimination and such, so that the cost you end up paying is negligible.
> >> How
> >> much of this is reality? As you, I'm not a VM guru - but I think it's a
> >> question worth asking. It seems a likely scenario that the JVM will do
> >> great in most cases, and have some bad performance cliffs in others - is
> >> that something we are willing to sign up for?
> >>
> >>   OK, that's fair. By allowing people to use value types boxed we allow
> >> them
> >> to shoot themselves in the foot perf-wise (assuming the VM can't help),
> >> because we make value types much more accessible. This is mostly due to
> >> `any T` being opt-in rather than the new default for `T` where `T
> extends
> >> Object` is not specified. If we also changed `T` to mean `any T` (source
> >> only, so for newly compiled code) then all generics code would be
> >> specializable by default and that means that users of generic code will
> >> always be able to use the specialised code as long as they can
> instantiate
> >> the generic type argument at compile-time (as is already the
> limitation).
> >> If they can't, well they already have to use `Object` and boxing and
> can't
> >> traverse collections because they're not `List<Object>` (ATM).
> >>
> >> Making it the default has down sides, probably in larger class files,
> but
> >> as long as it's not the default, there will always be incompatibilities
> in
> >> libraries that will forget to opt-in, which will mean that either (ATM)
> >> they can't be used with containers of value types, or that the
> containers
> >> have to be wrapped to box the value types (unless we fix that as I
> >> suggest). IMO that's already going to cause compatibility issues and
> there
> >> may arise a "coding guideline" (remember them from C++ sore points?)
> that
> >> people will be strongly encourage to use `val T` everywhere just in case
> >> someone wants to use an unboxed value type.
> >>
> >> The rift between primitives and Object is already a famous sore point in
> >> Java (not criticising, this was a choice made a long time ago, for valid
> >> reasons, we just have to deal with it) which has been "fixed" in most
> >> non-Java JVM-languages due to popular demand. I don't think the new
> >> default
> >> with value types should be that generics don't accept value types.
> >>
> >>    On the language-side unknowns, how much code out there is relying on
> >>
> >> being able to access fields on Foo<?> or raw Foo types? This is perhaps
> >> not
> >> common, but I think we need to gather data points on this i.e. by
> looking
> >> at existing open source projects (help welcome here!). Other possible
> weak
> >> points are that this doesn't necessarily address all the issue w.r.t.
> >> language uniformity - i.e. how is an ArrayList<int> supposed to answer
> to
> >> a
> >> question of the kind 'is instance of List<?>' ?
> >>
> >>   Fields that don't involve the `any T` are fine. Fields that do may
> have
> >> to
> >> revert to autoboxing _if_ we feel we _must_ accomodate that to autobox
> not
> >> just value types but their containers.
> >>
> >> `ArrayList<int>` is special, mostly due to the fact that I'm not sure we
> >> can retrofit `Integer` to be a value type, so existing primitives may
> not
> >> get the `List<int> === List<Integer>` that I suggest for value types
> >> (though I'd be very interested in making this work).
> >>
> >> If we take an easier example assuming a value type named `Date`, then
> yes
> >> I expect that:
> >>
> >> - `List<val Date> instanceof List<?>` == `true`
> >> - `List<__Boxed Date> instanceof List<?>` == `true`
> >>
> >>    I'm not denying there's something there worth exploring (as I have in
> >>
> >> fact already said), but it seems to me that, while you can go a long way
> >> with bridges, there are still questions that bridges alone simply do not
> >> have an answer for.
> >>
> >>   Thanks, and yes I agree there are still questions remaining, and more
> >> importantly an implementation lacking, but hopefully if the proposal
> >> sounds
> >> decent I can help with the proto.
> >>
> >> Thanks for your answers BTW :)
> >>
> >>
> >>
> >>
>
>
>
> End of valhalla-dev Digest, Vol 7, Issue 84
> *******************************************
>

From volker.simonis at gmail.com  Mon Feb  2 19:06:27 2015
From: volker.simonis at gmail.com (Volker Simonis)
Date: Mon, 2 Feb 2015 20:06:27 +0100
Subject: What's the status of / relation between "JEP 169: Value Objects"
	/ "Value Types for Java" / "Object Layout"
In-Reply-To: <54CA6853.2060601@oracle.com>
References: <CA+3eh122g3Ax+-PCL3EvX3V-cQkorva2j_c42_i4uRLOqw42Fg@mail.gmail.com>
	<54CA6853.2060601@oracle.com>
Message-ID: <CA+3eh13pg6+ZziNuzLNdPSc8AkA4o1N9Xx0kGYFNm7TV3t9yzQ@mail.gmail.com>

Hi Brian,

thanks a lot for your detailed answer and apologies for the late reply
(I was a little distracted by FOSDEM :)

All your comments have been clear and reasonable and are much
appreciated. Please find my additional answers inline:

On Thu, Jan 29, 2015 at 6:05 PM, Brian Goetz <brian.goetz at oracle.com> wrote:
>> Question: is JEP 169 still under active development or has it been
>> merged into the more general "Value types for Java" proposal below?
>
>
> It has been merged into the more general Value Types for Java proposal.
>

Then maybe this JEP should be closed to avoid further confusion?

>> The "Value types for Java" approach clearly seems to be the most
>> general but also the most complex proposal.
>
>
> For some meanings of "complex".  It is certainly the most intrusive and
> large; new bytecodes, new type signatures.  But from a user-model
> perspective, value types are actually fairly simple.
>
>> It's out of scope for Java
>> 9 and still questionable for Java 10 and above. The "PackedObject" and
>> "ObjectLayout" approaches are clearly simpler and more limited in
>> scope as they only concentrate on better object layout.
>
>
> To your list, I'd add: Project Panama, the sister project to Valhalla.
> Panama focuses on interop with native code and data, including layout
> specification.  A key goal of Packed was to be able to access off-heap
> native data in its native format, rather than marshalling it across the JNI
> boundary.  Panama is focused on this problem as well, but aims to treat it
> as a separate problem from Java object layout, resulting in what we believe
> to be a cleaner decomposition of the two concerns.
>

Your right. I somehow missed to look at Panama more deeply because I
always thought it is only about FFI. John Rose nicely explains the
various parts of Panama in this mail
http://mail.openjdk.java.net/pipermail/panama-dev/2014-October/000042.html
where he also mentions the intention of Panama to create new flatter
data layouts in the Heap and the relation of Panama to PackedObjects
and ObjectLayout.

> Packed is an interesting mix of memory density (object embedding and packed
> arrays) and native interop.  But mixing the two goals also has costs; our
> approach is to separate them into orthogonal concerns, and we think that
> Valhalla and Panama do just that.  So in many ways, while a larger project,
> the combination of Valhalla+Panama addresses the problem that Packed did, in
> a cleaner way.
>
>> Question: is there a chance to get a some sort of Java-only but
>> transparently optimizable structure package like "ObjectLayout" into
>> Java early (i.e. Java 9)?
>
>
> It would depend on a lot of things -- including the level of readiness of
> the design and implementation, and the overlap with anticipated future
> features.  We've reviewed some of the early design of ObjectLayout and
> provided feedback to the projects architects; currently, I think it's in the
> "promising exploration" stage, but I think multiple rounds of simplification
> are needed before it is ready to be considered for "everybody's Java."  But
> if the choice is to push something that's not ready into 9, or to wait
> longer -- there's not actually a choice to be made there.
>
> I appreciate the desire to "get something you can use now", but we have to
> be prepared to support whatever we push into Java for the next 20 years, and
> deal with the additional constraints it generates -- which can be an
> enormous cost.  (Even thought the direct cost is mostly borne by Oracle, the
> indirect cost is borne by everyone, in the form of slower progress on
> everything else.)  So I am very wary of the motivation of "well, something
> better is coming, but this works now, so can we push it in?"  I'd prefer to
> focus on answering whether this is right thing for Java for the next 20
> years.
>
>> In my eyes this wouldn't contradict with a more general solution like
>> the one proposed in the "Value types for Java" approach while still
>> offering quite significant performance improvements for quite a big
>> range of problems.
>
>
> The goals of the ObjectLayout effort has overlap with, but also differs
> from, the goals of Valhalla.  And herein is the problem; neither generalizes
> the other, and I don't think we do the user base a great favor by pursuing
> two separate neither-coincident-nor-orthogonal approaches.  I suspect,
> though, that after a few rounds of simplification, ObjectLayout could morph
> into something that fit either coincidently or orthogonally with the
> Valhalla work -- which would be great.  But, as you know, our resources are
> limited, so we (Oracle) can't really afford to invest in both.  And such
> simplification takes time -- getting to that "aha" moment when you realize
> you can simplify something is generally an incompressible process.
>
>> Question: what would be the right place to propose something like the
>> "ObjectLayout" library for Java 9/10? Would that fit within the
>> umbrella of the Valhalla project or would it be done within its own
>> project / under it's own JEP?
>
>
> Suggesting a version number at this point would be putting the cart before
> the horse (you'll note that we've not even proposed a version number for
> Valhalla; the closest we've gotten to that is "after 9".)
>
> OpenJDK Projects are a tool for building a community around a body of work;
> JEPs are a project-management tool for defining, scoping, and tracking the
> progress of a feature.  Given where OL is, it would be reasonable to start a
> Project, which would become the nexus of collaboration that could eventually
> produce a JEP.
>

That sounds reasonable. I'll speak with the ObjectLayout people to
hear what they think about starting a new OpenJDK project.

And after all you've said before, I also came to the conclusion that
investigating ways of new in-heap object layouts seem to be better of
in the Panama project. So I'll also ask there what they think about
it. Maybe ObjectLayout could become part of Panama or maybe we could
just start a new subproject of Panama with the same/similar goals.

> Hope this helps,
> -Brian

It really did!

Thanks again,
Volker

From vitalyd at gmail.com  Mon Feb  2 20:19:09 2015
From: vitalyd at gmail.com (Vitaly Davidovich)
Date: Mon, 2 Feb 2015 15:19:09 -0500
Subject: What's the status of / relation between "JEP 169: Value Objects"
	/ "Value Types for Java" / "Object Layout"
In-Reply-To: <CA+3eh13pg6+ZziNuzLNdPSc8AkA4o1N9Xx0kGYFNm7TV3t9yzQ@mail.gmail.com>
References: <CA+3eh122g3Ax+-PCL3EvX3V-cQkorva2j_c42_i4uRLOqw42Fg@mail.gmail.com>
	<54CA6853.2060601@oracle.com>
	<CA+3eh13pg6+ZziNuzLNdPSc8AkA4o1N9Xx0kGYFNm7TV3t9yzQ@mail.gmail.com>
Message-ID: <CAHjP37HO8b3kgaWy4pVXpY3ky=+ebhLHBafMeNnZ-QooA9bXkA@mail.gmail.com>

Volker (or anyone else for that matter),

Just curious -- do you envision "inline" layout of objects as something one
would have to opt-in or as the default layout for all objects in a heap? It
seems like this should be the default (assuming zero to minimal overhead
for loading the references) as I think wanting "out of line" allocations is
more rare.

Thanks

On Mon, Feb 2, 2015 at 2:06 PM, Volker Simonis <volker.simonis at gmail.com>
wrote:

> Hi Brian,
>
> thanks a lot for your detailed answer and apologies for the late reply
> (I was a little distracted by FOSDEM :)
>
> All your comments have been clear and reasonable and are much
> appreciated. Please find my additional answers inline:
>
> On Thu, Jan 29, 2015 at 6:05 PM, Brian Goetz <brian.goetz at oracle.com>
> wrote:
> >> Question: is JEP 169 still under active development or has it been
> >> merged into the more general "Value types for Java" proposal below?
> >
> >
> > It has been merged into the more general Value Types for Java proposal.
> >
>
> Then maybe this JEP should be closed to avoid further confusion?
>
> >> The "Value types for Java" approach clearly seems to be the most
> >> general but also the most complex proposal.
> >
> >
> > For some meanings of "complex".  It is certainly the most intrusive and
> > large; new bytecodes, new type signatures.  But from a user-model
> > perspective, value types are actually fairly simple.
> >
> >> It's out of scope for Java
> >> 9 and still questionable for Java 10 and above. The "PackedObject" and
> >> "ObjectLayout" approaches are clearly simpler and more limited in
> >> scope as they only concentrate on better object layout.
> >
> >
> > To your list, I'd add: Project Panama, the sister project to Valhalla.
> > Panama focuses on interop with native code and data, including layout
> > specification.  A key goal of Packed was to be able to access off-heap
> > native data in its native format, rather than marshalling it across the
> JNI
> > boundary.  Panama is focused on this problem as well, but aims to treat
> it
> > as a separate problem from Java object layout, resulting in what we
> believe
> > to be a cleaner decomposition of the two concerns.
> >
>
> Your right. I somehow missed to look at Panama more deeply because I
> always thought it is only about FFI. John Rose nicely explains the
> various parts of Panama in this mail
> http://mail.openjdk.java.net/pipermail/panama-dev/2014-October/000042.html
> where he also mentions the intention of Panama to create new flatter
> data layouts in the Heap and the relation of Panama to PackedObjects
> and ObjectLayout.
>
> > Packed is an interesting mix of memory density (object embedding and
> packed
> > arrays) and native interop.  But mixing the two goals also has costs; our
> > approach is to separate them into orthogonal concerns, and we think that
> > Valhalla and Panama do just that.  So in many ways, while a larger
> project,
> > the combination of Valhalla+Panama addresses the problem that Packed
> did, in
> > a cleaner way.
> >
> >> Question: is there a chance to get a some sort of Java-only but
> >> transparently optimizable structure package like "ObjectLayout" into
> >> Java early (i.e. Java 9)?
> >
> >
> > It would depend on a lot of things -- including the level of readiness of
> > the design and implementation, and the overlap with anticipated future
> > features.  We've reviewed some of the early design of ObjectLayout and
> > provided feedback to the projects architects; currently, I think it's in
> the
> > "promising exploration" stage, but I think multiple rounds of
> simplification
> > are needed before it is ready to be considered for "everybody's Java."
> But
> > if the choice is to push something that's not ready into 9, or to wait
> > longer -- there's not actually a choice to be made there.
> >
> > I appreciate the desire to "get something you can use now", but we have
> to
> > be prepared to support whatever we push into Java for the next 20 years,
> and
> > deal with the additional constraints it generates -- which can be an
> > enormous cost.  (Even thought the direct cost is mostly borne by Oracle,
> the
> > indirect cost is borne by everyone, in the form of slower progress on
> > everything else.)  So I am very wary of the motivation of "well,
> something
> > better is coming, but this works now, so can we push it in?"  I'd prefer
> to
> > focus on answering whether this is right thing for Java for the next 20
> > years.
> >
> >> In my eyes this wouldn't contradict with a more general solution like
> >> the one proposed in the "Value types for Java" approach while still
> >> offering quite significant performance improvements for quite a big
> >> range of problems.
> >
> >
> > The goals of the ObjectLayout effort has overlap with, but also differs
> > from, the goals of Valhalla.  And herein is the problem; neither
> generalizes
> > the other, and I don't think we do the user base a great favor by
> pursuing
> > two separate neither-coincident-nor-orthogonal approaches.  I suspect,
> > though, that after a few rounds of simplification, ObjectLayout could
> morph
> > into something that fit either coincidently or orthogonally with the
> > Valhalla work -- which would be great.  But, as you know, our resources
> are
> > limited, so we (Oracle) can't really afford to invest in both.  And such
> > simplification takes time -- getting to that "aha" moment when you
> realize
> > you can simplify something is generally an incompressible process.
> >
> >> Question: what would be the right place to propose something like the
> >> "ObjectLayout" library for Java 9/10? Would that fit within the
> >> umbrella of the Valhalla project or would it be done within its own
> >> project / under it's own JEP?
> >
> >
> > Suggesting a version number at this point would be putting the cart
> before
> > the horse (you'll note that we've not even proposed a version number for
> > Valhalla; the closest we've gotten to that is "after 9".)
> >
> > OpenJDK Projects are a tool for building a community around a body of
> work;
> > JEPs are a project-management tool for defining, scoping, and tracking
> the
> > progress of a feature.  Given where OL is, it would be reasonable to
> start a
> > Project, which would become the nexus of collaboration that could
> eventually
> > produce a JEP.
> >
>
> That sounds reasonable. I'll speak with the ObjectLayout people to
> hear what they think about starting a new OpenJDK project.
>
> And after all you've said before, I also came to the conclusion that
> investigating ways of new in-heap object layouts seem to be better of
> in the Panama project. So I'll also ask there what they think about
> it. Maybe ObjectLayout could become part of Panama or maybe we could
> just start a new subproject of Panama with the same/similar goals.
>
> > Hope this helps,
> > -Brian
>
> It really did!
>
> Thanks again,
> Volker
> _______________________________________________
> mlvm-dev mailing list
> mlvm-dev at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>

From maurizio.cimadamore at oracle.com  Mon Feb  2 22:19:10 2015
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Mon, 02 Feb 2015 22:19:10 +0000
Subject: CFV: New Valhalla Committer: Frederic Parain
Message-ID: <54CFF7DE.4050305@oracle.com>

I hereby nominate Frederic Parain (fparain) to Valhalla Committer.

Frederic is a member of the Hotspot Runtime team who's looking to join 
the dark forces @ Project Valhalla. Here's a list of his OpenJDK 
contributions.

HotSpot
     https://jbs.oracle.com/bugs/browse/JDK-8061618: Removed unused 
networking functions from os class
     https://jbs.oracle.com/bugs/browse/JDK-8057777: Cleanup of old and 
unused VM interfaces
     https://jbs.oracle.com/bugs/browse/JDK-8047714: Fix for JDK-6546236 
made Solaris os::yield() a no-op
     https://jbs.oracle.com/bugs/browse/JDK-JDK-8043630: Method 
os::yield_all() should be removed
     https://jbs.oracle.com/bugs/browse/JDK-JDK-8038473: Remove support 
for old T1 libthread
     https://jbs.oracle.com/bugs/browse/JDK-8036128: Remove deprecated 
VM flag UseVMInterruptibleIO
     https://jbs.oracle.com/bugs/browse/JDK-6546236: Thread interrupt() 
of Thread.sleep() can be lost on Solaris due to race with signal handler
     https://jbs.oracle.com/bugs/browse/JDK-8030808: 
dtrace/hotspot/Monitors/Monitors001 fails in product builds on solaris-sparc
     https://jbs.oracle.com/bugs/browse/JDK-8019845: Memory leak during 
class redefinition
     https://jbs.oracle.com/bugs/browse/JDK-7143807: ResourceMark 
nesting problem in stringStream
     https://jbs.oracle.com/bugs/browse/JDK-8016465: The hs_err file 
gets wrong name
     https://jbs.oracle.com/bugs/browse/JDK-7060111: race condition in 
VMError::report_and_die()
     https://jbs.oracle.com/bugs/browse/JDK-8004095: Add support for JMX 
interface to Diagnostic Framework and Commands
     https://jbs.oracle.com/bugs/browse/JDK-7183754: Test 
runtime/6294277/Test6294277.sh runs wrong JVM
     https://jbs.oracle.com/bugs/browse/JDK-6294277: java -Xdebug 
crashes on SourceDebugExtension attribute larger than 64K
     https://jbs.oracle.com/bugs/browse/JDK-7171703: JNI DefineClass 
crashes client VM when first parameter is NULL
     https://jbs.oracle.com/bugs/browse/JDK-7087969: 
GarbageCollectorMXBean notification contains ticks vs millis
     https://jbs.oracle.com/bugs/browse/JDK-7145243: Need additional 
specializations for argument parsing framework
     https://jbs.oracle.com/bugs/browse/JDK-7143760: Memory leak in 
GarbageCollectionNotifications
     https://jbs.oracle.com/bugs/browse/JDK-7131346: Parsing of boolean 
arguments to diagnostic commands is broken
     https://jbs.oracle.com/bugs/browse/JDK-7120511: Add diagnostic commands
     https://jbs.oracle.com/bugs/browse/JDK-7125594: C-heap growth issue 
in ThreadService::find_deadlocks_at_safepoint
     https://jbs.oracle.com/bugs/browse/JDK-7104647: Adding a diagnostic 
command framework
     https://jbs.oracle.com/bugs/browse/JDK-7036199: Adding a 
notification to the implementation of GarbageCollectorMXBeans
     https://jbs.oracle.com/bugs/browse/JDK-6935224: Adding new DTrace 
probes to work with Palantir

  JDK:
     https://jbs.oracle.com/bugs/browse/JDK-8057777: Cleanup of old and 
unused VM interfaces
     https://jbs.oracle.com/bugs/browse/JDK-8035952: Remove use of 
JVM_Open, JVM_Read and JVM_Close functions from serviceability code
     https://jbs.oracle.com/bugs/browse/JDK-7150256: Add back Diagnostic 
Command JMX API
     https://jbs.oracle.com/bugs/browse/JDK-7156831: The jcmd man page 
is not included in generated bundles
     https://jbs.oracle.com/bugs/browse/JDK-7074616: 
java.lang.management.ManagementFactory.getPlatformManagementInterfaces fails
     https://jbs.oracle.com/bugs/browse/JDK-7145925: Removing remote 
access to diagnostic commands in the HotSpotDiagnosticMBean
     https://jbs.oracle.com/bugs/browse/JDK-7144833: 
sun/tools/jcmd/jcmd-Defaults.sh failing intermittently
     https://jbs.oracle.com/bugs/browse/JDK-7140868: TEST_BUG: jcmd 
tests need to use -XX:+UsePerfData
     https://jbs.oracle.com/bugs/browse/JDK-7120974: 
ManagementPermission "control" needs clarification
     https://jbs.oracle.com/bugs/browse/JDK-7104647: Adding a diagnostic 
command framework
     https://jbs.oracle.com/bugs/browse/JDK-7036199: Adding a 
notification to the implementation of GarbageCollectorMXBeans
     https://jbs.oracle.com/bugs/browse/JDK-7031754: javax.management 
docs need to be updated to replace Java SE 6 occurrences
     https://jbs.oracle.com/bugs/browse/JDK-7028071: Add two attributes 
to the OperatingSystemMXBean to provide CPU Load info


Votes are due by February 15, 2015.

Only current Valhalla Committers [1] are eligible to vote on this
nomination.

For Lazy Consensus voting instructions, see [2].

Thank you,
Maurizio Cimadamore

[1] - http://openjdk.java.net/census
[2] - http://openjdk.java.net/projects/#committer-vote



From brian.goetz at oracle.com  Mon Feb  2 22:24:37 2015
From: brian.goetz at oracle.com (Brian Goetz)
Date: Mon, 2 Feb 2015 14:24:37 -0800
Subject: CFV: New Valhalla Committer: Frederic Parain
In-Reply-To: <54CFF7DE.4050305@oracle.com>
References: <54CFF7DE.4050305@oracle.com>
Message-ID: <A7F76A5D-43C7-46D5-A52C-3902C9D75244@oracle.com>

Vote: Yes

On Feb 2, 2015, at 2:19 PM, Maurizio Cimadamore <maurizio.cimadamore at oracle.com> wrote:

> I hereby nominate Frederic Parain (fparain) to Valhalla Committer.
> 
> Frederic is a member of the Hotspot Runtime team who's looking to join the dark forces @ Project Valhalla. Here's a list of his OpenJDK contributions.
> 
> HotSpot
>    https://jbs.oracle.com/bugs/browse/JDK-8061618: Removed unused networking functions from os class
>    https://jbs.oracle.com/bugs/browse/JDK-8057777: Cleanup of old and unused VM interfaces
>    https://jbs.oracle.com/bugs/browse/JDK-8047714: Fix for JDK-6546236 made Solaris os::yield() a no-op
>    https://jbs.oracle.com/bugs/browse/JDK-JDK-8043630: Method os::yield_all() should be removed
>    https://jbs.oracle.com/bugs/browse/JDK-JDK-8038473: Remove support for old T1 libthread
>    https://jbs.oracle.com/bugs/browse/JDK-8036128: Remove deprecated VM flag UseVMInterruptibleIO
>    https://jbs.oracle.com/bugs/browse/JDK-6546236: Thread interrupt() of Thread.sleep() can be lost on Solaris due to race with signal handler
>    https://jbs.oracle.com/bugs/browse/JDK-8030808: dtrace/hotspot/Monitors/Monitors001 fails in product builds on solaris-sparc
>    https://jbs.oracle.com/bugs/browse/JDK-8019845: Memory leak during class redefinition
>    https://jbs.oracle.com/bugs/browse/JDK-7143807: ResourceMark nesting problem in stringStream
>    https://jbs.oracle.com/bugs/browse/JDK-8016465: The hs_err file gets wrong name
>    https://jbs.oracle.com/bugs/browse/JDK-7060111: race condition in VMError::report_and_die()
>    https://jbs.oracle.com/bugs/browse/JDK-8004095: Add support for JMX interface to Diagnostic Framework and Commands
>    https://jbs.oracle.com/bugs/browse/JDK-7183754: Test runtime/6294277/Test6294277.sh runs wrong JVM
>    https://jbs.oracle.com/bugs/browse/JDK-6294277: java -Xdebug crashes on SourceDebugExtension attribute larger than 64K
>    https://jbs.oracle.com/bugs/browse/JDK-7171703: JNI DefineClass crashes client VM when first parameter is NULL
>    https://jbs.oracle.com/bugs/browse/JDK-7087969: GarbageCollectorMXBean notification contains ticks vs millis
>    https://jbs.oracle.com/bugs/browse/JDK-7145243: Need additional specializations for argument parsing framework
>    https://jbs.oracle.com/bugs/browse/JDK-7143760: Memory leak in GarbageCollectionNotifications
>    https://jbs.oracle.com/bugs/browse/JDK-7131346: Parsing of boolean arguments to diagnostic commands is broken
>    https://jbs.oracle.com/bugs/browse/JDK-7120511: Add diagnostic commands
>    https://jbs.oracle.com/bugs/browse/JDK-7125594: C-heap growth issue in ThreadService::find_deadlocks_at_safepoint
>    https://jbs.oracle.com/bugs/browse/JDK-7104647: Adding a diagnostic command framework
>    https://jbs.oracle.com/bugs/browse/JDK-7036199: Adding a notification to the implementation of GarbageCollectorMXBeans
>    https://jbs.oracle.com/bugs/browse/JDK-6935224: Adding new DTrace probes to work with Palantir
> 
> JDK:
>    https://jbs.oracle.com/bugs/browse/JDK-8057777: Cleanup of old and unused VM interfaces
>    https://jbs.oracle.com/bugs/browse/JDK-8035952: Remove use of JVM_Open, JVM_Read and JVM_Close functions from serviceability code
>    https://jbs.oracle.com/bugs/browse/JDK-7150256: Add back Diagnostic Command JMX API
>    https://jbs.oracle.com/bugs/browse/JDK-7156831: The jcmd man page is not included in generated bundles
>    https://jbs.oracle.com/bugs/browse/JDK-7074616: java.lang.management.ManagementFactory.getPlatformManagementInterfaces fails
>    https://jbs.oracle.com/bugs/browse/JDK-7145925: Removing remote access to diagnostic commands in the HotSpotDiagnosticMBean
>    https://jbs.oracle.com/bugs/browse/JDK-7144833: sun/tools/jcmd/jcmd-Defaults.sh failing intermittently
>    https://jbs.oracle.com/bugs/browse/JDK-7140868: TEST_BUG: jcmd tests need to use -XX:+UsePerfData
>    https://jbs.oracle.com/bugs/browse/JDK-7120974: ManagementPermission "control" needs clarification
>    https://jbs.oracle.com/bugs/browse/JDK-7104647: Adding a diagnostic command framework
>    https://jbs.oracle.com/bugs/browse/JDK-7036199: Adding a notification to the implementation of GarbageCollectorMXBeans
>    https://jbs.oracle.com/bugs/browse/JDK-7031754: javax.management docs need to be updated to replace Java SE 6 occurrences
>    https://jbs.oracle.com/bugs/browse/JDK-7028071: Add two attributes to the OperatingSystemMXBean to provide CPU Load info
> 
> 
> Votes are due by February 15, 2015.
> 
> Only current Valhalla Committers [1] are eligible to vote on this
> nomination.
> 
> For Lazy Consensus voting instructions, see [2].
> 
> Thank you,
> Maurizio Cimadamore
> 
> [1] - http://openjdk.java.net/census
> [2] - http://openjdk.java.net/projects/#committer-vote
> 
> 


From karen.kinnear at oracle.com  Mon Feb  2 23:28:37 2015
From: karen.kinnear at oracle.com (Karen Kinnear)
Date: Mon, 2 Feb 2015 18:28:37 -0500
Subject: CFV: New Valhalla Committer: Frederic Parain
In-Reply-To: <54CFF7DE.4050305@oracle.com>
References: <54CFF7DE.4050305@oracle.com>
Message-ID: <F29DA1E8-4ABE-4012-B88D-1B505E09D07D@oracle.com>

Vote: yes

thanks,
Karen

On Feb 2, 2015, at 5:19 PM, Maurizio Cimadamore wrote:

> I hereby nominate Frederic Parain (fparain) to Valhalla Committer.
> 
> Frederic is a member of the Hotspot Runtime team who's looking to join the dark forces @ Project Valhalla. Here's a list of his OpenJDK contributions.
> 
> HotSpot
>    https://jbs.oracle.com/bugs/browse/JDK-8061618: Removed unused networking functions from os class
>    https://jbs.oracle.com/bugs/browse/JDK-8057777: Cleanup of old and unused VM interfaces
>    https://jbs.oracle.com/bugs/browse/JDK-8047714: Fix for JDK-6546236 made Solaris os::yield() a no-op
>    https://jbs.oracle.com/bugs/browse/JDK-JDK-8043630: Method os::yield_all() should be removed
>    https://jbs.oracle.com/bugs/browse/JDK-JDK-8038473: Remove support for old T1 libthread
>    https://jbs.oracle.com/bugs/browse/JDK-8036128: Remove deprecated VM flag UseVMInterruptibleIO
>    https://jbs.oracle.com/bugs/browse/JDK-6546236: Thread interrupt() of Thread.sleep() can be lost on Solaris due to race with signal handler
>    https://jbs.oracle.com/bugs/browse/JDK-8030808: dtrace/hotspot/Monitors/Monitors001 fails in product builds on solaris-sparc
>    https://jbs.oracle.com/bugs/browse/JDK-8019845: Memory leak during class redefinition
>    https://jbs.oracle.com/bugs/browse/JDK-7143807: ResourceMark nesting problem in stringStream
>    https://jbs.oracle.com/bugs/browse/JDK-8016465: The hs_err file gets wrong name
>    https://jbs.oracle.com/bugs/browse/JDK-7060111: race condition in VMError::report_and_die()
>    https://jbs.oracle.com/bugs/browse/JDK-8004095: Add support for JMX interface to Diagnostic Framework and Commands
>    https://jbs.oracle.com/bugs/browse/JDK-7183754: Test runtime/6294277/Test6294277.sh runs wrong JVM
>    https://jbs.oracle.com/bugs/browse/JDK-6294277: java -Xdebug crashes on SourceDebugExtension attribute larger than 64K
>    https://jbs.oracle.com/bugs/browse/JDK-7171703: JNI DefineClass crashes client VM when first parameter is NULL
>    https://jbs.oracle.com/bugs/browse/JDK-7087969: GarbageCollectorMXBean notification contains ticks vs millis
>    https://jbs.oracle.com/bugs/browse/JDK-7145243: Need additional specializations for argument parsing framework
>    https://jbs.oracle.com/bugs/browse/JDK-7143760: Memory leak in GarbageCollectionNotifications
>    https://jbs.oracle.com/bugs/browse/JDK-7131346: Parsing of boolean arguments to diagnostic commands is broken
>    https://jbs.oracle.com/bugs/browse/JDK-7120511: Add diagnostic commands
>    https://jbs.oracle.com/bugs/browse/JDK-7125594: C-heap growth issue in ThreadService::find_deadlocks_at_safepoint
>    https://jbs.oracle.com/bugs/browse/JDK-7104647: Adding a diagnostic command framework
>    https://jbs.oracle.com/bugs/browse/JDK-7036199: Adding a notification to the implementation of GarbageCollectorMXBeans
>    https://jbs.oracle.com/bugs/browse/JDK-6935224: Adding new DTrace probes to work with Palantir
> 
> JDK:
>    https://jbs.oracle.com/bugs/browse/JDK-8057777: Cleanup of old and unused VM interfaces
>    https://jbs.oracle.com/bugs/browse/JDK-8035952: Remove use of JVM_Open, JVM_Read and JVM_Close functions from serviceability code
>    https://jbs.oracle.com/bugs/browse/JDK-7150256: Add back Diagnostic Command JMX API
>    https://jbs.oracle.com/bugs/browse/JDK-7156831: The jcmd man page is not included in generated bundles
>    https://jbs.oracle.com/bugs/browse/JDK-7074616: java.lang.management.ManagementFactory.getPlatformManagementInterfaces fails
>    https://jbs.oracle.com/bugs/browse/JDK-7145925: Removing remote access to diagnostic commands in the HotSpotDiagnosticMBean
>    https://jbs.oracle.com/bugs/browse/JDK-7144833: sun/tools/jcmd/jcmd-Defaults.sh failing intermittently
>    https://jbs.oracle.com/bugs/browse/JDK-7140868: TEST_BUG: jcmd tests need to use -XX:+UsePerfData
>    https://jbs.oracle.com/bugs/browse/JDK-7120974: ManagementPermission "control" needs clarification
>    https://jbs.oracle.com/bugs/browse/JDK-7104647: Adding a diagnostic command framework
>    https://jbs.oracle.com/bugs/browse/JDK-7036199: Adding a notification to the implementation of GarbageCollectorMXBeans
>    https://jbs.oracle.com/bugs/browse/JDK-7031754: javax.management docs need to be updated to replace Java SE 6 occurrences
>    https://jbs.oracle.com/bugs/browse/JDK-7028071: Add two attributes to the OperatingSystemMXBean to provide CPU Load info
> 
> 
> Votes are due by February 15, 2015.
> 
> Only current Valhalla Committers [1] are eligible to vote on this
> nomination.
> 
> For Lazy Consensus voting instructions, see [2].
> 
> Thank you,
> Maurizio Cimadamore
> 
> [1] - http://openjdk.java.net/census
> [2] - http://openjdk.java.net/projects/#committer-vote
> 
> 


From john.r.rose at oracle.com  Tue Feb  3 10:01:23 2015
From: john.r.rose at oracle.com (John Rose)
Date: Tue, 3 Feb 2015 11:01:23 +0100
Subject: CFV: New Valhalla Committer: Frederic Parain
In-Reply-To: <54CFF7DE.4050305@oracle.com>
References: <54CFF7DE.4050305@oracle.com>
Message-ID: <976D2687-FDE8-4C4F-8831-8C3EC8A19B5C@oracle.com>

Vote: yes

On Feb 2, 2015, at 11:19 PM, Maurizio Cimadamore <maurizio.cimadamore at oracle.com> wrote:
> 
> I hereby nominate Frederic Parain (fparain) to Valhalla Committer.
> 


From paul.sandoz at oracle.com  Tue Feb  3 11:56:37 2015
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Tue, 3 Feb 2015 12:56:37 +0100
Subject: CFV: New Valhalla Committer: Frederic Parain
In-Reply-To: <54CFF7DE.4050305@oracle.com>
References: <54CFF7DE.4050305@oracle.com>
Message-ID: <61887861-5780-4DA1-84C2-41BD3FDBC464@oracle.com>

Vote: yes

Paul.

From volker.simonis at gmail.com  Tue Feb  3 16:29:56 2015
From: volker.simonis at gmail.com (Volker Simonis)
Date: Tue, 3 Feb 2015 17:29:56 +0100
Subject: What's the status of / relation between "JEP 169: Value Objects"
	/ "Value Types for Java" / "Object Layout"
In-Reply-To: <CAHjP37HO8b3kgaWy4pVXpY3ky=+ebhLHBafMeNnZ-QooA9bXkA@mail.gmail.com>
References: <CA+3eh122g3Ax+-PCL3EvX3V-cQkorva2j_c42_i4uRLOqw42Fg@mail.gmail.com>
	<54CA6853.2060601@oracle.com>
	<CA+3eh13pg6+ZziNuzLNdPSc8AkA4o1N9Xx0kGYFNm7TV3t9yzQ@mail.gmail.com>
	<CAHjP37HO8b3kgaWy4pVXpY3ky=+ebhLHBafMeNnZ-QooA9bXkA@mail.gmail.com>
Message-ID: <CA+3eh135AOaiHoFQaA8Uc7WFsG-rHeeF30MBHw52J=fev9c1CQ@mail.gmail.com>

Hi Vitaly,

for PackedObjects/ObjectLayout you need to specially annotate the
classes and/or fields which you want to allocate "inline". Once you've
done that you have no choice with the PackedObjects approach.
ObjectLayout is a little special here, because it can run with any
Java VM in which case it will still use the default reference model.
But it can potentially be optimized by some VM's to provide the flat
object layout. I expect these optimizations to be controllable by a
command line option.

With the "Value Types for Java" [1] approach you'll have the
possiblitly to express the behavior right in Java like in the
following example from [1]:

final __ByValue class Point {
static Point origin = __MakeValue(0, 0);

I think the default will always be "reference semantics" in Java but
with various degrees of freedom to optionally choose value semantics.

Regards,
Volker

[1] http://cr.openjdk.java.net/~jrose/values/values-0.html

On Mon, Feb 2, 2015 at 9:19 PM, Vitaly Davidovich <vitalyd at gmail.com> wrote:
> Volker (or anyone else for that matter),
>
> Just curious -- do you envision "inline" layout of objects as something one
> would have to opt-in or as the default layout for all objects in a heap? It
> seems like this should be the default (assuming zero to minimal overhead for
> loading the references) as I think wanting "out of line" allocations is more
> rare.
>
> Thanks
>
> On Mon, Feb 2, 2015 at 2:06 PM, Volker Simonis <volker.simonis at gmail.com>
> wrote:
>>
>> Hi Brian,
>>
>> thanks a lot for your detailed answer and apologies for the late reply
>> (I was a little distracted by FOSDEM :)
>>
>> All your comments have been clear and reasonable and are much
>> appreciated. Please find my additional answers inline:
>>
>> On Thu, Jan 29, 2015 at 6:05 PM, Brian Goetz <brian.goetz at oracle.com>
>> wrote:
>> >> Question: is JEP 169 still under active development or has it been
>> >> merged into the more general "Value types for Java" proposal below?
>> >
>> >
>> > It has been merged into the more general Value Types for Java proposal.
>> >
>>
>> Then maybe this JEP should be closed to avoid further confusion?
>>
>> >> The "Value types for Java" approach clearly seems to be the most
>> >> general but also the most complex proposal.
>> >
>> >
>> > For some meanings of "complex".  It is certainly the most intrusive and
>> > large; new bytecodes, new type signatures.  But from a user-model
>> > perspective, value types are actually fairly simple.
>> >
>> >> It's out of scope for Java
>> >> 9 and still questionable for Java 10 and above. The "PackedObject" and
>> >> "ObjectLayout" approaches are clearly simpler and more limited in
>> >> scope as they only concentrate on better object layout.
>> >
>> >
>> > To your list, I'd add: Project Panama, the sister project to Valhalla.
>> > Panama focuses on interop with native code and data, including layout
>> > specification.  A key goal of Packed was to be able to access off-heap
>> > native data in its native format, rather than marshalling it across the
>> > JNI
>> > boundary.  Panama is focused on this problem as well, but aims to treat
>> > it
>> > as a separate problem from Java object layout, resulting in what we
>> > believe
>> > to be a cleaner decomposition of the two concerns.
>> >
>>
>> Your right. I somehow missed to look at Panama more deeply because I
>> always thought it is only about FFI. John Rose nicely explains the
>> various parts of Panama in this mail
>> http://mail.openjdk.java.net/pipermail/panama-dev/2014-October/000042.html
>> where he also mentions the intention of Panama to create new flatter
>> data layouts in the Heap and the relation of Panama to PackedObjects
>> and ObjectLayout.
>>
>> > Packed is an interesting mix of memory density (object embedding and
>> > packed
>> > arrays) and native interop.  But mixing the two goals also has costs;
>> > our
>> > approach is to separate them into orthogonal concerns, and we think that
>> > Valhalla and Panama do just that.  So in many ways, while a larger
>> > project,
>> > the combination of Valhalla+Panama addresses the problem that Packed
>> > did, in
>> > a cleaner way.
>> >
>> >> Question: is there a chance to get a some sort of Java-only but
>> >> transparently optimizable structure package like "ObjectLayout" into
>> >> Java early (i.e. Java 9)?
>> >
>> >
>> > It would depend on a lot of things -- including the level of readiness
>> > of
>> > the design and implementation, and the overlap with anticipated future
>> > features.  We've reviewed some of the early design of ObjectLayout and
>> > provided feedback to the projects architects; currently, I think it's in
>> > the
>> > "promising exploration" stage, but I think multiple rounds of
>> > simplification
>> > are needed before it is ready to be considered for "everybody's Java."
>> > But
>> > if the choice is to push something that's not ready into 9, or to wait
>> > longer -- there's not actually a choice to be made there.
>> >
>> > I appreciate the desire to "get something you can use now", but we have
>> > to
>> > be prepared to support whatever we push into Java for the next 20 years,
>> > and
>> > deal with the additional constraints it generates -- which can be an
>> > enormous cost.  (Even thought the direct cost is mostly borne by Oracle,
>> > the
>> > indirect cost is borne by everyone, in the form of slower progress on
>> > everything else.)  So I am very wary of the motivation of "well,
>> > something
>> > better is coming, but this works now, so can we push it in?"  I'd prefer
>> > to
>> > focus on answering whether this is right thing for Java for the next 20
>> > years.
>> >
>> >> In my eyes this wouldn't contradict with a more general solution like
>> >> the one proposed in the "Value types for Java" approach while still
>> >> offering quite significant performance improvements for quite a big
>> >> range of problems.
>> >
>> >
>> > The goals of the ObjectLayout effort has overlap with, but also differs
>> > from, the goals of Valhalla.  And herein is the problem; neither
>> > generalizes
>> > the other, and I don't think we do the user base a great favor by
>> > pursuing
>> > two separate neither-coincident-nor-orthogonal approaches.  I suspect,
>> > though, that after a few rounds of simplification, ObjectLayout could
>> > morph
>> > into something that fit either coincidently or orthogonally with the
>> > Valhalla work -- which would be great.  But, as you know, our resources
>> > are
>> > limited, so we (Oracle) can't really afford to invest in both.  And such
>> > simplification takes time -- getting to that "aha" moment when you
>> > realize
>> > you can simplify something is generally an incompressible process.
>> >
>> >> Question: what would be the right place to propose something like the
>> >> "ObjectLayout" library for Java 9/10? Would that fit within the
>> >> umbrella of the Valhalla project or would it be done within its own
>> >> project / under it's own JEP?
>> >
>> >
>> > Suggesting a version number at this point would be putting the cart
>> > before
>> > the horse (you'll note that we've not even proposed a version number for
>> > Valhalla; the closest we've gotten to that is "after 9".)
>> >
>> > OpenJDK Projects are a tool for building a community around a body of
>> > work;
>> > JEPs are a project-management tool for defining, scoping, and tracking
>> > the
>> > progress of a feature.  Given where OL is, it would be reasonable to
>> > start a
>> > Project, which would become the nexus of collaboration that could
>> > eventually
>> > produce a JEP.
>> >
>>
>> That sounds reasonable. I'll speak with the ObjectLayout people to
>> hear what they think about starting a new OpenJDK project.
>>
>> And after all you've said before, I also came to the conclusion that
>> investigating ways of new in-heap object layouts seem to be better of
>> in the Panama project. So I'll also ask there what they think about
>> it. Maybe ObjectLayout could become part of Panama or maybe we could
>> just start a new subproject of Panama with the same/similar goals.
>>
>> > Hope this helps,
>> > -Brian
>>
>> It really did!
>>
>> Thanks again,
>> Volker
>> _______________________________________________
>> mlvm-dev mailing list
>> mlvm-dev at openjdk.java.net
>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>
>
>
> _______________________________________________
> mlvm-dev mailing list
> mlvm-dev at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>

From vitalyd at gmail.com  Tue Feb  3 16:40:00 2015
From: vitalyd at gmail.com (Vitaly Davidovich)
Date: Tue, 3 Feb 2015 11:40:00 -0500
Subject: What's the status of / relation between "JEP 169: Value Objects"
	/ "Value Types for Java" / "Object Layout"
In-Reply-To: <CA+3eh135AOaiHoFQaA8Uc7WFsG-rHeeF30MBHw52J=fev9c1CQ@mail.gmail.com>
References: <CA+3eh122g3Ax+-PCL3EvX3V-cQkorva2j_c42_i4uRLOqw42Fg@mail.gmail.com>
	<54CA6853.2060601@oracle.com>
	<CA+3eh13pg6+ZziNuzLNdPSc8AkA4o1N9Xx0kGYFNm7TV3t9yzQ@mail.gmail.com>
	<CAHjP37HO8b3kgaWy4pVXpY3ky=+ebhLHBafMeNnZ-QooA9bXkA@mail.gmail.com>
	<CA+3eh135AOaiHoFQaA8Uc7WFsG-rHeeF30MBHw52J=fev9c1CQ@mail.gmail.com>
Message-ID: <CAHjP37GhZYk61Z12L6Yj24YCwmyaao2oWJGRva6+gw1wGzGpOQ@mail.gmail.com>

Hi Volker,

Sorry, I may have been unclear in my question.  As you say, ObjectLayout
requires that you annotate the fields that you'd like inlined and then also
use special API to construct those objects.  I'm wondering whether,
instead, all private final fields are automatically inlined, and only cases
where you'd like to layout the field out-of-band would require annotation.
This would be controlled via a cmdline flag, as you say, similar to perhaps
how compressed oops are enabled (or not).  Note that I'm talking about
purely layout of reference types, not value types.

The "concern" with having to explicitly annotate and use dedicated APIs to
opt-in is that adoption will be fairly low, whereas I think most of the
time one would want inlined storage layout.

Thanks


On Tue, Feb 3, 2015 at 11:29 AM, Volker Simonis <volker.simonis at gmail.com>
wrote:

> Hi Vitaly,
>
> for PackedObjects/ObjectLayout you need to specially annotate the
> classes and/or fields which you want to allocate "inline". Once you've
> done that you have no choice with the PackedObjects approach.
> ObjectLayout is a little special here, because it can run with any
> Java VM in which case it will still use the default reference model.
> But it can potentially be optimized by some VM's to provide the flat
> object layout. I expect these optimizations to be controllable by a
> command line option.
>
> With the "Value Types for Java" [1] approach you'll have the
> possiblitly to express the behavior right in Java like in the
> following example from [1]:
>
> final __ByValue class Point {
> static Point origin = __MakeValue(0, 0);
>
> I think the default will always be "reference semantics" in Java but
> with various degrees of freedom to optionally choose value semantics.
>
> Regards,
> Volker
>
> [1] http://cr.openjdk.java.net/~jrose/values/values-0.html
>
> On Mon, Feb 2, 2015 at 9:19 PM, Vitaly Davidovich <vitalyd at gmail.com>
> wrote:
> > Volker (or anyone else for that matter),
> >
> > Just curious -- do you envision "inline" layout of objects as something
> one
> > would have to opt-in or as the default layout for all objects in a heap?
> It
> > seems like this should be the default (assuming zero to minimal overhead
> for
> > loading the references) as I think wanting "out of line" allocations is
> more
> > rare.
> >
> > Thanks
> >
> > On Mon, Feb 2, 2015 at 2:06 PM, Volker Simonis <volker.simonis at gmail.com
> >
> > wrote:
> >>
> >> Hi Brian,
> >>
> >> thanks a lot for your detailed answer and apologies for the late reply
> >> (I was a little distracted by FOSDEM :)
> >>
> >> All your comments have been clear and reasonable and are much
> >> appreciated. Please find my additional answers inline:
> >>
> >> On Thu, Jan 29, 2015 at 6:05 PM, Brian Goetz <brian.goetz at oracle.com>
> >> wrote:
> >> >> Question: is JEP 169 still under active development or has it been
> >> >> merged into the more general "Value types for Java" proposal below?
> >> >
> >> >
> >> > It has been merged into the more general Value Types for Java
> proposal.
> >> >
> >>
> >> Then maybe this JEP should be closed to avoid further confusion?
> >>
> >> >> The "Value types for Java" approach clearly seems to be the most
> >> >> general but also the most complex proposal.
> >> >
> >> >
> >> > For some meanings of "complex".  It is certainly the most intrusive
> and
> >> > large; new bytecodes, new type signatures.  But from a user-model
> >> > perspective, value types are actually fairly simple.
> >> >
> >> >> It's out of scope for Java
> >> >> 9 and still questionable for Java 10 and above. The "PackedObject"
> and
> >> >> "ObjectLayout" approaches are clearly simpler and more limited in
> >> >> scope as they only concentrate on better object layout.
> >> >
> >> >
> >> > To your list, I'd add: Project Panama, the sister project to Valhalla.
> >> > Panama focuses on interop with native code and data, including layout
> >> > specification.  A key goal of Packed was to be able to access off-heap
> >> > native data in its native format, rather than marshalling it across
> the
> >> > JNI
> >> > boundary.  Panama is focused on this problem as well, but aims to
> treat
> >> > it
> >> > as a separate problem from Java object layout, resulting in what we
> >> > believe
> >> > to be a cleaner decomposition of the two concerns.
> >> >
> >>
> >> Your right. I somehow missed to look at Panama more deeply because I
> >> always thought it is only about FFI. John Rose nicely explains the
> >> various parts of Panama in this mail
> >>
> http://mail.openjdk.java.net/pipermail/panama-dev/2014-October/000042.html
> >> where he also mentions the intention of Panama to create new flatter
> >> data layouts in the Heap and the relation of Panama to PackedObjects
> >> and ObjectLayout.
> >>
> >> > Packed is an interesting mix of memory density (object embedding and
> >> > packed
> >> > arrays) and native interop.  But mixing the two goals also has costs;
> >> > our
> >> > approach is to separate them into orthogonal concerns, and we think
> that
> >> > Valhalla and Panama do just that.  So in many ways, while a larger
> >> > project,
> >> > the combination of Valhalla+Panama addresses the problem that Packed
> >> > did, in
> >> > a cleaner way.
> >> >
> >> >> Question: is there a chance to get a some sort of Java-only but
> >> >> transparently optimizable structure package like "ObjectLayout" into
> >> >> Java early (i.e. Java 9)?
> >> >
> >> >
> >> > It would depend on a lot of things -- including the level of readiness
> >> > of
> >> > the design and implementation, and the overlap with anticipated future
> >> > features.  We've reviewed some of the early design of ObjectLayout and
> >> > provided feedback to the projects architects; currently, I think it's
> in
> >> > the
> >> > "promising exploration" stage, but I think multiple rounds of
> >> > simplification
> >> > are needed before it is ready to be considered for "everybody's Java."
> >> > But
> >> > if the choice is to push something that's not ready into 9, or to wait
> >> > longer -- there's not actually a choice to be made there.
> >> >
> >> > I appreciate the desire to "get something you can use now", but we
> have
> >> > to
> >> > be prepared to support whatever we push into Java for the next 20
> years,
> >> > and
> >> > deal with the additional constraints it generates -- which can be an
> >> > enormous cost.  (Even thought the direct cost is mostly borne by
> Oracle,
> >> > the
> >> > indirect cost is borne by everyone, in the form of slower progress on
> >> > everything else.)  So I am very wary of the motivation of "well,
> >> > something
> >> > better is coming, but this works now, so can we push it in?"  I'd
> prefer
> >> > to
> >> > focus on answering whether this is right thing for Java for the next
> 20
> >> > years.
> >> >
> >> >> In my eyes this wouldn't contradict with a more general solution like
> >> >> the one proposed in the "Value types for Java" approach while still
> >> >> offering quite significant performance improvements for quite a big
> >> >> range of problems.
> >> >
> >> >
> >> > The goals of the ObjectLayout effort has overlap with, but also
> differs
> >> > from, the goals of Valhalla.  And herein is the problem; neither
> >> > generalizes
> >> > the other, and I don't think we do the user base a great favor by
> >> > pursuing
> >> > two separate neither-coincident-nor-orthogonal approaches.  I suspect,
> >> > though, that after a few rounds of simplification, ObjectLayout could
> >> > morph
> >> > into something that fit either coincidently or orthogonally with the
> >> > Valhalla work -- which would be great.  But, as you know, our
> resources
> >> > are
> >> > limited, so we (Oracle) can't really afford to invest in both.  And
> such
> >> > simplification takes time -- getting to that "aha" moment when you
> >> > realize
> >> > you can simplify something is generally an incompressible process.
> >> >
> >> >> Question: what would be the right place to propose something like the
> >> >> "ObjectLayout" library for Java 9/10? Would that fit within the
> >> >> umbrella of the Valhalla project or would it be done within its own
> >> >> project / under it's own JEP?
> >> >
> >> >
> >> > Suggesting a version number at this point would be putting the cart
> >> > before
> >> > the horse (you'll note that we've not even proposed a version number
> for
> >> > Valhalla; the closest we've gotten to that is "after 9".)
> >> >
> >> > OpenJDK Projects are a tool for building a community around a body of
> >> > work;
> >> > JEPs are a project-management tool for defining, scoping, and tracking
> >> > the
> >> > progress of a feature.  Given where OL is, it would be reasonable to
> >> > start a
> >> > Project, which would become the nexus of collaboration that could
> >> > eventually
> >> > produce a JEP.
> >> >
> >>
> >> That sounds reasonable. I'll speak with the ObjectLayout people to
> >> hear what they think about starting a new OpenJDK project.
> >>
> >> And after all you've said before, I also came to the conclusion that
> >> investigating ways of new in-heap object layouts seem to be better of
> >> in the Panama project. So I'll also ask there what they think about
> >> it. Maybe ObjectLayout could become part of Panama or maybe we could
> >> just start a new subproject of Panama with the same/similar goals.
> >>
> >> > Hope this helps,
> >> > -Brian
> >>
> >> It really did!
> >>
> >> Thanks again,
> >> Volker
> >> _______________________________________________
> >> mlvm-dev mailing list
> >> mlvm-dev at openjdk.java.net
> >> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
> >
> >
> >
> > _______________________________________________
> > mlvm-dev mailing list
> > mlvm-dev at openjdk.java.net
> > http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
> >
> _______________________________________________
> mlvm-dev mailing list
> mlvm-dev at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>

From volker.simonis at gmail.com  Tue Feb  3 16:53:06 2015
From: volker.simonis at gmail.com (Volker Simonis)
Date: Tue, 3 Feb 2015 17:53:06 +0100
Subject: What's the status of / relation between "JEP 169: Value Objects"
	/ "Value Types for Java" / "Object Layout"
In-Reply-To: <CAHjP37GhZYk61Z12L6Yj24YCwmyaao2oWJGRva6+gw1wGzGpOQ@mail.gmail.com>
References: <CA+3eh122g3Ax+-PCL3EvX3V-cQkorva2j_c42_i4uRLOqw42Fg@mail.gmail.com>
	<54CA6853.2060601@oracle.com>
	<CA+3eh13pg6+ZziNuzLNdPSc8AkA4o1N9Xx0kGYFNm7TV3t9yzQ@mail.gmail.com>
	<CAHjP37HO8b3kgaWy4pVXpY3ky=+ebhLHBafMeNnZ-QooA9bXkA@mail.gmail.com>
	<CA+3eh135AOaiHoFQaA8Uc7WFsG-rHeeF30MBHw52J=fev9c1CQ@mail.gmail.com>
	<CAHjP37GhZYk61Z12L6Yj24YCwmyaao2oWJGRva6+gw1wGzGpOQ@mail.gmail.com>
Message-ID: <CA+3eh12Os3mm0L3KE=RqjANNkFS8zh8+Ak-xVS88o6BVibHoAg@mail.gmail.com>

Hi Vitaly,

I don't think what you propose could be done in general. References
are polymorphic, i.e. you could have:

class Point { int x, y; }
class Line { Point p1, p2;}

Now how could you inline p1 and p2 into a Line object when you also have:

class Point3D extends Point { int z; }

You could of course only inline objects of final classes which are
directly derived from Object. But I think if you really carefully
reason about all the consequences (which doesn't imply that I've done
this :) you will finally get to something similar like the
ObjectLayout library.

Regards,
Volker


On Tue, Feb 3, 2015 at 5:40 PM, Vitaly Davidovich <vitalyd at gmail.com> wrote:
> Hi Volker,
>
> Sorry, I may have been unclear in my question.  As you say, ObjectLayout
> requires that you annotate the fields that you'd like inlined and then also
> use special API to construct those objects.  I'm wondering whether,
> instead, all private final fields are automatically inlined, and only cases
> where you'd like to layout the field out-of-band would require annotation.
> This would be controlled via a cmdline flag, as you say, similar to perhaps
> how compressed oops are enabled (or not).  Note that I'm talking about
> purely layout of reference types, not value types.
>
> The "concern" with having to explicitly annotate and use dedicated APIs to
> opt-in is that adoption will be fairly low, whereas I think most of the
> time one would want inlined storage layout.
>
> Thanks
>
>
> On Tue, Feb 3, 2015 at 11:29 AM, Volker Simonis <volker.simonis at gmail.com>
> wrote:
>
>> Hi Vitaly,
>>
>> for PackedObjects/ObjectLayout you need to specially annotate the
>> classes and/or fields which you want to allocate "inline". Once you've
>> done that you have no choice with the PackedObjects approach.
>> ObjectLayout is a little special here, because it can run with any
>> Java VM in which case it will still use the default reference model.
>> But it can potentially be optimized by some VM's to provide the flat
>> object layout. I expect these optimizations to be controllable by a
>> command line option.
>>
>> With the "Value Types for Java" [1] approach you'll have the
>> possiblitly to express the behavior right in Java like in the
>> following example from [1]:
>>
>> final __ByValue class Point {
>> static Point origin = __MakeValue(0, 0);
>>
>> I think the default will always be "reference semantics" in Java but
>> with various degrees of freedom to optionally choose value semantics.
>>
>> Regards,
>> Volker
>>
>> [1] http://cr.openjdk.java.net/~jrose/values/values-0.html
>>
>> On Mon, Feb 2, 2015 at 9:19 PM, Vitaly Davidovich <vitalyd at gmail.com>
>> wrote:
>> > Volker (or anyone else for that matter),
>> >
>> > Just curious -- do you envision "inline" layout of objects as something
>> one
>> > would have to opt-in or as the default layout for all objects in a heap?
>> It
>> > seems like this should be the default (assuming zero to minimal overhead
>> for
>> > loading the references) as I think wanting "out of line" allocations is
>> more
>> > rare.
>> >
>> > Thanks
>> >
>> > On Mon, Feb 2, 2015 at 2:06 PM, Volker Simonis <volker.simonis at gmail.com
>> >
>> > wrote:
>> >>
>> >> Hi Brian,
>> >>
>> >> thanks a lot for your detailed answer and apologies for the late reply
>> >> (I was a little distracted by FOSDEM :)
>> >>
>> >> All your comments have been clear and reasonable and are much
>> >> appreciated. Please find my additional answers inline:
>> >>
>> >> On Thu, Jan 29, 2015 at 6:05 PM, Brian Goetz <brian.goetz at oracle.com>
>> >> wrote:
>> >> >> Question: is JEP 169 still under active development or has it been
>> >> >> merged into the more general "Value types for Java" proposal below?
>> >> >
>> >> >
>> >> > It has been merged into the more general Value Types for Java
>> proposal.
>> >> >
>> >>
>> >> Then maybe this JEP should be closed to avoid further confusion?
>> >>
>> >> >> The "Value types for Java" approach clearly seems to be the most
>> >> >> general but also the most complex proposal.
>> >> >
>> >> >
>> >> > For some meanings of "complex".  It is certainly the most intrusive
>> and
>> >> > large; new bytecodes, new type signatures.  But from a user-model
>> >> > perspective, value types are actually fairly simple.
>> >> >
>> >> >> It's out of scope for Java
>> >> >> 9 and still questionable for Java 10 and above. The "PackedObject"
>> and
>> >> >> "ObjectLayout" approaches are clearly simpler and more limited in
>> >> >> scope as they only concentrate on better object layout.
>> >> >
>> >> >
>> >> > To your list, I'd add: Project Panama, the sister project to Valhalla.
>> >> > Panama focuses on interop with native code and data, including layout
>> >> > specification.  A key goal of Packed was to be able to access off-heap
>> >> > native data in its native format, rather than marshalling it across
>> the
>> >> > JNI
>> >> > boundary.  Panama is focused on this problem as well, but aims to
>> treat
>> >> > it
>> >> > as a separate problem from Java object layout, resulting in what we
>> >> > believe
>> >> > to be a cleaner decomposition of the two concerns.
>> >> >
>> >>
>> >> Your right. I somehow missed to look at Panama more deeply because I
>> >> always thought it is only about FFI. John Rose nicely explains the
>> >> various parts of Panama in this mail
>> >>
>> http://mail.openjdk.java.net/pipermail/panama-dev/2014-October/000042.html
>> >> where he also mentions the intention of Panama to create new flatter
>> >> data layouts in the Heap and the relation of Panama to PackedObjects
>> >> and ObjectLayout.
>> >>
>> >> > Packed is an interesting mix of memory density (object embedding and
>> >> > packed
>> >> > arrays) and native interop.  But mixing the two goals also has costs;
>> >> > our
>> >> > approach is to separate them into orthogonal concerns, and we think
>> that
>> >> > Valhalla and Panama do just that.  So in many ways, while a larger
>> >> > project,
>> >> > the combination of Valhalla+Panama addresses the problem that Packed
>> >> > did, in
>> >> > a cleaner way.
>> >> >
>> >> >> Question: is there a chance to get a some sort of Java-only but
>> >> >> transparently optimizable structure package like "ObjectLayout" into
>> >> >> Java early (i.e. Java 9)?
>> >> >
>> >> >
>> >> > It would depend on a lot of things -- including the level of readiness
>> >> > of
>> >> > the design and implementation, and the overlap with anticipated future
>> >> > features.  We've reviewed some of the early design of ObjectLayout and
>> >> > provided feedback to the projects architects; currently, I think it's
>> in
>> >> > the
>> >> > "promising exploration" stage, but I think multiple rounds of
>> >> > simplification
>> >> > are needed before it is ready to be considered for "everybody's Java."
>> >> > But
>> >> > if the choice is to push something that's not ready into 9, or to wait
>> >> > longer -- there's not actually a choice to be made there.
>> >> >
>> >> > I appreciate the desire to "get something you can use now", but we
>> have
>> >> > to
>> >> > be prepared to support whatever we push into Java for the next 20
>> years,
>> >> > and
>> >> > deal with the additional constraints it generates -- which can be an
>> >> > enormous cost.  (Even thought the direct cost is mostly borne by
>> Oracle,
>> >> > the
>> >> > indirect cost is borne by everyone, in the form of slower progress on
>> >> > everything else.)  So I am very wary of the motivation of "well,
>> >> > something
>> >> > better is coming, but this works now, so can we push it in?"  I'd
>> prefer
>> >> > to
>> >> > focus on answering whether this is right thing for Java for the next
>> 20
>> >> > years.
>> >> >
>> >> >> In my eyes this wouldn't contradict with a more general solution like
>> >> >> the one proposed in the "Value types for Java" approach while still
>> >> >> offering quite significant performance improvements for quite a big
>> >> >> range of problems.
>> >> >
>> >> >
>> >> > The goals of the ObjectLayout effort has overlap with, but also
>> differs
>> >> > from, the goals of Valhalla.  And herein is the problem; neither
>> >> > generalizes
>> >> > the other, and I don't think we do the user base a great favor by
>> >> > pursuing
>> >> > two separate neither-coincident-nor-orthogonal approaches.  I suspect,
>> >> > though, that after a few rounds of simplification, ObjectLayout could
>> >> > morph
>> >> > into something that fit either coincidently or orthogonally with the
>> >> > Valhalla work -- which would be great.  But, as you know, our
>> resources
>> >> > are
>> >> > limited, so we (Oracle) can't really afford to invest in both.  And
>> such
>> >> > simplification takes time -- getting to that "aha" moment when you
>> >> > realize
>> >> > you can simplify something is generally an incompressible process.
>> >> >
>> >> >> Question: what would be the right place to propose something like the
>> >> >> "ObjectLayout" library for Java 9/10? Would that fit within the
>> >> >> umbrella of the Valhalla project or would it be done within its own
>> >> >> project / under it's own JEP?
>> >> >
>> >> >
>> >> > Suggesting a version number at this point would be putting the cart
>> >> > before
>> >> > the horse (you'll note that we've not even proposed a version number
>> for
>> >> > Valhalla; the closest we've gotten to that is "after 9".)
>> >> >
>> >> > OpenJDK Projects are a tool for building a community around a body of
>> >> > work;
>> >> > JEPs are a project-management tool for defining, scoping, and tracking
>> >> > the
>> >> > progress of a feature.  Given where OL is, it would be reasonable to
>> >> > start a
>> >> > Project, which would become the nexus of collaboration that could
>> >> > eventually
>> >> > produce a JEP.
>> >> >
>> >>
>> >> That sounds reasonable. I'll speak with the ObjectLayout people to
>> >> hear what they think about starting a new OpenJDK project.
>> >>
>> >> And after all you've said before, I also came to the conclusion that
>> >> investigating ways of new in-heap object layouts seem to be better of
>> >> in the Panama project. So I'll also ask there what they think about
>> >> it. Maybe ObjectLayout could become part of Panama or maybe we could
>> >> just start a new subproject of Panama with the same/similar goals.
>> >>
>> >> > Hope this helps,
>> >> > -Brian
>> >>
>> >> It really did!
>> >>
>> >> Thanks again,
>> >> Volker
>> >> _______________________________________________
>> >> mlvm-dev mailing list
>> >> mlvm-dev at openjdk.java.net
>> >> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>> >
>> >
>> >
>> > _______________________________________________
>> > mlvm-dev mailing list
>> > mlvm-dev at openjdk.java.net
>> > http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>> >
>> _______________________________________________
>> mlvm-dev mailing list
>> mlvm-dev at openjdk.java.net
>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>>

From vitalyd at gmail.com  Tue Feb  3 16:57:53 2015
From: vitalyd at gmail.com (Vitaly Davidovich)
Date: Tue, 3 Feb 2015 11:57:53 -0500
Subject: What's the status of / relation between "JEP 169: Value Objects"
	/ "Value Types for Java" / "Object Layout"
In-Reply-To: <CA+3eh12Os3mm0L3KE=RqjANNkFS8zh8+Ak-xVS88o6BVibHoAg@mail.gmail.com>
References: <CA+3eh122g3Ax+-PCL3EvX3V-cQkorva2j_c42_i4uRLOqw42Fg@mail.gmail.com>
	<54CA6853.2060601@oracle.com>
	<CA+3eh13pg6+ZziNuzLNdPSc8AkA4o1N9Xx0kGYFNm7TV3t9yzQ@mail.gmail.com>
	<CAHjP37HO8b3kgaWy4pVXpY3ky=+ebhLHBafMeNnZ-QooA9bXkA@mail.gmail.com>
	<CA+3eh135AOaiHoFQaA8Uc7WFsG-rHeeF30MBHw52J=fev9c1CQ@mail.gmail.com>
	<CAHjP37GhZYk61Z12L6Yj24YCwmyaao2oWJGRva6+gw1wGzGpOQ@mail.gmail.com>
	<CA+3eh12Os3mm0L3KE=RqjANNkFS8zh8+Ak-xVS88o6BVibHoAg@mail.gmail.com>
Message-ID: <CAHjP37FFXxgnitzxPyewk+eUjEQe5onnM8f=sz8EuzEajbEJfQ@mail.gmail.com>

Right,  but I'm talking about using same restrictions that ObjectLayout
requires (private final fields initialized inside constructor).  I guess an
easy way to describe it as "do same thing automatically that would be done
manually using ObjectLayout".

sent from my phone
On Feb 3, 2015 11:53 AM, "Volker Simonis" <volker.simonis at gmail.com> wrote:

> Hi Vitaly,
>
> I don't think what you propose could be done in general. References
> are polymorphic, i.e. you could have:
>
> class Point { int x, y; }
> class Line { Point p1, p2;}
>
> Now how could you inline p1 and p2 into a Line object when you also have:
>
> class Point3D extends Point { int z; }
>
> You could of course only inline objects of final classes which are
> directly derived from Object. But I think if you really carefully
> reason about all the consequences (which doesn't imply that I've done
> this :) you will finally get to something similar like the
> ObjectLayout library.
>
> Regards,
> Volker
>
>
> On Tue, Feb 3, 2015 at 5:40 PM, Vitaly Davidovich <vitalyd at gmail.com>
> wrote:
> > Hi Volker,
> >
> > Sorry, I may have been unclear in my question.  As you say, ObjectLayout
> > requires that you annotate the fields that you'd like inlined and then
> also
> > use special API to construct those objects.  I'm wondering whether,
> > instead, all private final fields are automatically inlined, and only
> cases
> > where you'd like to layout the field out-of-band would require
> annotation.
> > This would be controlled via a cmdline flag, as you say, similar to
> perhaps
> > how compressed oops are enabled (or not).  Note that I'm talking about
> > purely layout of reference types, not value types.
> >
> > The "concern" with having to explicitly annotate and use dedicated APIs
> to
> > opt-in is that adoption will be fairly low, whereas I think most of the
> > time one would want inlined storage layout.
> >
> > Thanks
> >
> >
> > On Tue, Feb 3, 2015 at 11:29 AM, Volker Simonis <
> volker.simonis at gmail.com>
> > wrote:
> >
> >> Hi Vitaly,
> >>
> >> for PackedObjects/ObjectLayout you need to specially annotate the
> >> classes and/or fields which you want to allocate "inline". Once you've
> >> done that you have no choice with the PackedObjects approach.
> >> ObjectLayout is a little special here, because it can run with any
> >> Java VM in which case it will still use the default reference model.
> >> But it can potentially be optimized by some VM's to provide the flat
> >> object layout. I expect these optimizations to be controllable by a
> >> command line option.
> >>
> >> With the "Value Types for Java" [1] approach you'll have the
> >> possiblitly to express the behavior right in Java like in the
> >> following example from [1]:
> >>
> >> final __ByValue class Point {
> >> static Point origin = __MakeValue(0, 0);
> >>
> >> I think the default will always be "reference semantics" in Java but
> >> with various degrees of freedom to optionally choose value semantics.
> >>
> >> Regards,
> >> Volker
> >>
> >> [1] http://cr.openjdk.java.net/~jrose/values/values-0.html
> >>
> >> On Mon, Feb 2, 2015 at 9:19 PM, Vitaly Davidovich <vitalyd at gmail.com>
> >> wrote:
> >> > Volker (or anyone else for that matter),
> >> >
> >> > Just curious -- do you envision "inline" layout of objects as
> something
> >> one
> >> > would have to opt-in or as the default layout for all objects in a
> heap?
> >> It
> >> > seems like this should be the default (assuming zero to minimal
> overhead
> >> for
> >> > loading the references) as I think wanting "out of line" allocations
> is
> >> more
> >> > rare.
> >> >
> >> > Thanks
> >> >
> >> > On Mon, Feb 2, 2015 at 2:06 PM, Volker Simonis <
> volker.simonis at gmail.com
> >> >
> >> > wrote:
> >> >>
> >> >> Hi Brian,
> >> >>
> >> >> thanks a lot for your detailed answer and apologies for the late
> reply
> >> >> (I was a little distracted by FOSDEM :)
> >> >>
> >> >> All your comments have been clear and reasonable and are much
> >> >> appreciated. Please find my additional answers inline:
> >> >>
> >> >> On Thu, Jan 29, 2015 at 6:05 PM, Brian Goetz <brian.goetz at oracle.com
> >
> >> >> wrote:
> >> >> >> Question: is JEP 169 still under active development or has it been
> >> >> >> merged into the more general "Value types for Java" proposal
> below?
> >> >> >
> >> >> >
> >> >> > It has been merged into the more general Value Types for Java
> >> proposal.
> >> >> >
> >> >>
> >> >> Then maybe this JEP should be closed to avoid further confusion?
> >> >>
> >> >> >> The "Value types for Java" approach clearly seems to be the most
> >> >> >> general but also the most complex proposal.
> >> >> >
> >> >> >
> >> >> > For some meanings of "complex".  It is certainly the most intrusive
> >> and
> >> >> > large; new bytecodes, new type signatures.  But from a user-model
> >> >> > perspective, value types are actually fairly simple.
> >> >> >
> >> >> >> It's out of scope for Java
> >> >> >> 9 and still questionable for Java 10 and above. The "PackedObject"
> >> and
> >> >> >> "ObjectLayout" approaches are clearly simpler and more limited in
> >> >> >> scope as they only concentrate on better object layout.
> >> >> >
> >> >> >
> >> >> > To your list, I'd add: Project Panama, the sister project to
> Valhalla.
> >> >> > Panama focuses on interop with native code and data, including
> layout
> >> >> > specification.  A key goal of Packed was to be able to access
> off-heap
> >> >> > native data in its native format, rather than marshalling it across
> >> the
> >> >> > JNI
> >> >> > boundary.  Panama is focused on this problem as well, but aims to
> >> treat
> >> >> > it
> >> >> > as a separate problem from Java object layout, resulting in what we
> >> >> > believe
> >> >> > to be a cleaner decomposition of the two concerns.
> >> >> >
> >> >>
> >> >> Your right. I somehow missed to look at Panama more deeply because I
> >> >> always thought it is only about FFI. John Rose nicely explains the
> >> >> various parts of Panama in this mail
> >> >>
> >>
> http://mail.openjdk.java.net/pipermail/panama-dev/2014-October/000042.html
> >> >> where he also mentions the intention of Panama to create new flatter
> >> >> data layouts in the Heap and the relation of Panama to PackedObjects
> >> >> and ObjectLayout.
> >> >>
> >> >> > Packed is an interesting mix of memory density (object embedding
> and
> >> >> > packed
> >> >> > arrays) and native interop.  But mixing the two goals also has
> costs;
> >> >> > our
> >> >> > approach is to separate them into orthogonal concerns, and we think
> >> that
> >> >> > Valhalla and Panama do just that.  So in many ways, while a larger
> >> >> > project,
> >> >> > the combination of Valhalla+Panama addresses the problem that
> Packed
> >> >> > did, in
> >> >> > a cleaner way.
> >> >> >
> >> >> >> Question: is there a chance to get a some sort of Java-only but
> >> >> >> transparently optimizable structure package like "ObjectLayout"
> into
> >> >> >> Java early (i.e. Java 9)?
> >> >> >
> >> >> >
> >> >> > It would depend on a lot of things -- including the level of
> readiness
> >> >> > of
> >> >> > the design and implementation, and the overlap with anticipated
> future
> >> >> > features.  We've reviewed some of the early design of ObjectLayout
> and
> >> >> > provided feedback to the projects architects; currently, I think
> it's
> >> in
> >> >> > the
> >> >> > "promising exploration" stage, but I think multiple rounds of
> >> >> > simplification
> >> >> > are needed before it is ready to be considered for "everybody's
> Java."
> >> >> > But
> >> >> > if the choice is to push something that's not ready into 9, or to
> wait
> >> >> > longer -- there's not actually a choice to be made there.
> >> >> >
> >> >> > I appreciate the desire to "get something you can use now", but we
> >> have
> >> >> > to
> >> >> > be prepared to support whatever we push into Java for the next 20
> >> years,
> >> >> > and
> >> >> > deal with the additional constraints it generates -- which can be
> an
> >> >> > enormous cost.  (Even thought the direct cost is mostly borne by
> >> Oracle,
> >> >> > the
> >> >> > indirect cost is borne by everyone, in the form of slower progress
> on
> >> >> > everything else.)  So I am very wary of the motivation of "well,
> >> >> > something
> >> >> > better is coming, but this works now, so can we push it in?"  I'd
> >> prefer
> >> >> > to
> >> >> > focus on answering whether this is right thing for Java for the
> next
> >> 20
> >> >> > years.
> >> >> >
> >> >> >> In my eyes this wouldn't contradict with a more general solution
> like
> >> >> >> the one proposed in the "Value types for Java" approach while
> still
> >> >> >> offering quite significant performance improvements for quite a
> big
> >> >> >> range of problems.
> >> >> >
> >> >> >
> >> >> > The goals of the ObjectLayout effort has overlap with, but also
> >> differs
> >> >> > from, the goals of Valhalla.  And herein is the problem; neither
> >> >> > generalizes
> >> >> > the other, and I don't think we do the user base a great favor by
> >> >> > pursuing
> >> >> > two separate neither-coincident-nor-orthogonal approaches.  I
> suspect,
> >> >> > though, that after a few rounds of simplification, ObjectLayout
> could
> >> >> > morph
> >> >> > into something that fit either coincidently or orthogonally with
> the
> >> >> > Valhalla work -- which would be great.  But, as you know, our
> >> resources
> >> >> > are
> >> >> > limited, so we (Oracle) can't really afford to invest in both.  And
> >> such
> >> >> > simplification takes time -- getting to that "aha" moment when you
> >> >> > realize
> >> >> > you can simplify something is generally an incompressible process.
> >> >> >
> >> >> >> Question: what would be the right place to propose something like
> the
> >> >> >> "ObjectLayout" library for Java 9/10? Would that fit within the
> >> >> >> umbrella of the Valhalla project or would it be done within its
> own
> >> >> >> project / under it's own JEP?
> >> >> >
> >> >> >
> >> >> > Suggesting a version number at this point would be putting the cart
> >> >> > before
> >> >> > the horse (you'll note that we've not even proposed a version
> number
> >> for
> >> >> > Valhalla; the closest we've gotten to that is "after 9".)
> >> >> >
> >> >> > OpenJDK Projects are a tool for building a community around a body
> of
> >> >> > work;
> >> >> > JEPs are a project-management tool for defining, scoping, and
> tracking
> >> >> > the
> >> >> > progress of a feature.  Given where OL is, it would be reasonable
> to
> >> >> > start a
> >> >> > Project, which would become the nexus of collaboration that could
> >> >> > eventually
> >> >> > produce a JEP.
> >> >> >
> >> >>
> >> >> That sounds reasonable. I'll speak with the ObjectLayout people to
> >> >> hear what they think about starting a new OpenJDK project.
> >> >>
> >> >> And after all you've said before, I also came to the conclusion that
> >> >> investigating ways of new in-heap object layouts seem to be better of
> >> >> in the Panama project. So I'll also ask there what they think about
> >> >> it. Maybe ObjectLayout could become part of Panama or maybe we could
> >> >> just start a new subproject of Panama with the same/similar goals.
> >> >>
> >> >> > Hope this helps,
> >> >> > -Brian
> >> >>
> >> >> It really did!
> >> >>
> >> >> Thanks again,
> >> >> Volker
> >> >> _______________________________________________
> >> >> mlvm-dev mailing list
> >> >> mlvm-dev at openjdk.java.net
> >> >> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
> >> >
> >> >
> >> >
> >> > _______________________________________________
> >> > mlvm-dev mailing list
> >> > mlvm-dev at openjdk.java.net
> >> > http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
> >> >
> >> _______________________________________________
> >> mlvm-dev mailing list
> >> mlvm-dev at openjdk.java.net
> >> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
> >>
>

From gil at azulsystems.com  Tue Feb  3 16:58:23 2015
From: gil at azulsystems.com (Gil Tene)
Date: Tue, 3 Feb 2015 16:58:23 +0000
Subject: What's the status of / relation between "JEP 169: Value
	Objects"	/ "Value Types for Java" / "Object Layout"
In-Reply-To: <CAHjP37GhZYk61Z12L6Yj24YCwmyaao2oWJGRva6+gw1wGzGpOQ@mail.gmail.com>
References: <CA+3eh122g3Ax+-PCL3EvX3V-cQkorva2j_c42_i4uRLOqw42Fg@mail.gmail.com>
	<54CA6853.2060601@oracle.com>
	<CA+3eh13pg6+ZziNuzLNdPSc8AkA4o1N9Xx0kGYFNm7TV3t9yzQ@mail.gmail.com>
	<CAHjP37HO8b3kgaWy4pVXpY3ky=+ebhLHBafMeNnZ-QooA9bXkA@mail.gmail.com>
	<CA+3eh135AOaiHoFQaA8Uc7WFsG-rHeeF30MBHw52J=fev9c1CQ@mail.gmail.com>
	<CAHjP37GhZYk61Z12L6Yj24YCwmyaao2oWJGRva6+gw1wGzGpOQ@mail.gmail.com>
Message-ID: <505C1729-7E97-4D3B-8760-A3AF0BB76376@azulsystems.com>

A couple of point here, specific to org.ObjectLayout (http://objectlayout.org):

Declaration:
The ObjectLayout @Intrinsic (http://objectlayout.org) annotation is used for declaring what you refer to as "inline" objects. It is specifically not intended to be a layout control directive, but an optimization hint. Whether or not a JVM "inlines" the intrinsic object within the containing one, and how/where that "inlining" happens becomes a JVM-specific implementation concern, and no a semantic one.

Field initialization:

Implicit, undeclared choices to "inline" all final referenced fields fail very quickly when attempted in practice. E.g. final fields can (and often will) be set to refer to pre-existing-at-construction-time objects, which are (by definition) impossible to "inline". In addition, there are many common uses final reference fields where "inlining" is no possible because the actual object size of the referred-to object is not a global constant (e.g. it will be set to a construction-time or parameter-based choice of subclass).

We've given Intrinsic Object initialization a lot of thought in org.ObjectLayout. The dedicated initialization API is there to assure several things, including exact-type (the field's specific declared type) choice. It could turn into a less-verbose version in some future JDK if language support was added (e.g. to avoid mentioning the needed-only-to-conform-with-syntax things like "this", and the field name in the constructWithin() call), but I expect the semantics to need to be similar even if the syntax was made less verbose.

-- Gil.

> On Feb 3, 2015, at 8:40 AM, Vitaly Davidovich <vitalyd at gmail.com> wrote:
> 
> Hi Volker,
> 
> Sorry, I may have been unclear in my question.  As you say, ObjectLayout
> requires that you annotate the fields that you'd like inlined and then also
> use special API to construct those objects.  I'm wondering whether,
> instead, all private final fields are automatically inlined, and only cases
> where you'd like to layout the field out-of-band would require annotation.
> This would be controlled via a cmdline flag, as you say, similar to perhaps
> how compressed oops are enabled (or not).  Note that I'm talking about
> purely layout of reference types, not value types.
> 
> The "concern" with having to explicitly annotate and use dedicated APIs to
> opt-in is that adoption will be fairly low, whereas I think most of the
> time one would want inlined storage layout.
> 
> Thanks
> 
> 
> On Tue, Feb 3, 2015 at 11:29 AM, Volker Simonis <volker.simonis at gmail.com>
> wrote:
> 
>> Hi Vitaly,
>> 
>> for PackedObjects/ObjectLayout you need to specially annotate the
>> classes and/or fields which you want to allocate "inline". Once you've
>> done that you have no choice with the PackedObjects approach.
>> ObjectLayout is a little special here, because it can run with any
>> Java VM in which case it will still use the default reference model.
>> But it can potentially be optimized by some VM's to provide the flat
>> object layout. I expect these optimizations to be controllable by a
>> command line option.
>> 
>> With the "Value Types for Java" [1] approach you'll have the
>> possiblitly to express the behavior right in Java like in the
>> following example from [1]:
>> 
>> final __ByValue class Point {
>> static Point origin = __MakeValue(0, 0);
>> 
>> I think the default will always be "reference semantics" in Java but
>> with various degrees of freedom to optionally choose value semantics.
>> 
>> Regards,
>> Volker
>> 
>> [1] http://cr.openjdk.java.net/~jrose/values/values-0.html
>> 
>> On Mon, Feb 2, 2015 at 9:19 PM, Vitaly Davidovich <vitalyd at gmail.com>
>> wrote:
>>> Volker (or anyone else for that matter),
>>> 
>>> Just curious -- do you envision "inline" layout of objects as something
>> one
>>> would have to opt-in or as the default layout for all objects in a heap?
>> It
>>> seems like this should be the default (assuming zero to minimal overhead
>> for
>>> loading the references) as I think wanting "out of line" allocations is
>> more
>>> rare.
>>> 
>>> Thanks
>>> 
>>> On Mon, Feb 2, 2015 at 2:06 PM, Volker Simonis <volker.simonis at gmail.com
>>> 
>>> wrote:
>>>> 
>>>> Hi Brian,
>>>> 
>>>> thanks a lot for your detailed answer and apologies for the late reply
>>>> (I was a little distracted by FOSDEM :)
>>>> 
>>>> All your comments have been clear and reasonable and are much
>>>> appreciated. Please find my additional answers inline:
>>>> 
>>>> On Thu, Jan 29, 2015 at 6:05 PM, Brian Goetz <brian.goetz at oracle.com>
>>>> wrote:
>>>>>> Question: is JEP 169 still under active development or has it been
>>>>>> merged into the more general "Value types for Java" proposal below?
>>>>> 
>>>>> 
>>>>> It has been merged into the more general Value Types for Java
>> proposal.
>>>>> 
>>>> 
>>>> Then maybe this JEP should be closed to avoid further confusion?
>>>> 
>>>>>> The "Value types for Java" approach clearly seems to be the most
>>>>>> general but also the most complex proposal.
>>>>> 
>>>>> 
>>>>> For some meanings of "complex".  It is certainly the most intrusive
>> and
>>>>> large; new bytecodes, new type signatures.  But from a user-model
>>>>> perspective, value types are actually fairly simple.
>>>>> 
>>>>>> It's out of scope for Java
>>>>>> 9 and still questionable for Java 10 and above. The "PackedObject"
>> and
>>>>>> "ObjectLayout" approaches are clearly simpler and more limited in
>>>>>> scope as they only concentrate on better object layout.
>>>>> 
>>>>> 
>>>>> To your list, I'd add: Project Panama, the sister project to Valhalla.
>>>>> Panama focuses on interop with native code and data, including layout
>>>>> specification.  A key goal of Packed was to be able to access off-heap
>>>>> native data in its native format, rather than marshalling it across
>> the
>>>>> JNI
>>>>> boundary.  Panama is focused on this problem as well, but aims to
>> treat
>>>>> it
>>>>> as a separate problem from Java object layout, resulting in what we
>>>>> believe
>>>>> to be a cleaner decomposition of the two concerns.
>>>>> 
>>>> 
>>>> Your right. I somehow missed to look at Panama more deeply because I
>>>> always thought it is only about FFI. John Rose nicely explains the
>>>> various parts of Panama in this mail
>>>> 
>> http://mail.openjdk.java.net/pipermail/panama-dev/2014-October/000042.html
>>>> where he also mentions the intention of Panama to create new flatter
>>>> data layouts in the Heap and the relation of Panama to PackedObjects
>>>> and ObjectLayout.
>>>> 
>>>>> Packed is an interesting mix of memory density (object embedding and
>>>>> packed
>>>>> arrays) and native interop.  But mixing the two goals also has costs;
>>>>> our
>>>>> approach is to separate them into orthogonal concerns, and we think
>> that
>>>>> Valhalla and Panama do just that.  So in many ways, while a larger
>>>>> project,
>>>>> the combination of Valhalla+Panama addresses the problem that Packed
>>>>> did, in
>>>>> a cleaner way.
>>>>> 
>>>>>> Question: is there a chance to get a some sort of Java-only but
>>>>>> transparently optimizable structure package like "ObjectLayout" into
>>>>>> Java early (i.e. Java 9)?
>>>>> 
>>>>> 
>>>>> It would depend on a lot of things -- including the level of readiness
>>>>> of
>>>>> the design and implementation, and the overlap with anticipated future
>>>>> features.  We've reviewed some of the early design of ObjectLayout and
>>>>> provided feedback to the projects architects; currently, I think it's
>> in
>>>>> the
>>>>> "promising exploration" stage, but I think multiple rounds of
>>>>> simplification
>>>>> are needed before it is ready to be considered for "everybody's Java."
>>>>> But
>>>>> if the choice is to push something that's not ready into 9, or to wait
>>>>> longer -- there's not actually a choice to be made there.
>>>>> 
>>>>> I appreciate the desire to "get something you can use now", but we
>> have
>>>>> to
>>>>> be prepared to support whatever we push into Java for the next 20
>> years,
>>>>> and
>>>>> deal with the additional constraints it generates -- which can be an
>>>>> enormous cost.  (Even thought the direct cost is mostly borne by
>> Oracle,
>>>>> the
>>>>> indirect cost is borne by everyone, in the form of slower progress on
>>>>> everything else.)  So I am very wary of the motivation of "well,
>>>>> something
>>>>> better is coming, but this works now, so can we push it in?"  I'd
>> prefer
>>>>> to
>>>>> focus on answering whether this is right thing for Java for the next
>> 20
>>>>> years.
>>>>> 
>>>>>> In my eyes this wouldn't contradict with a more general solution like
>>>>>> the one proposed in the "Value types for Java" approach while still
>>>>>> offering quite significant performance improvements for quite a big
>>>>>> range of problems.
>>>>> 
>>>>> 
>>>>> The goals of the ObjectLayout effort has overlap with, but also
>> differs
>>>>> from, the goals of Valhalla.  And herein is the problem; neither
>>>>> generalizes
>>>>> the other, and I don't think we do the user base a great favor by
>>>>> pursuing
>>>>> two separate neither-coincident-nor-orthogonal approaches.  I suspect,
>>>>> though, that after a few rounds of simplification, ObjectLayout could
>>>>> morph
>>>>> into something that fit either coincidently or orthogonally with the
>>>>> Valhalla work -- which would be great.  But, as you know, our
>> resources
>>>>> are
>>>>> limited, so we (Oracle) can't really afford to invest in both.  And
>> such
>>>>> simplification takes time -- getting to that "aha" moment when you
>>>>> realize
>>>>> you can simplify something is generally an incompressible process.
>>>>> 
>>>>>> Question: what would be the right place to propose something like the
>>>>>> "ObjectLayout" library for Java 9/10? Would that fit within the
>>>>>> umbrella of the Valhalla project or would it be done within its own
>>>>>> project / under it's own JEP?
>>>>> 
>>>>> 
>>>>> Suggesting a version number at this point would be putting the cart
>>>>> before
>>>>> the horse (you'll note that we've not even proposed a version number
>> for
>>>>> Valhalla; the closest we've gotten to that is "after 9".)
>>>>> 
>>>>> OpenJDK Projects are a tool for building a community around a body of
>>>>> work;
>>>>> JEPs are a project-management tool for defining, scoping, and tracking
>>>>> the
>>>>> progress of a feature.  Given where OL is, it would be reasonable to
>>>>> start a
>>>>> Project, which would become the nexus of collaboration that could
>>>>> eventually
>>>>> produce a JEP.
>>>>> 
>>>> 
>>>> That sounds reasonable. I'll speak with the ObjectLayout people to
>>>> hear what they think about starting a new OpenJDK project.
>>>> 
>>>> And after all you've said before, I also came to the conclusion that
>>>> investigating ways of new in-heap object layouts seem to be better of
>>>> in the Panama project. So I'll also ask there what they think about
>>>> it. Maybe ObjectLayout could become part of Panama or maybe we could
>>>> just start a new subproject of Panama with the same/similar goals.
>>>> 
>>>>> Hope this helps,
>>>>> -Brian
>>>> 
>>>> It really did!
>>>> 
>>>> Thanks again,
>>>> Volker
>>>> _______________________________________________
>>>> mlvm-dev mailing list
>>>> mlvm-dev at openjdk.java.net
>>>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>>> 
>>> 
>>> 
>>> _______________________________________________
>>> mlvm-dev mailing list
>>> mlvm-dev at openjdk.java.net
>>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>>> 
>> _______________________________________________
>> mlvm-dev mailing list
>> mlvm-dev at openjdk.java.net
>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>> 


From vitalyd at gmail.com  Tue Feb  3 17:13:08 2015
From: vitalyd at gmail.com (Vitaly Davidovich)
Date: Tue, 3 Feb 2015 12:13:08 -0500
Subject: What's the status of / relation between "JEP 169: Value Objects"
	/ "Value Types for Java" / "Object Layout"
In-Reply-To: <505C1729-7E97-4D3B-8760-A3AF0BB76376@azulsystems.com>
References: <CA+3eh122g3Ax+-PCL3EvX3V-cQkorva2j_c42_i4uRLOqw42Fg@mail.gmail.com>
	<54CA6853.2060601@oracle.com>
	<CA+3eh13pg6+ZziNuzLNdPSc8AkA4o1N9Xx0kGYFNm7TV3t9yzQ@mail.gmail.com>
	<CAHjP37HO8b3kgaWy4pVXpY3ky=+ebhLHBafMeNnZ-QooA9bXkA@mail.gmail.com>
	<CA+3eh135AOaiHoFQaA8Uc7WFsG-rHeeF30MBHw52J=fev9c1CQ@mail.gmail.com>
	<CAHjP37GhZYk61Z12L6Yj24YCwmyaao2oWJGRva6+gw1wGzGpOQ@mail.gmail.com>
	<505C1729-7E97-4D3B-8760-A3AF0BB76376@azulsystems.com>
Message-ID: <CAHjP37F5SQfH41_FeDmB_0__887HqpcLMRUdVPuhFvMh9EA-EA@mail.gmail.com>

Gil, not sure if you saw my reply to Volker, but I agree -- I was simply
asking why request this optimization via explicit syntax and not do it
automatically in the runtime (with all the same restrictions, caveats, fine
print, etc).

On Tue, Feb 3, 2015 at 11:58 AM, Gil Tene <gil at azulsystems.com> wrote:

> A couple of point here, specific to org.ObjectLayout (
> http://objectlayout.org):
>
> Declaration:
> The ObjectLayout @Intrinsic (http://objectlayout.org) annotation is used
> for declaring what you refer to as "inline" objects. It is specifically not
> intended to be a layout control directive, but an optimization hint.
> Whether or not a JVM "inlines" the intrinsic object within the containing
> one, and how/where that "inlining" happens becomes a JVM-specific
> implementation concern, and no a semantic one.
>
> Field initialization:
>
> Implicit, undeclared choices to "inline" all final referenced fields fail
> very quickly when attempted in practice. E.g. final fields can (and often
> will) be set to refer to pre-existing-at-construction-time objects, which
> are (by definition) impossible to "inline". In addition, there are many
> common uses final reference fields where "inlining" is no possible because
> the actual object size of the referred-to object is not a global constant
> (e.g. it will be set to a construction-time or parameter-based choice of
> subclass).
>
> We've given Intrinsic Object initialization a lot of thought in
> org.ObjectLayout. The dedicated initialization API is there to assure
> several things, including exact-type (the field's specific declared type)
> choice. It could turn into a less-verbose version in some future JDK if
> language support was added (e.g. to avoid mentioning the
> needed-only-to-conform-with-syntax things like "this", and the field name
> in the constructWithin() call), but I expect the semantics to need to be
> similar even if the syntax was made less verbose.
>
> -- Gil.
>
> > On Feb 3, 2015, at 8:40 AM, Vitaly Davidovich <vitalyd at gmail.com> wrote:
> >
> > Hi Volker,
> >
> > Sorry, I may have been unclear in my question.  As you say, ObjectLayout
> > requires that you annotate the fields that you'd like inlined and then
> also
> > use special API to construct those objects.  I'm wondering whether,
> > instead, all private final fields are automatically inlined, and only
> cases
> > where you'd like to layout the field out-of-band would require
> annotation.
> > This would be controlled via a cmdline flag, as you say, similar to
> perhaps
> > how compressed oops are enabled (or not).  Note that I'm talking about
> > purely layout of reference types, not value types.
> >
> > The "concern" with having to explicitly annotate and use dedicated APIs
> to
> > opt-in is that adoption will be fairly low, whereas I think most of the
> > time one would want inlined storage layout.
> >
> > Thanks
> >
> >
> > On Tue, Feb 3, 2015 at 11:29 AM, Volker Simonis <
> volker.simonis at gmail.com>
> > wrote:
> >
> >> Hi Vitaly,
> >>
> >> for PackedObjects/ObjectLayout you need to specially annotate the
> >> classes and/or fields which you want to allocate "inline". Once you've
> >> done that you have no choice with the PackedObjects approach.
> >> ObjectLayout is a little special here, because it can run with any
> >> Java VM in which case it will still use the default reference model.
> >> But it can potentially be optimized by some VM's to provide the flat
> >> object layout. I expect these optimizations to be controllable by a
> >> command line option.
> >>
> >> With the "Value Types for Java" [1] approach you'll have the
> >> possiblitly to express the behavior right in Java like in the
> >> following example from [1]:
> >>
> >> final __ByValue class Point {
> >> static Point origin = __MakeValue(0, 0);
> >>
> >> I think the default will always be "reference semantics" in Java but
> >> with various degrees of freedom to optionally choose value semantics.
> >>
> >> Regards,
> >> Volker
> >>
> >> [1] http://cr.openjdk.java.net/~jrose/values/values-0.html
> >>
> >> On Mon, Feb 2, 2015 at 9:19 PM, Vitaly Davidovich <vitalyd at gmail.com>
> >> wrote:
> >>> Volker (or anyone else for that matter),
> >>>
> >>> Just curious -- do you envision "inline" layout of objects as something
> >> one
> >>> would have to opt-in or as the default layout for all objects in a
> heap?
> >> It
> >>> seems like this should be the default (assuming zero to minimal
> overhead
> >> for
> >>> loading the references) as I think wanting "out of line" allocations is
> >> more
> >>> rare.
> >>>
> >>> Thanks
> >>>
> >>> On Mon, Feb 2, 2015 at 2:06 PM, Volker Simonis <
> volker.simonis at gmail.com
> >>>
> >>> wrote:
> >>>>
> >>>> Hi Brian,
> >>>>
> >>>> thanks a lot for your detailed answer and apologies for the late reply
> >>>> (I was a little distracted by FOSDEM :)
> >>>>
> >>>> All your comments have been clear and reasonable and are much
> >>>> appreciated. Please find my additional answers inline:
> >>>>
> >>>> On Thu, Jan 29, 2015 at 6:05 PM, Brian Goetz <brian.goetz at oracle.com>
> >>>> wrote:
> >>>>>> Question: is JEP 169 still under active development or has it been
> >>>>>> merged into the more general "Value types for Java" proposal below?
> >>>>>
> >>>>>
> >>>>> It has been merged into the more general Value Types for Java
> >> proposal.
> >>>>>
> >>>>
> >>>> Then maybe this JEP should be closed to avoid further confusion?
> >>>>
> >>>>>> The "Value types for Java" approach clearly seems to be the most
> >>>>>> general but also the most complex proposal.
> >>>>>
> >>>>>
> >>>>> For some meanings of "complex".  It is certainly the most intrusive
> >> and
> >>>>> large; new bytecodes, new type signatures.  But from a user-model
> >>>>> perspective, value types are actually fairly simple.
> >>>>>
> >>>>>> It's out of scope for Java
> >>>>>> 9 and still questionable for Java 10 and above. The "PackedObject"
> >> and
> >>>>>> "ObjectLayout" approaches are clearly simpler and more limited in
> >>>>>> scope as they only concentrate on better object layout.
> >>>>>
> >>>>>
> >>>>> To your list, I'd add: Project Panama, the sister project to
> Valhalla.
> >>>>> Panama focuses on interop with native code and data, including layout
> >>>>> specification.  A key goal of Packed was to be able to access
> off-heap
> >>>>> native data in its native format, rather than marshalling it across
> >> the
> >>>>> JNI
> >>>>> boundary.  Panama is focused on this problem as well, but aims to
> >> treat
> >>>>> it
> >>>>> as a separate problem from Java object layout, resulting in what we
> >>>>> believe
> >>>>> to be a cleaner decomposition of the two concerns.
> >>>>>
> >>>>
> >>>> Your right. I somehow missed to look at Panama more deeply because I
> >>>> always thought it is only about FFI. John Rose nicely explains the
> >>>> various parts of Panama in this mail
> >>>>
> >>
> http://mail.openjdk.java.net/pipermail/panama-dev/2014-October/000042.html
> >>>> where he also mentions the intention of Panama to create new flatter
> >>>> data layouts in the Heap and the relation of Panama to PackedObjects
> >>>> and ObjectLayout.
> >>>>
> >>>>> Packed is an interesting mix of memory density (object embedding and
> >>>>> packed
> >>>>> arrays) and native interop.  But mixing the two goals also has costs;
> >>>>> our
> >>>>> approach is to separate them into orthogonal concerns, and we think
> >> that
> >>>>> Valhalla and Panama do just that.  So in many ways, while a larger
> >>>>> project,
> >>>>> the combination of Valhalla+Panama addresses the problem that Packed
> >>>>> did, in
> >>>>> a cleaner way.
> >>>>>
> >>>>>> Question: is there a chance to get a some sort of Java-only but
> >>>>>> transparently optimizable structure package like "ObjectLayout" into
> >>>>>> Java early (i.e. Java 9)?
> >>>>>
> >>>>>
> >>>>> It would depend on a lot of things -- including the level of
> readiness
> >>>>> of
> >>>>> the design and implementation, and the overlap with anticipated
> future
> >>>>> features.  We've reviewed some of the early design of ObjectLayout
> and
> >>>>> provided feedback to the projects architects; currently, I think it's
> >> in
> >>>>> the
> >>>>> "promising exploration" stage, but I think multiple rounds of
> >>>>> simplification
> >>>>> are needed before it is ready to be considered for "everybody's
> Java."
> >>>>> But
> >>>>> if the choice is to push something that's not ready into 9, or to
> wait
> >>>>> longer -- there's not actually a choice to be made there.
> >>>>>
> >>>>> I appreciate the desire to "get something you can use now", but we
> >> have
> >>>>> to
> >>>>> be prepared to support whatever we push into Java for the next 20
> >> years,
> >>>>> and
> >>>>> deal with the additional constraints it generates -- which can be an
> >>>>> enormous cost.  (Even thought the direct cost is mostly borne by
> >> Oracle,
> >>>>> the
> >>>>> indirect cost is borne by everyone, in the form of slower progress on
> >>>>> everything else.)  So I am very wary of the motivation of "well,
> >>>>> something
> >>>>> better is coming, but this works now, so can we push it in?"  I'd
> >> prefer
> >>>>> to
> >>>>> focus on answering whether this is right thing for Java for the next
> >> 20
> >>>>> years.
> >>>>>
> >>>>>> In my eyes this wouldn't contradict with a more general solution
> like
> >>>>>> the one proposed in the "Value types for Java" approach while still
> >>>>>> offering quite significant performance improvements for quite a big
> >>>>>> range of problems.
> >>>>>
> >>>>>
> >>>>> The goals of the ObjectLayout effort has overlap with, but also
> >> differs
> >>>>> from, the goals of Valhalla.  And herein is the problem; neither
> >>>>> generalizes
> >>>>> the other, and I don't think we do the user base a great favor by
> >>>>> pursuing
> >>>>> two separate neither-coincident-nor-orthogonal approaches.  I
> suspect,
> >>>>> though, that after a few rounds of simplification, ObjectLayout could
> >>>>> morph
> >>>>> into something that fit either coincidently or orthogonally with the
> >>>>> Valhalla work -- which would be great.  But, as you know, our
> >> resources
> >>>>> are
> >>>>> limited, so we (Oracle) can't really afford to invest in both.  And
> >> such
> >>>>> simplification takes time -- getting to that "aha" moment when you
> >>>>> realize
> >>>>> you can simplify something is generally an incompressible process.
> >>>>>
> >>>>>> Question: what would be the right place to propose something like
> the
> >>>>>> "ObjectLayout" library for Java 9/10? Would that fit within the
> >>>>>> umbrella of the Valhalla project or would it be done within its own
> >>>>>> project / under it's own JEP?
> >>>>>
> >>>>>
> >>>>> Suggesting a version number at this point would be putting the cart
> >>>>> before
> >>>>> the horse (you'll note that we've not even proposed a version number
> >> for
> >>>>> Valhalla; the closest we've gotten to that is "after 9".)
> >>>>>
> >>>>> OpenJDK Projects are a tool for building a community around a body of
> >>>>> work;
> >>>>> JEPs are a project-management tool for defining, scoping, and
> tracking
> >>>>> the
> >>>>> progress of a feature.  Given where OL is, it would be reasonable to
> >>>>> start a
> >>>>> Project, which would become the nexus of collaboration that could
> >>>>> eventually
> >>>>> produce a JEP.
> >>>>>
> >>>>
> >>>> That sounds reasonable. I'll speak with the ObjectLayout people to
> >>>> hear what they think about starting a new OpenJDK project.
> >>>>
> >>>> And after all you've said before, I also came to the conclusion that
> >>>> investigating ways of new in-heap object layouts seem to be better of
> >>>> in the Panama project. So I'll also ask there what they think about
> >>>> it. Maybe ObjectLayout could become part of Panama or maybe we could
> >>>> just start a new subproject of Panama with the same/similar goals.
> >>>>
> >>>>> Hope this helps,
> >>>>> -Brian
> >>>>
> >>>> It really did!
> >>>>
> >>>> Thanks again,
> >>>> Volker
> >>>> _______________________________________________
> >>>> mlvm-dev mailing list
> >>>> mlvm-dev at openjdk.java.net
> >>>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
> >>>
> >>>
> >>>
> >>> _______________________________________________
> >>> mlvm-dev mailing list
> >>> mlvm-dev at openjdk.java.net
> >>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
> >>>
> >> _______________________________________________
> >> mlvm-dev mailing list
> >> mlvm-dev at openjdk.java.net
> >> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
> >>
>
>

From peter.levart at gmail.com  Tue Feb  3 18:30:50 2015
From: peter.levart at gmail.com (Peter Levart)
Date: Tue, 03 Feb 2015 19:30:50 +0100
Subject: experiences with prototype
Message-ID: <54D113DA.9060509@gmail.com>

Hi,

I wanted to see what it would take to any-fy a basic subset of java 
collections library. I thought I would start with basic interfaces 
(Iterator, Iterable, Collection, List, ...) and then add basic 
implementations. I quickly learned that any-fying is a viral process. 
Any dependency that uses a generic type variable from any-fied class has 
to be any-fied too. So I had to make a cut somewhere or I would be 
any-fying the whole JDK (exaggerating) :-o ... I removed the methods 
that have connection to Streams API (stream(), splitterator(), ...) and 
parallel methods (FJPool) and got a manageable isolated subset. I though 
I was mostly finished with basic interfaces (which are not just 
interfaces as from Java 8 on they contain implementation too) as I fixed 
the last compilation error when javac crashed.

Since I can't easily isolate the part that crashes javac, here are the 
sources (19 classes) that I'm trying to compile in one go:

http://cr.openjdk.java.net/~plevart/misc/valhala-hacks/javany-src.jar

Perhaps Maurizio could look at it.

In general, the process of any-fying up to this point went mostly 
straightforward. A couple of pain points I encountered are:


__WhereVal(T)/__WhereRef(T) only works when 'T' is a type variable from 
enclosing generic class. It doesn't work for example with type variables 
from generic methods.


The Object-like methods (equals, hashCode, toString) work for <any T> 
typed receiver, but not when enclosed in __WhereVal(T) { ... } block. 
Why this restriction?


Especially tricky for any-fying was the following AbstractCollection method:

public abstract class AbstractCollection<any E> implements Collection<E> {
     public <any T> T[] toArray(T[] a)

...as it must support things like:

Collection<int> intCol = ...;

long[] longArray = intCol.toArray(new long[0]);

The tricky part was how to code a conversion from values of  type E to 
values of type T when E and T are both any type variables. The 
conversions one would like to support are: identity, widening, boxing, 
unboxing. These tricks are all hacked in javany.util.Any class If 
anybody is interested.

Currently there's no support in prototype to solve the List.remove(int) 
problem, so in my experiment I just renamed the method to removeByIndex.

All in all I was surprised with the level of completeness in the 
prototype as early as this. I'll try to go further with this anyfication 
experiment when the compiler lets me...


Regards, Peter


From maurizio.cimadamore at oracle.com  Tue Feb  3 18:46:42 2015
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Tue, 03 Feb 2015 18:46:42 +0000
Subject: experiences with prototype
In-Reply-To: <54D113DA.9060509@gmail.com>
References: <54D113DA.9060509@gmail.com>
Message-ID: <54D11792.3090504@oracle.com>

Thanks Peter,
this is great feedback; regarding  the support for WhereRef/Val(Z) where 
Z is a method type-variable, I'm not sure it will be done in this 
current form/iteration. I've explored this, and looks not that 
straightforward to implement; given we don't want to support __Where in 
the long run, we might as well, replace the whole thing altogether.

The Object methods on val T seems a plain bug.

I will also investigate on the crash you are getting...

Regarding your choices for any-fying collection, I think they makes most 
sense - this is the road that both Brian and I ended up following in 
separate attempts after few failed attempt; the Stream string is quite a 
big one to pull :-)

Maurizio


On 03/02/15 18:30, Peter Levart wrote:
> Hi,
>
> I wanted to see what it would take to any-fy a basic subset of java 
> collections library. I thought I would start with basic interfaces 
> (Iterator, Iterable, Collection, List, ...) and then add basic 
> implementations. I quickly learned that any-fying is a viral process. 
> Any dependency that uses a generic type variable from any-fied class 
> has to be any-fied too. So I had to make a cut somewhere or I would be 
> any-fying the whole JDK (exaggerating) :-o ... I removed the methods 
> that have connection to Streams API (stream(), splitterator(), ...) 
> and parallel methods (FJPool) and got a manageable isolated subset. I 
> though I was mostly finished with basic interfaces (which are not just 
> interfaces as from Java 8 on they contain implementation too) as I 
> fixed the last compilation error when javac crashed.
>
> Since I can't easily isolate the part that crashes javac, here are the 
> sources (19 classes) that I'm trying to compile in one go:
>
> http://cr.openjdk.java.net/~plevart/misc/valhala-hacks/javany-src.jar
>
> Perhaps Maurizio could look at it.
>
> In general, the process of any-fying up to this point went mostly 
> straightforward. A couple of pain points I encountered are:
>
>
> __WhereVal(T)/__WhereRef(T) only works when 'T' is a type variable 
> from enclosing generic class. It doesn't work for example with type 
> variables from generic methods.
>
>
> The Object-like methods (equals, hashCode, toString) work for <any T> 
> typed receiver, but not when enclosed in __WhereVal(T) { ... } block. 
> Why this restriction?
>
>
> Especially tricky for any-fying was the following AbstractCollection 
> method:
>
> public abstract class AbstractCollection<any E> implements 
> Collection<E> {
>     public <any T> T[] toArray(T[] a)
>
> ...as it must support things like:
>
> Collection<int> intCol = ...;
>
> long[] longArray = intCol.toArray(new long[0]);
>
> The tricky part was how to code a conversion from values of  type E to 
> values of type T when E and T are both any type variables. The 
> conversions one would like to support are: identity, widening, boxing, 
> unboxing. These tricks are all hacked in javany.util.Any class If 
> anybody is interested.
>
> Currently there's no support in prototype to solve the 
> List.remove(int) problem, so in my experiment I just renamed the 
> method to removeByIndex.
>
> All in all I was surprised with the level of completeness in the 
> prototype as early as this. I'll try to go further with this 
> anyfication experiment when the compiler lets me...
>
>
> Regards, Peter
>


From maurizio.cimadamore at oracle.com  Tue Feb  3 19:09:32 2015
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Tue, 03 Feb 2015 19:09:32 +0000
Subject: experiences with prototype
In-Reply-To: <54D11792.3090504@oracle.com>
References: <54D113DA.9060509@gmail.com> <54D11792.3090504@oracle.com>
Message-ID: <54D11CEC.6080009@oracle.com>


On 03/02/15 18:46, Maurizio Cimadamore wrote:
> I will also investigate on the crash you are getting... 
Hi Peter,
the crash is coming from this code in AbstractCollection (see code in bold):

     public boolean contains(Object o) {
         __WhereVal(E) {
             Iterator<E> it = iterator();
             if (o == null) {
                 return false;
             } else {
                 while (it.hasNext())
*if (o.equals((Object) it.next()))*
                         return true;
             }
             return false;
         }
         __WhereRef(E) {
             Iterator<E> it = iterator();
             if (o == null) {
                 while (it.hasNext())
                     if (it.next() == null)
                         return true;
             } else {
                 while (it.hasNext())
                     if (o.equals(it.next()))
                         return true;
             }
             return false;
         }
     }

I believe that, apart from the obvious javac bug, the code has an issue, 
as it.next() is supposed to return a value there, but you are casting to 
Object?

For the records - a simpler test case for the bug is this:

class Foo<any E> {
     E e;
     E get() { return e; }

     void test() {
         __WhereVal(E) {
             Object o = (Object)get();
         }
     }
}

Maurizio

From gil at azulsystems.com  Tue Feb  3 19:10:54 2015
From: gil at azulsystems.com (Gil Tene)
Date: Tue, 3 Feb 2015 19:10:54 +0000
Subject: What's the status of / relation between "JEP 169: Value
	Objects" / "Value Types for Java" / "Object Layout"
In-Reply-To: <CAHjP37F5SQfH41_FeDmB_0__887HqpcLMRUdVPuhFvMh9EA-EA@mail.gmail.com>
References: <CA+3eh122g3Ax+-PCL3EvX3V-cQkorva2j_c42_i4uRLOqw42Fg@mail.gmail.com>
	<54CA6853.2060601@oracle.com>
	<CA+3eh13pg6+ZziNuzLNdPSc8AkA4o1N9Xx0kGYFNm7TV3t9yzQ@mail.gmail.com>
	<CAHjP37HO8b3kgaWy4pVXpY3ky=+ebhLHBafMeNnZ-QooA9bXkA@mail.gmail.com>
	<CA+3eh135AOaiHoFQaA8Uc7WFsG-rHeeF30MBHw52J=fev9c1CQ@mail.gmail.com>
	<CAHjP37GhZYk61Z12L6Yj24YCwmyaao2oWJGRva6+gw1wGzGpOQ@mail.gmail.com>
	<505C1729-7E97-4D3B-8760-A3AF0BB76376@azulsystems.com>
	<CAHjP37F5SQfH41_FeDmB_0__887HqpcLMRUdVPuhFvMh9EA-EA@mail.gmail.com>
Message-ID: <0A138385-7F6E-4A2E-A9A2-79240B39D0A2@azulsystems.com>


On Feb 3, 2015, at 9:13 AM, Vitaly Davidovich <vitalyd at gmail.com<mailto:vitalyd at gmail.com>> wrote:

Gil, not sure if you saw my reply to Volker, but I agree -- I was simply asking why request this optimization via explicit syntax and not do it automatically in the runtime (with all the same restrictions, caveats, fine print, etc).

Well, there are a few reasons why this is "very hard":

1. It's "hard" to discern at runtime that an object assigned to a final reference field can/should be "inlined" into it's containing object:

1.1 Object-Inlining decisions are not instance-specific, they are class-global.

1.1.1 If a single containing instance inlines such an object, all instances of the same class must spend the space.

1.1.2 If a single "containing" instance does not inline the object referred to by the final reference field, NONE of the instances of the containing class can gain from the deference-avoiding (dead reckoning) optimization.

1.1.3 If we tried to inline all final reference declared objects without a specific declaration of intent, we'd likely end up with a lot  of wasted space.

2. Construction semantics: Even if you could auto-discern the need to "inline" the objects, you face a hard problem is dealing with how the "inlined" object (not the final ref pointing to it) is constructed and initialized:

2.1 Any factory-based instantiation of the referred to object is by-definition not inline-able (the factory based creation with a specific inlining intent and enough provided context, such as constructWithin(), being an exception).

2.2 Current non-factory based instantiation options in Java (new, reflection, methodHandles) all perform their own internal allocation of instance storage, and would not be able to use the "inlined" space without some serious surgery. Even if such surgery to all internal instantiation forms was done, getting the target instance location to the instantiation code is also "very hard", given that instantiation logically occurs before the assignment of the resulting reference to the final field, and many operations can happen in between. Approaches that attempt to override a sequence of operations (e.g. the simplest stuff like new; dup; push arg1; push arg2; invoke_special; putfield (into final ref field), or much more complicated ones...) such that the intermediate heap reference is never exposed and can be replaced with a reference to the already-allocated space only work in trivial situations, and tend to fail in all sorts of interesting common-case ways (e.g. when perfectly innocent instrumentation is involved).

Now "very hard" probably does not mean impossible. But it has so much open ended stuff that makes it a huge thing to tackle safely. Given the fact that much of the context of org.ObjectLayout benefits will never be auto-discernible without using explicit statements to describe the expected semantics (using a StructuredArray to state optimization-enabling limiting semantics, for example), adding explicit intent declaration (as opposed to auto-optimization) for intrinsic objects seems natural there.


On Tue, Feb 3, 2015 at 11:58 AM, Gil Tene <gil at azulsystems.com<mailto:gil at azulsystems.com>> wrote:
A couple of point here, specific to org.ObjectLayout (http://objectlayout.org<http://objectlayout.org/>):

Declaration:
The ObjectLayout @Intrinsic (http://objectlayout.org<http://objectlayout.org/>) annotation is used for declaring what you refer to as "inline" objects. It is specifically not intended to be a layout control directive, but an optimization hint. Whether or not a JVM "inlines" the intrinsic object within the containing one, and how/where that "inlining" happens becomes a JVM-specific implementation concern, and no a semantic one.

Field initialization:

Implicit, undeclared choices to "inline" all final referenced fields fail very quickly when attempted in practice. E.g. final fields can (and often will) be set to refer to pre-existing-at-construction-time objects, which are (by definition) impossible to "inline". In addition, there are many common uses final reference fields where "inlining" is no possible because the actual object size of the referred-to object is not a global constant (e.g. it will be set to a construction-time or parameter-based choice of subclass).

We've given Intrinsic Object initialization a lot of thought in org.ObjectLayout. The dedicated initialization API is there to assure several things, including exact-type (the field's specific declared type) choice. It could turn into a less-verbose version in some future JDK if language support was added (e.g. to avoid mentioning the needed-only-to-conform-with-syntax things like "this", and the field name in the constructWithin() call), but I expect the semantics to need to be similar even if the syntax was made less verbose.

-- Gil.

> On Feb 3, 2015, at 8:40 AM, Vitaly Davidovich <vitalyd at gmail.com<mailto:vitalyd at gmail.com>> wrote:
>
> Hi Volker,
>
> Sorry, I may have been unclear in my question.  As you say, ObjectLayout
> requires that you annotate the fields that you'd like inlined and then also
> use special API to construct those objects.  I'm wondering whether,
> instead, all private final fields are automatically inlined, and only cases
> where you'd like to layout the field out-of-band would require annotation.
> This would be controlled via a cmdline flag, as you say, similar to perhaps
> how compressed oops are enabled (or not).  Note that I'm talking about
> purely layout of reference types, not value types.
>
> The "concern" with having to explicitly annotate and use dedicated APIs to
> opt-in is that adoption will be fairly low, whereas I think most of the
> time one would want inlined storage layout.
>
> Thanks
>
>
> On Tue, Feb 3, 2015 at 11:29 AM, Volker Simonis <volker.simonis at gmail.com<mailto:volker.simonis at gmail.com>>
> wrote:
>
>> Hi Vitaly,
>>
>> for PackedObjects/ObjectLayout you need to specially annotate the
>> classes and/or fields which you want to allocate "inline". Once you've
>> done that you have no choice with the PackedObjects approach.
>> ObjectLayout is a little special here, because it can run with any
>> Java VM in which case it will still use the default reference model.
>> But it can potentially be optimized by some VM's to provide the flat
>> object layout. I expect these optimizations to be controllable by a
>> command line option.
>>
>> With the "Value Types for Java" [1] approach you'll have the
>> possiblitly to express the behavior right in Java like in the
>> following example from [1]:
>>
>> final __ByValue class Point {
>> static Point origin = __MakeValue(0, 0);
>>
>> I think the default will always be "reference semantics" in Java but
>> with various degrees of freedom to optionally choose value semantics.
>>
>> Regards,
>> Volker
>>
>> [1] http://cr.openjdk.java.net/~jrose/values/values-0.html
>>
>> On Mon, Feb 2, 2015 at 9:19 PM, Vitaly Davidovich <vitalyd at gmail.com<mailto:vitalyd at gmail.com>>
>> wrote:
>>> Volker (or anyone else for that matter),
>>>
>>> Just curious -- do you envision "inline" layout of objects as something
>> one
>>> would have to opt-in or as the default layout for all objects in a heap?
>> It
>>> seems like this should be the default (assuming zero to minimal overhead
>> for
>>> loading the references) as I think wanting "out of line" allocations is
>> more
>>> rare.
>>>
>>> Thanks
>>>
>>> On Mon, Feb 2, 2015 at 2:06 PM, Volker Simonis <volker.simonis at gmail.com<mailto:volker.simonis at gmail.com>
>>>
>>> wrote:
>>>>
>>>> Hi Brian,
>>>>
>>>> thanks a lot for your detailed answer and apologies for the late reply
>>>> (I was a little distracted by FOSDEM :)
>>>>
>>>> All your comments have been clear and reasonable and are much
>>>> appreciated. Please find my additional answers inline:
>>>>
>>>> On Thu, Jan 29, 2015 at 6:05 PM, Brian Goetz <brian.goetz at oracle.com<mailto:brian.goetz at oracle.com>>
>>>> wrote:
>>>>>> Question: is JEP 169 still under active development or has it been
>>>>>> merged into the more general "Value types for Java" proposal below?
>>>>>
>>>>>
>>>>> It has been merged into the more general Value Types for Java
>> proposal.
>>>>>
>>>>
>>>> Then maybe this JEP should be closed to avoid further confusion?
>>>>
>>>>>> The "Value types for Java" approach clearly seems to be the most
>>>>>> general but also the most complex proposal.
>>>>>
>>>>>
>>>>> For some meanings of "complex".  It is certainly the most intrusive
>> and
>>>>> large; new bytecodes, new type signatures.  But from a user-model
>>>>> perspective, value types are actually fairly simple.
>>>>>
>>>>>> It's out of scope for Java
>>>>>> 9 and still questionable for Java 10 and above. The "PackedObject"
>> and
>>>>>> "ObjectLayout" approaches are clearly simpler and more limited in
>>>>>> scope as they only concentrate on better object layout.
>>>>>
>>>>>
>>>>> To your list, I'd add: Project Panama, the sister project to Valhalla.
>>>>> Panama focuses on interop with native code and data, including layout
>>>>> specification.  A key goal of Packed was to be able to access off-heap
>>>>> native data in its native format, rather than marshalling it across
>> the
>>>>> JNI
>>>>> boundary.  Panama is focused on this problem as well, but aims to
>> treat
>>>>> it
>>>>> as a separate problem from Java object layout, resulting in what we
>>>>> believe
>>>>> to be a cleaner decomposition of the two concerns.
>>>>>
>>>>
>>>> Your right. I somehow missed to look at Panama more deeply because I
>>>> always thought it is only about FFI. John Rose nicely explains the
>>>> various parts of Panama in this mail
>>>>
>> http://mail.openjdk.java.net/pipermail/panama-dev/2014-October/000042.html
>>>> where he also mentions the intention of Panama to create new flatter
>>>> data layouts in the Heap and the relation of Panama to PackedObjects
>>>> and ObjectLayout.
>>>>
>>>>> Packed is an interesting mix of memory density (object embedding and
>>>>> packed
>>>>> arrays) and native interop.  But mixing the two goals also has costs;
>>>>> our
>>>>> approach is to separate them into orthogonal concerns, and we think
>> that
>>>>> Valhalla and Panama do just that.  So in many ways, while a larger
>>>>> project,
>>>>> the combination of Valhalla+Panama addresses the problem that Packed
>>>>> did, in
>>>>> a cleaner way.
>>>>>
>>>>>> Question: is there a chance to get a some sort of Java-only but
>>>>>> transparently optimizable structure package like "ObjectLayout" into
>>>>>> Java early (i.e. Java 9)?
>>>>>
>>>>>
>>>>> It would depend on a lot of things -- including the level of readiness
>>>>> of
>>>>> the design and implementation, and the overlap with anticipated future
>>>>> features.  We've reviewed some of the early design of ObjectLayout and
>>>>> provided feedback to the projects architects; currently, I think it's
>> in
>>>>> the
>>>>> "promising exploration" stage, but I think multiple rounds of
>>>>> simplification
>>>>> are needed before it is ready to be considered for "everybody's Java."
>>>>> But
>>>>> if the choice is to push something that's not ready into 9, or to wait
>>>>> longer -- there's not actually a choice to be made there.
>>>>>
>>>>> I appreciate the desire to "get something you can use now", but we
>> have
>>>>> to
>>>>> be prepared to support whatever we push into Java for the next 20
>> years,
>>>>> and
>>>>> deal with the additional constraints it generates -- which can be an
>>>>> enormous cost.  (Even thought the direct cost is mostly borne by
>> Oracle,
>>>>> the
>>>>> indirect cost is borne by everyone, in the form of slower progress on
>>>>> everything else.)  So I am very wary of the motivation of "well,
>>>>> something
>>>>> better is coming, but this works now, so can we push it in?"  I'd
>> prefer
>>>>> to
>>>>> focus on answering whether this is right thing for Java for the next
>> 20
>>>>> years.
>>>>>
>>>>>> In my eyes this wouldn't contradict with a more general solution like
>>>>>> the one proposed in the "Value types for Java" approach while still
>>>>>> offering quite significant performance improvements for quite a big
>>>>>> range of problems.
>>>>>
>>>>>
>>>>> The goals of the ObjectLayout effort has overlap with, but also
>> differs
>>>>> from, the goals of Valhalla.  And herein is the problem; neither
>>>>> generalizes
>>>>> the other, and I don't think we do the user base a great favor by
>>>>> pursuing
>>>>> two separate neither-coincident-nor-orthogonal approaches.  I suspect,
>>>>> though, that after a few rounds of simplification, ObjectLayout could
>>>>> morph
>>>>> into something that fit either coincidently or orthogonally with the
>>>>> Valhalla work -- which would be great.  But, as you know, our
>> resources
>>>>> are
>>>>> limited, so we (Oracle) can't really afford to invest in both.  And
>> such
>>>>> simplification takes time -- getting to that "aha" moment when you
>>>>> realize
>>>>> you can simplify something is generally an incompressible process.
>>>>>
>>>>>> Question: what would be the right place to propose something like the
>>>>>> "ObjectLayout" library for Java 9/10? Would that fit within the
>>>>>> umbrella of the Valhalla project or would it be done within its own
>>>>>> project / under it's own JEP?
>>>>>
>>>>>
>>>>> Suggesting a version number at this point would be putting the cart
>>>>> before
>>>>> the horse (you'll note that we've not even proposed a version number
>> for
>>>>> Valhalla; the closest we've gotten to that is "after 9".)
>>>>>
>>>>> OpenJDK Projects are a tool for building a community around a body of
>>>>> work;
>>>>> JEPs are a project-management tool for defining, scoping, and tracking
>>>>> the
>>>>> progress of a feature.  Given where OL is, it would be reasonable to
>>>>> start a
>>>>> Project, which would become the nexus of collaboration that could
>>>>> eventually
>>>>> produce a JEP.
>>>>>
>>>>
>>>> That sounds reasonable. I'll speak with the ObjectLayout people to
>>>> hear what they think about starting a new OpenJDK project.
>>>>
>>>> And after all you've said before, I also came to the conclusion that
>>>> investigating ways of new in-heap object layouts seem to be better of
>>>> in the Panama project. So I'll also ask there what they think about
>>>> it. Maybe ObjectLayout could become part of Panama or maybe we could
>>>> just start a new subproject of Panama with the same/similar goals.
>>>>
>>>>> Hope this helps,
>>>>> -Brian
>>>>
>>>> It really did!
>>>>
>>>> Thanks again,
>>>> Volker
>>>> _______________________________________________
>>>> mlvm-dev mailing list
>>>> mlvm-dev at openjdk.java.net<mailto:mlvm-dev at openjdk.java.net>
>>>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>>>
>>>
>>>
>>> _______________________________________________
>>> mlvm-dev mailing list
>>> mlvm-dev at openjdk.java.net<mailto:mlvm-dev at openjdk.java.net>
>>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>>>
>> _______________________________________________
>> mlvm-dev mailing list
>> mlvm-dev at openjdk.java.net<mailto:mlvm-dev at openjdk.java.net>
>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>>




From vitalyd at gmail.com  Tue Feb  3 20:20:14 2015
From: vitalyd at gmail.com (Vitaly Davidovich)
Date: Tue, 3 Feb 2015 15:20:14 -0500
Subject: What's the status of / relation between "JEP 169: Value Objects"
	/ "Value Types for Java" / "Object Layout"
In-Reply-To: <0A138385-7F6E-4A2E-A9A2-79240B39D0A2@azulsystems.com>
References: <CA+3eh122g3Ax+-PCL3EvX3V-cQkorva2j_c42_i4uRLOqw42Fg@mail.gmail.com>
	<54CA6853.2060601@oracle.com>
	<CA+3eh13pg6+ZziNuzLNdPSc8AkA4o1N9Xx0kGYFNm7TV3t9yzQ@mail.gmail.com>
	<CAHjP37HO8b3kgaWy4pVXpY3ky=+ebhLHBafMeNnZ-QooA9bXkA@mail.gmail.com>
	<CA+3eh135AOaiHoFQaA8Uc7WFsG-rHeeF30MBHw52J=fev9c1CQ@mail.gmail.com>
	<CAHjP37GhZYk61Z12L6Yj24YCwmyaao2oWJGRva6+gw1wGzGpOQ@mail.gmail.com>
	<505C1729-7E97-4D3B-8760-A3AF0BB76376@azulsystems.com>
	<CAHjP37F5SQfH41_FeDmB_0__887HqpcLMRUdVPuhFvMh9EA-EA@mail.gmail.com>
	<0A138385-7F6E-4A2E-A9A2-79240B39D0A2@azulsystems.com>
Message-ID: <CAHjP37Ey-w7AeqJ8isoOov4K2Z6vG5o2bdRcJmn0Pp_WEesGKQ@mail.gmail.com>

1) My assumption is that vast majority of classes will benefit from inline
storage of their collaborators.  The waste would be in classes that have
hot and cold members together, but I think that's the minority? But, for
those cases, it would be beneficial to allow out-of-band layout (i.e.
today's layout) via annotation or whatever.  Why do you say this would
likely lead to memory waste? If the object is allocated one way or another,
it's going to take up space.  The only case i see being problematic is when
null is stored in majority of instances.  I guess this boils down to
whether one thinks it's more likely, by default, that objects embedded in
others are hot/cold in terms of access; I think it's more common for them
to be accessed together and so default should cater to that.

When/if inlined layout is available, next logical thing one may request is
specifying layout *order* to try and place commonly accessed data on same
cacheline.  This is less important for streaming cases, but would be nice
for random walks.

2) yes, this basically requires a placement-new like thing to be
implemented in the VM.  No disagreement that it's intrusive.

sent from my phone
On Feb 3, 2015 2:10 PM, "Gil Tene" <gil at azulsystems.com> wrote:

>
>  On Feb 3, 2015, at 9:13 AM, Vitaly Davidovich <vitalyd at gmail.com> wrote:
>
>  Gil, not sure if you saw my reply to Volker, but I agree -- I was simply
> asking why request this optimization via explicit syntax and not do it
> automatically in the runtime (with all the same restrictions, caveats, fine
> print, etc).
>
>
>  Well, there are a few reasons why this is "very hard":
>
>  1. It's "hard" to discern at runtime that an object assigned to a final
> reference field can/should be "inlined" into it's containing object:
>
>  1.1 Object-Inlining decisions are not instance-specific, they are
> class-global.
>
>  1.1.1 If a single containing instance inlines such an object, all
> instances of the same class must spend the space.
>
>  1.1.2 If a single "containing" instance does not inline the object
> referred to by the final reference field, NONE of the instances of the
> containing class can gain from the deference-avoiding (dead reckoning)
> optimization.
>
>  1.1.3 If we tried to inline all final reference declared objects without
> a specific declaration of intent, we'd likely end up with a lot  of wasted
> space.
>
>  2. Construction semantics: Even if you could auto-discern the need to
> "inline" the objects, you face a hard problem is dealing with how the
> "inlined" object (not the final ref pointing to it) is constructed and
> initialized:
>
>  2.1 Any factory-based instantiation of the referred to object is
> by-definition not inline-able (the factory based creation with a specific
> inlining intent and enough provided context, such as constructWithin(),
> being an exception).
>
>  2.2 Current non-factory based instantiation options in Java (new,
> reflection, methodHandles) all perform their own internal allocation of
> instance storage, and would not be able to use the "inlined" space without
> some serious surgery. Even if such surgery to all internal instantiation
> forms was done, getting the target instance location to the instantiation
> code is also "very hard", given that instantiation logically occurs before
> the assignment of the resulting reference to the final field, and many
> operations can happen in between. Approaches that attempt to override a
> sequence of operations (e.g. the simplest stuff like new; dup; push arg1;
> push arg2; invoke_special; putfield (into final ref field), or much more
> complicated ones...) such that the intermediate heap reference is never
> exposed and can be replaced with a reference to the already-allocated space
> only work in trivial situations, and tend to fail in all sorts of
> interesting common-case ways (e.g. when perfectly innocent instrumentation
> is involved).
>
>  Now "very hard" probably does not mean impossible. But it has so much
> open ended stuff that makes it a huge thing to tackle safely. Given the
> fact that much of the context of org.ObjectLayout benefits will never be
> auto-discernible without using explicit statements to describe the expected
> semantics (using a StructuredArray to state optimization-enabling limiting
> semantics, for example), adding explicit intent declaration (as opposed to
> auto-optimization) for intrinsic objects seems natural there.
>
>
> On Tue, Feb 3, 2015 at 11:58 AM, Gil Tene <gil at azulsystems.com> wrote:
>
>> A couple of point here, specific to org.ObjectLayout (
>> http://objectlayout.org):
>>
>> Declaration:
>> The ObjectLayout @Intrinsic (http://objectlayout.org) annotation is used
>> for declaring what you refer to as "inline" objects. It is specifically not
>> intended to be a layout control directive, but an optimization hint.
>> Whether or not a JVM "inlines" the intrinsic object within the containing
>> one, and how/where that "inlining" happens becomes a JVM-specific
>> implementation concern, and no a semantic one.
>>
>> Field initialization:
>>
>> Implicit, undeclared choices to "inline" all final referenced fields fail
>> very quickly when attempted in practice. E.g. final fields can (and often
>> will) be set to refer to pre-existing-at-construction-time objects, which
>> are (by definition) impossible to "inline". In addition, there are many
>> common uses final reference fields where "inlining" is no possible because
>> the actual object size of the referred-to object is not a global constant
>> (e.g. it will be set to a construction-time or parameter-based choice of
>> subclass).
>>
>> We've given Intrinsic Object initialization a lot of thought in
>> org.ObjectLayout. The dedicated initialization API is there to assure
>> several things, including exact-type (the field's specific declared type)
>> choice. It could turn into a less-verbose version in some future JDK if
>> language support was added (e.g. to avoid mentioning the
>> needed-only-to-conform-with-syntax things like "this", and the field name
>> in the constructWithin() call), but I expect the semantics to need to be
>> similar even if the syntax was made less verbose.
>>
>> -- Gil.
>>
>> > On Feb 3, 2015, at 8:40 AM, Vitaly Davidovich <vitalyd at gmail.com>
>> wrote:
>> >
>> > Hi Volker,
>> >
>> > Sorry, I may have been unclear in my question.  As you say, ObjectLayout
>> > requires that you annotate the fields that you'd like inlined and then
>> also
>> > use special API to construct those objects.  I'm wondering whether,
>> > instead, all private final fields are automatically inlined, and only
>> cases
>> > where you'd like to layout the field out-of-band would require
>> annotation.
>> > This would be controlled via a cmdline flag, as you say, similar to
>> perhaps
>> > how compressed oops are enabled (or not).  Note that I'm talking about
>> > purely layout of reference types, not value types.
>> >
>> > The "concern" with having to explicitly annotate and use dedicated APIs
>> to
>> > opt-in is that adoption will be fairly low, whereas I think most of the
>> > time one would want inlined storage layout.
>> >
>> > Thanks
>> >
>> >
>> > On Tue, Feb 3, 2015 at 11:29 AM, Volker Simonis <
>> volker.simonis at gmail.com>
>> > wrote:
>> >
>> >> Hi Vitaly,
>> >>
>> >> for PackedObjects/ObjectLayout you need to specially annotate the
>> >> classes and/or fields which you want to allocate "inline". Once you've
>> >> done that you have no choice with the PackedObjects approach.
>> >> ObjectLayout is a little special here, because it can run with any
>> >> Java VM in which case it will still use the default reference model.
>> >> But it can potentially be optimized by some VM's to provide the flat
>> >> object layout. I expect these optimizations to be controllable by a
>> >> command line option.
>> >>
>> >> With the "Value Types for Java" [1] approach you'll have the
>> >> possiblitly to express the behavior right in Java like in the
>> >> following example from [1]:
>> >>
>> >> final __ByValue class Point {
>> >> static Point origin = __MakeValue(0, 0);
>> >>
>> >> I think the default will always be "reference semantics" in Java but
>> >> with various degrees of freedom to optionally choose value semantics.
>> >>
>> >> Regards,
>> >> Volker
>> >>
>> >> [1] http://cr.openjdk.java.net/~jrose/values/values-0.html
>> >>
>> >> On Mon, Feb 2, 2015 at 9:19 PM, Vitaly Davidovich <vitalyd at gmail.com>
>> >> wrote:
>> >>> Volker (or anyone else for that matter),
>> >>>
>> >>> Just curious -- do you envision "inline" layout of objects as
>> something
>> >> one
>> >>> would have to opt-in or as the default layout for all objects in a
>> heap?
>> >> It
>> >>> seems like this should be the default (assuming zero to minimal
>> overhead
>> >> for
>> >>> loading the references) as I think wanting "out of line" allocations
>> is
>> >> more
>> >>> rare.
>> >>>
>> >>> Thanks
>> >>>
>> >>> On Mon, Feb 2, 2015 at 2:06 PM, Volker Simonis <
>> volker.simonis at gmail.com
>> >>>
>> >>> wrote:
>> >>>>
>> >>>> Hi Brian,
>> >>>>
>> >>>> thanks a lot for your detailed answer and apologies for the late
>> reply
>> >>>> (I was a little distracted by FOSDEM :)
>> >>>>
>> >>>> All your comments have been clear and reasonable and are much
>> >>>> appreciated. Please find my additional answers inline:
>> >>>>
>> >>>> On Thu, Jan 29, 2015 at 6:05 PM, Brian Goetz <brian.goetz at oracle.com
>> >
>> >>>> wrote:
>> >>>>>> Question: is JEP 169 still under active development or has it been
>> >>>>>> merged into the more general "Value types for Java" proposal below?
>> >>>>>
>> >>>>>
>> >>>>> It has been merged into the more general Value Types for Java
>> >> proposal.
>> >>>>>
>> >>>>
>> >>>> Then maybe this JEP should be closed to avoid further confusion?
>> >>>>
>> >>>>>> The "Value types for Java" approach clearly seems to be the most
>> >>>>>> general but also the most complex proposal.
>> >>>>>
>> >>>>>
>> >>>>> For some meanings of "complex".  It is certainly the most intrusive
>> >> and
>> >>>>> large; new bytecodes, new type signatures.  But from a user-model
>> >>>>> perspective, value types are actually fairly simple.
>> >>>>>
>> >>>>>> It's out of scope for Java
>> >>>>>> 9 and still questionable for Java 10 and above. The "PackedObject"
>> >> and
>> >>>>>> "ObjectLayout" approaches are clearly simpler and more limited in
>> >>>>>> scope as they only concentrate on better object layout.
>> >>>>>
>> >>>>>
>> >>>>> To your list, I'd add: Project Panama, the sister project to
>> Valhalla.
>> >>>>> Panama focuses on interop with native code and data, including
>> layout
>> >>>>> specification.  A key goal of Packed was to be able to access
>> off-heap
>> >>>>> native data in its native format, rather than marshalling it across
>> >> the
>> >>>>> JNI
>> >>>>> boundary.  Panama is focused on this problem as well, but aims to
>> >> treat
>> >>>>> it
>> >>>>> as a separate problem from Java object layout, resulting in what we
>> >>>>> believe
>> >>>>> to be a cleaner decomposition of the two concerns.
>> >>>>>
>> >>>>
>> >>>> Your right. I somehow missed to look at Panama more deeply because I
>> >>>> always thought it is only about FFI. John Rose nicely explains the
>> >>>> various parts of Panama in this mail
>> >>>>
>> >>
>> http://mail.openjdk.java.net/pipermail/panama-dev/2014-October/000042.html
>> >>>> where he also mentions the intention of Panama to create new flatter
>> >>>> data layouts in the Heap and the relation of Panama to PackedObjects
>> >>>> and ObjectLayout.
>> >>>>
>> >>>>> Packed is an interesting mix of memory density (object embedding and
>> >>>>> packed
>> >>>>> arrays) and native interop.  But mixing the two goals also has
>> costs;
>> >>>>> our
>> >>>>> approach is to separate them into orthogonal concerns, and we think
>> >> that
>> >>>>> Valhalla and Panama do just that.  So in many ways, while a larger
>> >>>>> project,
>> >>>>> the combination of Valhalla+Panama addresses the problem that Packed
>> >>>>> did, in
>> >>>>> a cleaner way.
>> >>>>>
>> >>>>>> Question: is there a chance to get a some sort of Java-only but
>> >>>>>> transparently optimizable structure package like "ObjectLayout"
>> into
>> >>>>>> Java early (i.e. Java 9)?
>> >>>>>
>> >>>>>
>> >>>>> It would depend on a lot of things -- including the level of
>> readiness
>> >>>>> of
>> >>>>> the design and implementation, and the overlap with anticipated
>> future
>> >>>>> features.  We've reviewed some of the early design of ObjectLayout
>> and
>> >>>>> provided feedback to the projects architects; currently, I think
>> it's
>> >> in
>> >>>>> the
>> >>>>> "promising exploration" stage, but I think multiple rounds of
>> >>>>> simplification
>> >>>>> are needed before it is ready to be considered for "everybody's
>> Java."
>> >>>>> But
>> >>>>> if the choice is to push something that's not ready into 9, or to
>> wait
>> >>>>> longer -- there's not actually a choice to be made there.
>> >>>>>
>> >>>>> I appreciate the desire to "get something you can use now", but we
>> >> have
>> >>>>> to
>> >>>>> be prepared to support whatever we push into Java for the next 20
>> >> years,
>> >>>>> and
>> >>>>> deal with the additional constraints it generates -- which can be an
>> >>>>> enormous cost.  (Even thought the direct cost is mostly borne by
>> >> Oracle,
>> >>>>> the
>> >>>>> indirect cost is borne by everyone, in the form of slower progress
>> on
>> >>>>> everything else.)  So I am very wary of the motivation of "well,
>> >>>>> something
>> >>>>> better is coming, but this works now, so can we push it in?"  I'd
>> >> prefer
>> >>>>> to
>> >>>>> focus on answering whether this is right thing for Java for the next
>> >> 20
>> >>>>> years.
>> >>>>>
>> >>>>>> In my eyes this wouldn't contradict with a more general solution
>> like
>> >>>>>> the one proposed in the "Value types for Java" approach while still
>> >>>>>> offering quite significant performance improvements for quite a big
>> >>>>>> range of problems.
>> >>>>>
>> >>>>>
>> >>>>> The goals of the ObjectLayout effort has overlap with, but also
>> >> differs
>> >>>>> from, the goals of Valhalla.  And herein is the problem; neither
>> >>>>> generalizes
>> >>>>> the other, and I don't think we do the user base a great favor by
>> >>>>> pursuing
>> >>>>> two separate neither-coincident-nor-orthogonal approaches.  I
>> suspect,
>> >>>>> though, that after a few rounds of simplification, ObjectLayout
>> could
>> >>>>> morph
>> >>>>> into something that fit either coincidently or orthogonally with the
>> >>>>> Valhalla work -- which would be great.  But, as you know, our
>> >> resources
>> >>>>> are
>> >>>>> limited, so we (Oracle) can't really afford to invest in both.  And
>> >> such
>> >>>>> simplification takes time -- getting to that "aha" moment when you
>> >>>>> realize
>> >>>>> you can simplify something is generally an incompressible process.
>> >>>>>
>> >>>>>> Question: what would be the right place to propose something like
>> the
>> >>>>>> "ObjectLayout" library for Java 9/10? Would that fit within the
>> >>>>>> umbrella of the Valhalla project or would it be done within its own
>> >>>>>> project / under it's own JEP?
>> >>>>>
>> >>>>>
>> >>>>> Suggesting a version number at this point would be putting the cart
>> >>>>> before
>> >>>>> the horse (you'll note that we've not even proposed a version number
>> >> for
>> >>>>> Valhalla; the closest we've gotten to that is "after 9".)
>> >>>>>
>> >>>>> OpenJDK Projects are a tool for building a community around a body
>> of
>> >>>>> work;
>> >>>>> JEPs are a project-management tool for defining, scoping, and
>> tracking
>> >>>>> the
>> >>>>> progress of a feature.  Given where OL is, it would be reasonable to
>> >>>>> start a
>> >>>>> Project, which would become the nexus of collaboration that could
>> >>>>> eventually
>> >>>>> produce a JEP.
>> >>>>>
>> >>>>
>> >>>> That sounds reasonable. I'll speak with the ObjectLayout people to
>> >>>> hear what they think about starting a new OpenJDK project.
>> >>>>
>> >>>> And after all you've said before, I also came to the conclusion that
>> >>>> investigating ways of new in-heap object layouts seem to be better of
>> >>>> in the Panama project. So I'll also ask there what they think about
>> >>>> it. Maybe ObjectLayout could become part of Panama or maybe we could
>> >>>> just start a new subproject of Panama with the same/similar goals.
>> >>>>
>> >>>>> Hope this helps,
>> >>>>> -Brian
>> >>>>
>> >>>> It really did!
>> >>>>
>> >>>> Thanks again,
>> >>>> Volker
>> >>>> _______________________________________________
>> >>>> mlvm-dev mailing list
>> >>>> mlvm-dev at openjdk.java.net
>> >>>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>> >>>
>> >>>
>> >>>
>> >>> _______________________________________________
>> >>> mlvm-dev mailing list
>> >>> mlvm-dev at openjdk.java.net
>> >>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>> >>>
>> >> _______________________________________________
>> >> mlvm-dev mailing list
>> >> mlvm-dev at openjdk.java.net
>> >> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>> >>
>>
>>
>
>

From peter.levart at gmail.com  Tue Feb  3 21:05:14 2015
From: peter.levart at gmail.com (Peter Levart)
Date: Tue, 03 Feb 2015 22:05:14 +0100
Subject: experiences with prototype
In-Reply-To: <54D11CEC.6080009@oracle.com>
References: <54D113DA.9060509@gmail.com> <54D11792.3090504@oracle.com>
	<54D11CEC.6080009@oracle.com>
Message-ID: <54D1380A.5070503@gmail.com>

Hi Maurizio,

I see. I thought this could be a nice idiom for boxing, since the following:

(Object) 42

...is legal and results in an Integer object at runtime.

But I don't know if a checkcast is actually inserted for (Object). Could 
javac redundantly do it in case casting to Object is from expression of 
<any> type and also equip checkcast with BMA indicating the type of 
expression so that specialization could replace it with boxing code?

Regards, Peter


On 02/03/2015 08:09 PM, Maurizio Cimadamore wrote:
>
> On 03/02/15 18:46, Maurizio Cimadamore wrote:
>> I will also investigate on the crash you are getting... 
> Hi Peter,
> the crash is coming from this code in AbstractCollection (see code in 
> bold):
>
>     public boolean contains(Object o) {
>         __WhereVal(E) {
>             Iterator<E> it = iterator();
>             if (o == null) {
>                 return false;
>             } else {
>                 while (it.hasNext())
> *if (o.equals((Object) it.next()))*
>                         return true;
>             }
>             return false;
>         }
>         __WhereRef(E) {
>             Iterator<E> it = iterator();
>             if (o == null) {
>                 while (it.hasNext())
>                     if (it.next() == null)
>                         return true;
>             } else {
>                 while (it.hasNext())
>                     if (o.equals(it.next()))
>                         return true;
>             }
>             return false;
>         }
>     }
>
> I believe that, apart from the obvious javac bug, the code has an 
> issue, as it.next() is supposed to return a value there, but you are 
> casting to Object?
>
> For the records - a simpler test case for the bug is this:
>
> class Foo<any E> {
>     E e;
>     E get() { return e; }
>
>     void test() {
>         __WhereVal(E) {
>             Object o = (Object)get();
>         }
>     }
> }
>
> Maurizio


From gil at azulsystems.com  Tue Feb  3 21:08:43 2015
From: gil at azulsystems.com (Gil Tene)
Date: Tue, 3 Feb 2015 21:08:43 +0000
Subject: What's the status of / relation between "JEP 169: Value
	Objects" / "Value Types for Java" / "Object Layout"
In-Reply-To: <CAHjP37Ey-w7AeqJ8isoOov4K2Z6vG5o2bdRcJmn0Pp_WEesGKQ@mail.gmail.com>
References: <CA+3eh122g3Ax+-PCL3EvX3V-cQkorva2j_c42_i4uRLOqw42Fg@mail.gmail.com>
	<54CA6853.2060601@oracle.com>
	<CA+3eh13pg6+ZziNuzLNdPSc8AkA4o1N9Xx0kGYFNm7TV3t9yzQ@mail.gmail.com>
	<CAHjP37HO8b3kgaWy4pVXpY3ky=+ebhLHBafMeNnZ-QooA9bXkA@mail.gmail.com>
	<CA+3eh135AOaiHoFQaA8Uc7WFsG-rHeeF30MBHw52J=fev9c1CQ@mail.gmail.com>
	<CAHjP37GhZYk61Z12L6Yj24YCwmyaao2oWJGRva6+gw1wGzGpOQ@mail.gmail.com>
	<505C1729-7E97-4D3B-8760-A3AF0BB76376@azulsystems.com>
	<CAHjP37F5SQfH41_FeDmB_0__887HqpcLMRUdVPuhFvMh9EA-EA@mail.gmail.com>
	<0A138385-7F6E-4A2E-A9A2-79240B39D0A2@azulsystems.com>
	<CAHjP37Ey-w7AeqJ8isoOov4K2Z6vG5o2bdRcJmn0Pp_WEesGKQ@mail.gmail.com>
Message-ID: <5C9C5691-B447-46D9-A4D5-7778E2BFA774@azulsystems.com>


> On Feb 3, 2015, at 12:20 PM, Vitaly Davidovich <vitalyd at gmail.com> wrote:
> 
> 1) My assumption is that vast majority of classes will benefit from inline storage of their collaborators.  The waste would be in classes that have hot and cold members together, but I think that's the minority? But, for those cases, it would be beneficial to allow out-of-band layout (i.e. today's layout) via annotation or whatever.  

I *think* it's a small (and fragile) minority, rather than the vast majority. That is, I think that auto-magic (implicit optimization without a special factory based instantiation) is limited in potential scope and impact: Without special magic (see #2 below), 100% of current final field assignments are non-inlineable because they (by definition) assign the field to a reference value that refers to an already-allocated-at-assignment-time object.

#2 is hard enough to do for trivial case (those that initialize the field to refer to an in-place instantiated object that does not escape ahead of the assignment, and whose all escaping post-assignemnt references can be safely hunted down and converted). And it is much harder for the plentiful non-trivial cases (the allocation/instantiation are done "far enough apart" such that interleaved operations complexify the analysis).

But even if #2 is solved somehow for some subset of instantiations that the VM can auto-magically determine can be converted to a "placement-new" sort of allocation / construction sequence, there are plenty of final field assignment cases where the final ref field refers to an object that is already referred to elsewhere (before the assignment). E.g. it is quite common for private final ref fields to be initialized from constructor parameters (e.g. HashMap.Entry). It is also common to gave final ref fields assigned to the return values of method calls. And for those final fields to refer to object that are common across multiple referring instances. All those cases are non-inlineable by definition.

> Why do you say this would likely lead to memory waste? If the object is allocated one way or another, it's going to take up space.  The only case i see being problematic is when null is stored in majority of instances.

It's not nulls. It's a reference to something that isn't "inlineable". Like setting the final field to a reference to an already-allocated-at-assignement-time object.

If the VM can determine ahead of time (ahead of instantiating any instances of a containing class) that ALL final field assignments performed in the future will be made from in-place instantiations that can be safely and auto-magically converted to placement-news, then there will be no wasted space. However, for every class where some late-discovered (discovered after an instance has been created) situation leads to a loss of the optimization, potential memory waste is also there.

It's true that the memory waste would be contained to the cases where the reference is assigned to a non-"placement-new" result, and that in mixed situations the pre-allocated space in the containing object will still be used. It's just hard to guess how much of each case there is in real code. 

> I guess this boils down to whether one thinks it's more likely, by default, that objects embedded in others are hot/cold in terms of access; I think it's more common for them to be accessed together and so default should cater to that.
> 
> When/if inlined layout is available, next logical thing one may request is specifying layout *order* to try and place commonly accessed data on same cacheline.  This is less important for streaming cases, but would be nice for random walks.

I think cache line co-placement makes sense for fields (e.g. an @Together thing to mirror the @Contended stuff), but I doubt that it makes much sense for objects, mostly due to size (not that many realistic objects will fit together in one cache line anyway). A natural "inline in declaration order" approach for intrinsic objects is probably just as good as anything else, and will require no special annotation.

> 
> 2) yes, this basically requires a placement-new like thing to be implemented in the VM.  No disagreement that it's intrusive.
> 

It's more than intrusive,. It's incompatible with the semantics of all current object-instntiating code, making auto-magical optimization that coverts current instantiations to placement-new a "very hard" thing.

Unfortunately, both #1 and #2 would need to be addressed for an optimization to apply, so it's enough for one to be "too hard" for non of it to fly.

> sent from my phone
> 
> On Feb 3, 2015 2:10 PM, "Gil Tene" <gil at azulsystems.com> wrote:
> 
>> On Feb 3, 2015, at 9:13 AM, Vitaly Davidovich <vitalyd at gmail.com> wrote:
>> 
>> Gil, not sure if you saw my reply to Volker, but I agree -- I was simply asking why request this optimization via explicit syntax and not do it automatically in the runtime (with all the same restrictions, caveats, fine print, etc).
> 
> Well, there are a few reasons why this is "very hard":
> 
> 1. It's "hard" to discern at runtime that an object assigned to a final reference field can/should be "inlined" into it's containing object:
> 
> 1.1 Object-Inlining decisions are not instance-specific, they are class-global.
> 
> 1.1.1 If a single containing instance inlines such an object, all instances of the same class must spend the space.
> 
> 1.1.2 If a single "containing" instance does not inline the object referred to by the final reference field, NONE of the instances of the containing class can gain from the deference-avoiding (dead reckoning) optimization.
> 
> 1.1.3 If we tried to inline all final reference declared objects without a specific declaration of intent, we'd likely end up with a lot  of wasted space.
> 
> 2. Construction semantics: Even if you could auto-discern the need to "inline" the objects, you face a hard problem is dealing with how the "inlined" object (not the final ref pointing to it) is constructed and initialized:
> 
> 2.1 Any factory-based instantiation of the referred to object is by-definition not inline-able (the factory based creation with a specific inlining intent and enough provided context, such as constructWithin(), being an exception).
> 
> 2.2 Current non-factory based instantiation options in Java (new, reflection, methodHandles) all perform their own internal allocation of instance storage, and would not be able to use the "inlined" space without some serious surgery. Even if such surgery to all internal instantiation forms was done, getting the target instance location to the instantiation code is also "very hard", given that instantiation logically occurs before the assignment of the resulting reference to the final field, and many operations can happen in between. Approaches that attempt to override a sequence of operations (e.g. the simplest stuff like new; dup; push arg1; push arg2; invoke_special; putfield (into final ref field), or much more complicated ones...) such that the intermediate heap reference is never exposed and can be replaced with a reference to the already-allocated space only work in trivial situations, and tend to fail in all sorts of interesting common-case ways (e.g. when perfectly innocent instrumentation is involved).
> 
> Now "very hard" probably does not mean impossible. But it has so much open ended stuff that makes it a huge thing to tackle safely. Given the fact that much of the context of org.ObjectLayout benefits will never be auto-discernible without using explicit statements to describe the expected semantics (using a StructuredArray to state optimization-enabling limiting semantics, for example), adding explicit intent declaration (as opposed to auto-optimization) for intrinsic objects seems natural there.
> 
>> 
>> On Tue, Feb 3, 2015 at 11:58 AM, Gil Tene <gil at azulsystems.com> wrote:
>> A couple of point here, specific to org.ObjectLayout (http://objectlayout.org):
>> 
>> Declaration:
>> The ObjectLayout @Intrinsic (http://objectlayout.org) annotation is used for declaring what you refer to as "inline" objects. It is specifically not intended to be a layout control directive, but an optimization hint. Whether or not a JVM "inlines" the intrinsic object within the containing one, and how/where that "inlining" happens becomes a JVM-specific implementation concern, and no a semantic one.
>> 
>> Field initialization:
>> 
>> Implicit, undeclared choices to "inline" all final referenced fields fail very quickly when attempted in practice. E.g. final fields can (and often will) be set to refer to pre-existing-at-construction-time objects, which are (by definition) impossible to "inline". In addition, there are many common uses final reference fields where "inlining" is no possible because the actual object size of the referred-to object is not a global constant (e.g. it will be set to a construction-time or parameter-based choice of subclass).
>> 
>> We've given Intrinsic Object initialization a lot of thought in org.ObjectLayout. The dedicated initialization API is there to assure several things, including exact-type (the field's specific declared type) choice. It could turn into a less-verbose version in some future JDK if language support was added (e.g. to avoid mentioning the needed-only-to-conform-with-syntax things like "this", and the field name in the constructWithin() call), but I expect the semantics to need to be similar even if the syntax was made less verbose.
>> 
>> -- Gil.
>> 
>> > On Feb 3, 2015, at 8:40 AM, Vitaly Davidovich <vitalyd at gmail.com> wrote:
>> >
>> > Hi Volker,
>> >
>> > Sorry, I may have been unclear in my question.  As you say, ObjectLayout
>> > requires that you annotate the fields that you'd like inlined and then also
>> > use special API to construct those objects.  I'm wondering whether,
>> > instead, all private final fields are automatically inlined, and only cases
>> > where you'd like to layout the field out-of-band would require annotation.
>> > This would be controlled via a cmdline flag, as you say, similar to perhaps
>> > how compressed oops are enabled (or not).  Note that I'm talking about
>> > purely layout of reference types, not value types.
>> >
>> > The "concern" with having to explicitly annotate and use dedicated APIs to
>> > opt-in is that adoption will be fairly low, whereas I think most of the
>> > time one would want inlined storage layout.
>> >
>> > Thanks
>> >
>> >
>> > On Tue, Feb 3, 2015 at 11:29 AM, Volker Simonis <volker.simonis at gmail.com>
>> > wrote:
>> >
>> >> Hi Vitaly,
>> >>
>> >> for PackedObjects/ObjectLayout you need to specially annotate the
>> >> classes and/or fields which you want to allocate "inline". Once you've
>> >> done that you have no choice with the PackedObjects approach.
>> >> ObjectLayout is a little special here, because it can run with any
>> >> Java VM in which case it will still use the default reference model.
>> >> But it can potentially be optimized by some VM's to provide the flat
>> >> object layout. I expect these optimizations to be controllable by a
>> >> command line option.
>> >>
>> >> With the "Value Types for Java" [1] approach you'll have the
>> >> possiblitly to express the behavior right in Java like in the
>> >> following example from [1]:
>> >>
>> >> final __ByValue class Point {
>> >> static Point origin = __MakeValue(0, 0);
>> >>
>> >> I think the default will always be "reference semantics" in Java but
>> >> with various degrees of freedom to optionally choose value semantics.
>> >>
>> >> Regards,
>> >> Volker
>> >>
>> >> [1] http://cr.openjdk.java.net/~jrose/values/values-0.html
>> >>
>> >> On Mon, Feb 2, 2015 at 9:19 PM, Vitaly Davidovich <vitalyd at gmail.com>
>> >> wrote:
>> >>> Volker (or anyone else for that matter),
>> >>>
>> >>> Just curious -- do you envision "inline" layout of objects as something
>> >> one
>> >>> would have to opt-in or as the default layout for all objects in a heap?
>> >> It
>> >>> seems like this should be the default (assuming zero to minimal overhead
>> >> for
>> >>> loading the references) as I think wanting "out of line" allocations is
>> >> more
>> >>> rare.
>> >>>
>> >>> Thanks
>> >>>
>> >>> On Mon, Feb 2, 2015 at 2:06 PM, Volker Simonis <volker.simonis at gmail.com
>> >>>
>> >>> wrote:
>> >>>>
>> >>>> Hi Brian,
>> >>>>
>> >>>> thanks a lot for your detailed answer and apologies for the late reply
>> >>>> (I was a little distracted by FOSDEM :)
>> >>>>
>> >>>> All your comments have been clear and reasonable and are much
>> >>>> appreciated. Please find my additional answers inline:
>> >>>>
>> >>>> On Thu, Jan 29, 2015 at 6:05 PM, Brian Goetz <brian.goetz at oracle.com>
>> >>>> wrote:
>> >>>>>> Question: is JEP 169 still under active development or has it been
>> >>>>>> merged into the more general "Value types for Java" proposal below?
>> >>>>>
>> >>>>>
>> >>>>> It has been merged into the more general Value Types for Java
>> >> proposal.
>> >>>>>
>> >>>>
>> >>>> Then maybe this JEP should be closed to avoid further confusion?
>> >>>>
>> >>>>>> The "Value types for Java" approach clearly seems to be the most
>> >>>>>> general but also the most complex proposal.
>> >>>>>
>> >>>>>
>> >>>>> For some meanings of "complex".  It is certainly the most intrusive
>> >> and
>> >>>>> large; new bytecodes, new type signatures.  But from a user-model
>> >>>>> perspective, value types are actually fairly simple.
>> >>>>>
>> >>>>>> It's out of scope for Java
>> >>>>>> 9 and still questionable for Java 10 and above. The "PackedObject"
>> >> and
>> >>>>>> "ObjectLayout" approaches are clearly simpler and more limited in
>> >>>>>> scope as they only concentrate on better object layout.
>> >>>>>
>> >>>>>
>> >>>>> To your list, I'd add: Project Panama, the sister project to Valhalla.
>> >>>>> Panama focuses on interop with native code and data, including layout
>> >>>>> specification.  A key goal of Packed was to be able to access off-heap
>> >>>>> native data in its native format, rather than marshalling it across
>> >> the
>> >>>>> JNI
>> >>>>> boundary.  Panama is focused on this problem as well, but aims to
>> >> treat
>> >>>>> it
>> >>>>> as a separate problem from Java object layout, resulting in what we
>> >>>>> believe
>> >>>>> to be a cleaner decomposition of the two concerns.
>> >>>>>
>> >>>>
>> >>>> Your right. I somehow missed to look at Panama more deeply because I
>> >>>> always thought it is only about FFI. John Rose nicely explains the
>> >>>> various parts of Panama in this mail
>> >>>>
>> >> http://mail.openjdk.java.net/pipermail/panama-dev/2014-October/000042.html
>> >>>> where he also mentions the intention of Panama to create new flatter
>> >>>> data layouts in the Heap and the relation of Panama to PackedObjects
>> >>>> and ObjectLayout.
>> >>>>
>> >>>>> Packed is an interesting mix of memory density (object embedding and
>> >>>>> packed
>> >>>>> arrays) and native interop.  But mixing the two goals also has costs;
>> >>>>> our
>> >>>>> approach is to separate them into orthogonal concerns, and we think
>> >> that
>> >>>>> Valhalla and Panama do just that.  So in many ways, while a larger
>> >>>>> project,
>> >>>>> the combination of Valhalla+Panama addresses the problem that Packed
>> >>>>> did, in
>> >>>>> a cleaner way.
>> >>>>>
>> >>>>>> Question: is there a chance to get a some sort of Java-only but
>> >>>>>> transparently optimizable structure package like "ObjectLayout" into
>> >>>>>> Java early (i.e. Java 9)?
>> >>>>>
>> >>>>>
>> >>>>> It would depend on a lot of things -- including the level of readiness
>> >>>>> of
>> >>>>> the design and implementation, and the overlap with anticipated future
>> >>>>> features.  We've reviewed some of the early design of ObjectLayout and
>> >>>>> provided feedback to the projects architects; currently, I think it's
>> >> in
>> >>>>> the
>> >>>>> "promising exploration" stage, but I think multiple rounds of
>> >>>>> simplification
>> >>>>> are needed before it is ready to be considered for "everybody's Java."
>> >>>>> But
>> >>>>> if the choice is to push something that's not ready into 9, or to wait
>> >>>>> longer -- there's not actually a choice to be made there.
>> >>>>>
>> >>>>> I appreciate the desire to "get something you can use now", but we
>> >> have
>> >>>>> to
>> >>>>> be prepared to support whatever we push into Java for the next 20
>> >> years,
>> >>>>> and
>> >>>>> deal with the additional constraints it generates -- which can be an
>> >>>>> enormous cost.  (Even thought the direct cost is mostly borne by
>> >> Oracle,
>> >>>>> the
>> >>>>> indirect cost is borne by everyone, in the form of slower progress on
>> >>>>> everything else.)  So I am very wary of the motivation of "well,
>> >>>>> something
>> >>>>> better is coming, but this works now, so can we push it in?"  I'd
>> >> prefer
>> >>>>> to
>> >>>>> focus on answering whether this is right thing for Java for the next
>> >> 20
>> >>>>> years.
>> >>>>>
>> >>>>>> In my eyes this wouldn't contradict with a more general solution like
>> >>>>>> the one proposed in the "Value types for Java" approach while still
>> >>>>>> offering quite significant performance improvements for quite a big
>> >>>>>> range of problems.
>> >>>>>
>> >>>>>
>> >>>>> The goals of the ObjectLayout effort has overlap with, but also
>> >> differs
>> >>>>> from, the goals of Valhalla.  And herein is the problem; neither
>> >>>>> generalizes
>> >>>>> the other, and I don't think we do the user base a great favor by
>> >>>>> pursuing
>> >>>>> two separate neither-coincident-nor-orthogonal approaches.  I suspect,
>> >>>>> though, that after a few rounds of simplification, ObjectLayout could
>> >>>>> morph
>> >>>>> into something that fit either coincidently or orthogonally with the
>> >>>>> Valhalla work -- which would be great.  But, as you know, our
>> >> resources
>> >>>>> are
>> >>>>> limited, so we (Oracle) can't really afford to invest in both.  And
>> >> such
>> >>>>> simplification takes time -- getting to that "aha" moment when you
>> >>>>> realize
>> >>>>> you can simplify something is generally an incompressible process.
>> >>>>>
>> >>>>>> Question: what would be the right place to propose something like the
>> >>>>>> "ObjectLayout" library for Java 9/10? Would that fit within the
>> >>>>>> umbrella of the Valhalla project or would it be done within its own
>> >>>>>> project / under it's own JEP?
>> >>>>>
>> >>>>>
>> >>>>> Suggesting a version number at this point would be putting the cart
>> >>>>> before
>> >>>>> the horse (you'll note that we've not even proposed a version number
>> >> for
>> >>>>> Valhalla; the closest we've gotten to that is "after 9".)
>> >>>>>
>> >>>>> OpenJDK Projects are a tool for building a community around a body of
>> >>>>> work;
>> >>>>> JEPs are a project-management tool for defining, scoping, and tracking
>> >>>>> the
>> >>>>> progress of a feature.  Given where OL is, it would be reasonable to
>> >>>>> start a
>> >>>>> Project, which would become the nexus of collaboration that could
>> >>>>> eventually
>> >>>>> produce a JEP.
>> >>>>>
>> >>>>
>> >>>> That sounds reasonable. I'll speak with the ObjectLayout people to
>> >>>> hear what they think about starting a new OpenJDK project.
>> >>>>
>> >>>> And after all you've said before, I also came to the conclusion that
>> >>>> investigating ways of new in-heap object layouts seem to be better of
>> >>>> in the Panama project. So I'll also ask there what they think about
>> >>>> it. Maybe ObjectLayout could become part of Panama or maybe we could
>> >>>> just start a new subproject of Panama with the same/similar goals.
>> >>>>
>> >>>>> Hope this helps,
>> >>>>> -Brian
>> >>>>
>> >>>> It really did!
>> >>>>
>> >>>> Thanks again,
>> >>>> Volker
>> >>>> _______________________________________________
>> >>>> mlvm-dev mailing list
>> >>>> mlvm-dev at openjdk.java.net
>> >>>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>> >>>
>> >>>
>> >>>
>> >>> _______________________________________________
>> >>> mlvm-dev mailing list
>> >>> mlvm-dev at openjdk.java.net
>> >>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>> >>>
>> >> _______________________________________________
>> >> mlvm-dev mailing list
>> >> mlvm-dev at openjdk.java.net
>> >> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>> >>
>> 
>> 
> 


From maurizio.cimadamore at oracle.com  Tue Feb  3 21:34:19 2015
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Tue, 03 Feb 2015 21:34:19 +0000
Subject: experiences with prototype
In-Reply-To: <54D1380A.5070503@gmail.com>
References: <54D113DA.9060509@gmail.com> <54D11792.3090504@oracle.com>
	<54D11CEC.6080009@oracle.com> <54D1380A.5070503@gmail.com>
Message-ID: <54D13EDB.5050704@oracle.com>


On 03/02/15 21:05, Peter Levart wrote:
> Hi Maurizio,
>
> I see. I thought this could be a nice idiom for boxing, since the 
> following:
>
> (Object) 42
>
> ...is legal and results in an Integer object at runtime.
I'm not saying this will never work - actually the compiler is currently 
accepting this kind of idioms, but the specializer does nothing with it, 
so you'll get runtime errors.
>
> But I don't know if a checkcast is actually inserted for (Object). 
> Could javac redundantly do it in case casting to Object is from 
> expression of <any> type and also equip checkcast with BMA indicating 
> the type of expression so that specialization could replace it with 
> boxing code?
That will be the way forward, yes

Maurizio
>
> Regards, Peter
>
>
> On 02/03/2015 08:09 PM, Maurizio Cimadamore wrote:
>>
>> On 03/02/15 18:46, Maurizio Cimadamore wrote:
>>> I will also investigate on the crash you are getting... 
>> Hi Peter,
>> the crash is coming from this code in AbstractCollection (see code in 
>> bold):
>>
>>     public boolean contains(Object o) {
>>         __WhereVal(E) {
>>             Iterator<E> it = iterator();
>>             if (o == null) {
>>                 return false;
>>             } else {
>>                 while (it.hasNext())
>> *if (o.equals((Object) it.next()))*
>>                         return true;
>>             }
>>             return false;
>>         }
>>         __WhereRef(E) {
>>             Iterator<E> it = iterator();
>>             if (o == null) {
>>                 while (it.hasNext())
>>                     if (it.next() == null)
>>                         return true;
>>             } else {
>>                 while (it.hasNext())
>>                     if (o.equals(it.next()))
>>                         return true;
>>             }
>>             return false;
>>         }
>>     }
>>
>> I believe that, apart from the obvious javac bug, the code has an 
>> issue, as it.next() is supposed to return a value there, but you are 
>> casting to Object?
>>
>> For the records - a simpler test case for the bug is this:
>>
>> class Foo<any E> {
>>     E e;
>>     E get() { return e; }
>>
>>     void test() {
>>         __WhereVal(E) {
>>             Object o = (Object)get();
>>         }
>>     }
>> }
>>
>> Maurizio
>


From vitalyd at gmail.com  Tue Feb  3 21:44:02 2015
From: vitalyd at gmail.com (Vitaly Davidovich)
Date: Tue, 3 Feb 2015 16:44:02 -0500
Subject: What's the status of / relation between "JEP 169: Value Objects"
	/ "Value Types for Java" / "Object Layout"
In-Reply-To: <5C9C5691-B447-46D9-A4D5-7778E2BFA774@azulsystems.com>
References: <CA+3eh122g3Ax+-PCL3EvX3V-cQkorva2j_c42_i4uRLOqw42Fg@mail.gmail.com>
	<54CA6853.2060601@oracle.com>
	<CA+3eh13pg6+ZziNuzLNdPSc8AkA4o1N9Xx0kGYFNm7TV3t9yzQ@mail.gmail.com>
	<CAHjP37HO8b3kgaWy4pVXpY3ky=+ebhLHBafMeNnZ-QooA9bXkA@mail.gmail.com>
	<CA+3eh135AOaiHoFQaA8Uc7WFsG-rHeeF30MBHw52J=fev9c1CQ@mail.gmail.com>
	<CAHjP37GhZYk61Z12L6Yj24YCwmyaao2oWJGRva6+gw1wGzGpOQ@mail.gmail.com>
	<505C1729-7E97-4D3B-8760-A3AF0BB76376@azulsystems.com>
	<CAHjP37F5SQfH41_FeDmB_0__887HqpcLMRUdVPuhFvMh9EA-EA@mail.gmail.com>
	<0A138385-7F6E-4A2E-A9A2-79240B39D0A2@azulsystems.com>
	<CAHjP37Ey-w7AeqJ8isoOov4K2Z6vG5o2bdRcJmn0Pp_WEesGKQ@mail.gmail.com>
	<5C9C5691-B447-46D9-A4D5-7778E2BFA774@azulsystems.com>
Message-ID: <CAHjP37GFdxf25XZj=2YiXZiTJ2nxoEroKoy3Xngm7h8jJx14Vw@mail.gmail.com>

>
> But even if #2 is solved somehow for some subset of instantiations that
> the VM can auto-magically determine can be converted to a "placement-new"
> sort of allocation / construction sequence, there are plenty of final field
> assignment cases where the final ref field refers to an object that is
> already referred to elsewhere (before the assignment). E.g. it is quite
> common for private final ref fields to be initialized from constructor
> parameters (e.g. HashMap.Entry). It is also common to gave final ref fields
> assigned to the return values of method calls. And for those final fields
> to refer to object that are common across multiple referring instances. All
> those cases are non-inlineable by definition.


I'm probably not making myself clear here.  The automagic optimization I'm
talking about right now is purely for the same situations where one would
use ObjectLayout.createXXX instead, except it's determined by the runtime.
Now, I'm handwaving over the "determined by the runtime" aspect, and for
sake of this discussion, considering only the "obvious" candidates (say, an
explicit "new" operator and the field is not set from constructor argument
at all).

> It's not nulls. It's a reference to something that isn't "inlineable".
> Like setting the final field to a reference to an already-allocated-at-assignement-time
> object.


> If the VM can determine ahead of time (ahead of instantiating any
> instances of a containing class) that ALL final field assignments performed
> in the future will be made from in-place instantiations that can be safely
> and auto-magically converted to placement-news, then there will be no
> wasted space. However, for every class where some late-discovered
> (discovered after an instance has been created) situation leads to a loss
> of the optimization, potential memory waste is also there.


> It's true that the memory waste would be contained to the cases where the
> reference is assigned to a non-"placement-new" result, and that in mixed
> situations the pre-allocated space in the containing object will still be
> used. It's just hard to guess how much of each case there is in real code.


Hmm, I wasn't even thinking of doing any dynamic optimizations here --
without giving it too much thought, it would be something determined at
class load time.  A "verification" pass would be done of the class that
would determine whether this class is suitable for this optimization
(again, let's assume very straightforward requirement -- final field set
with explicit "new" call).  This is somewhat analogous to using compressed
oops -- at heap init time, VM checks what the max heap size could be, and
then determines whether it can use compressed oops and what kind -- it
doesn't matter whether the heap actually gets to that size or not, this is
not toggled dynamically.  So, in this case, the only waste I could see is
if nulls are stored -- class layout would contain reserved space for the
embedded object, but it's actually null for most instances of the class,
leading to waste.

I think cache line co-placement makes sense for fields (e.g. an @Together
> thing to mirror the @Contended stuff), but I doubt that it makes much sense
> for objects, mostly due to size (not that many realistic objects will fit
> together in one cache line anyway). A natural "inline in declaration order"
> approach for intrinsic objects is probably just as good as anything else,
> and will require no special annotation.


Right, this does depend on size of the objects and you're probably right
here.  Given that object headers would still be present, 4 objects' worth
of headers is already a cacheline on most 64bit architectures.  But for
those cases where the cpu uses 2 lines as the coherence unit and/or
cacheline sizes grow, it could be useful.


On Tue, Feb 3, 2015 at 4:08 PM, Gil Tene <gil at azulsystems.com> wrote:

>
> > On Feb 3, 2015, at 12:20 PM, Vitaly Davidovich <vitalyd at gmail.com>
> wrote:
> >
> > 1) My assumption is that vast majority of classes will benefit from
> inline storage of their collaborators.  The waste would be in classes that
> have hot and cold members together, but I think that's the minority? But,
> for those cases, it would be beneficial to allow out-of-band layout (i.e.
> today's layout) via annotation or whatever.
>
> I *think* it's a small (and fragile) minority, rather than the vast
> majority. That is, I think that auto-magic (implicit optimization without a
> special factory based instantiation) is limited in potential scope and
> impact: Without special magic (see #2 below), 100% of current final field
> assignments are non-inlineable because they (by definition) assign the
> field to a reference value that refers to an
> already-allocated-at-assignment-time object.
>
> #2 is hard enough to do for trivial case (those that initialize the field
> to refer to an in-place instantiated object that does not escape ahead of
> the assignment, and whose all escaping post-assignemnt references can be
> safely hunted down and converted). And it is much harder for the plentiful
> non-trivial cases (the allocation/instantiation are done "far enough apart"
> such that interleaved operations complexify the analysis).
>
> But even if #2 is solved somehow for some subset of instantiations that
> the VM can auto-magically determine can be converted to a "placement-new"
> sort of allocation / construction sequence, there are plenty of final field
> assignment cases where the final ref field refers to an object that is
> already referred to elsewhere (before the assignment). E.g. it is quite
> common for private final ref fields to be initialized from constructor
> parameters (e.g. HashMap.Entry). It is also common to gave final ref fields
> assigned to the return values of method calls. And for those final fields
> to refer to object that are common across multiple referring instances. All
> those cases are non-inlineable by definition.
>
> > Why do you say this would likely lead to memory waste? If the object is
> allocated one way or another, it's going to take up space.  The only case i
> see being problematic is when null is stored in majority of instances.
>
> It's not nulls. It's a reference to something that isn't "inlineable".
> Like setting the final field to a reference to an
> already-allocated-at-assignement-time object.
>
> If the VM can determine ahead of time (ahead of instantiating any
> instances of a containing class) that ALL final field assignments performed
> in the future will be made from in-place instantiations that can be safely
> and auto-magically converted to placement-news, then there will be no
> wasted space. However, for every class where some late-discovered
> (discovered after an instance has been created) situation leads to a loss
> of the optimization, potential memory waste is also there.
>
> It's true that the memory waste would be contained to the cases where the
> reference is assigned to a non-"placement-new" result, and that in mixed
> situations the pre-allocated space in the containing object will still be
> used. It's just hard to guess how much of each case there is in real code.
>
> > I guess this boils down to whether one thinks it's more likely, by
> default, that objects embedded in others are hot/cold in terms of access; I
> think it's more common for them to be accessed together and so default
> should cater to that.
> >
> > When/if inlined layout is available, next logical thing one may request
> is specifying layout *order* to try and place commonly accessed data on
> same cacheline.  This is less important for streaming cases, but would be
> nice for random walks.
>
> I think cache line co-placement makes sense for fields (e.g. an @Together
> thing to mirror the @Contended stuff), but I doubt that it makes much sense
> for objects, mostly due to size (not that many realistic objects will fit
> together in one cache line anyway). A natural "inline in declaration order"
> approach for intrinsic objects is probably just as good as anything else,
> and will require no special annotation.
>
> >
> > 2) yes, this basically requires a placement-new like thing to be
> implemented in the VM.  No disagreement that it's intrusive.
> >
>
> It's more than intrusive,. It's incompatible with the semantics of all
> current object-instntiating code, making auto-magical optimization that
> coverts current instantiations to placement-new a "very hard" thing.
>
> Unfortunately, both #1 and #2 would need to be addressed for an
> optimization to apply, so it's enough for one to be "too hard" for non of
> it to fly.
>
> > sent from my phone
> >
> > On Feb 3, 2015 2:10 PM, "Gil Tene" <gil at azulsystems.com> wrote:
> >
> >> On Feb 3, 2015, at 9:13 AM, Vitaly Davidovich <vitalyd at gmail.com>
> wrote:
> >>
> >> Gil, not sure if you saw my reply to Volker, but I agree -- I was
> simply asking why request this optimization via explicit syntax and not do
> it automatically in the runtime (with all the same restrictions, caveats,
> fine print, etc).
> >
> > Well, there are a few reasons why this is "very hard":
> >
> > 1. It's "hard" to discern at runtime that an object assigned to a final
> reference field can/should be "inlined" into it's containing object:
> >
> > 1.1 Object-Inlining decisions are not instance-specific, they are
> class-global.
> >
> > 1.1.1 If a single containing instance inlines such an object, all
> instances of the same class must spend the space.
> >
> > 1.1.2 If a single "containing" instance does not inline the object
> referred to by the final reference field, NONE of the instances of the
> containing class can gain from the deference-avoiding (dead reckoning)
> optimization.
> >
> > 1.1.3 If we tried to inline all final reference declared objects without
> a specific declaration of intent, we'd likely end up with a lot  of wasted
> space.
> >
> > 2. Construction semantics: Even if you could auto-discern the need to
> "inline" the objects, you face a hard problem is dealing with how the
> "inlined" object (not the final ref pointing to it) is constructed and
> initialized:
> >
> > 2.1 Any factory-based instantiation of the referred to object is
> by-definition not inline-able (the factory based creation with a specific
> inlining intent and enough provided context, such as constructWithin(),
> being an exception).
> >
> > 2.2 Current non-factory based instantiation options in Java (new,
> reflection, methodHandles) all perform their own internal allocation of
> instance storage, and would not be able to use the "inlined" space without
> some serious surgery. Even if such surgery to all internal instantiation
> forms was done, getting the target instance location to the instantiation
> code is also "very hard", given that instantiation logically occurs before
> the assignment of the resulting reference to the final field, and many
> operations can happen in between. Approaches that attempt to override a
> sequence of operations (e.g. the simplest stuff like new; dup; push arg1;
> push arg2; invoke_special; putfield (into final ref field), or much more
> complicated ones...) such that the intermediate heap reference is never
> exposed and can be replaced with a reference to the already-allocated space
> only work in trivial situations, and tend to fail in all sorts of
> interesting common-case ways (e.g. when perfectly innocent instrumentation
> is involved).
> >
> > Now "very hard" probably does not mean impossible. But it has so much
> open ended stuff that makes it a huge thing to tackle safely. Given the
> fact that much of the context of org.ObjectLayout benefits will never be
> auto-discernible without using explicit statements to describe the expected
> semantics (using a StructuredArray to state optimization-enabling limiting
> semantics, for example), adding explicit intent declaration (as opposed to
> auto-optimization) for intrinsic objects seems natural there.
> >
> >>
> >> On Tue, Feb 3, 2015 at 11:58 AM, Gil Tene <gil at azulsystems.com> wrote:
> >> A couple of point here, specific to org.ObjectLayout (
> http://objectlayout.org):
> >>
> >> Declaration:
> >> The ObjectLayout @Intrinsic (http://objectlayout.org) annotation is
> used for declaring what you refer to as "inline" objects. It is
> specifically not intended to be a layout control directive, but an
> optimization hint. Whether or not a JVM "inlines" the intrinsic object
> within the containing one, and how/where that "inlining" happens becomes a
> JVM-specific implementation concern, and no a semantic one.
> >>
> >> Field initialization:
> >>
> >> Implicit, undeclared choices to "inline" all final referenced fields
> fail very quickly when attempted in practice. E.g. final fields can (and
> often will) be set to refer to pre-existing-at-construction-time objects,
> which are (by definition) impossible to "inline". In addition, there are
> many common uses final reference fields where "inlining" is no possible
> because the actual object size of the referred-to object is not a global
> constant (e.g. it will be set to a construction-time or parameter-based
> choice of subclass).
> >>
> >> We've given Intrinsic Object initialization a lot of thought in
> org.ObjectLayout. The dedicated initialization API is there to assure
> several things, including exact-type (the field's specific declared type)
> choice. It could turn into a less-verbose version in some future JDK if
> language support was added (e.g. to avoid mentioning the
> needed-only-to-conform-with-syntax things like "this", and the field name
> in the constructWithin() call), but I expect the semantics to need to be
> similar even if the syntax was made less verbose.
> >>
> >> -- Gil.
> >>
> >> > On Feb 3, 2015, at 8:40 AM, Vitaly Davidovich <vitalyd at gmail.com>
> wrote:
> >> >
> >> > Hi Volker,
> >> >
> >> > Sorry, I may have been unclear in my question.  As you say,
> ObjectLayout
> >> > requires that you annotate the fields that you'd like inlined and
> then also
> >> > use special API to construct those objects.  I'm wondering whether,
> >> > instead, all private final fields are automatically inlined, and only
> cases
> >> > where you'd like to layout the field out-of-band would require
> annotation.
> >> > This would be controlled via a cmdline flag, as you say, similar to
> perhaps
> >> > how compressed oops are enabled (or not).  Note that I'm talking about
> >> > purely layout of reference types, not value types.
> >> >
> >> > The "concern" with having to explicitly annotate and use dedicated
> APIs to
> >> > opt-in is that adoption will be fairly low, whereas I think most of
> the
> >> > time one would want inlined storage layout.
> >> >
> >> > Thanks
> >> >
> >> >
> >> > On Tue, Feb 3, 2015 at 11:29 AM, Volker Simonis <
> volker.simonis at gmail.com>
> >> > wrote:
> >> >
> >> >> Hi Vitaly,
> >> >>
> >> >> for PackedObjects/ObjectLayout you need to specially annotate the
> >> >> classes and/or fields which you want to allocate "inline". Once
> you've
> >> >> done that you have no choice with the PackedObjects approach.
> >> >> ObjectLayout is a little special here, because it can run with any
> >> >> Java VM in which case it will still use the default reference model.
> >> >> But it can potentially be optimized by some VM's to provide the flat
> >> >> object layout. I expect these optimizations to be controllable by a
> >> >> command line option.
> >> >>
> >> >> With the "Value Types for Java" [1] approach you'll have the
> >> >> possiblitly to express the behavior right in Java like in the
> >> >> following example from [1]:
> >> >>
> >> >> final __ByValue class Point {
> >> >> static Point origin = __MakeValue(0, 0);
> >> >>
> >> >> I think the default will always be "reference semantics" in Java but
> >> >> with various degrees of freedom to optionally choose value semantics.
> >> >>
> >> >> Regards,
> >> >> Volker
> >> >>
> >> >> [1] http://cr.openjdk.java.net/~jrose/values/values-0.html
> >> >>
> >> >> On Mon, Feb 2, 2015 at 9:19 PM, Vitaly Davidovich <vitalyd at gmail.com
> >
> >> >> wrote:
> >> >>> Volker (or anyone else for that matter),
> >> >>>
> >> >>> Just curious -- do you envision "inline" layout of objects as
> something
> >> >> one
> >> >>> would have to opt-in or as the default layout for all objects in a
> heap?
> >> >> It
> >> >>> seems like this should be the default (assuming zero to minimal
> overhead
> >> >> for
> >> >>> loading the references) as I think wanting "out of line"
> allocations is
> >> >> more
> >> >>> rare.
> >> >>>
> >> >>> Thanks
> >> >>>
> >> >>> On Mon, Feb 2, 2015 at 2:06 PM, Volker Simonis <
> volker.simonis at gmail.com
> >> >>>
> >> >>> wrote:
> >> >>>>
> >> >>>> Hi Brian,
> >> >>>>
> >> >>>> thanks a lot for your detailed answer and apologies for the late
> reply
> >> >>>> (I was a little distracted by FOSDEM :)
> >> >>>>
> >> >>>> All your comments have been clear and reasonable and are much
> >> >>>> appreciated. Please find my additional answers inline:
> >> >>>>
> >> >>>> On Thu, Jan 29, 2015 at 6:05 PM, Brian Goetz <
> brian.goetz at oracle.com>
> >> >>>> wrote:
> >> >>>>>> Question: is JEP 169 still under active development or has it
> been
> >> >>>>>> merged into the more general "Value types for Java" proposal
> below?
> >> >>>>>
> >> >>>>>
> >> >>>>> It has been merged into the more general Value Types for Java
> >> >> proposal.
> >> >>>>>
> >> >>>>
> >> >>>> Then maybe this JEP should be closed to avoid further confusion?
> >> >>>>
> >> >>>>>> The "Value types for Java" approach clearly seems to be the most
> >> >>>>>> general but also the most complex proposal.
> >> >>>>>
> >> >>>>>
> >> >>>>> For some meanings of "complex".  It is certainly the most
> intrusive
> >> >> and
> >> >>>>> large; new bytecodes, new type signatures.  But from a user-model
> >> >>>>> perspective, value types are actually fairly simple.
> >> >>>>>
> >> >>>>>> It's out of scope for Java
> >> >>>>>> 9 and still questionable for Java 10 and above. The
> "PackedObject"
> >> >> and
> >> >>>>>> "ObjectLayout" approaches are clearly simpler and more limited in
> >> >>>>>> scope as they only concentrate on better object layout.
> >> >>>>>
> >> >>>>>
> >> >>>>> To your list, I'd add: Project Panama, the sister project to
> Valhalla.
> >> >>>>> Panama focuses on interop with native code and data, including
> layout
> >> >>>>> specification.  A key goal of Packed was to be able to access
> off-heap
> >> >>>>> native data in its native format, rather than marshalling it
> across
> >> >> the
> >> >>>>> JNI
> >> >>>>> boundary.  Panama is focused on this problem as well, but aims to
> >> >> treat
> >> >>>>> it
> >> >>>>> as a separate problem from Java object layout, resulting in what
> we
> >> >>>>> believe
> >> >>>>> to be a cleaner decomposition of the two concerns.
> >> >>>>>
> >> >>>>
> >> >>>> Your right. I somehow missed to look at Panama more deeply because
> I
> >> >>>> always thought it is only about FFI. John Rose nicely explains the
> >> >>>> various parts of Panama in this mail
> >> >>>>
> >> >>
> http://mail.openjdk.java.net/pipermail/panama-dev/2014-October/000042.html
> >> >>>> where he also mentions the intention of Panama to create new
> flatter
> >> >>>> data layouts in the Heap and the relation of Panama to
> PackedObjects
> >> >>>> and ObjectLayout.
> >> >>>>
> >> >>>>> Packed is an interesting mix of memory density (object embedding
> and
> >> >>>>> packed
> >> >>>>> arrays) and native interop.  But mixing the two goals also has
> costs;
> >> >>>>> our
> >> >>>>> approach is to separate them into orthogonal concerns, and we
> think
> >> >> that
> >> >>>>> Valhalla and Panama do just that.  So in many ways, while a larger
> >> >>>>> project,
> >> >>>>> the combination of Valhalla+Panama addresses the problem that
> Packed
> >> >>>>> did, in
> >> >>>>> a cleaner way.
> >> >>>>>
> >> >>>>>> Question: is there a chance to get a some sort of Java-only but
> >> >>>>>> transparently optimizable structure package like "ObjectLayout"
> into
> >> >>>>>> Java early (i.e. Java 9)?
> >> >>>>>
> >> >>>>>
> >> >>>>> It would depend on a lot of things -- including the level of
> readiness
> >> >>>>> of
> >> >>>>> the design and implementation, and the overlap with anticipated
> future
> >> >>>>> features.  We've reviewed some of the early design of
> ObjectLayout and
> >> >>>>> provided feedback to the projects architects; currently, I think
> it's
> >> >> in
> >> >>>>> the
> >> >>>>> "promising exploration" stage, but I think multiple rounds of
> >> >>>>> simplification
> >> >>>>> are needed before it is ready to be considered for "everybody's
> Java."
> >> >>>>> But
> >> >>>>> if the choice is to push something that's not ready into 9, or to
> wait
> >> >>>>> longer -- there's not actually a choice to be made there.
> >> >>>>>
> >> >>>>> I appreciate the desire to "get something you can use now", but we
> >> >> have
> >> >>>>> to
> >> >>>>> be prepared to support whatever we push into Java for the next 20
> >> >> years,
> >> >>>>> and
> >> >>>>> deal with the additional constraints it generates -- which can be
> an
> >> >>>>> enormous cost.  (Even thought the direct cost is mostly borne by
> >> >> Oracle,
> >> >>>>> the
> >> >>>>> indirect cost is borne by everyone, in the form of slower
> progress on
> >> >>>>> everything else.)  So I am very wary of the motivation of "well,
> >> >>>>> something
> >> >>>>> better is coming, but this works now, so can we push it in?"  I'd
> >> >> prefer
> >> >>>>> to
> >> >>>>> focus on answering whether this is right thing for Java for the
> next
> >> >> 20
> >> >>>>> years.
> >> >>>>>
> >> >>>>>> In my eyes this wouldn't contradict with a more general solution
> like
> >> >>>>>> the one proposed in the "Value types for Java" approach while
> still
> >> >>>>>> offering quite significant performance improvements for quite a
> big
> >> >>>>>> range of problems.
> >> >>>>>
> >> >>>>>
> >> >>>>> The goals of the ObjectLayout effort has overlap with, but also
> >> >> differs
> >> >>>>> from, the goals of Valhalla.  And herein is the problem; neither
> >> >>>>> generalizes
> >> >>>>> the other, and I don't think we do the user base a great favor by
> >> >>>>> pursuing
> >> >>>>> two separate neither-coincident-nor-orthogonal approaches.  I
> suspect,
> >> >>>>> though, that after a few rounds of simplification, ObjectLayout
> could
> >> >>>>> morph
> >> >>>>> into something that fit either coincidently or orthogonally with
> the
> >> >>>>> Valhalla work -- which would be great.  But, as you know, our
> >> >> resources
> >> >>>>> are
> >> >>>>> limited, so we (Oracle) can't really afford to invest in both.
> And
> >> >> such
> >> >>>>> simplification takes time -- getting to that "aha" moment when you
> >> >>>>> realize
> >> >>>>> you can simplify something is generally an incompressible process.
> >> >>>>>
> >> >>>>>> Question: what would be the right place to propose something
> like the
> >> >>>>>> "ObjectLayout" library for Java 9/10? Would that fit within the
> >> >>>>>> umbrella of the Valhalla project or would it be done within its
> own
> >> >>>>>> project / under it's own JEP?
> >> >>>>>
> >> >>>>>
> >> >>>>> Suggesting a version number at this point would be putting the
> cart
> >> >>>>> before
> >> >>>>> the horse (you'll note that we've not even proposed a version
> number
> >> >> for
> >> >>>>> Valhalla; the closest we've gotten to that is "after 9".)
> >> >>>>>
> >> >>>>> OpenJDK Projects are a tool for building a community around a
> body of
> >> >>>>> work;
> >> >>>>> JEPs are a project-management tool for defining, scoping, and
> tracking
> >> >>>>> the
> >> >>>>> progress of a feature.  Given where OL is, it would be reasonable
> to
> >> >>>>> start a
> >> >>>>> Project, which would become the nexus of collaboration that could
> >> >>>>> eventually
> >> >>>>> produce a JEP.
> >> >>>>>
> >> >>>>
> >> >>>> That sounds reasonable. I'll speak with the ObjectLayout people to
> >> >>>> hear what they think about starting a new OpenJDK project.
> >> >>>>
> >> >>>> And after all you've said before, I also came to the conclusion
> that
> >> >>>> investigating ways of new in-heap object layouts seem to be better
> of
> >> >>>> in the Panama project. So I'll also ask there what they think about
> >> >>>> it. Maybe ObjectLayout could become part of Panama or maybe we
> could
> >> >>>> just start a new subproject of Panama with the same/similar goals.
> >> >>>>
> >> >>>>> Hope this helps,
> >> >>>>> -Brian
> >> >>>>
> >> >>>> It really did!
> >> >>>>
> >> >>>> Thanks again,
> >> >>>> Volker
> >> >>>> _______________________________________________
> >> >>>> mlvm-dev mailing list
> >> >>>> mlvm-dev at openjdk.java.net
> >> >>>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
> >> >>>
> >> >>>
> >> >>>
> >> >>> _______________________________________________
> >> >>> mlvm-dev mailing list
> >> >>> mlvm-dev at openjdk.java.net
> >> >>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
> >> >>>
> >> >> _______________________________________________
> >> >> mlvm-dev mailing list
> >> >> mlvm-dev at openjdk.java.net
> >> >> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
> >> >>
> >>
> >>
> >
>
>

From peter.levart at gmail.com  Wed Feb  4 08:46:21 2015
From: peter.levart at gmail.com (Peter Levart)
Date: Wed, 04 Feb 2015 09:46:21 +0100
Subject: experiences with prototype
In-Reply-To: <54D13EDB.5050704@oracle.com>
References: <54D113DA.9060509@gmail.com> <54D11792.3090504@oracle.com>
	<54D11CEC.6080009@oracle.com> <54D1380A.5070503@gmail.com>
	<54D13EDB.5050704@oracle.com>
Message-ID: <54D1DC5D.7060408@gmail.com>

Hi Maurizio,

I have now managed to successfully compile the code. Here's the updated 
source:

http://cr.openjdk.java.net/~plevart/misc/valhala-hacks/javany-src.jar


But there's a StringIndexOutOfBoundsException thrown from specializer 
when running the following Test:

public class Test {
     public static void main(String[] args) {
         List<int> ints = Arrays.asList(new int[]{1, 2, 3, 4, 5, 6, 7, 8});
         Iterator<int> it = ints.iterator();
         while (it.hasNext()) {
             System.out.println(it.next());
         }
     }
}


Specializing javany.util.List${0=I}; searching for 
javany/util/List.class (not found)
Specializing javany.util.List${0=I}; searching for 
javany/util/List.class (found)
Specializing javany.util.Collection${0=I}; searching for 
javany/util/Collection.class (not found)
Specializing javany.util.Collection${0=I}; searching for 
javany/util/Collection.class (found)
Specializing javany.lang.Iterable${0=I}; searching for 
javany/lang/Iterable.class (not found)
Specializing javany.lang.Iterable${0=I}; searching for 
javany/lang/Iterable.class (found)
Specializing method 
javany/util/Arrays$asList${0=I}.asList([Ljava/lang/Object;)Ljavany/util/List; 
with class=[] and method=[I]
Specializing javany.util.Arrays$ArrayList${0=I}; searching for 
javany/util/Arrays$ArrayList.class (not found)
Specializing javany.util.Arrays$ArrayList${0=I}; searching for 
javany/util/Arrays$ArrayList.class (found)
Exception in thread "main" java.lang.StringIndexOutOfBoundsException: 
String index out of range: 0
         at java.lang.String.charAt(String.java:646)
         at 
jdk.internal.org.objectweb.asm.signature.SignatureReader.accept(SignatureReader.java:107)
         at 
valhalla.specializer.SignatureSpecializer.forType(SignatureSpecializer.java:72)
         at 
valhalla.specializer.Specializer$ManglingMethodVisitor.visitInvokeDynamicInsn(Specializer.java:679)
         at 
jdk.internal.org.objectweb.asm.ClassReader.readCode(ClassReader.java:1507)
         at 
jdk.internal.org.objectweb.asm.ClassReader.readMethod(ClassReader.java:1084)
         at 
jdk.internal.org.objectweb.asm.ClassReader.accept(ClassReader.java:729)
         at valhalla.specializer.Specializer.specialize(Specializer.java:79)
         at java.net.URLClassLoader$1.run(URLClassLoader.java:409)
         at java.net.URLClassLoader$1.run(URLClassLoader.java:386)
         at java.security.AccessController.doPrivileged(Native Method)
         at java.net.URLClassLoader.findClass(URLClassLoader.java:385)
         at java.lang.ClassLoader.loadClass(ClassLoader.java:426)
         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:317)
         at java.lang.ClassLoader.loadClass(ClassLoader.java:359)
         at 
javany.util.Arrays$asList${0=I}/511754216.asList(Arrays.java:810)
         at Test.main(Test.java:10)


Appart from that, I learned that when the component type of vararg array 
is an <any> type variable (for example: <any T> T[] Arrays.asList(T ... 
a)), the invocation doesn't compile:

src/Test.java:10: error: method invoked with incorrect number of 
arguments; expected 3, found 1
         List<int> ints = Arrays.<int>asList(1, 2, 3);
                          ^
1 error


Non-specialized code also has problems at runtime:

public class Test {
     public static void main(String[] args) {
         List<String> strings = Arrays.<String>asList("a", "b", "c");
         Iterator<String> it = strings.iterator();
         while (it.hasNext()) {
             System.out.println(it.next());
         }
     }
}


Exception in thread "main" java.lang.ClassFormatError: Absent Code 
attribute in method that is not native or abstract in class file 
javany/util/AbstractList
         at java.lang.ClassLoader.defineClass1(Native Method)
         at java.lang.ClassLoader.defineClass(ClassLoader.java:762)
         at 
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
         at java.net.URLClassLoader.defineClass(URLClassLoader.java:537)
         at java.net.URLClassLoader.access$300(URLClassLoader.java:78)
         at java.net.URLClassLoader$1.run(URLClassLoader.java:438)
         at java.net.URLClassLoader$1.run(URLClassLoader.java:386)
         at java.security.AccessController.doPrivileged(Native Method)
         at java.net.URLClassLoader.findClass(URLClassLoader.java:385)
         at java.lang.ClassLoader.loadClass(ClassLoader.java:426)
         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:317)
         at java.lang.ClassLoader.loadClass(ClassLoader.java:359)
         at java.lang.ClassLoader.defineClass1(Native Method)
         at java.lang.ClassLoader.defineClass(ClassLoader.java:762)
         at 
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
         at java.net.URLClassLoader.defineClass(URLClassLoader.java:537)
         at java.net.URLClassLoader.access$300(URLClassLoader.java:78)
         at java.net.URLClassLoader$1.run(URLClassLoader.java:438)
         at java.net.URLClassLoader$1.run(URLClassLoader.java:386)
         at java.security.AccessController.doPrivileged(Native Method)
         at java.net.URLClassLoader.findClass(URLClassLoader.java:385)
         at java.lang.ClassLoader.loadClass(ClassLoader.java:426)
         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:317)
         at java.lang.ClassLoader.loadClass(ClassLoader.java:359)
         at javany.util.Arrays.asList(Arrays.java:810)
         at Test.main(Test.java:10)



Regards, Peter

On 02/03/2015 10:34 PM, Maurizio Cimadamore wrote:
>
> On 03/02/15 21:05, Peter Levart wrote:
>> Hi Maurizio,
>>
>> I see. I thought this could be a nice idiom for boxing, since the 
>> following:
>>
>> (Object) 42
>>
>> ...is legal and results in an Integer object at runtime.
> I'm not saying this will never work - actually the compiler is 
> currently accepting this kind of idioms, but the specializer does 
> nothing with it, so you'll get runtime errors.
>>
>> But I don't know if a checkcast is actually inserted for (Object). 
>> Could javac redundantly do it in case casting to Object is from 
>> expression of <any> type and also equip checkcast with BMA indicating 
>> the type of expression so that specialization could replace it with 
>> boxing code?
> That will be the way forward, yes
>
> Maurizio
>>
>> Regards, Peter
>>
>>
>> On 02/03/2015 08:09 PM, Maurizio Cimadamore wrote:
>>>
>>> On 03/02/15 18:46, Maurizio Cimadamore wrote:
>>>> I will also investigate on the crash you are getting... 
>>> Hi Peter,
>>> the crash is coming from this code in AbstractCollection (see code 
>>> in bold):
>>>
>>>     public boolean contains(Object o) {
>>>         __WhereVal(E) {
>>>             Iterator<E> it = iterator();
>>>             if (o == null) {
>>>                 return false;
>>>             } else {
>>>                 while (it.hasNext())
>>> *if (o.equals((Object) it.next()))*
>>>                         return true;
>>>             }
>>>             return false;
>>>         }
>>>         __WhereRef(E) {
>>>             Iterator<E> it = iterator();
>>>             if (o == null) {
>>>                 while (it.hasNext())
>>>                     if (it.next() == null)
>>>                         return true;
>>>             } else {
>>>                 while (it.hasNext())
>>>                     if (o.equals(it.next()))
>>>                         return true;
>>>             }
>>>             return false;
>>>         }
>>>     }
>>>
>>> I believe that, apart from the obvious javac bug, the code has an 
>>> issue, as it.next() is supposed to return a value there, but you are 
>>> casting to Object?
>>>
>>> For the records - a simpler test case for the bug is this:
>>>
>>> class Foo<any E> {
>>>     E e;
>>>     E get() { return e; }
>>>
>>>     void test() {
>>>         __WhereVal(E) {
>>>             Object o = (Object)get();
>>>         }
>>>     }
>>> }
>>>
>>> Maurizio
>>
>
>


From peter.levart at gmail.com  Wed Feb  4 09:53:09 2015
From: peter.levart at gmail.com (Peter Levart)
Date: Wed, 04 Feb 2015 10:53:09 +0100
Subject: experiences with prototype
In-Reply-To: <54D1DC5D.7060408@gmail.com>
References: <54D113DA.9060509@gmail.com> <54D11792.3090504@oracle.com>
	<54D11CEC.6080009@oracle.com> <54D1380A.5070503@gmail.com>
	<54D13EDB.5050704@oracle.com> <54D1DC5D.7060408@gmail.com>
Message-ID: <54D1EC05.5050504@gmail.com>

Hi,

Anybody interested in changes to original JDK code? Here's a webrev:

http://cr.openjdk.java.net/~plevart/misc/valhala-hacks/javany-webrev.01/

Peter

On 02/04/2015 09:46 AM, Peter Levart wrote:
> Hi Maurizio,
>
> I have now managed to successfully compile the code. Here's the 
> updated source:
>
> http://cr.openjdk.java.net/~plevart/misc/valhala-hacks/javany-src.jar
>
>
> But there's a StringIndexOutOfBoundsException thrown from specializer 
> when running the following Test:
>
> public class Test {
>     public static void main(String[] args) {
>         List<int> ints = Arrays.asList(new int[]{1, 2, 3, 4, 5, 6, 7, 
> 8});
>         Iterator<int> it = ints.iterator();
>         while (it.hasNext()) {
>             System.out.println(it.next());
>         }
>     }
> }
>
>
> Specializing javany.util.List${0=I}; searching for 
> javany/util/List.class (not found)
> Specializing javany.util.List${0=I}; searching for 
> javany/util/List.class (found)
> Specializing javany.util.Collection${0=I}; searching for 
> javany/util/Collection.class (not found)
> Specializing javany.util.Collection${0=I}; searching for 
> javany/util/Collection.class (found)
> Specializing javany.lang.Iterable${0=I}; searching for 
> javany/lang/Iterable.class (not found)
> Specializing javany.lang.Iterable${0=I}; searching for 
> javany/lang/Iterable.class (found)
> Specializing method 
> javany/util/Arrays$asList${0=I}.asList([Ljava/lang/Object;)Ljavany/util/List; 
> with class=[] and method=[I]
> Specializing javany.util.Arrays$ArrayList${0=I}; searching for 
> javany/util/Arrays$ArrayList.class (not found)
> Specializing javany.util.Arrays$ArrayList${0=I}; searching for 
> javany/util/Arrays$ArrayList.class (found)
> Exception in thread "main" java.lang.StringIndexOutOfBoundsException: 
> String index out of range: 0
>         at java.lang.String.charAt(String.java:646)
>         at 
> jdk.internal.org.objectweb.asm.signature.SignatureReader.accept(SignatureReader.java:107)
>         at 
> valhalla.specializer.SignatureSpecializer.forType(SignatureSpecializer.java:72)
>         at 
> valhalla.specializer.Specializer$ManglingMethodVisitor.visitInvokeDynamicInsn(Specializer.java:679)
>         at 
> jdk.internal.org.objectweb.asm.ClassReader.readCode(ClassReader.java:1507)
>         at 
> jdk.internal.org.objectweb.asm.ClassReader.readMethod(ClassReader.java:1084)
>         at 
> jdk.internal.org.objectweb.asm.ClassReader.accept(ClassReader.java:729)
>         at 
> valhalla.specializer.Specializer.specialize(Specializer.java:79)
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:409)
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:386)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:385)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:426)
>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:317)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:359)
>         at 
> javany.util.Arrays$asList${0=I}/511754216.asList(Arrays.java:810)
>         at Test.main(Test.java:10)
>
>
> Appart from that, I learned that when the component type of vararg 
> array is an <any> type variable (for example: <any T> T[] 
> Arrays.asList(T ... a)), the invocation doesn't compile:
>
> src/Test.java:10: error: method invoked with incorrect number of 
> arguments; expected 3, found 1
>         List<int> ints = Arrays.<int>asList(1, 2, 3);
>                          ^
> 1 error
>
>
> Non-specialized code also has problems at runtime:
>
> public class Test {
>     public static void main(String[] args) {
>         List<String> strings = Arrays.<String>asList("a", "b", "c");
>         Iterator<String> it = strings.iterator();
>         while (it.hasNext()) {
>             System.out.println(it.next());
>         }
>     }
> }
>
>
> Exception in thread "main" java.lang.ClassFormatError: Absent Code 
> attribute in method that is not native or abstract in class file 
> javany/util/AbstractList
>         at java.lang.ClassLoader.defineClass1(Native Method)
>         at java.lang.ClassLoader.defineClass(ClassLoader.java:762)
>         at 
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>         at java.net.URLClassLoader.defineClass(URLClassLoader.java:537)
>         at java.net.URLClassLoader.access$300(URLClassLoader.java:78)
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:438)
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:386)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:385)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:426)
>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:317)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:359)
>         at java.lang.ClassLoader.defineClass1(Native Method)
>         at java.lang.ClassLoader.defineClass(ClassLoader.java:762)
>         at 
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>         at java.net.URLClassLoader.defineClass(URLClassLoader.java:537)
>         at java.net.URLClassLoader.access$300(URLClassLoader.java:78)
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:438)
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:386)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:385)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:426)
>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:317)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:359)
>         at javany.util.Arrays.asList(Arrays.java:810)
>         at Test.main(Test.java:10)
>
>
>
> Regards, Peter
>
> On 02/03/2015 10:34 PM, Maurizio Cimadamore wrote:
>>
>> On 03/02/15 21:05, Peter Levart wrote:
>>> Hi Maurizio,
>>>
>>> I see. I thought this could be a nice idiom for boxing, since the 
>>> following:
>>>
>>> (Object) 42
>>>
>>> ...is legal and results in an Integer object at runtime.
>> I'm not saying this will never work - actually the compiler is 
>> currently accepting this kind of idioms, but the specializer does 
>> nothing with it, so you'll get runtime errors.
>>>
>>> But I don't know if a checkcast is actually inserted for (Object). 
>>> Could javac redundantly do it in case casting to Object is from 
>>> expression of <any> type and also equip checkcast with BMA 
>>> indicating the type of expression so that specialization could 
>>> replace it with boxing code?
>> That will be the way forward, yes
>>
>> Maurizio
>>>
>>> Regards, Peter
>>>
>>>
>>> On 02/03/2015 08:09 PM, Maurizio Cimadamore wrote:
>>>>
>>>> On 03/02/15 18:46, Maurizio Cimadamore wrote:
>>>>> I will also investigate on the crash you are getting... 
>>>> Hi Peter,
>>>> the crash is coming from this code in AbstractCollection (see code 
>>>> in bold):
>>>>
>>>>     public boolean contains(Object o) {
>>>>         __WhereVal(E) {
>>>>             Iterator<E> it = iterator();
>>>>             if (o == null) {
>>>>                 return false;
>>>>             } else {
>>>>                 while (it.hasNext())
>>>> *if (o.equals((Object) it.next()))*
>>>>                         return true;
>>>>             }
>>>>             return false;
>>>>         }
>>>>         __WhereRef(E) {
>>>>             Iterator<E> it = iterator();
>>>>             if (o == null) {
>>>>                 while (it.hasNext())
>>>>                     if (it.next() == null)
>>>>                         return true;
>>>>             } else {
>>>>                 while (it.hasNext())
>>>>                     if (o.equals(it.next()))
>>>>                         return true;
>>>>             }
>>>>             return false;
>>>>         }
>>>>     }
>>>>
>>>> I believe that, apart from the obvious javac bug, the code has an 
>>>> issue, as it.next() is supposed to return a value there, but you 
>>>> are casting to Object?
>>>>
>>>> For the records - a simpler test case for the bug is this:
>>>>
>>>> class Foo<any E> {
>>>>     E e;
>>>>     E get() { return e; }
>>>>
>>>>     void test() {
>>>>         __WhereVal(E) {
>>>>             Object o = (Object)get();
>>>>         }
>>>>     }
>>>> }
>>>>
>>>> Maurizio
>>>
>>
>>
>
>


From maurizio.cimadamore at oracle.com  Wed Feb  4 10:18:11 2015
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Wed, 04 Feb 2015 10:18:11 +0000
Subject: experiences with prototype
In-Reply-To: <54D1DC5D.7060408@gmail.com>
References: <54D113DA.9060509@gmail.com> <54D11792.3090504@oracle.com>
	<54D11CEC.6080009@oracle.com> <54D1380A.5070503@gmail.com>
	<54D13EDB.5050704@oracle.com> <54D1DC5D.7060408@gmail.com>
Message-ID: <54D1F1E3.10209@oracle.com>

Thanks for the additional feedback, I'll try to get at the bottom of 
those issues.

Maurizio

On 04/02/15 08:46, Peter Levart wrote:
> Hi Maurizio,
>
> I have now managed to successfully compile the code. Here's the 
> updated source:
>
> http://cr.openjdk.java.net/~plevart/misc/valhala-hacks/javany-src.jar
>
>
> But there's a StringIndexOutOfBoundsException thrown from specializer 
> when running the following Test:
>
> public class Test {
>     public static void main(String[] args) {
>         List<int> ints = Arrays.asList(new int[]{1, 2, 3, 4, 5, 6, 7, 8});
>         Iterator<int> it = ints.iterator();
>         while (it.hasNext()) {
>             System.out.println(it.next());
>         }
>     }
> }
>
>
> Specializing javany.util.List${0=I}; searching for 
> javany/util/List.class (not found)
> Specializing javany.util.List${0=I}; searching for 
> javany/util/List.class (found)
> Specializing javany.util.Collection${0=I}; searching for 
> javany/util/Collection.class (not found)
> Specializing javany.util.Collection${0=I}; searching for 
> javany/util/Collection.class (found)
> Specializing javany.lang.Iterable${0=I}; searching for 
> javany/lang/Iterable.class (not found)
> Specializing javany.lang.Iterable${0=I}; searching for 
> javany/lang/Iterable.class (found)
> Specializing method 
> javany/util/Arrays$asList${0=I}.asList([Ljava/lang/Object;)Ljavany/util/List; 
> with class=[] and method=[I]
> Specializing javany.util.Arrays$ArrayList${0=I}; searching for 
> javany/util/Arrays$ArrayList.class (not found)
> Specializing javany.util.Arrays$ArrayList${0=I}; searching for 
> javany/util/Arrays$ArrayList.class (found)
> Exception in thread "main" java.lang.StringIndexOutOfBoundsException: 
> String index out of range: 0
>         at java.lang.String.charAt(String.java:646)
>         at 
> jdk.internal.org.objectweb.asm.signature.SignatureReader.accept(SignatureReader.java:107)
>         at 
> valhalla.specializer.SignatureSpecializer.forType(SignatureSpecializer.java:72)
>         at 
> valhalla.specializer.Specializer$ManglingMethodVisitor.visitInvokeDynamicInsn(Specializer.java:679)
>         at 
> jdk.internal.org.objectweb.asm.ClassReader.readCode(ClassReader.java:1507)
>         at 
> jdk.internal.org.objectweb.asm.ClassReader.readMethod(ClassReader.java:1084)
>         at 
> jdk.internal.org.objectweb.asm.ClassReader.accept(ClassReader.java:729)
>         at 
> valhalla.specializer.Specializer.specialize(Specializer.java:79)
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:409)
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:386)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:385)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:426)
>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:317)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:359)
>         at 
> javany.util.Arrays$asList${0=I}/511754216.asList(Arrays.java:810)
>         at Test.main(Test.java:10)
>
>
> Appart from that, I learned that when the component type of vararg 
> array is an <any> type variable (for example: <any T> T[] 
> Arrays.asList(T ... a)), the invocation doesn't compile:
>
> src/Test.java:10: error: method invoked with incorrect number of 
> arguments; expected 3, found 1
>         List<int> ints = Arrays.<int>asList(1, 2, 3);
>                          ^
> 1 error
>
>
> Non-specialized code also has problems at runtime:
>
> public class Test {
>     public static void main(String[] args) {
>         List<String> strings = Arrays.<String>asList("a", "b", "c");
>         Iterator<String> it = strings.iterator();
>         while (it.hasNext()) {
>             System.out.println(it.next());
>         }
>     }
> }
>
>
> Exception in thread "main" java.lang.ClassFormatError: Absent Code 
> attribute in method that is not native or abstract in class file 
> javany/util/AbstractList
>         at java.lang.ClassLoader.defineClass1(Native Method)
>         at java.lang.ClassLoader.defineClass(ClassLoader.java:762)
>         at 
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>         at java.net.URLClassLoader.defineClass(URLClassLoader.java:537)
>         at java.net.URLClassLoader.access$300(URLClassLoader.java:78)
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:438)
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:386)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:385)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:426)
>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:317)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:359)
>         at java.lang.ClassLoader.defineClass1(Native Method)
>         at java.lang.ClassLoader.defineClass(ClassLoader.java:762)
>         at 
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>         at java.net.URLClassLoader.defineClass(URLClassLoader.java:537)
>         at java.net.URLClassLoader.access$300(URLClassLoader.java:78)
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:438)
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:386)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:385)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:426)
>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:317)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:359)
>         at javany.util.Arrays.asList(Arrays.java:810)
>         at Test.main(Test.java:10)
>
>
>
> Regards, Peter
>
> On 02/03/2015 10:34 PM, Maurizio Cimadamore wrote:
>>
>> On 03/02/15 21:05, Peter Levart wrote:
>>> Hi Maurizio,
>>>
>>> I see. I thought this could be a nice idiom for boxing, since the 
>>> following:
>>>
>>> (Object) 42
>>>
>>> ...is legal and results in an Integer object at runtime.
>> I'm not saying this will never work - actually the compiler is 
>> currently accepting this kind of idioms, but the specializer does 
>> nothing with it, so you'll get runtime errors.
>>>
>>> But I don't know if a checkcast is actually inserted for (Object). 
>>> Could javac redundantly do it in case casting to Object is from 
>>> expression of <any> type and also equip checkcast with BMA 
>>> indicating the type of expression so that specialization could 
>>> replace it with boxing code?
>> That will be the way forward, yes
>>
>> Maurizio
>>>
>>> Regards, Peter
>>>
>>>
>>> On 02/03/2015 08:09 PM, Maurizio Cimadamore wrote:
>>>>
>>>> On 03/02/15 18:46, Maurizio Cimadamore wrote:
>>>>> I will also investigate on the crash you are getting... 
>>>> Hi Peter,
>>>> the crash is coming from this code in AbstractCollection (see code 
>>>> in bold):
>>>>
>>>>     public boolean contains(Object o) {
>>>>         __WhereVal(E) {
>>>>             Iterator<E> it = iterator();
>>>>             if (o == null) {
>>>>                 return false;
>>>>             } else {
>>>>                 while (it.hasNext())
>>>> *if (o.equals((Object) it.next()))*
>>>>                         return true;
>>>>             }
>>>>             return false;
>>>>         }
>>>>         __WhereRef(E) {
>>>>             Iterator<E> it = iterator();
>>>>             if (o == null) {
>>>>                 while (it.hasNext())
>>>>                     if (it.next() == null)
>>>>                         return true;
>>>>             } else {
>>>>                 while (it.hasNext())
>>>>                     if (o.equals(it.next()))
>>>>                         return true;
>>>>             }
>>>>             return false;
>>>>         }
>>>>     }
>>>>
>>>> I believe that, apart from the obvious javac bug, the code has an 
>>>> issue, as it.next() is supposed to return a value there, but you 
>>>> are casting to Object?
>>>>
>>>> For the records - a simpler test case for the bug is this:
>>>>
>>>> class Foo<any E> {
>>>>     E e;
>>>>     E get() { return e; }
>>>>
>>>>     void test() {
>>>>         __WhereVal(E) {
>>>>             Object o = (Object)get();
>>>>         }
>>>>     }
>>>> }
>>>>
>>>> Maurizio
>>>
>>
>>
>


From maurizio.cimadamore at oracle.com  Wed Feb  4 12:37:54 2015
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Wed, 04 Feb 2015 12:37:54 +0000
Subject: hg: valhalla/valhalla/langtools: Fixes:
Message-ID: <201502041237.t14CbsL0016882@aojmv0008>

Changeset: d4008ed705bd
Author:    mcimadamore
Date:      2015-02-04 12:37 +0000
URL:       http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/d4008ed705bd

Fixes:
* subtyping/cast routines broken for value-constrained tvars
* cannot call Object methods on value-constrained tvars
* specialized generic method calls fail to copy varargs call info

! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Types.java
! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Attr.java
! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/SpecializeTypes.java
! test/tools/javac/valhalla/typespec/Cast01.java
! test/tools/javac/valhalla/typespec/Cast01.out
+ test/tools/javac/valhalla/typespec/Inference06.java
! test/tools/javac/valhalla/typespec/ObjectMethods.java
! test/tools/javac/valhalla/typespec/ObjectMethods.out


From maurizio.cimadamore at oracle.com  Wed Feb  4 12:42:42 2015
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Wed, 04 Feb 2015 12:42:42 +0000
Subject: experiences with prototype
In-Reply-To: <54D1F1E3.10209@oracle.com>
References: <54D113DA.9060509@gmail.com>
	<54D11792.3090504@oracle.com>	<54D11CEC.6080009@oracle.com>
	<54D1380A.5070503@gmail.com>	<54D13EDB.5050704@oracle.com>
	<54D1DC5D.7060408@gmail.com> <54D1F1E3.10209@oracle.com>
Message-ID: <54D213C2.3020603@oracle.com>

Compiler fixes have been pushed, I will now look into the runtime issues 
you are getting...

Maurizio

On 04/02/15 10:18, Maurizio Cimadamore wrote:
> Thanks for the additional feedback, I'll try to get at the bottom of 
> those issues.
>
> Maurizio
>
> On 04/02/15 08:46, Peter Levart wrote:
>> Hi Maurizio,
>>
>> I have now managed to successfully compile the code. Here's the 
>> updated source:
>>
>> http://cr.openjdk.java.net/~plevart/misc/valhala-hacks/javany-src.jar
>>
>>
>> But there's a StringIndexOutOfBoundsException thrown from specializer 
>> when running the following Test:
>>
>> public class Test {
>>     public static void main(String[] args) {
>>         List<int> ints = Arrays.asList(new int[]{1, 2, 3, 4, 5, 6, 7, 
>> 8});
>>         Iterator<int> it = ints.iterator();
>>         while (it.hasNext()) {
>>             System.out.println(it.next());
>>         }
>>     }
>> }
>>
>>
>> Specializing javany.util.List${0=I}; searching for 
>> javany/util/List.class (not found)
>> Specializing javany.util.List${0=I}; searching for 
>> javany/util/List.class (found)
>> Specializing javany.util.Collection${0=I}; searching for 
>> javany/util/Collection.class (not found)
>> Specializing javany.util.Collection${0=I}; searching for 
>> javany/util/Collection.class (found)
>> Specializing javany.lang.Iterable${0=I}; searching for 
>> javany/lang/Iterable.class (not found)
>> Specializing javany.lang.Iterable${0=I}; searching for 
>> javany/lang/Iterable.class (found)
>> Specializing method 
>> javany/util/Arrays$asList${0=I}.asList([Ljava/lang/Object;)Ljavany/util/List; 
>> with class=[] and method=[I]
>> Specializing javany.util.Arrays$ArrayList${0=I}; searching for 
>> javany/util/Arrays$ArrayList.class (not found)
>> Specializing javany.util.Arrays$ArrayList${0=I}; searching for 
>> javany/util/Arrays$ArrayList.class (found)
>> Exception in thread "main" java.lang.StringIndexOutOfBoundsException: 
>> String index out of range: 0
>>         at java.lang.String.charAt(String.java:646)
>>         at 
>> jdk.internal.org.objectweb.asm.signature.SignatureReader.accept(SignatureReader.java:107)
>>         at 
>> valhalla.specializer.SignatureSpecializer.forType(SignatureSpecializer.java:72)
>>         at 
>> valhalla.specializer.Specializer$ManglingMethodVisitor.visitInvokeDynamicInsn(Specializer.java:679)
>>         at 
>> jdk.internal.org.objectweb.asm.ClassReader.readCode(ClassReader.java:1507)
>>         at 
>> jdk.internal.org.objectweb.asm.ClassReader.readMethod(ClassReader.java:1084)
>>         at 
>> jdk.internal.org.objectweb.asm.ClassReader.accept(ClassReader.java:729)
>>         at 
>> valhalla.specializer.Specializer.specialize(Specializer.java:79)
>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:409)
>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:386)
>>         at java.security.AccessController.doPrivileged(Native Method)
>>         at java.net.URLClassLoader.findClass(URLClassLoader.java:385)
>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:426)
>>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:317)
>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:359)
>>         at 
>> javany.util.Arrays$asList${0=I}/511754216.asList(Arrays.java:810)
>>         at Test.main(Test.java:10)
>>
>>
>> Appart from that, I learned that when the component type of vararg 
>> array is an <any> type variable (for example: <any T> T[] 
>> Arrays.asList(T ... a)), the invocation doesn't compile:
>>
>> src/Test.java:10: error: method invoked with incorrect number of 
>> arguments; expected 3, found 1
>>         List<int> ints = Arrays.<int>asList(1, 2, 3);
>>                          ^
>> 1 error
>>
>>
>> Non-specialized code also has problems at runtime:
>>
>> public class Test {
>>     public static void main(String[] args) {
>>         List<String> strings = Arrays.<String>asList("a", "b", "c");
>>         Iterator<String> it = strings.iterator();
>>         while (it.hasNext()) {
>>             System.out.println(it.next());
>>         }
>>     }
>> }
>>
>>
>> Exception in thread "main" java.lang.ClassFormatError: Absent Code 
>> attribute in method that is not native or abstract in class file 
>> javany/util/AbstractList
>>         at java.lang.ClassLoader.defineClass1(Native Method)
>>         at java.lang.ClassLoader.defineClass(ClassLoader.java:762)
>>         at 
>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>>         at java.net.URLClassLoader.defineClass(URLClassLoader.java:537)
>>         at java.net.URLClassLoader.access$300(URLClassLoader.java:78)
>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:438)
>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:386)
>>         at java.security.AccessController.doPrivileged(Native Method)
>>         at java.net.URLClassLoader.findClass(URLClassLoader.java:385)
>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:426)
>>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:317)
>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:359)
>>         at java.lang.ClassLoader.defineClass1(Native Method)
>>         at java.lang.ClassLoader.defineClass(ClassLoader.java:762)
>>         at 
>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>>         at java.net.URLClassLoader.defineClass(URLClassLoader.java:537)
>>         at java.net.URLClassLoader.access$300(URLClassLoader.java:78)
>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:438)
>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:386)
>>         at java.security.AccessController.doPrivileged(Native Method)
>>         at java.net.URLClassLoader.findClass(URLClassLoader.java:385)
>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:426)
>>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:317)
>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:359)
>>         at javany.util.Arrays.asList(Arrays.java:810)
>>         at Test.main(Test.java:10)
>>
>>
>>
>> Regards, Peter
>>
>> On 02/03/2015 10:34 PM, Maurizio Cimadamore wrote:
>>>
>>> On 03/02/15 21:05, Peter Levart wrote:
>>>> Hi Maurizio,
>>>>
>>>> I see. I thought this could be a nice idiom for boxing, since the 
>>>> following:
>>>>
>>>> (Object) 42
>>>>
>>>> ...is legal and results in an Integer object at runtime.
>>> I'm not saying this will never work - actually the compiler is 
>>> currently accepting this kind of idioms, but the specializer does 
>>> nothing with it, so you'll get runtime errors.
>>>>
>>>> But I don't know if a checkcast is actually inserted for (Object). 
>>>> Could javac redundantly do it in case casting to Object is from 
>>>> expression of <any> type and also equip checkcast with BMA 
>>>> indicating the type of expression so that specialization could 
>>>> replace it with boxing code?
>>> That will be the way forward, yes
>>>
>>> Maurizio
>>>>
>>>> Regards, Peter
>>>>
>>>>
>>>> On 02/03/2015 08:09 PM, Maurizio Cimadamore wrote:
>>>>>
>>>>> On 03/02/15 18:46, Maurizio Cimadamore wrote:
>>>>>> I will also investigate on the crash you are getting... 
>>>>> Hi Peter,
>>>>> the crash is coming from this code in AbstractCollection (see code 
>>>>> in bold):
>>>>>
>>>>>     public boolean contains(Object o) {
>>>>>         __WhereVal(E) {
>>>>>             Iterator<E> it = iterator();
>>>>>             if (o == null) {
>>>>>                 return false;
>>>>>             } else {
>>>>>                 while (it.hasNext())
>>>>> *if (o.equals((Object) it.next()))*
>>>>>                         return true;
>>>>>             }
>>>>>             return false;
>>>>>         }
>>>>>         __WhereRef(E) {
>>>>>             Iterator<E> it = iterator();
>>>>>             if (o == null) {
>>>>>                 while (it.hasNext())
>>>>>                     if (it.next() == null)
>>>>>                         return true;
>>>>>             } else {
>>>>>                 while (it.hasNext())
>>>>>                     if (o.equals(it.next()))
>>>>>                         return true;
>>>>>             }
>>>>>             return false;
>>>>>         }
>>>>>     }
>>>>>
>>>>> I believe that, apart from the obvious javac bug, the code has an 
>>>>> issue, as it.next() is supposed to return a value there, but you 
>>>>> are casting to Object?
>>>>>
>>>>> For the records - a simpler test case for the bug is this:
>>>>>
>>>>> class Foo<any E> {
>>>>>     E e;
>>>>>     E get() { return e; }
>>>>>
>>>>>     void test() {
>>>>>         __WhereVal(E) {
>>>>>             Object o = (Object)get();
>>>>>         }
>>>>>     }
>>>>> }
>>>>>
>>>>> Maurizio
>>>>
>>>
>>>
>>
>


From maurizio.cimadamore at oracle.com  Wed Feb  4 18:48:53 2015
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Wed, 04 Feb 2015 18:48:53 +0000
Subject: hg: valhalla/valhalla/langtools: Fix: javac generates redundant
	bytecode mapping for object method calls on array receivers
Message-ID: <201502041848.t14ImraS000461@aojmv0008>

Changeset: dd1bada7ea9f
Author:    mcimadamore
Date:      2015-02-04 18:48 +0000
URL:       http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/dd1bada7ea9f

Fix: javac generates redundant bytecode mapping for object method calls on array receivers

! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Items.java
! test/tools/javac/valhalla/typespec/items/tests/TestAnyMembers.java


From maurizio.cimadamore at oracle.com  Thu Feb  5 16:57:41 2015
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Thu, 05 Feb 2015 16:57:41 +0000
Subject: hg: valhalla/valhalla/langtools: Make BytecodeMaping signatures more
	uniform
Message-ID: <201502051657.t15GvfiJ020430@aojmv0008>

Changeset: 0176195716d5
Author:    mcimadamore
Date:      2015-02-05 16:57 +0000
URL:       http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/0176195716d5

Make BytecodeMaping signatures more uniform
* indy-related signatures do not mention owner anymore
* the signature type similar to the erased signature except for the fact that it mentions specializable types (any tvars and primitives typeargs)

! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Types.java
! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/SpecializeTypes.java
! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Items.java
! test/tools/javac/valhalla/typespec/items/tests/TestGenericSpecializedConstructor.java
! test/tools/javac/valhalla/typespec/items/tests/TestIndy.java
! test/tools/javac/valhalla/typespec/items/tests/TestIndyFactory.java
! test/tools/javac/valhalla/typespec/items/tests/TestRespecialization.java


From maurizio.cimadamore at oracle.com  Thu Feb  5 17:01:00 2015
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Thu, 05 Feb 2015 17:01:00 +0000
Subject: hg: valhalla/valhalla/jdk: Fix specializer bugs:
Message-ID: <201502051701.t15H16BA020846@aojmv0008>

Changeset: f589e6657bc2
Author:    mcimadamore
Date:      2015-02-05 17:00 +0000
URL:       http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/f589e6657bc2

Fix specializer bugs:
* specialization crashes in places (i.e. CHECKCAST) if an array descriptor is found
* specializaton of indy doesn't handle dummy args
* specialization of indy doesn't specialize receiver type
* specialization of indy should work with new format (w/o the '::')

! src/java.base/share/classes/valhalla/specializer/SignatureSpecializer.java
! src/java.base/share/classes/valhalla/specializer/Specializer.java
! test/valhalla/test/valhalla/specializer/GenericMethodsTest.java


From maurizio.cimadamore at oracle.com  Fri Feb  6 14:04:48 2015
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Fri, 06 Feb 2015 14:04:48 +0000
Subject: hg: valhalla/valhalla/langtools: Fix: list of pending specialized
	definition is not saved/restored correctly during visit
Message-ID: <201502061404.t16E4muW028292@aojmv0008>

Changeset: 31a4ae08c730
Author:    mcimadamore
Date:      2015-02-06 14:04 +0000
URL:       http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/31a4ae08c730

Fix: list of pending specialized definition is not saved/restored correctly during visit

! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/SpecializeTypes.java


From maurizio.cimadamore at oracle.com  Fri Feb  6 14:31:51 2015
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Fri, 06 Feb 2015 14:31:51 +0000
Subject: hg: valhalla/valhalla/jdk: Tweak stackmap generation:
Message-ID: <201502061431.t16EVpoZ006305@aojmv0008>

Changeset: d122437798d9
Author:    mcimadamore
Date:      2015-02-06 14:31 +0000
URL:       http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/d122437798d9

Tweak stackmap generation:
* The code for merging two stack frame types might require additional specialization; merge as Object (for now)

! src/java.base/share/classes/valhalla/specializer/Specializer.java


From maurizio.cimadamore at oracle.com  Fri Feb  6 14:38:56 2015
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Fri, 06 Feb 2015 14:38:56 +0000
Subject: experiences with prototype
In-Reply-To: <54D213C2.3020603@oracle.com>
References: <54D113DA.9060509@gmail.com>	<54D11792.3090504@oracle.com>	<54D11CEC.6080009@oracle.com>	<54D1380A.5070503@gmail.com>	<54D13EDB.5050704@oracle.com>	<54D1DC5D.7060408@gmail.com>
	<54D1F1E3.10209@oracle.com> <54D213C2.3020603@oracle.com>
Message-ID: <54D4D200.9090906@oracle.com>

Peter,
I think I fixed most of the issues in the way; I've been able to compile 
and run some tests successfully.
Attached is a patch of the changes that are required for the code to run 
correctly; I had to desugar some constructs (i.e. for-each, non-static 
inner class) as all desugared stuff currently has an issue in that javac 
won't emit the special sauce that is required by the specializer to 
correctly specialize a class. That's an issue with timing of the 
compilation pipeline - we are aware of it; but, as it's a no trivial 
fix, it means that, for now, it's better not to rely too much of 
compiler-desugared code.

Maurizio

On 04/02/15 12:42, Maurizio Cimadamore wrote:
> Compiler fixes have been pushed, I will now look into the runtime 
> issues you are getting...
>
> Maurizio
>
> On 04/02/15 10:18, Maurizio Cimadamore wrote:
>> Thanks for the additional feedback, I'll try to get at the bottom of 
>> those issues.
>>
>> Maurizio
>>
>> On 04/02/15 08:46, Peter Levart wrote:
>>> Hi Maurizio,
>>>
>>> I have now managed to successfully compile the code. Here's the 
>>> updated source:
>>>
>>> http://cr.openjdk.java.net/~plevart/misc/valhala-hacks/javany-src.jar
>>>
>>>
>>> But there's a StringIndexOutOfBoundsException thrown from 
>>> specializer when running the following Test:
>>>
>>> public class Test {
>>>     public static void main(String[] args) {
>>>         List<int> ints = Arrays.asList(new int[]{1, 2, 3, 4, 5, 6, 
>>> 7, 8});
>>>         Iterator<int> it = ints.iterator();
>>>         while (it.hasNext()) {
>>>             System.out.println(it.next());
>>>         }
>>>     }
>>> }
>>>
>>>
>>> Specializing javany.util.List${0=I}; searching for 
>>> javany/util/List.class (not found)
>>> Specializing javany.util.List${0=I}; searching for 
>>> javany/util/List.class (found)
>>> Specializing javany.util.Collection${0=I}; searching for 
>>> javany/util/Collection.class (not found)
>>> Specializing javany.util.Collection${0=I}; searching for 
>>> javany/util/Collection.class (found)
>>> Specializing javany.lang.Iterable${0=I}; searching for 
>>> javany/lang/Iterable.class (not found)
>>> Specializing javany.lang.Iterable${0=I}; searching for 
>>> javany/lang/Iterable.class (found)
>>> Specializing method 
>>> javany/util/Arrays$asList${0=I}.asList([Ljava/lang/Object;)Ljavany/util/List; 
>>> with class=[] and method=[I]
>>> Specializing javany.util.Arrays$ArrayList${0=I}; searching for 
>>> javany/util/Arrays$ArrayList.class (not found)
>>> Specializing javany.util.Arrays$ArrayList${0=I}; searching for 
>>> javany/util/Arrays$ArrayList.class (found)
>>> Exception in thread "main" 
>>> java.lang.StringIndexOutOfBoundsException: String index out of range: 0
>>>         at java.lang.String.charAt(String.java:646)
>>>         at 
>>> jdk.internal.org.objectweb.asm.signature.SignatureReader.accept(SignatureReader.java:107)
>>>         at 
>>> valhalla.specializer.SignatureSpecializer.forType(SignatureSpecializer.java:72)
>>>         at 
>>> valhalla.specializer.Specializer$ManglingMethodVisitor.visitInvokeDynamicInsn(Specializer.java:679)
>>>         at 
>>> jdk.internal.org.objectweb.asm.ClassReader.readCode(ClassReader.java:1507)
>>>         at 
>>> jdk.internal.org.objectweb.asm.ClassReader.readMethod(ClassReader.java:1084)
>>>         at 
>>> jdk.internal.org.objectweb.asm.ClassReader.accept(ClassReader.java:729)
>>>         at 
>>> valhalla.specializer.Specializer.specialize(Specializer.java:79)
>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:409)
>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:386)
>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>         at java.net.URLClassLoader.findClass(URLClassLoader.java:385)
>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:426)
>>>         at 
>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:317)
>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:359)
>>>         at 
>>> javany.util.Arrays$asList${0=I}/511754216.asList(Arrays.java:810)
>>>         at Test.main(Test.java:10)
>>>
>>>
>>> Appart from that, I learned that when the component type of vararg 
>>> array is an <any> type variable (for example: <any T> T[] 
>>> Arrays.asList(T ... a)), the invocation doesn't compile:
>>>
>>> src/Test.java:10: error: method invoked with incorrect number of 
>>> arguments; expected 3, found 1
>>>         List<int> ints = Arrays.<int>asList(1, 2, 3);
>>>                          ^
>>> 1 error
>>>
>>>
>>> Non-specialized code also has problems at runtime:
>>>
>>> public class Test {
>>>     public static void main(String[] args) {
>>>         List<String> strings = Arrays.<String>asList("a", "b", "c");
>>>         Iterator<String> it = strings.iterator();
>>>         while (it.hasNext()) {
>>>             System.out.println(it.next());
>>>         }
>>>     }
>>> }
>>>
>>>
>>> Exception in thread "main" java.lang.ClassFormatError: Absent Code 
>>> attribute in method that is not native or abstract in class file 
>>> javany/util/AbstractList
>>>         at java.lang.ClassLoader.defineClass1(Native Method)
>>>         at java.lang.ClassLoader.defineClass(ClassLoader.java:762)
>>>         at 
>>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>>>         at java.net.URLClassLoader.defineClass(URLClassLoader.java:537)
>>>         at java.net.URLClassLoader.access$300(URLClassLoader.java:78)
>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:438)
>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:386)
>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>         at java.net.URLClassLoader.findClass(URLClassLoader.java:385)
>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:426)
>>>         at 
>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:317)
>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:359)
>>>         at java.lang.ClassLoader.defineClass1(Native Method)
>>>         at java.lang.ClassLoader.defineClass(ClassLoader.java:762)
>>>         at 
>>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>>>         at java.net.URLClassLoader.defineClass(URLClassLoader.java:537)
>>>         at java.net.URLClassLoader.access$300(URLClassLoader.java:78)
>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:438)
>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:386)
>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>         at java.net.URLClassLoader.findClass(URLClassLoader.java:385)
>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:426)
>>>         at 
>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:317)
>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:359)
>>>         at javany.util.Arrays.asList(Arrays.java:810)
>>>         at Test.main(Test.java:10)
>>>
>>>
>>>
>>> Regards, Peter
>>>
>>> On 02/03/2015 10:34 PM, Maurizio Cimadamore wrote:
>>>>
>>>> On 03/02/15 21:05, Peter Levart wrote:
>>>>> Hi Maurizio,
>>>>>
>>>>> I see. I thought this could be a nice idiom for boxing, since the 
>>>>> following:
>>>>>
>>>>> (Object) 42
>>>>>
>>>>> ...is legal and results in an Integer object at runtime.
>>>> I'm not saying this will never work - actually the compiler is 
>>>> currently accepting this kind of idioms, but the specializer does 
>>>> nothing with it, so you'll get runtime errors.
>>>>>
>>>>> But I don't know if a checkcast is actually inserted for (Object). 
>>>>> Could javac redundantly do it in case casting to Object is from 
>>>>> expression of <any> type and also equip checkcast with BMA 
>>>>> indicating the type of expression so that specialization could 
>>>>> replace it with boxing code?
>>>> That will be the way forward, yes
>>>>
>>>> Maurizio
>>>>>
>>>>> Regards, Peter
>>>>>
>>>>>
>>>>> On 02/03/2015 08:09 PM, Maurizio Cimadamore wrote:
>>>>>>
>>>>>> On 03/02/15 18:46, Maurizio Cimadamore wrote:
>>>>>>> I will also investigate on the crash you are getting... 
>>>>>> Hi Peter,
>>>>>> the crash is coming from this code in AbstractCollection (see 
>>>>>> code in bold):
>>>>>>
>>>>>>     public boolean contains(Object o) {
>>>>>>         __WhereVal(E) {
>>>>>>             Iterator<E> it = iterator();
>>>>>>             if (o == null) {
>>>>>>                 return false;
>>>>>>             } else {
>>>>>>                 while (it.hasNext())
>>>>>> *if (o.equals((Object) it.next()))*
>>>>>>                         return true;
>>>>>>             }
>>>>>>             return false;
>>>>>>         }
>>>>>>         __WhereRef(E) {
>>>>>>             Iterator<E> it = iterator();
>>>>>>             if (o == null) {
>>>>>>                 while (it.hasNext())
>>>>>>                     if (it.next() == null)
>>>>>>                         return true;
>>>>>>             } else {
>>>>>>                 while (it.hasNext())
>>>>>>                     if (o.equals(it.next()))
>>>>>>                         return true;
>>>>>>             }
>>>>>>             return false;
>>>>>>         }
>>>>>>     }
>>>>>>
>>>>>> I believe that, apart from the obvious javac bug, the code has an 
>>>>>> issue, as it.next() is supposed to return a value there, but you 
>>>>>> are casting to Object?
>>>>>>
>>>>>> For the records - a simpler test case for the bug is this:
>>>>>>
>>>>>> class Foo<any E> {
>>>>>>     E e;
>>>>>>     E get() { return e; }
>>>>>>
>>>>>>     void test() {
>>>>>>         __WhereVal(E) {
>>>>>>             Object o = (Object)get();
>>>>>>         }
>>>>>>     }
>>>>>> }
>>>>>>
>>>>>> Maurizio
>>>>>
>>>>
>>>>
>>>
>>
>

-------------- next part --------------
diff -r src-old/javany/util/AbstractList.java src-new/javany/util/AbstractList.java
331c331
<         return new Itr();
---
>         return new Itr<>(this);
372c372
<         return new ListItr(index);
---
>         return new ListItr<>(this, index);
375c375,383
<     private class Itr implements Iterator<E> {
---
>     private static class Itr<any E> implements Iterator<E> {
> 
> 		AbstractList<E> this$0;
> 
> 		Itr(AbstractList<E> this$0) {
> 			this.this$0 = this$0;
> 			expectedModCount = this$0.modCount;
>         }
> 
393c401
<         int expectedModCount = modCount;
---
>         int expectedModCount;
396c404
<             return cursor != size();
---
>             return cursor != this$0.size();
403c411
<                 E next = get(i);
---
>                 E next = this$0.get(i);
419c427
<                 AbstractList.this.remove(lastRet);
---
>                 this$0.remove(lastRet);
423c431
<                 expectedModCount = modCount;
---
>                 expectedModCount = this$0.modCount;
430c438
<             if (modCount != expectedModCount)
---
>             if (this$0.modCount != expectedModCount)
435,436c443,446
<     private class ListItr extends Itr implements ListIterator<E> {
<         ListItr(int index) {
---
>     private static class ListItr<any E> extends Itr<E> implements ListIterator<E> {
> 
>         ListItr(AbstractList<E> this$0, int index) {
> 			super(this$0);
448c458
<                 E previous = get(i);
---
>                 E previous = this$0.get(i);
471,472c481,482
<                 AbstractList.this.set(lastRet, e);
<                 expectedModCount = modCount;
---
>                 this$0.set(lastRet, e);
>                 expectedModCount = this$0.modCount;
483c493
<                 AbstractList.this.add(i, e);
---
>                 this$0.add(i, e);
486c496
<                 expectedModCount = modCount;
---
>                 expectedModCount = this$0.modCount;
Only in src-new/javany/util: AbstractList.java~
diff -r src-old/javany/util/Arrays.java src-new/javany/util/Arrays.java
823c823
<             a = Objects.requireNonNull(array);
---
>             a = (E[])Objects.requireNonNull(array);
916c916,917
<             for (E e : a) {
---
>             for (int i = 0; i < a.length ; i++) {
> 				E e = a[i];
Only in src-new/javany/util: Arrays.java~
diff -r src-old/javany/util/List.java src-new/javany/util/List.java
482c482,483
<         for (E e : a) {
---
>         for (int idx = 0; idx < a.length ; idx++) {
>             E e = a[idx];
Only in src-new/javany/util: List.java~
diff -r src-old/Test.java src-new/Test.java
9,10c9,18
<     public static void main(String[] args) {
<         List<String> strings = Arrays.<String>asList("a", "b", "c");
---
> 	static void testInt() {
> 		List<int> ints = Arrays.asList(1, 2, 3);
>         Iterator<int> it = ints.iterator();
>         while (it.hasNext()) {
>             System.out.println(it.next());
>         }
>     }
> 
> 	static void testString() {
> 		List<String> strings = Arrays.asList("a", "b", "c");
14a23,27
>     }
> 
>     public static void main(String[] args) {
>         testString();
> 		testInt();

From maurizio.cimadamore at oracle.com  Fri Feb  6 18:38:30 2015
From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com)
Date: Fri, 06 Feb 2015 18:38:30 +0000
Subject: hg: valhalla/valhalla/langtools: Fix: missing BMAs in various kinds
	of synthetic cast generated by TransTypes
Message-ID: <201502061838.t16IcVLB001119@aojmv0008>

Changeset: 99ab651be669
Author:    mcimadamore
Date:      2015-02-06 18:35 +0000
URL:       http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/99ab651be669

Fix: missing BMAs in various kinds of synthetic cast generated by TransTypes

! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TransTypes.java
! test/tools/javac/valhalla/typespec/items/tests/TestSyntheticCast.java


From peter.levart at gmail.com  Sun Feb  8 10:27:42 2015
From: peter.levart at gmail.com (Peter Levart)
Date: Sun, 08 Feb 2015 11:27:42 +0100
Subject: experiences with prototype
In-Reply-To: <54D4D200.9090906@oracle.com>
References: <54D113DA.9060509@gmail.com>	<54D11792.3090504@oracle.com>	<54D11CEC.6080009@oracle.com>	<54D1380A.5070503@gmail.com>	<54D13EDB.5050704@oracle.com>	<54D1DC5D.7060408@gmail.com>
	<54D1F1E3.10209@oracle.com> <54D213C2.3020603@oracle.com>
	<54D4D200.9090906@oracle.com>
Message-ID: <54D73A1E.9020102@gmail.com>

Hi Maurizio,

This is great. I replaced all new-style for loops on <any T> T[] arrays 
with classic on-index iteration and applied your patch selectively 
without the redundant cast in javany/util/Arrays.java:823:

     a = (E[])Objects.requireNonNull(array)

... as I think your last langtools patch (99ab651be669) fixes that, right?


Now with the following code:

http://cr.openjdk.java.net/~plevart/misc/valhala-hacks/javany-src.02.jar

And this test:

public class Test {

     static void testStrings() {
         List<String> strings = Arrays.<String>asList("a", "b", "c");
         System.out.println(strings);
     }

     static void testInts() {
         List<int> ints = Arrays.<int>asList(1, 2, 3);
         System.out.println(ints);
     }

     public static void main(String[] args) {
         testStrings();
         testInts();
     }
}


I get:

[a, b, c]
Specializing javany.util.List${0=I}; searching for 
javany/util/List.class (not found)
Specializing javany.util.List${0=I}; searching for 
javany/util/List.class (found)
Specializing javany.util.Collection${0=I}; searching for 
javany/util/Collection.class (not found)
Specializing javany.util.Collection${0=I}; searching for 
javany/util/Collection.class (found)
Specializing javany.lang.Iterable${0=I}; searching for 
javany/lang/Iterable.class (not found)
Specializing javany.lang.Iterable${0=I}; searching for 
javany/lang/Iterable.class (found)
Specializing method 
javany/util/Arrays$asList${0=I}.asList([Ljava/lang/Object;)Ljavany/util/List; 
with class=[] and method=[I]
Specializing javany.util.Arrays$ArrayList${0=I}; searching for 
javany/util/Arrays$ArrayList.class (not found)
Specializing javany.util.Arrays$ArrayList${0=I}; searching for 
javany/util/Arrays$ArrayList.class (found)
Specializing javany.util.AbstractList${0=I}; searching for 
javany/util/AbstractList.class (not found)
Specializing javany.util.AbstractList${0=I}; searching for 
javany/util/AbstractList.class (found)
Specializing javany.util.AbstractCollection${0=I}; searching for 
javany/util/AbstractCollection.class (not found)
Specializing javany.util.AbstractCollection${0=I}; searching for 
javany/util/AbstractCollection.class (found)
Specializing javany.util.Iterator${0=I}; searching for 
javany/util/Iterator.class (not found)
Specializing javany.util.Iterator${0=I}; searching for 
javany/util/Iterator.class (found)
Specializing javany.util.ListIterator${0=I}; searching for 
javany/util/ListIterator.class (not found)
Specializing javany.util.ListIterator${0=I}; searching for 
javany/util/ListIterator.class (found)
Specializing javany.util.AbstractList$Itr${0=I}; searching for 
javany/util/AbstractList$Itr.class (not found)
Specializing javany.util.AbstractList$Itr${0=I}; searching for 
javany/util/AbstractList$Itr.class (found)
Specializing method 
javany/util/Any$toString${0=I}.toString(Ljava/lang/Object;)Ljava/lang/String; 
with class=[] and method=[I]
Specializing javany.util.Any${0=I}; searching for javany/util/Any.class 
(not found)
Specializing javany.util.Any${0=I}; searching for javany/util/Any.class 
(found)
Exception in thread "main" java.lang.BootstrapMethodError: call site 
initialization exception
         at java.lang.invoke.CallSite.makeSite(CallSite.java:341)
         at 
java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:307)
         at 
java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:297)
         at 
javany.util.AbstractCollection${0=I}.toString(AbstractCollection.java:514)
         at java.lang.String.valueOf(String.java:2987)
         at java.io.PrintStream.println(PrintStream.java:821)
         at Test.testInts(Test.java:17)
         at Test.main(Test.java:22)
Caused by: java.lang.VerifyError: Bad invokespecial instruction: current 
class isn't assignable to reference class.
Exception Details:
   Location:
     javany/util/Any$toString${0=I}.toString(I)Ljava/lang/String; @8: 
invokespecial
   Reason:
     Error exists in the bytecode
   Bytecode:
     0000000: bb00 0959 b700 0d1a b700 10b0

         at sun.misc.Unsafe.defineAnonymousClass(Native Method)
         at 
java.lang.invoke.GenericMethodSpecializer.metafactory(GenericMethodSpecializer.java:98)
         at java.lang.invoke.CallSite.makeSite(CallSite.java:302)
         ... 7 more


But looking at generated bytecode:


Classfile 
/home/peter/work/local/valhalla-test/dump/javany.util.Any$toString${0=I}_ERROR.class
   Last modified Feb 8, 2015; size 321 bytes
   MD5 checksum 7dac6f4f515394e0aa2ed1990a092697
   Compiled from "Any.java"
public class javany.util.Any$toString${0=I}
   minor version: 0
   major version: 52
   flags: ACC_PUBLIC, ACC_SUPER
Constant pool:
    #1 = Utf8               javany/util/Any$toString${0=I}
    #2 = Class              #1             // 
"javany/util/Any$toString${0=I}"
    #3 = Utf8               java/lang/Object
    #4 = Class              #3             // java/lang/Object
    #5 = Utf8               Any.java
    #6 = Utf8               toString
    #7 = Utf8               (I)Ljava/lang/String;
    #8 = Utf8               javany/util/Any${0=I}
    #9 = Class              #8             // "javany/util/Any${0=I}"
   #10 = Utf8               <init>
   #11 = Utf8               ()V
   #12 = NameAndType        #10:#11        // "<init>":()V
   #13 = Methodref          #9.#12         // 
"javany/util/Any${0=I}"."<init>":()V
   #14 = Utf8               toStringImpl
   #15 = NameAndType        #14:#7         // 
toStringImpl:(I)Ljava/lang/String;
   #16 = Methodref          #9.#15         // 
"javany/util/Any${0=I}".toStringImpl:(I)Ljava/lang/String;
   #17 = Utf8               Code
   #18 = Utf8               LineNumberTable
   #19 = Utf8               Signature
   #20 = Utf8               SourceFile
{
   public static java.lang.String toString(int);
     descriptor: (I)Ljava/lang/String;
     flags: ACC_PUBLIC, ACC_STATIC
     Code:
       stack=2, locals=1, args_size=1
          0: new           #9                  // class 
"javany/util/Any${0=I}"
          3: dup
          4: invokespecial #13                 // Method 
"javany/util/Any${0=I}"."<init>":()V
          7: iload_0
          8: invokespecial #16                 // Method 
"javany/util/Any${0=I}".toStringImpl:(I)Ljava/lang/String;
         11: areturn
       LineNumberTable:
         line 31: 0
     Signature: #7                           // (I)Ljava/lang/String;
}
SourceFile: "Any.java"


Comparing it to original Any.class bytecode:


   public static <T extends java.lang.Object> java.lang.String toString(T);
     descriptor: (Ljava/lang/Object;)Ljava/lang/String;
     flags: ACC_PUBLIC, ACC_STATIC
     Code:
       stack=2, locals=1, args_size=1
          0: new           #3                  // class javany/util/Any
          3: dup
          4: invokespecial #4                  // Method "<init>":()V
          7: aload_0
          8: invokespecial #8                  // Method 
toStringImpl:(Ljava/lang/Object;)Ljava/lang/String;
         11: areturn
       LineNumberTable:
         line 31: 0
Error: unknown attribute
       BytecodeMapping: length = 0x12
        00 04 00 00 00 8C 00 04 00 8D 00 07 00 87 00 08
        00 9F
     Signature: #160                         // 
<T:Ljava/lang/Object;>(TT;)Ljava/lang/String;
Error: unknown attribute
       TypeVariablesMap: length = 0x9
        01 00 A1 01 01 00 92 00 86


And also peeking at Any${0=I}.class:


   private java.lang.String toStringImpl(int);
     descriptor: (I)Ljava/lang/String;
     flags: ACC_PRIVATE
     Code:
       stack=2, locals=2, args_size=2
          0: aload_0
          1: iload_1
          2: invokespecial #45                 // Method 
toStringNonNullImpl:(I)Ljava/lang/String;
          5: areturn
       LineNumberTable:
         line 181: 0
     Signature: #33                          // (I)Ljava/lang/String;


...I can't spot what's wrong. Can you?


Regards, Peter


On 02/06/2015 03:38 PM, Maurizio Cimadamore wrote:
> Peter,
> I think I fixed most of the issues in the way; I've been able to 
> compile and run some tests successfully.
> Attached is a patch of the changes that are required for the code to 
> run correctly; I had to desugar some constructs (i.e. for-each, 
> non-static inner class) as all desugared stuff currently has an issue 
> in that javac won't emit the special sauce that is required by the 
> specializer to correctly specialize a class. That's an issue with 
> timing of the compilation pipeline - we are aware of it; but, as it's 
> a no trivial fix, it means that, for now, it's better not to rely too 
> much of compiler-desugared code.
>
> Maurizio
>
> On 04/02/15 12:42, Maurizio Cimadamore wrote:
>> Compiler fixes have been pushed, I will now look into the runtime 
>> issues you are getting...
>>
>> Maurizio
>>
>> On 04/02/15 10:18, Maurizio Cimadamore wrote:
>>> Thanks for the additional feedback, I'll try to get at the bottom of 
>>> those issues.
>>>
>>> Maurizio
>>>
>>> On 04/02/15 08:46, Peter Levart wrote:
>>>> Hi Maurizio,
>>>>
>>>> I have now managed to successfully compile the code. Here's the 
>>>> updated source:
>>>>
>>>> http://cr.openjdk.java.net/~plevart/misc/valhala-hacks/javany-src.jar
>>>>
>>>>
>>>> But there's a StringIndexOutOfBoundsException thrown from 
>>>> specializer when running the following Test:
>>>>
>>>> public class Test {
>>>>     public static void main(String[] args) {
>>>>         List<int> ints = Arrays.asList(new int[]{1, 2, 3, 4, 5, 6, 
>>>> 7, 8});
>>>>         Iterator<int> it = ints.iterator();
>>>>         while (it.hasNext()) {
>>>>             System.out.println(it.next());
>>>>         }
>>>>     }
>>>> }
>>>>
>>>>
>>>> Specializing javany.util.List${0=I}; searching for 
>>>> javany/util/List.class (not found)
>>>> Specializing javany.util.List${0=I}; searching for 
>>>> javany/util/List.class (found)
>>>> Specializing javany.util.Collection${0=I}; searching for 
>>>> javany/util/Collection.class (not found)
>>>> Specializing javany.util.Collection${0=I}; searching for 
>>>> javany/util/Collection.class (found)
>>>> Specializing javany.lang.Iterable${0=I}; searching for 
>>>> javany/lang/Iterable.class (not found)
>>>> Specializing javany.lang.Iterable${0=I}; searching for 
>>>> javany/lang/Iterable.class (found)
>>>> Specializing method 
>>>> javany/util/Arrays$asList${0=I}.asList([Ljava/lang/Object;)Ljavany/util/List; 
>>>> with class=[] and method=[I]
>>>> Specializing javany.util.Arrays$ArrayList${0=I}; searching for 
>>>> javany/util/Arrays$ArrayList.class (not found)
>>>> Specializing javany.util.Arrays$ArrayList${0=I}; searching for 
>>>> javany/util/Arrays$ArrayList.class (found)
>>>> Exception in thread "main" 
>>>> java.lang.StringIndexOutOfBoundsException: String index out of 
>>>> range: 0
>>>>         at java.lang.String.charAt(String.java:646)
>>>>         at 
>>>> jdk.internal.org.objectweb.asm.signature.SignatureReader.accept(SignatureReader.java:107)
>>>>         at 
>>>> valhalla.specializer.SignatureSpecializer.forType(SignatureSpecializer.java:72)
>>>>         at 
>>>> valhalla.specializer.Specializer$ManglingMethodVisitor.visitInvokeDynamicInsn(Specializer.java:679)
>>>>         at 
>>>> jdk.internal.org.objectweb.asm.ClassReader.readCode(ClassReader.java:1507)
>>>>         at 
>>>> jdk.internal.org.objectweb.asm.ClassReader.readMethod(ClassReader.java:1084)
>>>>         at 
>>>> jdk.internal.org.objectweb.asm.ClassReader.accept(ClassReader.java:729) 
>>>>
>>>>         at 
>>>> valhalla.specializer.Specializer.specialize(Specializer.java:79)
>>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:409)
>>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:386)
>>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>>         at java.net.URLClassLoader.findClass(URLClassLoader.java:385)
>>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:426)
>>>>         at 
>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:317)
>>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:359)
>>>>         at 
>>>> javany.util.Arrays$asList${0=I}/511754216.asList(Arrays.java:810)
>>>>         at Test.main(Test.java:10)
>>>>
>>>>
>>>> Appart from that, I learned that when the component type of vararg 
>>>> array is an <any> type variable (for example: <any T> T[] 
>>>> Arrays.asList(T ... a)), the invocation doesn't compile:
>>>>
>>>> src/Test.java:10: error: method invoked with incorrect number of 
>>>> arguments; expected 3, found 1
>>>>         List<int> ints = Arrays.<int>asList(1, 2, 3);
>>>>                          ^
>>>> 1 error
>>>>
>>>>
>>>> Non-specialized code also has problems at runtime:
>>>>
>>>> public class Test {
>>>>     public static void main(String[] args) {
>>>>         List<String> strings = Arrays.<String>asList("a", "b", "c");
>>>>         Iterator<String> it = strings.iterator();
>>>>         while (it.hasNext()) {
>>>>             System.out.println(it.next());
>>>>         }
>>>>     }
>>>> }
>>>>
>>>>
>>>> Exception in thread "main" java.lang.ClassFormatError: Absent Code 
>>>> attribute in method that is not native or abstract in class file 
>>>> javany/util/AbstractList
>>>>         at java.lang.ClassLoader.defineClass1(Native Method)
>>>>         at java.lang.ClassLoader.defineClass(ClassLoader.java:762)
>>>>         at 
>>>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) 
>>>>
>>>>         at 
>>>> java.net.URLClassLoader.defineClass(URLClassLoader.java:537)
>>>>         at java.net.URLClassLoader.access$300(URLClassLoader.java:78)
>>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:438)
>>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:386)
>>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>>         at java.net.URLClassLoader.findClass(URLClassLoader.java:385)
>>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:426)
>>>>         at 
>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:317)
>>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:359)
>>>>         at java.lang.ClassLoader.defineClass1(Native Method)
>>>>         at java.lang.ClassLoader.defineClass(ClassLoader.java:762)
>>>>         at 
>>>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) 
>>>>
>>>>         at 
>>>> java.net.URLClassLoader.defineClass(URLClassLoader.java:537)
>>>>         at java.net.URLClassLoader.access$300(URLClassLoader.java:78)
>>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:438)
>>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:386)
>>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>>         at java.net.URLClassLoader.findClass(URLClassLoader.java:385)
>>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:426)
>>>>         at 
>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:317)
>>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:359)
>>>>         at javany.util.Arrays.asList(Arrays.java:810)
>>>>         at Test.main(Test.java:10)
>>>>
>>>>
>>>>
>>>> Regards, Peter
>>>>
>>>> On 02/03/2015 10:34 PM, Maurizio Cimadamore wrote:
>>>>>
>>>>> On 03/02/15 21:05, Peter Levart wrote:
>>>>>> Hi Maurizio,
>>>>>>
>>>>>> I see. I thought this could be a nice idiom for boxing, since the 
>>>>>> following:
>>>>>>
>>>>>> (Object) 42
>>>>>>
>>>>>> ...is legal and results in an Integer object at runtime.
>>>>> I'm not saying this will never work - actually the compiler is 
>>>>> currently accepting this kind of idioms, but the specializer does 
>>>>> nothing with it, so you'll get runtime errors.
>>>>>>
>>>>>> But I don't know if a checkcast is actually inserted for 
>>>>>> (Object). Could javac redundantly do it in case casting to Object 
>>>>>> is from expression of <any> type and also equip checkcast with 
>>>>>> BMA indicating the type of expression so that specialization 
>>>>>> could replace it with boxing code?
>>>>> That will be the way forward, yes
>>>>>
>>>>> Maurizio
>>>>>>
>>>>>> Regards, Peter
>>>>>>
>>>>>>
>>>>>> On 02/03/2015 08:09 PM, Maurizio Cimadamore wrote:
>>>>>>>
>>>>>>> On 03/02/15 18:46, Maurizio Cimadamore wrote:
>>>>>>>> I will also investigate on the crash you are getting... 
>>>>>>> Hi Peter,
>>>>>>> the crash is coming from this code in AbstractCollection (see 
>>>>>>> code in bold):
>>>>>>>
>>>>>>>     public boolean contains(Object o) {
>>>>>>>         __WhereVal(E) {
>>>>>>>             Iterator<E> it = iterator();
>>>>>>>             if (o == null) {
>>>>>>>                 return false;
>>>>>>>             } else {
>>>>>>>                 while (it.hasNext())
>>>>>>> *if (o.equals((Object) it.next()))*
>>>>>>>                         return true;
>>>>>>>             }
>>>>>>>             return false;
>>>>>>>         }
>>>>>>>         __WhereRef(E) {
>>>>>>>             Iterator<E> it = iterator();
>>>>>>>             if (o == null) {
>>>>>>>                 while (it.hasNext())
>>>>>>>                     if (it.next() == null)
>>>>>>>                         return true;
>>>>>>>             } else {
>>>>>>>                 while (it.hasNext())
>>>>>>>                     if (o.equals(it.next()))
>>>>>>>                         return true;
>>>>>>>             }
>>>>>>>             return false;
>>>>>>>         }
>>>>>>>     }
>>>>>>>
>>>>>>> I believe that, apart from the obvious javac bug, the code has 
>>>>>>> an issue, as it.next() is supposed to return a value there, but 
>>>>>>> you are casting to Object?
>>>>>>>
>>>>>>> For the records - a simpler test case for the bug is this:
>>>>>>>
>>>>>>> class Foo<any E> {
>>>>>>>     E e;
>>>>>>>     E get() { return e; }
>>>>>>>
>>>>>>>     void test() {
>>>>>>>         __WhereVal(E) {
>>>>>>>             Object o = (Object)get();
>>>>>>>         }
>>>>>>>     }
>>>>>>> }
>>>>>>>
>>>>>>> Maurizio
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>


From peter.levart at gmail.com  Sun Feb  8 10:43:20 2015
From: peter.levart at gmail.com (Peter Levart)
Date: Sun, 08 Feb 2015 11:43:20 +0100
Subject: experiences with prototype
In-Reply-To: <54D73A1E.9020102@gmail.com>
References: <54D113DA.9060509@gmail.com>	<54D11792.3090504@oracle.com>	<54D11CEC.6080009@oracle.com>	<54D1380A.5070503@gmail.com>	<54D13EDB.5050704@oracle.com>	<54D1DC5D.7060408@gmail.com>
	<54D1F1E3.10209@oracle.com> <54D213C2.3020603@oracle.com>
	<54D4D200.9090906@oracle.com> <54D73A1E.9020102@gmail.com>
Message-ID: <54D73DC8.6000900@gmail.com>

I've got it.

The problem was that Any.<int>toStringImpl() instance method was 
private. Since it is located in a specialized (Any${0=I}) class, it is 
not accessible from Any class (or specialized VM anonymous class 
generated for a specialized generic static method). The VerifyError: 
"Bad invokespecial instruction: current class isn't assignable to 
reference class." is not very informative though.

By making the method(s) package-private, my test now runs correctly.

Regards, Peter

On 02/08/2015 11:27 AM, Peter Levart wrote:
> Hi Maurizio,
>
> This is great. I replaced all new-style for loops on <any T> T[] 
> arrays with classic on-index iteration and applied your patch 
> selectively without the redundant cast in javany/util/Arrays.java:823:
>
>     a = (E[])Objects.requireNonNull(array)
>
> ... as I think your last langtools patch (99ab651be669) fixes that, right?
>
>
> Now with the following code:
>
> http://cr.openjdk.java.net/~plevart/misc/valhala-hacks/javany-src.02.jar
>
> And this test:
>
> public class Test {
>
>     static void testStrings() {
>         List<String> strings = Arrays.<String>asList("a", "b", "c");
>         System.out.println(strings);
>     }
>
>     static void testInts() {
>         List<int> ints = Arrays.<int>asList(1, 2, 3);
>         System.out.println(ints);
>     }
>
>     public static void main(String[] args) {
>         testStrings();
>         testInts();
>     }
> }
>
>
> I get:
>
> [a, b, c]
> Specializing javany.util.List${0=I}; searching for 
> javany/util/List.class (not found)
> Specializing javany.util.List${0=I}; searching for 
> javany/util/List.class (found)
> Specializing javany.util.Collection${0=I}; searching for 
> javany/util/Collection.class (not found)
> Specializing javany.util.Collection${0=I}; searching for 
> javany/util/Collection.class (found)
> Specializing javany.lang.Iterable${0=I}; searching for 
> javany/lang/Iterable.class (not found)
> Specializing javany.lang.Iterable${0=I}; searching for 
> javany/lang/Iterable.class (found)
> Specializing method 
> javany/util/Arrays$asList${0=I}.asList([Ljava/lang/Object;)Ljavany/util/List; 
> with class=[] and method=[I]
> Specializing javany.util.Arrays$ArrayList${0=I}; searching for 
> javany/util/Arrays$ArrayList.class (not found)
> Specializing javany.util.Arrays$ArrayList${0=I}; searching for 
> javany/util/Arrays$ArrayList.class (found)
> Specializing javany.util.AbstractList${0=I}; searching for 
> javany/util/AbstractList.class (not found)
> Specializing javany.util.AbstractList${0=I}; searching for 
> javany/util/AbstractList.class (found)
> Specializing javany.util.AbstractCollection${0=I}; searching for 
> javany/util/AbstractCollection.class (not found)
> Specializing javany.util.AbstractCollection${0=I}; searching for 
> javany/util/AbstractCollection.class (found)
> Specializing javany.util.Iterator${0=I}; searching for 
> javany/util/Iterator.class (not found)
> Specializing javany.util.Iterator${0=I}; searching for 
> javany/util/Iterator.class (found)
> Specializing javany.util.ListIterator${0=I}; searching for 
> javany/util/ListIterator.class (not found)
> Specializing javany.util.ListIterator${0=I}; searching for 
> javany/util/ListIterator.class (found)
> Specializing javany.util.AbstractList$Itr${0=I}; searching for 
> javany/util/AbstractList$Itr.class (not found)
> Specializing javany.util.AbstractList$Itr${0=I}; searching for 
> javany/util/AbstractList$Itr.class (found)
> Specializing method 
> javany/util/Any$toString${0=I}.toString(Ljava/lang/Object;)Ljava/lang/String; 
> with class=[] and method=[I]
> Specializing javany.util.Any${0=I}; searching for 
> javany/util/Any.class (not found)
> Specializing javany.util.Any${0=I}; searching for 
> javany/util/Any.class (found)
> Exception in thread "main" java.lang.BootstrapMethodError: call site 
> initialization exception
>         at java.lang.invoke.CallSite.makeSite(CallSite.java:341)
>         at 
> java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:307)
>         at 
> java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:297)
>         at 
> javany.util.AbstractCollection${0=I}.toString(AbstractCollection.java:514)
>         at java.lang.String.valueOf(String.java:2987)
>         at java.io.PrintStream.println(PrintStream.java:821)
>         at Test.testInts(Test.java:17)
>         at Test.main(Test.java:22)
> Caused by: java.lang.VerifyError: Bad invokespecial instruction: 
> current class isn't assignable to reference class.
> Exception Details:
>   Location:
>     javany/util/Any$toString${0=I}.toString(I)Ljava/lang/String; @8: 
> invokespecial
>   Reason:
>     Error exists in the bytecode
>   Bytecode:
>     0000000: bb00 0959 b700 0d1a b700 10b0
>
>         at sun.misc.Unsafe.defineAnonymousClass(Native Method)
>         at 
> java.lang.invoke.GenericMethodSpecializer.metafactory(GenericMethodSpecializer.java:98)
>         at java.lang.invoke.CallSite.makeSite(CallSite.java:302)
>         ... 7 more
>
>
> But looking at generated bytecode:
>
>
> Classfile 
> /home/peter/work/local/valhalla-test/dump/javany.util.Any$toString${0=I}_ERROR.class
>   Last modified Feb 8, 2015; size 321 bytes
>   MD5 checksum 7dac6f4f515394e0aa2ed1990a092697
>   Compiled from "Any.java"
> public class javany.util.Any$toString${0=I}
>   minor version: 0
>   major version: 52
>   flags: ACC_PUBLIC, ACC_SUPER
> Constant pool:
>    #1 = Utf8               javany/util/Any$toString${0=I}
>    #2 = Class              #1             // 
> "javany/util/Any$toString${0=I}"
>    #3 = Utf8               java/lang/Object
>    #4 = Class              #3             // java/lang/Object
>    #5 = Utf8               Any.java
>    #6 = Utf8               toString
>    #7 = Utf8               (I)Ljava/lang/String;
>    #8 = Utf8               javany/util/Any${0=I}
>    #9 = Class              #8             // "javany/util/Any${0=I}"
>   #10 = Utf8               <init>
>   #11 = Utf8               ()V
>   #12 = NameAndType        #10:#11        // "<init>":()V
>   #13 = Methodref          #9.#12         // 
> "javany/util/Any${0=I}"."<init>":()V
>   #14 = Utf8               toStringImpl
>   #15 = NameAndType        #14:#7         // 
> toStringImpl:(I)Ljava/lang/String;
>   #16 = Methodref          #9.#15         // 
> "javany/util/Any${0=I}".toStringImpl:(I)Ljava/lang/String;
>   #17 = Utf8               Code
>   #18 = Utf8               LineNumberTable
>   #19 = Utf8               Signature
>   #20 = Utf8               SourceFile
> {
>   public static java.lang.String toString(int);
>     descriptor: (I)Ljava/lang/String;
>     flags: ACC_PUBLIC, ACC_STATIC
>     Code:
>       stack=2, locals=1, args_size=1
>          0: new           #9                  // class 
> "javany/util/Any${0=I}"
>          3: dup
>          4: invokespecial #13                 // Method 
> "javany/util/Any${0=I}"."<init>":()V
>          7: iload_0
>          8: invokespecial #16                 // Method 
> "javany/util/Any${0=I}".toStringImpl:(I)Ljava/lang/String;
>         11: areturn
>       LineNumberTable:
>         line 31: 0
>     Signature: #7                           // (I)Ljava/lang/String;
> }
> SourceFile: "Any.java"
>
>
> Comparing it to original Any.class bytecode:
>
>
>   public static <T extends java.lang.Object> java.lang.String toString(T);
>     descriptor: (Ljava/lang/Object;)Ljava/lang/String;
>     flags: ACC_PUBLIC, ACC_STATIC
>     Code:
>       stack=2, locals=1, args_size=1
>          0: new           #3                  // class javany/util/Any
>          3: dup
>          4: invokespecial #4                  // Method "<init>":()V
>          7: aload_0
>          8: invokespecial #8                  // Method 
> toStringImpl:(Ljava/lang/Object;)Ljava/lang/String;
>         11: areturn
>       LineNumberTable:
>         line 31: 0
> Error: unknown attribute
>       BytecodeMapping: length = 0x12
>        00 04 00 00 00 8C 00 04 00 8D 00 07 00 87 00 08
>        00 9F
>     Signature: #160                         // 
> <T:Ljava/lang/Object;>(TT;)Ljava/lang/String;
> Error: unknown attribute
>       TypeVariablesMap: length = 0x9
>        01 00 A1 01 01 00 92 00 86
>
>
> And also peeking at Any${0=I}.class:
>
>
>   private java.lang.String toStringImpl(int);
>     descriptor: (I)Ljava/lang/String;
>     flags: ACC_PRIVATE
>     Code:
>       stack=2, locals=2, args_size=2
>          0: aload_0
>          1: iload_1
>          2: invokespecial #45                 // Method 
> toStringNonNullImpl:(I)Ljava/lang/String;
>          5: areturn
>       LineNumberTable:
>         line 181: 0
>     Signature: #33                          // (I)Ljava/lang/String;
>
>
> ...I can't spot what's wrong. Can you?
>
>
> Regards, Peter
>
>
> On 02/06/2015 03:38 PM, Maurizio Cimadamore wrote:
>> Peter,
>> I think I fixed most of the issues in the way; I've been able to 
>> compile and run some tests successfully.
>> Attached is a patch of the changes that are required for the code to 
>> run correctly; I had to desugar some constructs (i.e. for-each, 
>> non-static inner class) as all desugared stuff currently has an issue 
>> in that javac won't emit the special sauce that is required by the 
>> specializer to correctly specialize a class. That's an issue with 
>> timing of the compilation pipeline - we are aware of it; but, as it's 
>> a no trivial fix, it means that, for now, it's better not to rely too 
>> much of compiler-desugared code.
>>
>> Maurizio
>>
>> On 04/02/15 12:42, Maurizio Cimadamore wrote:
>>> Compiler fixes have been pushed, I will now look into the runtime 
>>> issues you are getting...
>>>
>>> Maurizio
>>>
>>> On 04/02/15 10:18, Maurizio Cimadamore wrote:
>>>> Thanks for the additional feedback, I'll try to get at the bottom 
>>>> of those issues.
>>>>
>>>> Maurizio
>>>>
>>>> On 04/02/15 08:46, Peter Levart wrote:
>>>>> Hi Maurizio,
>>>>>
>>>>> I have now managed to successfully compile the code. Here's the 
>>>>> updated source:
>>>>>
>>>>> http://cr.openjdk.java.net/~plevart/misc/valhala-hacks/javany-src.jar
>>>>>
>>>>>
>>>>> But there's a StringIndexOutOfBoundsException thrown from 
>>>>> specializer when running the following Test:
>>>>>
>>>>> public class Test {
>>>>>     public static void main(String[] args) {
>>>>>         List<int> ints = Arrays.asList(new int[]{1, 2, 3, 4, 5, 6, 
>>>>> 7, 8});
>>>>>         Iterator<int> it = ints.iterator();
>>>>>         while (it.hasNext()) {
>>>>>             System.out.println(it.next());
>>>>>         }
>>>>>     }
>>>>> }
>>>>>
>>>>>
>>>>> Specializing javany.util.List${0=I}; searching for 
>>>>> javany/util/List.class (not found)
>>>>> Specializing javany.util.List${0=I}; searching for 
>>>>> javany/util/List.class (found)
>>>>> Specializing javany.util.Collection${0=I}; searching for 
>>>>> javany/util/Collection.class (not found)
>>>>> Specializing javany.util.Collection${0=I}; searching for 
>>>>> javany/util/Collection.class (found)
>>>>> Specializing javany.lang.Iterable${0=I}; searching for 
>>>>> javany/lang/Iterable.class (not found)
>>>>> Specializing javany.lang.Iterable${0=I}; searching for 
>>>>> javany/lang/Iterable.class (found)
>>>>> Specializing method 
>>>>> javany/util/Arrays$asList${0=I}.asList([Ljava/lang/Object;)Ljavany/util/List; 
>>>>> with class=[] and method=[I]
>>>>> Specializing javany.util.Arrays$ArrayList${0=I}; searching for 
>>>>> javany/util/Arrays$ArrayList.class (not found)
>>>>> Specializing javany.util.Arrays$ArrayList${0=I}; searching for 
>>>>> javany/util/Arrays$ArrayList.class (found)
>>>>> Exception in thread "main" 
>>>>> java.lang.StringIndexOutOfBoundsException: String index out of 
>>>>> range: 0
>>>>>         at java.lang.String.charAt(String.java:646)
>>>>>         at 
>>>>> jdk.internal.org.objectweb.asm.signature.SignatureReader.accept(SignatureReader.java:107)
>>>>>         at 
>>>>> valhalla.specializer.SignatureSpecializer.forType(SignatureSpecializer.java:72)
>>>>>         at 
>>>>> valhalla.specializer.Specializer$ManglingMethodVisitor.visitInvokeDynamicInsn(Specializer.java:679)
>>>>>         at 
>>>>> jdk.internal.org.objectweb.asm.ClassReader.readCode(ClassReader.java:1507)
>>>>>         at 
>>>>> jdk.internal.org.objectweb.asm.ClassReader.readMethod(ClassReader.java:1084)
>>>>>         at 
>>>>> jdk.internal.org.objectweb.asm.ClassReader.accept(ClassReader.java:729) 
>>>>>
>>>>>         at 
>>>>> valhalla.specializer.Specializer.specialize(Specializer.java:79)
>>>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:409)
>>>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:386)
>>>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>>>         at java.net.URLClassLoader.findClass(URLClassLoader.java:385)
>>>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:426)
>>>>>         at 
>>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:317)
>>>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:359)
>>>>>         at 
>>>>> javany.util.Arrays$asList${0=I}/511754216.asList(Arrays.java:810)
>>>>>         at Test.main(Test.java:10)
>>>>>
>>>>>
>>>>> Appart from that, I learned that when the component type of vararg 
>>>>> array is an <any> type variable (for example: <any T> T[] 
>>>>> Arrays.asList(T ... a)), the invocation doesn't compile:
>>>>>
>>>>> src/Test.java:10: error: method invoked with incorrect number of 
>>>>> arguments; expected 3, found 1
>>>>>         List<int> ints = Arrays.<int>asList(1, 2, 3);
>>>>>                          ^
>>>>> 1 error
>>>>>
>>>>>
>>>>> Non-specialized code also has problems at runtime:
>>>>>
>>>>> public class Test {
>>>>>     public static void main(String[] args) {
>>>>>         List<String> strings = Arrays.<String>asList("a", "b", "c");
>>>>>         Iterator<String> it = strings.iterator();
>>>>>         while (it.hasNext()) {
>>>>>             System.out.println(it.next());
>>>>>         }
>>>>>     }
>>>>> }
>>>>>
>>>>>
>>>>> Exception in thread "main" java.lang.ClassFormatError: Absent Code 
>>>>> attribute in method that is not native or abstract in class file 
>>>>> javany/util/AbstractList
>>>>>         at java.lang.ClassLoader.defineClass1(Native Method)
>>>>>         at java.lang.ClassLoader.defineClass(ClassLoader.java:762)
>>>>>         at 
>>>>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) 
>>>>>
>>>>>         at 
>>>>> java.net.URLClassLoader.defineClass(URLClassLoader.java:537)
>>>>>         at java.net.URLClassLoader.access$300(URLClassLoader.java:78)
>>>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:438)
>>>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:386)
>>>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>>>         at java.net.URLClassLoader.findClass(URLClassLoader.java:385)
>>>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:426)
>>>>>         at 
>>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:317)
>>>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:359)
>>>>>         at java.lang.ClassLoader.defineClass1(Native Method)
>>>>>         at java.lang.ClassLoader.defineClass(ClassLoader.java:762)
>>>>>         at 
>>>>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) 
>>>>>
>>>>>         at 
>>>>> java.net.URLClassLoader.defineClass(URLClassLoader.java:537)
>>>>>         at java.net.URLClassLoader.access$300(URLClassLoader.java:78)
>>>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:438)
>>>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:386)
>>>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>>>         at java.net.URLClassLoader.findClass(URLClassLoader.java:385)
>>>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:426)
>>>>>         at 
>>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:317)
>>>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:359)
>>>>>         at javany.util.Arrays.asList(Arrays.java:810)
>>>>>         at Test.main(Test.java:10)
>>>>>
>>>>>
>>>>>
>>>>> Regards, Peter
>>>>>
>>>>> On 02/03/2015 10:34 PM, Maurizio Cimadamore wrote:
>>>>>>
>>>>>> On 03/02/15 21:05, Peter Levart wrote:
>>>>>>> Hi Maurizio,
>>>>>>>
>>>>>>> I see. I thought this could be a nice idiom for boxing, since 
>>>>>>> the following:
>>>>>>>
>>>>>>> (Object) 42
>>>>>>>
>>>>>>> ...is legal and results in an Integer object at runtime.
>>>>>> I'm not saying this will never work - actually the compiler is 
>>>>>> currently accepting this kind of idioms, but the specializer does 
>>>>>> nothing with it, so you'll get runtime errors.
>>>>>>>
>>>>>>> But I don't know if a checkcast is actually inserted for 
>>>>>>> (Object). Could javac redundantly do it in case casting to 
>>>>>>> Object is from expression of <any> type and also equip checkcast 
>>>>>>> with BMA indicating the type of expression so that 
>>>>>>> specialization could replace it with boxing code?
>>>>>> That will be the way forward, yes
>>>>>>
>>>>>> Maurizio
>>>>>>>
>>>>>>> Regards, Peter
>>>>>>>
>>>>>>>
>>>>>>> On 02/03/2015 08:09 PM, Maurizio Cimadamore wrote:
>>>>>>>>
>>>>>>>> On 03/02/15 18:46, Maurizio Cimadamore wrote:
>>>>>>>>> I will also investigate on the crash you are getting... 
>>>>>>>> Hi Peter,
>>>>>>>> the crash is coming from this code in AbstractCollection (see 
>>>>>>>> code in bold):
>>>>>>>>
>>>>>>>>     public boolean contains(Object o) {
>>>>>>>>         __WhereVal(E) {
>>>>>>>>             Iterator<E> it = iterator();
>>>>>>>>             if (o == null) {
>>>>>>>>                 return false;
>>>>>>>>             } else {
>>>>>>>>                 while (it.hasNext())
>>>>>>>> *if (o.equals((Object) it.next()))*
>>>>>>>>                         return true;
>>>>>>>>             }
>>>>>>>>             return false;
>>>>>>>>         }
>>>>>>>>         __WhereRef(E) {
>>>>>>>>             Iterator<E> it = iterator();
>>>>>>>>             if (o == null) {
>>>>>>>>                 while (it.hasNext())
>>>>>>>>                     if (it.next() == null)
>>>>>>>>                         return true;
>>>>>>>>             } else {
>>>>>>>>                 while (it.hasNext())
>>>>>>>>                     if (o.equals(it.next()))
>>>>>>>>                         return true;
>>>>>>>>             }
>>>>>>>>             return false;
>>>>>>>>         }
>>>>>>>>     }
>>>>>>>>
>>>>>>>> I believe that, apart from the obvious javac bug, the code has 
>>>>>>>> an issue, as it.next() is supposed to return a value there, but 
>>>>>>>> you are casting to Object?
>>>>>>>>
>>>>>>>> For the records - a simpler test case for the bug is this:
>>>>>>>>
>>>>>>>> class Foo<any E> {
>>>>>>>>     E e;
>>>>>>>>     E get() { return e; }
>>>>>>>>
>>>>>>>>     void test() {
>>>>>>>>         __WhereVal(E) {
>>>>>>>>             Object o = (Object)get();
>>>>>>>>         }
>>>>>>>>     }
>>>>>>>> }
>>>>>>>>
>>>>>>>> Maurizio
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>


From peter.levart at gmail.com  Sun Feb  8 11:28:10 2015
From: peter.levart at gmail.com (Peter Levart)
Date: Sun, 08 Feb 2015 12:28:10 +0100
Subject: experiences with prototype
In-Reply-To: <54D73DC8.6000900@gmail.com>
References: <54D113DA.9060509@gmail.com>	<54D11792.3090504@oracle.com>	<54D11CEC.6080009@oracle.com>	<54D1380A.5070503@gmail.com>	<54D13EDB.5050704@oracle.com>	<54D1DC5D.7060408@gmail.com>
	<54D1F1E3.10209@oracle.com> <54D213C2.3020603@oracle.com>
	<54D4D200.9090906@oracle.com> <54D73A1E.9020102@gmail.com>
	<54D73DC8.6000900@gmail.com>
Message-ID: <54D7484A.3080705@gmail.com>


On 02/08/2015 11:43 AM, Peter Levart wrote:
> I've got it.
>
> The problem was that Any.<int>toStringImpl() instance method was 
> private. Since it is located in a specialized (Any${0=I}) class, it is 
> not accessible from Any class (or specialized VM anonymous class 
> generated for a specialized generic static method). The VerifyError: 
> "Bad invokespecial instruction: current class isn't assignable to 
> reference class." is not very informative though.

Well, it is, if you think about it. "invokespecial" is always used 
internally where it invokes methods of "current" or "super" class. So 
this is not an illegal access error (yet), but a verify error indicating 
that wrong target for "invokespecial" instruction was used. Synthetic 
access/bridge method would be needed here (like with inner classes), but 
as you say, javac de-sugaring phase is executed too late currently.

But couldn't verifier for "invokespecial" be relaxed a bit so it would 
allow calls to private methods of unrelated classes and rather rely on 
access permissions checking at link-time. This and some notion of 
"protection domain" shared by a group of classes, where groups would be 
formed of:
- top-level class + all nested/inner classes
- master class + derived specialized classes

...would also make a secure platform without using synthetic access 
methods. Wouldn't it?

Regards, Peter

>
> By making the method(s) package-private, my test now runs correctly.
>
> Regards, Peter
>
> On 02/08/2015 11:27 AM, Peter Levart wrote:
>> Hi Maurizio,
>>
>> This is great. I replaced all new-style for loops on <any T> T[] 
>> arrays with classic on-index iteration and applied your patch 
>> selectively without the redundant cast in javany/util/Arrays.java:823:
>>
>>     a = (E[])Objects.requireNonNull(array)
>>
>> ... as I think your last langtools patch (99ab651be669) fixes that, 
>> right?
>>
>>
>> Now with the following code:
>>
>> http://cr.openjdk.java.net/~plevart/misc/valhala-hacks/javany-src.02.jar
>>
>> And this test:
>>
>> public class Test {
>>
>>     static void testStrings() {
>>         List<String> strings = Arrays.<String>asList("a", "b", "c");
>>         System.out.println(strings);
>>     }
>>
>>     static void testInts() {
>>         List<int> ints = Arrays.<int>asList(1, 2, 3);
>>         System.out.println(ints);
>>     }
>>
>>     public static void main(String[] args) {
>>         testStrings();
>>         testInts();
>>     }
>> }
>>
>>
>> I get:
>>
>> [a, b, c]
>> Specializing javany.util.List${0=I}; searching for 
>> javany/util/List.class (not found)
>> Specializing javany.util.List${0=I}; searching for 
>> javany/util/List.class (found)
>> Specializing javany.util.Collection${0=I}; searching for 
>> javany/util/Collection.class (not found)
>> Specializing javany.util.Collection${0=I}; searching for 
>> javany/util/Collection.class (found)
>> Specializing javany.lang.Iterable${0=I}; searching for 
>> javany/lang/Iterable.class (not found)
>> Specializing javany.lang.Iterable${0=I}; searching for 
>> javany/lang/Iterable.class (found)
>> Specializing method 
>> javany/util/Arrays$asList${0=I}.asList([Ljava/lang/Object;)Ljavany/util/List; 
>> with class=[] and method=[I]
>> Specializing javany.util.Arrays$ArrayList${0=I}; searching for 
>> javany/util/Arrays$ArrayList.class (not found)
>> Specializing javany.util.Arrays$ArrayList${0=I}; searching for 
>> javany/util/Arrays$ArrayList.class (found)
>> Specializing javany.util.AbstractList${0=I}; searching for 
>> javany/util/AbstractList.class (not found)
>> Specializing javany.util.AbstractList${0=I}; searching for 
>> javany/util/AbstractList.class (found)
>> Specializing javany.util.AbstractCollection${0=I}; searching for 
>> javany/util/AbstractCollection.class (not found)
>> Specializing javany.util.AbstractCollection${0=I}; searching for 
>> javany/util/AbstractCollection.class (found)
>> Specializing javany.util.Iterator${0=I}; searching for 
>> javany/util/Iterator.class (not found)
>> Specializing javany.util.Iterator${0=I}; searching for 
>> javany/util/Iterator.class (found)
>> Specializing javany.util.ListIterator${0=I}; searching for 
>> javany/util/ListIterator.class (not found)
>> Specializing javany.util.ListIterator${0=I}; searching for 
>> javany/util/ListIterator.class (found)
>> Specializing javany.util.AbstractList$Itr${0=I}; searching for 
>> javany/util/AbstractList$Itr.class (not found)
>> Specializing javany.util.AbstractList$Itr${0=I}; searching for 
>> javany/util/AbstractList$Itr.class (found)
>> Specializing method 
>> javany/util/Any$toString${0=I}.toString(Ljava/lang/Object;)Ljava/lang/String; 
>> with class=[] and method=[I]
>> Specializing javany.util.Any${0=I}; searching for 
>> javany/util/Any.class (not found)
>> Specializing javany.util.Any${0=I}; searching for 
>> javany/util/Any.class (found)
>> Exception in thread "main" java.lang.BootstrapMethodError: call site 
>> initialization exception
>>         at java.lang.invoke.CallSite.makeSite(CallSite.java:341)
>>         at 
>> java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:307)
>>         at 
>> java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:297)
>>         at 
>> javany.util.AbstractCollection${0=I}.toString(AbstractCollection.java:514)
>>         at java.lang.String.valueOf(String.java:2987)
>>         at java.io.PrintStream.println(PrintStream.java:821)
>>         at Test.testInts(Test.java:17)
>>         at Test.main(Test.java:22)
>> Caused by: java.lang.VerifyError: Bad invokespecial instruction: 
>> current class isn't assignable to reference class.
>> Exception Details:
>>   Location:
>>     javany/util/Any$toString${0=I}.toString(I)Ljava/lang/String; @8: 
>> invokespecial
>>   Reason:
>>     Error exists in the bytecode
>>   Bytecode:
>>     0000000: bb00 0959 b700 0d1a b700 10b0
>>
>>         at sun.misc.Unsafe.defineAnonymousClass(Native Method)
>>         at 
>> java.lang.invoke.GenericMethodSpecializer.metafactory(GenericMethodSpecializer.java:98)
>>         at java.lang.invoke.CallSite.makeSite(CallSite.java:302)
>>         ... 7 more
>>
>>
>> But looking at generated bytecode:
>>
>>
>> Classfile 
>> /home/peter/work/local/valhalla-test/dump/javany.util.Any$toString${0=I}_ERROR.class
>>   Last modified Feb 8, 2015; size 321 bytes
>>   MD5 checksum 7dac6f4f515394e0aa2ed1990a092697
>>   Compiled from "Any.java"
>> public class javany.util.Any$toString${0=I}
>>   minor version: 0
>>   major version: 52
>>   flags: ACC_PUBLIC, ACC_SUPER
>> Constant pool:
>>    #1 = Utf8               javany/util/Any$toString${0=I}
>>    #2 = Class              #1             // 
>> "javany/util/Any$toString${0=I}"
>>    #3 = Utf8               java/lang/Object
>>    #4 = Class              #3             // java/lang/Object
>>    #5 = Utf8               Any.java
>>    #6 = Utf8               toString
>>    #7 = Utf8               (I)Ljava/lang/String;
>>    #8 = Utf8               javany/util/Any${0=I}
>>    #9 = Class              #8             // "javany/util/Any${0=I}"
>>   #10 = Utf8               <init>
>>   #11 = Utf8               ()V
>>   #12 = NameAndType        #10:#11        // "<init>":()V
>>   #13 = Methodref          #9.#12         // 
>> "javany/util/Any${0=I}"."<init>":()V
>>   #14 = Utf8               toStringImpl
>>   #15 = NameAndType        #14:#7         // 
>> toStringImpl:(I)Ljava/lang/String;
>>   #16 = Methodref          #9.#15         // 
>> "javany/util/Any${0=I}".toStringImpl:(I)Ljava/lang/String;
>>   #17 = Utf8               Code
>>   #18 = Utf8               LineNumberTable
>>   #19 = Utf8               Signature
>>   #20 = Utf8               SourceFile
>> {
>>   public static java.lang.String toString(int);
>>     descriptor: (I)Ljava/lang/String;
>>     flags: ACC_PUBLIC, ACC_STATIC
>>     Code:
>>       stack=2, locals=1, args_size=1
>>          0: new           #9                  // class 
>> "javany/util/Any${0=I}"
>>          3: dup
>>          4: invokespecial #13                 // Method 
>> "javany/util/Any${0=I}"."<init>":()V
>>          7: iload_0
>>          8: invokespecial #16                 // Method 
>> "javany/util/Any${0=I}".toStringImpl:(I)Ljava/lang/String;
>>         11: areturn
>>       LineNumberTable:
>>         line 31: 0
>>     Signature: #7                           // (I)Ljava/lang/String;
>> }
>> SourceFile: "Any.java"
>>
>>
>> Comparing it to original Any.class bytecode:
>>
>>
>>   public static <T extends java.lang.Object> java.lang.String 
>> toString(T);
>>     descriptor: (Ljava/lang/Object;)Ljava/lang/String;
>>     flags: ACC_PUBLIC, ACC_STATIC
>>     Code:
>>       stack=2, locals=1, args_size=1
>>          0: new           #3                  // class javany/util/Any
>>          3: dup
>>          4: invokespecial #4                  // Method "<init>":()V
>>          7: aload_0
>>          8: invokespecial #8                  // Method 
>> toStringImpl:(Ljava/lang/Object;)Ljava/lang/String;
>>         11: areturn
>>       LineNumberTable:
>>         line 31: 0
>> Error: unknown attribute
>>       BytecodeMapping: length = 0x12
>>        00 04 00 00 00 8C 00 04 00 8D 00 07 00 87 00 08
>>        00 9F
>>     Signature: #160                         // 
>> <T:Ljava/lang/Object;>(TT;)Ljava/lang/String;
>> Error: unknown attribute
>>       TypeVariablesMap: length = 0x9
>>        01 00 A1 01 01 00 92 00 86
>>
>>
>> And also peeking at Any${0=I}.class:
>>
>>
>>   private java.lang.String toStringImpl(int);
>>     descriptor: (I)Ljava/lang/String;
>>     flags: ACC_PRIVATE
>>     Code:
>>       stack=2, locals=2, args_size=2
>>          0: aload_0
>>          1: iload_1
>>          2: invokespecial #45                 // Method 
>> toStringNonNullImpl:(I)Ljava/lang/String;
>>          5: areturn
>>       LineNumberTable:
>>         line 181: 0
>>     Signature: #33                          // (I)Ljava/lang/String;
>>
>>
>> ...I can't spot what's wrong. Can you?
>>
>>
>> Regards, Peter
>>
>>
>> On 02/06/2015 03:38 PM, Maurizio Cimadamore wrote:
>>> Peter,
>>> I think I fixed most of the issues in the way; I've been able to 
>>> compile and run some tests successfully.
>>> Attached is a patch of the changes that are required for the code to 
>>> run correctly; I had to desugar some constructs (i.e. for-each, 
>>> non-static inner class) as all desugared stuff currently has an 
>>> issue in that javac won't emit the special sauce that is required by 
>>> the specializer to correctly specialize a class. That's an issue 
>>> with timing of the compilation pipeline - we are aware of it; but, 
>>> as it's a no trivial fix, it means that, for now, it's better not to 
>>> rely too much of compiler-desugared code.
>>>
>>> Maurizio
>>>
>>> On 04/02/15 12:42, Maurizio Cimadamore wrote:
>>>> Compiler fixes have been pushed, I will now look into the runtime 
>>>> issues you are getting...
>>>>
>>>> Maurizio
>>>>
>>>> On 04/02/15 10:18, Maurizio Cimadamore wrote:
>>>>> Thanks for the additional feedback, I'll try to get at the bottom 
>>>>> of those issues.
>>>>>
>>>>> Maurizio
>>>>>
>>>>> On 04/02/15 08:46, Peter Levart wrote:
>>>>>> Hi Maurizio,
>>>>>>
>>>>>> I have now managed to successfully compile the code. Here's the 
>>>>>> updated source:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~plevart/misc/valhala-hacks/javany-src.jar 
>>>>>>
>>>>>>
>>>>>>
>>>>>> But there's a StringIndexOutOfBoundsException thrown from 
>>>>>> specializer when running the following Test:
>>>>>>
>>>>>> public class Test {
>>>>>>     public static void main(String[] args) {
>>>>>>         List<int> ints = Arrays.asList(new int[]{1, 2, 3, 4, 5, 
>>>>>> 6, 7, 8});
>>>>>>         Iterator<int> it = ints.iterator();
>>>>>>         while (it.hasNext()) {
>>>>>>             System.out.println(it.next());
>>>>>>         }
>>>>>>     }
>>>>>> }
>>>>>>
>>>>>>
>>>>>> Specializing javany.util.List${0=I}; searching for 
>>>>>> javany/util/List.class (not found)
>>>>>> Specializing javany.util.List${0=I}; searching for 
>>>>>> javany/util/List.class (found)
>>>>>> Specializing javany.util.Collection${0=I}; searching for 
>>>>>> javany/util/Collection.class (not found)
>>>>>> Specializing javany.util.Collection${0=I}; searching for 
>>>>>> javany/util/Collection.class (found)
>>>>>> Specializing javany.lang.Iterable${0=I}; searching for 
>>>>>> javany/lang/Iterable.class (not found)
>>>>>> Specializing javany.lang.Iterable${0=I}; searching for 
>>>>>> javany/lang/Iterable.class (found)
>>>>>> Specializing method 
>>>>>> javany/util/Arrays$asList${0=I}.asList([Ljava/lang/Object;)Ljavany/util/List; 
>>>>>> with class=[] and method=[I]
>>>>>> Specializing javany.util.Arrays$ArrayList${0=I}; searching for 
>>>>>> javany/util/Arrays$ArrayList.class (not found)
>>>>>> Specializing javany.util.Arrays$ArrayList${0=I}; searching for 
>>>>>> javany/util/Arrays$ArrayList.class (found)
>>>>>> Exception in thread "main" 
>>>>>> java.lang.StringIndexOutOfBoundsException: String index out of 
>>>>>> range: 0
>>>>>>         at java.lang.String.charAt(String.java:646)
>>>>>>         at 
>>>>>> jdk.internal.org.objectweb.asm.signature.SignatureReader.accept(SignatureReader.java:107)
>>>>>>         at 
>>>>>> valhalla.specializer.SignatureSpecializer.forType(SignatureSpecializer.java:72)
>>>>>>         at 
>>>>>> valhalla.specializer.Specializer$ManglingMethodVisitor.visitInvokeDynamicInsn(Specializer.java:679)
>>>>>>         at 
>>>>>> jdk.internal.org.objectweb.asm.ClassReader.readCode(ClassReader.java:1507)
>>>>>>         at 
>>>>>> jdk.internal.org.objectweb.asm.ClassReader.readMethod(ClassReader.java:1084)
>>>>>>         at 
>>>>>> jdk.internal.org.objectweb.asm.ClassReader.accept(ClassReader.java:729) 
>>>>>>
>>>>>>         at 
>>>>>> valhalla.specializer.Specializer.specialize(Specializer.java:79)
>>>>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:409)
>>>>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:386)
>>>>>>         at java.security.AccessController.doPrivileged(Native 
>>>>>> Method)
>>>>>>         at 
>>>>>> java.net.URLClassLoader.findClass(URLClassLoader.java:385)
>>>>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:426)
>>>>>>         at 
>>>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:317)
>>>>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:359)
>>>>>>         at 
>>>>>> javany.util.Arrays$asList${0=I}/511754216.asList(Arrays.java:810)
>>>>>>         at Test.main(Test.java:10)
>>>>>>
>>>>>>
>>>>>> Appart from that, I learned that when the component type of 
>>>>>> vararg array is an <any> type variable (for example: <any T> T[] 
>>>>>> Arrays.asList(T ... a)), the invocation doesn't compile:
>>>>>>
>>>>>> src/Test.java:10: error: method invoked with incorrect number of 
>>>>>> arguments; expected 3, found 1
>>>>>>         List<int> ints = Arrays.<int>asList(1, 2, 3);
>>>>>>                          ^
>>>>>> 1 error
>>>>>>
>>>>>>
>>>>>> Non-specialized code also has problems at runtime:
>>>>>>
>>>>>> public class Test {
>>>>>>     public static void main(String[] args) {
>>>>>>         List<String> strings = Arrays.<String>asList("a", "b", "c");
>>>>>>         Iterator<String> it = strings.iterator();
>>>>>>         while (it.hasNext()) {
>>>>>>             System.out.println(it.next());
>>>>>>         }
>>>>>>     }
>>>>>> }
>>>>>>
>>>>>>
>>>>>> Exception in thread "main" java.lang.ClassFormatError: Absent 
>>>>>> Code attribute in method that is not native or abstract in class 
>>>>>> file javany/util/AbstractList
>>>>>>         at java.lang.ClassLoader.defineClass1(Native Method)
>>>>>>         at java.lang.ClassLoader.defineClass(ClassLoader.java:762)
>>>>>>         at 
>>>>>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) 
>>>>>>
>>>>>>         at 
>>>>>> java.net.URLClassLoader.defineClass(URLClassLoader.java:537)
>>>>>>         at 
>>>>>> java.net.URLClassLoader.access$300(URLClassLoader.java:78)
>>>>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:438)
>>>>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:386)
>>>>>>         at java.security.AccessController.doPrivileged(Native 
>>>>>> Method)
>>>>>>         at 
>>>>>> java.net.URLClassLoader.findClass(URLClassLoader.java:385)
>>>>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:426)
>>>>>>         at 
>>>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:317)
>>>>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:359)
>>>>>>         at java.lang.ClassLoader.defineClass1(Native Method)
>>>>>>         at java.lang.ClassLoader.defineClass(ClassLoader.java:762)
>>>>>>         at 
>>>>>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) 
>>>>>>
>>>>>>         at 
>>>>>> java.net.URLClassLoader.defineClass(URLClassLoader.java:537)
>>>>>>         at 
>>>>>> java.net.URLClassLoader.access$300(URLClassLoader.java:78)
>>>>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:438)
>>>>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:386)
>>>>>>         at java.security.AccessController.doPrivileged(Native 
>>>>>> Method)
>>>>>>         at 
>>>>>> java.net.URLClassLoader.findClass(URLClassLoader.java:385)
>>>>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:426)
>>>>>>         at 
>>>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:317)
>>>>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:359)
>>>>>>         at javany.util.Arrays.asList(Arrays.java:810)
>>>>>>         at Test.main(Test.java:10)
>>>>>>
>>>>>>
>>>>>>
>>>>>> Regards, Peter
>>>>>>
>>>>>> On 02/03/2015 10:34 PM, Maurizio Cimadamore wrote:
>>>>>>>
>>>>>>> On 03/02/15 21:05, Peter Levart wrote:
>>>>>>>> Hi Maurizio,
>>>>>>>>
>>>>>>>> I see. I thought this could be a nice idiom for boxing, since 
>>>>>>>> the following:
>>>>>>>>
>>>>>>>> (Object) 42
>>>>>>>>
>>>>>>>> ...is legal and results in an Integer object at runtime.
>>>>>>> I'm not saying this will never work - actually the compiler is 
>>>>>>> currently accepting this kind of idioms, but the specializer 
>>>>>>> does nothing with it, so you'll get runtime errors.
>>>>>>>>
>>>>>>>> But I don't know if a checkcast is actually inserted for 
>>>>>>>> (Object). Could javac redundantly do it in case casting to 
>>>>>>>> Object is from expression of <any> type and also equip 
>>>>>>>> checkcast with BMA indicating the type of expression so that 
>>>>>>>> specialization could replace it with boxing code?
>>>>>>> That will be the way forward, yes
>>>>>>>
>>>>>>> Maurizio
>>>>>>>>
>>>>>>>> Regards, Peter
>>>>>>>>
>>>>>>>>
>>>>>>>> On 02/03/2015 08:09 PM, Maurizio Cimadamore wrote:
>>>>>>>>>
>>>>>>>>> On 03/02/15 18:46, Maurizio Cimadamore wrote:
>>>>>>>>>> I will also investigate on the crash you are getting... 
>>>>>>>>> Hi Peter,
>>>>>>>>> the crash is coming from this code in AbstractCollection (see 
>>>>>>>>> code in bold):
>>>>>>>>>
>>>>>>>>>     public boolean contains(Object o) {
>>>>>>>>>         __WhereVal(E) {
>>>>>>>>>             Iterator<E> it = iterator();
>>>>>>>>>             if (o == null) {
>>>>>>>>>                 return false;
>>>>>>>>>             } else {
>>>>>>>>>                 while (it.hasNext())
>>>>>>>>> *if (o.equals((Object) it.next()))*
>>>>>>>>>                         return true;
>>>>>>>>>             }
>>>>>>>>>             return false;
>>>>>>>>>         }
>>>>>>>>>         __WhereRef(E) {
>>>>>>>>>             Iterator<E> it = iterator();
>>>>>>>>>             if (o == null) {
>>>>>>>>>                 while (it.hasNext())
>>>>>>>>>                     if (it.next() == null)
>>>>>>>>>                         return true;
>>>>>>>>>             } else {
>>>>>>>>>                 while (it.hasNext())
>>>>>>>>>                     if (o.equals(it.next()))
>>>>>>>>>                         return true;
>>>>>>>>>             }
>>>>>>>>>             return false;
>>>>>>>>>         }
>>>>>>>>>     }
>>>>>>>>>
>>>>>>>>> I believe that, apart from the obvious javac bug, the code has 
>>>>>>>>> an issue, as it.next() is supposed to return a value there, 
>>>>>>>>> but you are casting to Object?
>>>>>>>>>
>>>>>>>>> For the records - a simpler test case for the bug is this:
>>>>>>>>>
>>>>>>>>> class Foo<any E> {
>>>>>>>>>     E e;
>>>>>>>>>     E get() { return e; }
>>>>>>>>>
>>>>>>>>>     void test() {
>>>>>>>>>         __WhereVal(E) {
>>>>>>>>>             Object o = (Object)get();
>>>>>>>>>         }
>>>>>>>>>     }
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> Maurizio
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>


From peter.levart at gmail.com  Sun Feb  8 11:50:35 2015
From: peter.levart at gmail.com (Peter Levart)
Date: Sun, 08 Feb 2015 12:50:35 +0100
Subject: experiences with prototype
In-Reply-To: <54D73DC8.6000900@gmail.com>
References: <54D113DA.9060509@gmail.com>	<54D11792.3090504@oracle.com>	<54D11CEC.6080009@oracle.com>	<54D1380A.5070503@gmail.com>	<54D13EDB.5050704@oracle.com>	<54D1DC5D.7060408@gmail.com>
	<54D1F1E3.10209@oracle.com> <54D213C2.3020603@oracle.com>
	<54D4D200.9090906@oracle.com> <54D73A1E.9020102@gmail.com>
	<54D73DC8.6000900@gmail.com>
Message-ID: <54D74D8B.7030200@gmail.com>

Hi Maurizzio,

Testing javany.util further, I found that the following compiles without 
errors:


     default Comparator<T> thenComparing(Comparator<? super T> other) {
         Objects.requireNonNull(other);
         return (Comparator<T> & Serializable) (c1, c2) -> {
             int res = compare(c1, c2);
             return (res != 0) ? res : other.compare(c1, c2);
         };
     }


...but fails at runtime:


Specializing method 
javany/util/Comparators$naturalOrder${}.naturalOrder()Ljavany/util/Comparator; 
with class=[] and method=[TT;]
Specializing javany.util.Comparator${0=Z}; searching for 
javany/util/Comparator.class (not found)
Specializing javany.util.Comparator${0=Z}; searching for 
javany/util/Comparator.class (found)
Exception in thread "main" java.lang.BootstrapMethodError: call site 
initialization exception
         at java.lang.invoke.CallSite.makeSite(CallSite.java:341)
         at 
java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:307)
         at 
java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:297)
         at javany.util.Comparators.<clinit>(Comparators.java:55)
         at 
javany.util.Comparators$naturalOrder${}/1831932724.naturalOrder(Comparators.java:44)
         at javany.util.Comparator.reverseOrder(Comparator.java:337)
         at Test.testStrings(Test.java:14)
         at Test.main(Test.java:26)
Caused by: java.lang.VerifyError: Bad type on operand stack
Exception Details:
   Location:
javany/util/Comparator${0=Z}.lambda$thenComparing$10d9995b$1(Ljavany/util/Comparator;Ljava/lang/Object;Ljava/lang/Object;)I 
@3: invokeinterface
   Reason:
     Type 'java/lang/Object' (current frame, stack[2]) is not assignable 
to integer
   Current Frame:
     bci: @3
     flags: { }
     locals: { 'javany/util/Comparator${0=Z}', 'javany/util/Comparator', 
'java/lang/Object', 'java/lang/Object' }
     stack: { 'javany/util/Comparator${0=Z}', 'java/lang/Object', 
'java/lang/Object' }
   Bytecode:
     0000000: 2a2c 2db9 007b 0300 3604 1504 9900 0815
     0000010: 04a7 000b 2b2c 2db9 007d 0300 ac
   Stackmap Table:
     append_frame(@20,Integer)
     same_locals_1_stack_item_frame(@28,Integer)

         at sun.misc.Unsafe.defineAnonymousClass(Native Method)
         at 
java.lang.invoke.InnerClassLambdaMetafactory.spinInnerClass(InnerClassLambdaMetafactory.java:324)
         at 
java.lang.invoke.InnerClassLambdaMetafactory.buildCallSite(InnerClassLambdaMetafactory.java:194)
         at 
java.lang.invoke.LambdaMetafactory.metafactory(LambdaMetafactory.java:304)
         at java.lang.invoke.CallSite.makeSite(CallSite.java:302)
         ... 7 more


Are lambdas/method references also not compatible with specialization 
currently?


Peter


On 02/08/2015 11:43 AM, Peter Levart wrote:
> I've got it.
>
> The problem was that Any.<int>toStringImpl() instance method was 
> private. Since it is located in a specialized (Any${0=I}) class, it is 
> not accessible from Any class (or specialized VM anonymous class 
> generated for a specialized generic static method). The VerifyError: 
> "Bad invokespecial instruction: current class isn't assignable to 
> reference class." is not very informative though.
>
> By making the method(s) package-private, my test now runs correctly.
>
> Regards, Peter
>
> On 02/08/2015 11:27 AM, Peter Levart wrote:
>> Hi Maurizio,
>>
>> This is great. I replaced all new-style for loops on <any T> T[] 
>> arrays with classic on-index iteration and applied your patch 
>> selectively without the redundant cast in javany/util/Arrays.java:823:
>>
>>     a = (E[])Objects.requireNonNull(array)
>>
>> ... as I think your last langtools patch (99ab651be669) fixes that, 
>> right?
>>
>>
>> Now with the following code:
>>
>> http://cr.openjdk.java.net/~plevart/misc/valhala-hacks/javany-src.02.jar
>>
>> And this test:
>>
>> public class Test {
>>
>>     static void testStrings() {
>>         List<String> strings = Arrays.<String>asList("a", "b", "c");
>>         System.out.println(strings);
>>     }
>>
>>     static void testInts() {
>>         List<int> ints = Arrays.<int>asList(1, 2, 3);
>>         System.out.println(ints);
>>     }
>>
>>     public static void main(String[] args) {
>>         testStrings();
>>         testInts();
>>     }
>> }
>>
>>
>> I get:
>>
>> [a, b, c]
>> Specializing javany.util.List${0=I}; searching for 
>> javany/util/List.class (not found)
>> Specializing javany.util.List${0=I}; searching for 
>> javany/util/List.class (found)
>> Specializing javany.util.Collection${0=I}; searching for 
>> javany/util/Collection.class (not found)
>> Specializing javany.util.Collection${0=I}; searching for 
>> javany/util/Collection.class (found)
>> Specializing javany.lang.Iterable${0=I}; searching for 
>> javany/lang/Iterable.class (not found)
>> Specializing javany.lang.Iterable${0=I}; searching for 
>> javany/lang/Iterable.class (found)
>> Specializing method 
>> javany/util/Arrays$asList${0=I}.asList([Ljava/lang/Object;)Ljavany/util/List; 
>> with class=[] and method=[I]
>> Specializing javany.util.Arrays$ArrayList${0=I}; searching for 
>> javany/util/Arrays$ArrayList.class (not found)
>> Specializing javany.util.Arrays$ArrayList${0=I}; searching for 
>> javany/util/Arrays$ArrayList.class (found)
>> Specializing javany.util.AbstractList${0=I}; searching for 
>> javany/util/AbstractList.class (not found)
>> Specializing javany.util.AbstractList${0=I}; searching for 
>> javany/util/AbstractList.class (found)
>> Specializing javany.util.AbstractCollection${0=I}; searching for 
>> javany/util/AbstractCollection.class (not found)
>> Specializing javany.util.AbstractCollection${0=I}; searching for 
>> javany/util/AbstractCollection.class (found)
>> Specializing javany.util.Iterator${0=I}; searching for 
>> javany/util/Iterator.class (not found)
>> Specializing javany.util.Iterator${0=I}; searching for 
>> javany/util/Iterator.class (found)
>> Specializing javany.util.ListIterator${0=I}; searching for 
>> javany/util/ListIterator.class (not found)
>> Specializing javany.util.ListIterator${0=I}; searching for 
>> javany/util/ListIterator.class (found)
>> Specializing javany.util.AbstractList$Itr${0=I}; searching for 
>> javany/util/AbstractList$Itr.class (not found)
>> Specializing javany.util.AbstractList$Itr${0=I}; searching for 
>> javany/util/AbstractList$Itr.class (found)
>> Specializing method 
>> javany/util/Any$toString${0=I}.toString(Ljava/lang/Object;)Ljava/lang/String; 
>> with class=[] and method=[I]
>> Specializing javany.util.Any${0=I}; searching for 
>> javany/util/Any.class (not found)
>> Specializing javany.util.Any${0=I}; searching for 
>> javany/util/Any.class (found)
>> Exception in thread "main" java.lang.BootstrapMethodError: call site 
>> initialization exception
>>         at java.lang.invoke.CallSite.makeSite(CallSite.java:341)
>>         at 
>> java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:307)
>>         at 
>> java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:297)
>>         at 
>> javany.util.AbstractCollection${0=I}.toString(AbstractCollection.java:514)
>>         at java.lang.String.valueOf(String.java:2987)
>>         at java.io.PrintStream.println(PrintStream.java:821)
>>         at Test.testInts(Test.java:17)
>>         at Test.main(Test.java:22)
>> Caused by: java.lang.VerifyError: Bad invokespecial instruction: 
>> current class isn't assignable to reference class.
>> Exception Details:
>>   Location:
>>     javany/util/Any$toString${0=I}.toString(I)Ljava/lang/String; @8: 
>> invokespecial
>>   Reason:
>>     Error exists in the bytecode
>>   Bytecode:
>>     0000000: bb00 0959 b700 0d1a b700 10b0
>>
>>         at sun.misc.Unsafe.defineAnonymousClass(Native Method)
>>         at 
>> java.lang.invoke.GenericMethodSpecializer.metafactory(GenericMethodSpecializer.java:98)
>>         at java.lang.invoke.CallSite.makeSite(CallSite.java:302)
>>         ... 7 more
>>
>>
>> But looking at generated bytecode:
>>
>>
>> Classfile 
>> /home/peter/work/local/valhalla-test/dump/javany.util.Any$toString${0=I}_ERROR.class
>>   Last modified Feb 8, 2015; size 321 bytes
>>   MD5 checksum 7dac6f4f515394e0aa2ed1990a092697
>>   Compiled from "Any.java"
>> public class javany.util.Any$toString${0=I}
>>   minor version: 0
>>   major version: 52
>>   flags: ACC_PUBLIC, ACC_SUPER
>> Constant pool:
>>    #1 = Utf8               javany/util/Any$toString${0=I}
>>    #2 = Class              #1             // 
>> "javany/util/Any$toString${0=I}"
>>    #3 = Utf8               java/lang/Object
>>    #4 = Class              #3             // java/lang/Object
>>    #5 = Utf8               Any.java
>>    #6 = Utf8               toString
>>    #7 = Utf8               (I)Ljava/lang/String;
>>    #8 = Utf8               javany/util/Any${0=I}
>>    #9 = Class              #8             // "javany/util/Any${0=I}"
>>   #10 = Utf8               <init>
>>   #11 = Utf8               ()V
>>   #12 = NameAndType        #10:#11        // "<init>":()V
>>   #13 = Methodref          #9.#12         // 
>> "javany/util/Any${0=I}"."<init>":()V
>>   #14 = Utf8               toStringImpl
>>   #15 = NameAndType        #14:#7         // 
>> toStringImpl:(I)Ljava/lang/String;
>>   #16 = Methodref          #9.#15         // 
>> "javany/util/Any${0=I}".toStringImpl:(I)Ljava/lang/String;
>>   #17 = Utf8               Code
>>   #18 = Utf8               LineNumberTable
>>   #19 = Utf8               Signature
>>   #20 = Utf8               SourceFile
>> {
>>   public static java.lang.String toString(int);
>>     descriptor: (I)Ljava/lang/String;
>>     flags: ACC_PUBLIC, ACC_STATIC
>>     Code:
>>       stack=2, locals=1, args_size=1
>>          0: new           #9                  // class 
>> "javany/util/Any${0=I}"
>>          3: dup
>>          4: invokespecial #13                 // Method 
>> "javany/util/Any${0=I}"."<init>":()V
>>          7: iload_0
>>          8: invokespecial #16                 // Method 
>> "javany/util/Any${0=I}".toStringImpl:(I)Ljava/lang/String;
>>         11: areturn
>>       LineNumberTable:
>>         line 31: 0
>>     Signature: #7                           // (I)Ljava/lang/String;
>> }
>> SourceFile: "Any.java"
>>
>>
>> Comparing it to original Any.class bytecode:
>>
>>
>>   public static <T extends java.lang.Object> java.lang.String 
>> toString(T);
>>     descriptor: (Ljava/lang/Object;)Ljava/lang/String;
>>     flags: ACC_PUBLIC, ACC_STATIC
>>     Code:
>>       stack=2, locals=1, args_size=1
>>          0: new           #3                  // class javany/util/Any
>>          3: dup
>>          4: invokespecial #4                  // Method "<init>":()V
>>          7: aload_0
>>          8: invokespecial #8                  // Method 
>> toStringImpl:(Ljava/lang/Object;)Ljava/lang/String;
>>         11: areturn
>>       LineNumberTable:
>>         line 31: 0
>> Error: unknown attribute
>>       BytecodeMapping: length = 0x12
>>        00 04 00 00 00 8C 00 04 00 8D 00 07 00 87 00 08
>>        00 9F
>>     Signature: #160                         // 
>> <T:Ljava/lang/Object;>(TT;)Ljava/lang/String;
>> Error: unknown attribute
>>       TypeVariablesMap: length = 0x9
>>        01 00 A1 01 01 00 92 00 86
>>
>>
>> And also peeking at Any${0=I}.class:
>>
>>
>>   private java.lang.String toStringImpl(int);
>>     descriptor: (I)Ljava/lang/String;
>>     flags: ACC_PRIVATE
>>     Code:
>>       stack=2, locals=2, args_size=2
>>          0: aload_0
>>          1: iload_1
>>          2: invokespecial #45                 // Method 
>> toStringNonNullImpl:(I)Ljava/lang/String;
>>          5: areturn
>>       LineNumberTable:
>>         line 181: 0
>>     Signature: #33                          // (I)Ljava/lang/String;
>>
>>
>> ...I can't spot what's wrong. Can you?
>>
>>
>> Regards, Peter
>>
>>
>> On 02/06/2015 03:38 PM, Maurizio Cimadamore wrote:
>>> Peter,
>>> I think I fixed most of the issues in the way; I've been able to 
>>> compile and run some tests successfully.
>>> Attached is a patch of the changes that are required for the code to 
>>> run correctly; I had to desugar some constructs (i.e. for-each, 
>>> non-static inner class) as all desugared stuff currently has an 
>>> issue in that javac won't emit the special sauce that is required by 
>>> the specializer to correctly specialize a class. That's an issue 
>>> with timing of the compilation pipeline - we are aware of it; but, 
>>> as it's a no trivial fix, it means that, for now, it's better not to 
>>> rely too much of compiler-desugared code.
>>>
>>> Maurizio
>>>
>>> On 04/02/15 12:42, Maurizio Cimadamore wrote:
>>>> Compiler fixes have been pushed, I will now look into the runtime 
>>>> issues you are getting...
>>>>
>>>> Maurizio
>>>>
>>>> On 04/02/15 10:18, Maurizio Cimadamore wrote:
>>>>> Thanks for the additional feedback, I'll try to get at the bottom 
>>>>> of those issues.
>>>>>
>>>>> Maurizio
>>>>>
>>>>> On 04/02/15 08:46, Peter Levart wrote:
>>>>>> Hi Maurizio,
>>>>>>
>>>>>> I have now managed to successfully compile the code. Here's the 
>>>>>> updated source:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~plevart/misc/valhala-hacks/javany-src.jar 
>>>>>>
>>>>>>
>>>>>>
>>>>>> But there's a StringIndexOutOfBoundsException thrown from 
>>>>>> specializer when running the following Test:
>>>>>>
>>>>>> public class Test {
>>>>>>     public static void main(String[] args) {
>>>>>>         List<int> ints = Arrays.asList(new int[]{1, 2, 3, 4, 5, 
>>>>>> 6, 7, 8});
>>>>>>         Iterator<int> it = ints.iterator();
>>>>>>         while (it.hasNext()) {
>>>>>>             System.out.println(it.next());
>>>>>>         }
>>>>>>     }
>>>>>> }
>>>>>>
>>>>>>
>>>>>> Specializing javany.util.List${0=I}; searching for 
>>>>>> javany/util/List.class (not found)
>>>>>> Specializing javany.util.List${0=I}; searching for 
>>>>>> javany/util/List.class (found)
>>>>>> Specializing javany.util.Collection${0=I}; searching for 
>>>>>> javany/util/Collection.class (not found)
>>>>>> Specializing javany.util.Collection${0=I}; searching for 
>>>>>> javany/util/Collection.class (found)
>>>>>> Specializing javany.lang.Iterable${0=I}; searching for 
>>>>>> javany/lang/Iterable.class (not found)
>>>>>> Specializing javany.lang.Iterable${0=I}; searching for 
>>>>>> javany/lang/Iterable.class (found)
>>>>>> Specializing method 
>>>>>> javany/util/Arrays$asList${0=I}.asList([Ljava/lang/Object;)Ljavany/util/List; 
>>>>>> with class=[] and method=[I]
>>>>>> Specializing javany.util.Arrays$ArrayList${0=I}; searching for 
>>>>>> javany/util/Arrays$ArrayList.class (not found)
>>>>>> Specializing javany.util.Arrays$ArrayList${0=I}; searching for 
>>>>>> javany/util/Arrays$ArrayList.class (found)
>>>>>> Exception in thread "main" 
>>>>>> java.lang.StringIndexOutOfBoundsException: String index out of 
>>>>>> range: 0
>>>>>>         at java.lang.String.charAt(String.java:646)
>>>>>>         at 
>>>>>> jdk.internal.org.objectweb.asm.signature.SignatureReader.accept(SignatureReader.java:107)
>>>>>>         at 
>>>>>> valhalla.specializer.SignatureSpecializer.forType(SignatureSpecializer.java:72)
>>>>>>         at 
>>>>>> valhalla.specializer.Specializer$ManglingMethodVisitor.visitInvokeDynamicInsn(Specializer.java:679)
>>>>>>         at 
>>>>>> jdk.internal.org.objectweb.asm.ClassReader.readCode(ClassReader.java:1507)
>>>>>>         at 
>>>>>> jdk.internal.org.objectweb.asm.ClassReader.readMethod(ClassReader.java:1084)
>>>>>>         at 
>>>>>> jdk.internal.org.objectweb.asm.ClassReader.accept(ClassReader.java:729) 
>>>>>>
>>>>>>         at 
>>>>>> valhalla.specializer.Specializer.specialize(Specializer.java:79)
>>>>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:409)
>>>>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:386)
>>>>>>         at java.security.AccessController.doPrivileged(Native 
>>>>>> Method)
>>>>>>         at 
>>>>>> java.net.URLClassLoader.findClass(URLClassLoader.java:385)
>>>>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:426)
>>>>>>         at 
>>>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:317)
>>>>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:359)
>>>>>>         at 
>>>>>> javany.util.Arrays$asList${0=I}/511754216.asList(Arrays.java:810)
>>>>>>         at Test.main(Test.java:10)
>>>>>>
>>>>>>
>>>>>> Appart from that, I learned that when the component type of 
>>>>>> vararg array is an <any> type variable (for example: <any T> T[] 
>>>>>> Arrays.asList(T ... a)), the invocation doesn't compile:
>>>>>>
>>>>>> src/Test.java:10: error: method invoked with incorrect number of 
>>>>>> arguments; expected 3, found 1
>>>>>>         List<int> ints = Arrays.<int>asList(1, 2, 3);
>>>>>>                          ^
>>>>>> 1 error
>>>>>>
>>>>>>
>>>>>> Non-specialized code also has problems at runtime:
>>>>>>
>>>>>> public class Test {
>>>>>>     public static void main(String[] args) {
>>>>>>         List<String> strings = Arrays.<String>asList("a", "b", "c");
>>>>>>         Iterator<String> it = strings.iterator();
>>>>>>         while (it.hasNext()) {
>>>>>>             System.out.println(it.next());
>>>>>>         }
>>>>>>     }
>>>>>> }
>>>>>>
>>>>>>
>>>>>> Exception in thread "main" java.lang.ClassFormatError: Absent 
>>>>>> Code attribute in method that is not native or abstract in class 
>>>>>> file javany/util/AbstractList
>>>>>>         at java.lang.ClassLoader.defineClass1(Native Method)
>>>>>>         at java.lang.ClassLoader.defineClass(ClassLoader.java:762)
>>>>>>         at 
>>>>>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) 
>>>>>>
>>>>>>         at 
>>>>>> java.net.URLClassLoader.defineClass(URLClassLoader.java:537)
>>>>>>         at 
>>>>>> java.net.URLClassLoader.access$300(URLClassLoader.java:78)
>>>>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:438)
>>>>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:386)
>>>>>>         at java.security.AccessController.doPrivileged(Native 
>>>>>> Method)
>>>>>>         at 
>>>>>> java.net.URLClassLoader.findClass(URLClassLoader.java:385)
>>>>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:426)
>>>>>>         at 
>>>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:317)
>>>>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:359)
>>>>>>         at java.lang.ClassLoader.defineClass1(Native Method)
>>>>>>         at java.lang.ClassLoader.defineClass(ClassLoader.java:762)
>>>>>>         at 
>>>>>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) 
>>>>>>
>>>>>>         at 
>>>>>> java.net.URLClassLoader.defineClass(URLClassLoader.java:537)
>>>>>>         at 
>>>>>> java.net.URLClassLoader.access$300(URLClassLoader.java:78)
>>>>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:438)
>>>>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:386)
>>>>>>         at java.security.AccessController.doPrivileged(Native 
>>>>>> Method)
>>>>>>         at 
>>>>>> java.net.URLClassLoader.findClass(URLClassLoader.java:385)
>>>>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:426)
>>>>>>         at 
>>>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:317)
>>>>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:359)
>>>>>>         at javany.util.Arrays.asList(Arrays.java:810)
>>>>>>         at Test.main(Test.java:10)
>>>>>>
>>>>>>
>>>>>>
>>>>>> Regards, Peter
>>>>>>
>>>>>> On 02/03/2015 10:34 PM, Maurizio Cimadamore wrote:
>>>>>>>
>>>>>>> On 03/02/15 21:05, Peter Levart wrote:
>>>>>>>> Hi Maurizio,
>>>>>>>>
>>>>>>>> I see. I thought this could be a nice idiom for boxing, since 
>>>>>>>> the following:
>>>>>>>>
>>>>>>>> (Object) 42
>>>>>>>>
>>>>>>>> ...is legal and results in an Integer object at runtime.
>>>>>>> I'm not saying this will never work - actually the compiler is 
>>>>>>> currently accepting this kind of idioms, but the specializer 
>>>>>>> does nothing with it, so you'll get runtime errors.
>>>>>>>>
>>>>>>>> But I don't know if a checkcast is actually inserted for 
>>>>>>>> (Object). Could javac redundantly do it in case casting to 
>>>>>>>> Object is from expression of <any> type and also equip 
>>>>>>>> checkcast with BMA indicating the type of expression so that 
>>>>>>>> specialization could replace it with boxing code?
>>>>>>> That will be the way forward, yes
>>>>>>>
>>>>>>> Maurizio
>>>>>>>>
>>>>>>>> Regards, Peter
>>>>>>>>
>>>>>>>>
>>>>>>>> On 02/03/2015 08:09 PM, Maurizio Cimadamore wrote:
>>>>>>>>>
>>>>>>>>> On 03/02/15 18:46, Maurizio Cimadamore wrote:
>>>>>>>>>> I will also investigate on the crash you are getting... 
>>>>>>>>> Hi Peter,
>>>>>>>>> the crash is coming from this code in AbstractCollection (see 
>>>>>>>>> code in bold):
>>>>>>>>>
>>>>>>>>>     public boolean contains(Object o) {
>>>>>>>>>         __WhereVal(E) {
>>>>>>>>>             Iterator<E> it = iterator();
>>>>>>>>>             if (o == null) {
>>>>>>>>>                 return false;
>>>>>>>>>             } else {
>>>>>>>>>                 while (it.hasNext())
>>>>>>>>> *if (o.equals((Object) it.next()))*
>>>>>>>>>                         return true;
>>>>>>>>>             }
>>>>>>>>>             return false;
>>>>>>>>>         }
>>>>>>>>>         __WhereRef(E) {
>>>>>>>>>             Iterator<E> it = iterator();
>>>>>>>>>             if (o == null) {
>>>>>>>>>                 while (it.hasNext())
>>>>>>>>>                     if (it.next() == null)
>>>>>>>>>                         return true;
>>>>>>>>>             } else {
>>>>>>>>>                 while (it.hasNext())
>>>>>>>>>                     if (o.equals(it.next()))
>>>>>>>>>                         return true;
>>>>>>>>>             }
>>>>>>>>>             return false;
>>>>>>>>>         }
>>>>>>>>>     }
>>>>>>>>>
>>>>>>>>> I believe that, apart from the obvious javac bug, the code has 
>>>>>>>>> an issue, as it.next() is supposed to return a value there, 
>>>>>>>>> but you are casting to Object?
>>>>>>>>>
>>>>>>>>> For the records - a simpler test case for the bug is this:
>>>>>>>>>
>>>>>>>>> class Foo<any E> {
>>>>>>>>>     E e;
>>>>>>>>>     E get() { return e; }
>>>>>>>>>
>>>>>>>>>     void test() {
>>>>>>>>>         __WhereVal(E) {
>>>>>>>>>             Object o = (Object)get();
>>>>>>>>>         }
>>>>>>>>>     }
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> Maurizio
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>


From forax at univ-mlv.fr  Sun Feb  8 17:50:00 2015
From: forax at univ-mlv.fr (Remi Forax)
Date: Sun, 08 Feb 2015 18:50:00 +0100
Subject: experiences with prototype
In-Reply-To: <54D74D8B.7030200@gmail.com>
References: <54D113DA.9060509@gmail.com>	<54D11792.3090504@oracle.com>	<54D11CEC.6080009@oracle.com>	<54D1380A.5070503@gmail.com>	<54D13EDB.5050704@oracle.com>	<54D1DC5D.7060408@gmail.com>	<54D1F1E3.10209@oracle.com>
	<54D213C2.3020603@oracle.com>	<54D4D200.9090906@oracle.com>
	<54D73A1E.9020102@gmail.com>	<54D73DC8.6000900@gmail.com>
	<54D74D8B.7030200@gmail.com>
Message-ID: <54D7A1C8.4050608@univ-mlv.fr>

Hi Peter,
lambdas/method reference are desugared to an invokedynamic + a specific 
bootstrap method,
so you need
1) javac to emit enough information for the specializer
     javac as to indicate if the functional interface (the return value 
of invokedynamic)
     is a specilized type and if some captured values are also of a 
specialized type.
     This part is the same as when specializing invokedynamic.
2) javac also emit 3 informations,
     two method types that correspond to the descriptor of the abstract 
method of the
     functional interface, the first one is erased, the second one is 
not erased.
     Both these method descriptors need to be specialized thus required 
to generate
     2 supplementary signatures in the bytecode mapping.
     The last information is a method handle corresponding to the 
implementation of
     the body of the lambda.
3) javac currently doesn't emit any generics information for the body of 
the lambda,
     javac should emit such information and add the information in order 
to specialize
     the body if the lambda use some type variable bound by any.
4) now that we have all the information available, we need the bootstrap 
method
     (the lambda metafactory) to generate not the classical lambda proxy but
    a specialized version of the lambda proxy. Given that the lambda 
metafactory
    use ASM to geenrate the lambda proxy, and the specializer use ASM too,
    the idea is just to use the specializer as an adapter visitor 
between the code
    that generate the lambda proxy and the ASM bytecode writer.
    This will generate a specialized lambda proxy.

As a corner case, while a lambda can not declare a type variable
(in retrospect, I should have pushed for it more during EG discussions 
because
some of the function design patterns are harder to implement because of 
that),
a method referenced by a method reference can declare a type variable so
either the code of the method inside the lambda proxy should emit an 
invokedynamic
for such generics method reference call or the lambda metafactory should
be changed to support this kind of call.

R?mi

On 02/08/2015 12:50 PM, Peter Levart wrote:
> Hi Maurizzio,
>
> Testing javany.util further, I found that the following compiles 
> without errors:
>
>
>     default Comparator<T> thenComparing(Comparator<? super T> other) {
>         Objects.requireNonNull(other);
>         return (Comparator<T> & Serializable) (c1, c2) -> {
>             int res = compare(c1, c2);
>             return (res != 0) ? res : other.compare(c1, c2);
>         };
>     }
>
>
> ...but fails at runtime:
>
>
> Specializing method 
> javany/util/Comparators$naturalOrder${}.naturalOrder()Ljavany/util/Comparator; 
> with class=[] and method=[TT;]
> Specializing javany.util.Comparator${0=Z}; searching for 
> javany/util/Comparator.class (not found)
> Specializing javany.util.Comparator${0=Z}; searching for 
> javany/util/Comparator.class (found)
> Exception in thread "main" java.lang.BootstrapMethodError: call site 
> initialization exception
>         at java.lang.invoke.CallSite.makeSite(CallSite.java:341)
>         at 
> java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:307)
>         at 
> java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:297)
>         at javany.util.Comparators.<clinit>(Comparators.java:55)
>         at 
> javany.util.Comparators$naturalOrder${}/1831932724.naturalOrder(Comparators.java:44)
>         at javany.util.Comparator.reverseOrder(Comparator.java:337)
>         at Test.testStrings(Test.java:14)
>         at Test.main(Test.java:26)
> Caused by: java.lang.VerifyError: Bad type on operand stack
> Exception Details:
>   Location:
> javany/util/Comparator${0=Z}.lambda$thenComparing$10d9995b$1(Ljavany/util/Comparator;Ljava/lang/Object;Ljava/lang/Object;)I 
> @3: invokeinterface
>   Reason:
>     Type 'java/lang/Object' (current frame, stack[2]) is not 
> assignable to integer
>   Current Frame:
>     bci: @3
>     flags: { }
>     locals: { 'javany/util/Comparator${0=Z}', 
> 'javany/util/Comparator', 'java/lang/Object', 'java/lang/Object' }
>     stack: { 'javany/util/Comparator${0=Z}', 'java/lang/Object', 
> 'java/lang/Object' }
>   Bytecode:
>     0000000: 2a2c 2db9 007b 0300 3604 1504 9900 0815
>     0000010: 04a7 000b 2b2c 2db9 007d 0300 ac
>   Stackmap Table:
>     append_frame(@20,Integer)
>     same_locals_1_stack_item_frame(@28,Integer)
>
>         at sun.misc.Unsafe.defineAnonymousClass(Native Method)
>         at 
> java.lang.invoke.InnerClassLambdaMetafactory.spinInnerClass(InnerClassLambdaMetafactory.java:324)
>         at 
> java.lang.invoke.InnerClassLambdaMetafactory.buildCallSite(InnerClassLambdaMetafactory.java:194)
>         at 
> java.lang.invoke.LambdaMetafactory.metafactory(LambdaMetafactory.java:304)
>         at java.lang.invoke.CallSite.makeSite(CallSite.java:302)
>         ... 7 more
>
>
> Are lambdas/method references also not compatible with specialization 
> currently?
>
>
> Peter
>
>
> On 02/08/2015 11:43 AM, Peter Levart wrote:
>> I've got it.
>>
>> The problem was that Any.<int>toStringImpl() instance method was 
>> private. Since it is located in a specialized (Any${0=I}) class, it 
>> is not accessible from Any class (or specialized VM anonymous class 
>> generated for a specialized generic static method). The VerifyError: 
>> "Bad invokespecial instruction: current class isn't assignable to 
>> reference class." is not very informative though.
>>
>> By making the method(s) package-private, my test now runs correctly.
>>
>> Regards, Peter
>>
>> On 02/08/2015 11:27 AM, Peter Levart wrote:
>>> Hi Maurizio,
>>>
>>> This is great. I replaced all new-style for loops on <any T> T[] 
>>> arrays with classic on-index iteration and applied your patch 
>>> selectively without the redundant cast in javany/util/Arrays.java:823:
>>>
>>>     a = (E[])Objects.requireNonNull(array)
>>>
>>> ... as I think your last langtools patch (99ab651be669) fixes that, 
>>> right?
>>>
>>>
>>> Now with the following code:
>>>
>>> http://cr.openjdk.java.net/~plevart/misc/valhala-hacks/javany-src.02.jar 
>>>
>>>
>>> And this test:
>>>
>>> public class Test {
>>>
>>>     static void testStrings() {
>>>         List<String> strings = Arrays.<String>asList("a", "b", "c");
>>>         System.out.println(strings);
>>>     }
>>>
>>>     static void testInts() {
>>>         List<int> ints = Arrays.<int>asList(1, 2, 3);
>>>         System.out.println(ints);
>>>     }
>>>
>>>     public static void main(String[] args) {
>>>         testStrings();
>>>         testInts();
>>>     }
>>> }
>>>
>>>
>>> I get:
>>>
>>> [a, b, c]
>>> Specializing javany.util.List${0=I}; searching for 
>>> javany/util/List.class (not found)
>>> Specializing javany.util.List${0=I}; searching for 
>>> javany/util/List.class (found)
>>> Specializing javany.util.Collection${0=I}; searching for 
>>> javany/util/Collection.class (not found)
>>> Specializing javany.util.Collection${0=I}; searching for 
>>> javany/util/Collection.class (found)
>>> Specializing javany.lang.Iterable${0=I}; searching for 
>>> javany/lang/Iterable.class (not found)
>>> Specializing javany.lang.Iterable${0=I}; searching for 
>>> javany/lang/Iterable.class (found)
>>> Specializing method 
>>> javany/util/Arrays$asList${0=I}.asList([Ljava/lang/Object;)Ljavany/util/List; 
>>> with class=[] and method=[I]
>>> Specializing javany.util.Arrays$ArrayList${0=I}; searching for 
>>> javany/util/Arrays$ArrayList.class (not found)
>>> Specializing javany.util.Arrays$ArrayList${0=I}; searching for 
>>> javany/util/Arrays$ArrayList.class (found)
>>> Specializing javany.util.AbstractList${0=I}; searching for 
>>> javany/util/AbstractList.class (not found)
>>> Specializing javany.util.AbstractList${0=I}; searching for 
>>> javany/util/AbstractList.class (found)
>>> Specializing javany.util.AbstractCollection${0=I}; searching for 
>>> javany/util/AbstractCollection.class (not found)
>>> Specializing javany.util.AbstractCollection${0=I}; searching for 
>>> javany/util/AbstractCollection.class (found)
>>> Specializing javany.util.Iterator${0=I}; searching for 
>>> javany/util/Iterator.class (not found)
>>> Specializing javany.util.Iterator${0=I}; searching for 
>>> javany/util/Iterator.class (found)
>>> Specializing javany.util.ListIterator${0=I}; searching for 
>>> javany/util/ListIterator.class (not found)
>>> Specializing javany.util.ListIterator${0=I}; searching for 
>>> javany/util/ListIterator.class (found)
>>> Specializing javany.util.AbstractList$Itr${0=I}; searching for 
>>> javany/util/AbstractList$Itr.class (not found)
>>> Specializing javany.util.AbstractList$Itr${0=I}; searching for 
>>> javany/util/AbstractList$Itr.class (found)
>>> Specializing method 
>>> javany/util/Any$toString${0=I}.toString(Ljava/lang/Object;)Ljava/lang/String; 
>>> with class=[] and method=[I]
>>> Specializing javany.util.Any${0=I}; searching for 
>>> javany/util/Any.class (not found)
>>> Specializing javany.util.Any${0=I}; searching for 
>>> javany/util/Any.class (found)
>>> Exception in thread "main" java.lang.BootstrapMethodError: call site 
>>> initialization exception
>>>         at java.lang.invoke.CallSite.makeSite(CallSite.java:341)
>>>         at 
>>> java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:307)
>>>         at 
>>> java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:297)
>>>         at 
>>> javany.util.AbstractCollection${0=I}.toString(AbstractCollection.java:514)
>>>         at java.lang.String.valueOf(String.java:2987)
>>>         at java.io.PrintStream.println(PrintStream.java:821)
>>>         at Test.testInts(Test.java:17)
>>>         at Test.main(Test.java:22)
>>> Caused by: java.lang.VerifyError: Bad invokespecial instruction: 
>>> current class isn't assignable to reference class.
>>> Exception Details:
>>>   Location:
>>> javany/util/Any$toString${0=I}.toString(I)Ljava/lang/String; @8: 
>>> invokespecial
>>>   Reason:
>>>     Error exists in the bytecode
>>>   Bytecode:
>>>     0000000: bb00 0959 b700 0d1a b700 10b0
>>>
>>>         at sun.misc.Unsafe.defineAnonymousClass(Native Method)
>>>         at 
>>> java.lang.invoke.GenericMethodSpecializer.metafactory(GenericMethodSpecializer.java:98)
>>>         at java.lang.invoke.CallSite.makeSite(CallSite.java:302)
>>>         ... 7 more
>>>
>>>
>>> But looking at generated bytecode:
>>>
>>>
>>> Classfile 
>>> /home/peter/work/local/valhalla-test/dump/javany.util.Any$toString${0=I}_ERROR.class
>>>   Last modified Feb 8, 2015; size 321 bytes
>>>   MD5 checksum 7dac6f4f515394e0aa2ed1990a092697
>>>   Compiled from "Any.java"
>>> public class javany.util.Any$toString${0=I}
>>>   minor version: 0
>>>   major version: 52
>>>   flags: ACC_PUBLIC, ACC_SUPER
>>> Constant pool:
>>>    #1 = Utf8               javany/util/Any$toString${0=I}
>>>    #2 = Class              #1             // 
>>> "javany/util/Any$toString${0=I}"
>>>    #3 = Utf8               java/lang/Object
>>>    #4 = Class              #3             // java/lang/Object
>>>    #5 = Utf8               Any.java
>>>    #6 = Utf8               toString
>>>    #7 = Utf8               (I)Ljava/lang/String;
>>>    #8 = Utf8               javany/util/Any${0=I}
>>>    #9 = Class              #8             // "javany/util/Any${0=I}"
>>>   #10 = Utf8               <init>
>>>   #11 = Utf8               ()V
>>>   #12 = NameAndType        #10:#11        // "<init>":()V
>>>   #13 = Methodref          #9.#12         // 
>>> "javany/util/Any${0=I}"."<init>":()V
>>>   #14 = Utf8               toStringImpl
>>>   #15 = NameAndType        #14:#7         // 
>>> toStringImpl:(I)Ljava/lang/String;
>>>   #16 = Methodref          #9.#15         // 
>>> "javany/util/Any${0=I}".toStringImpl:(I)Ljava/lang/String;
>>>   #17 = Utf8               Code
>>>   #18 = Utf8               LineNumberTable
>>>   #19 = Utf8               Signature
>>>   #20 = Utf8               SourceFile
>>> {
>>>   public static java.lang.String toString(int);
>>>     descriptor: (I)Ljava/lang/String;
>>>     flags: ACC_PUBLIC, ACC_STATIC
>>>     Code:
>>>       stack=2, locals=1, args_size=1
>>>          0: new           #9                  // class 
>>> "javany/util/Any${0=I}"
>>>          3: dup
>>>          4: invokespecial #13                 // Method 
>>> "javany/util/Any${0=I}"."<init>":()V
>>>          7: iload_0
>>>          8: invokespecial #16                 // Method 
>>> "javany/util/Any${0=I}".toStringImpl:(I)Ljava/lang/String;
>>>         11: areturn
>>>       LineNumberTable:
>>>         line 31: 0
>>>     Signature: #7                           // (I)Ljava/lang/String;
>>> }
>>> SourceFile: "Any.java"
>>>
>>>
>>> Comparing it to original Any.class bytecode:
>>>
>>>
>>>   public static <T extends java.lang.Object> java.lang.String 
>>> toString(T);
>>>     descriptor: (Ljava/lang/Object;)Ljava/lang/String;
>>>     flags: ACC_PUBLIC, ACC_STATIC
>>>     Code:
>>>       stack=2, locals=1, args_size=1
>>>          0: new           #3                  // class javany/util/Any
>>>          3: dup
>>>          4: invokespecial #4                  // Method "<init>":()V
>>>          7: aload_0
>>>          8: invokespecial #8                  // Method 
>>> toStringImpl:(Ljava/lang/Object;)Ljava/lang/String;
>>>         11: areturn
>>>       LineNumberTable:
>>>         line 31: 0
>>> Error: unknown attribute
>>>       BytecodeMapping: length = 0x12
>>>        00 04 00 00 00 8C 00 04 00 8D 00 07 00 87 00 08
>>>        00 9F
>>>     Signature: #160                         // 
>>> <T:Ljava/lang/Object;>(TT;)Ljava/lang/String;
>>> Error: unknown attribute
>>>       TypeVariablesMap: length = 0x9
>>>        01 00 A1 01 01 00 92 00 86
>>>
>>>
>>> And also peeking at Any${0=I}.class:
>>>
>>>
>>>   private java.lang.String toStringImpl(int);
>>>     descriptor: (I)Ljava/lang/String;
>>>     flags: ACC_PRIVATE
>>>     Code:
>>>       stack=2, locals=2, args_size=2
>>>          0: aload_0
>>>          1: iload_1
>>>          2: invokespecial #45                 // Method 
>>> toStringNonNullImpl:(I)Ljava/lang/String;
>>>          5: areturn
>>>       LineNumberTable:
>>>         line 181: 0
>>>     Signature: #33                          // (I)Ljava/lang/String;
>>>
>>>
>>> ...I can't spot what's wrong. Can you?
>>>
>>>
>>> Regards, Peter
>>>
>>>
>>> On 02/06/2015 03:38 PM, Maurizio Cimadamore wrote:
>>>> Peter,
>>>> I think I fixed most of the issues in the way; I've been able to 
>>>> compile and run some tests successfully.
>>>> Attached is a patch of the changes that are required for the code 
>>>> to run correctly; I had to desugar some constructs (i.e. for-each, 
>>>> non-static inner class) as all desugared stuff currently has an 
>>>> issue in that javac won't emit the special sauce that is required 
>>>> by the specializer to correctly specialize a class. That's an issue 
>>>> with timing of the compilation pipeline - we are aware of it; but, 
>>>> as it's a no trivial fix, it means that, for now, it's better not 
>>>> to rely too much of compiler-desugared code.
>>>>
>>>> Maurizio
>>>>
>>>> On 04/02/15 12:42, Maurizio Cimadamore wrote:
>>>>> Compiler fixes have been pushed, I will now look into the runtime 
>>>>> issues you are getting...
>>>>>
>>>>> Maurizio
>>>>>
>>>>> On 04/02/15 10:18, Maurizio Cimadamore wrote:
>>>>>> Thanks for the additional feedback, I'll try to get at the bottom 
>>>>>> of those issues.
>>>>>>
>>>>>> Maurizio
>>>>>>
>>>>>> On 04/02/15 08:46, Peter Levart wrote:
>>>>>>> Hi Maurizio,
>>>>>>>
>>>>>>> I have now managed to successfully compile the code. Here's the 
>>>>>>> updated source:
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~plevart/misc/valhala-hacks/javany-src.jar 
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> But there's a StringIndexOutOfBoundsException thrown from 
>>>>>>> specializer when running the following Test:
>>>>>>>
>>>>>>> public class Test {
>>>>>>>     public static void main(String[] args) {
>>>>>>>         List<int> ints = Arrays.asList(new int[]{1, 2, 3, 4, 5, 
>>>>>>> 6, 7, 8});
>>>>>>>         Iterator<int> it = ints.iterator();
>>>>>>>         while (it.hasNext()) {
>>>>>>>             System.out.println(it.next());
>>>>>>>         }
>>>>>>>     }
>>>>>>> }
>>>>>>>
>>>>>>>
>>>>>>> Specializing javany.util.List${0=I}; searching for 
>>>>>>> javany/util/List.class (not found)
>>>>>>> Specializing javany.util.List${0=I}; searching for 
>>>>>>> javany/util/List.class (found)
>>>>>>> Specializing javany.util.Collection${0=I}; searching for 
>>>>>>> javany/util/Collection.class (not found)
>>>>>>> Specializing javany.util.Collection${0=I}; searching for 
>>>>>>> javany/util/Collection.class (found)
>>>>>>> Specializing javany.lang.Iterable${0=I}; searching for 
>>>>>>> javany/lang/Iterable.class (not found)
>>>>>>> Specializing javany.lang.Iterable${0=I}; searching for 
>>>>>>> javany/lang/Iterable.class (found)
>>>>>>> Specializing method 
>>>>>>> javany/util/Arrays$asList${0=I}.asList([Ljava/lang/Object;)Ljavany/util/List; 
>>>>>>> with class=[] and method=[I]
>>>>>>> Specializing javany.util.Arrays$ArrayList${0=I}; searching for 
>>>>>>> javany/util/Arrays$ArrayList.class (not found)
>>>>>>> Specializing javany.util.Arrays$ArrayList${0=I}; searching for 
>>>>>>> javany/util/Arrays$ArrayList.class (found)
>>>>>>> Exception in thread "main" 
>>>>>>> java.lang.StringIndexOutOfBoundsException: String index out of 
>>>>>>> range: 0
>>>>>>>         at java.lang.String.charAt(String.java:646)
>>>>>>>         at 
>>>>>>> jdk.internal.org.objectweb.asm.signature.SignatureReader.accept(SignatureReader.java:107)
>>>>>>>         at 
>>>>>>> valhalla.specializer.SignatureSpecializer.forType(SignatureSpecializer.java:72)
>>>>>>>         at 
>>>>>>> valhalla.specializer.Specializer$ManglingMethodVisitor.visitInvokeDynamicInsn(Specializer.java:679)
>>>>>>>         at 
>>>>>>> jdk.internal.org.objectweb.asm.ClassReader.readCode(ClassReader.java:1507)
>>>>>>>         at 
>>>>>>> jdk.internal.org.objectweb.asm.ClassReader.readMethod(ClassReader.java:1084)
>>>>>>>         at 
>>>>>>> jdk.internal.org.objectweb.asm.ClassReader.accept(ClassReader.java:729) 
>>>>>>>
>>>>>>>         at 
>>>>>>> valhalla.specializer.Specializer.specialize(Specializer.java:79)
>>>>>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:409)
>>>>>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:386)
>>>>>>>         at java.security.AccessController.doPrivileged(Native 
>>>>>>> Method)
>>>>>>>         at 
>>>>>>> java.net.URLClassLoader.findClass(URLClassLoader.java:385)
>>>>>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:426)
>>>>>>>         at 
>>>>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:317)
>>>>>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:359)
>>>>>>>         at 
>>>>>>> javany.util.Arrays$asList${0=I}/511754216.asList(Arrays.java:810)
>>>>>>>         at Test.main(Test.java:10)
>>>>>>>
>>>>>>>
>>>>>>> Appart from that, I learned that when the component type of 
>>>>>>> vararg array is an <any> type variable (for example: <any T> T[] 
>>>>>>> Arrays.asList(T ... a)), the invocation doesn't compile:
>>>>>>>
>>>>>>> src/Test.java:10: error: method invoked with incorrect number of 
>>>>>>> arguments; expected 3, found 1
>>>>>>>         List<int> ints = Arrays.<int>asList(1, 2, 3);
>>>>>>>                          ^
>>>>>>> 1 error
>>>>>>>
>>>>>>>
>>>>>>> Non-specialized code also has problems at runtime:
>>>>>>>
>>>>>>> public class Test {
>>>>>>>     public static void main(String[] args) {
>>>>>>>         List<String> strings = Arrays.<String>asList("a", "b", 
>>>>>>> "c");
>>>>>>>         Iterator<String> it = strings.iterator();
>>>>>>>         while (it.hasNext()) {
>>>>>>>             System.out.println(it.next());
>>>>>>>         }
>>>>>>>     }
>>>>>>> }
>>>>>>>
>>>>>>>
>>>>>>> Exception in thread "main" java.lang.ClassFormatError: Absent 
>>>>>>> Code attribute in method that is not native or abstract in class 
>>>>>>> file javany/util/AbstractList
>>>>>>>         at java.lang.ClassLoader.defineClass1(Native Method)
>>>>>>>         at java.lang.ClassLoader.defineClass(ClassLoader.java:762)
>>>>>>>         at 
>>>>>>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) 
>>>>>>>
>>>>>>>         at 
>>>>>>> java.net.URLClassLoader.defineClass(URLClassLoader.java:537)
>>>>>>>         at 
>>>>>>> java.net.URLClassLoader.access$300(URLClassLoader.java:78)
>>>>>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:438)
>>>>>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:386)
>>>>>>>         at java.security.AccessController.doPrivileged(Native 
>>>>>>> Method)
>>>>>>>         at 
>>>>>>> java.net.URLClassLoader.findClass(URLClassLoader.java:385)
>>>>>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:426)
>>>>>>>         at 
>>>>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:317)
>>>>>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:359)
>>>>>>>         at java.lang.ClassLoader.defineClass1(Native Method)
>>>>>>>         at java.lang.ClassLoader.defineClass(ClassLoader.java:762)
>>>>>>>         at 
>>>>>>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) 
>>>>>>>
>>>>>>>         at 
>>>>>>> java.net.URLClassLoader.defineClass(URLClassLoader.java:537)
>>>>>>>         at 
>>>>>>> java.net.URLClassLoader.access$300(URLClassLoader.java:78)
>>>>>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:438)
>>>>>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:386)
>>>>>>>         at java.security.AccessController.doPrivileged(Native 
>>>>>>> Method)
>>>>>>>         at 
>>>>>>> java.net.URLClassLoader.findClass(URLClassLoader.java:385)
>>>>>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:426)
>>>>>>>         at 
>>>>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:317)
>>>>>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:359)
>>>>>>>         at javany.util.Arrays.asList(Arrays.java:810)
>>>>>>>         at Test.main(Test.java:10)
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Regards, Peter
>>>>>>>
>>>>>>> On 02/03/2015 10:34 PM, Maurizio Cimadamore wrote:
>>>>>>>>
>>>>>>>> On 03/02/15 21:05, Peter Levart wrote:
>>>>>>>>> Hi Maurizio,
>>>>>>>>>
>>>>>>>>> I see. I thought this could be a nice idiom for boxing, since 
>>>>>>>>> the following:
>>>>>>>>>
>>>>>>>>> (Object) 42
>>>>>>>>>
>>>>>>>>> ...is legal and results in an Integer object at runtime.
>>>>>>>> I'm not saying this will never work - actually the compiler is 
>>>>>>>> currently accepting this kind of idioms, but the specializer 
>>>>>>>> does nothing with it, so you'll get runtime errors.
>>>>>>>>>
>>>>>>>>> But I don't know if a checkcast is actually inserted for 
>>>>>>>>> (Object). Could javac redundantly do it in case casting to 
>>>>>>>>> Object is from expression of <any> type and also equip 
>>>>>>>>> checkcast with BMA indicating the type of expression so that 
>>>>>>>>> specialization could replace it with boxing code?
>>>>>>>> That will be the way forward, yes
>>>>>>>>
>>>>>>>> Maurizio
>>>>>>>>>
>>>>>>>>> Regards, Peter
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 02/03/2015 08:09 PM, Maurizio Cimadamore wrote:
>>>>>>>>>>
>>>>>>>>>> On 03/02/15 18:46, Maurizio Cimadamore wrote:
>>>>>>>>>>> I will also investigate on the crash you are getting... 
>>>>>>>>>> Hi Peter,
>>>>>>>>>> the crash is coming from this code in AbstractCollection (see 
>>>>>>>>>> code in bold):
>>>>>>>>>>
>>>>>>>>>>     public boolean contains(Object o) {
>>>>>>>>>>         __WhereVal(E) {
>>>>>>>>>>             Iterator<E> it = iterator();
>>>>>>>>>>             if (o == null) {
>>>>>>>>>>                 return false;
>>>>>>>>>>             } else {
>>>>>>>>>>                 while (it.hasNext())
>>>>>>>>>> *if (o.equals((Object) it.next()))*
>>>>>>>>>>                         return true;
>>>>>>>>>>             }
>>>>>>>>>>             return false;
>>>>>>>>>>         }
>>>>>>>>>>         __WhereRef(E) {
>>>>>>>>>>             Iterator<E> it = iterator();
>>>>>>>>>>             if (o == null) {
>>>>>>>>>>                 while (it.hasNext())
>>>>>>>>>>                     if (it.next() == null)
>>>>>>>>>>                         return true;
>>>>>>>>>>             } else {
>>>>>>>>>>                 while (it.hasNext())
>>>>>>>>>>                     if (o.equals(it.next()))
>>>>>>>>>>                         return true;
>>>>>>>>>>             }
>>>>>>>>>>             return false;
>>>>>>>>>>         }
>>>>>>>>>>     }
>>>>>>>>>>
>>>>>>>>>> I believe that, apart from the obvious javac bug, the code 
>>>>>>>>>> has an issue, as it.next() is supposed to return a value 
>>>>>>>>>> there, but you are casting to Object?
>>>>>>>>>>
>>>>>>>>>> For the records - a simpler test case for the bug is this:
>>>>>>>>>>
>>>>>>>>>> class Foo<any E> {
>>>>>>>>>>     E e;
>>>>>>>>>>     E get() { return e; }
>>>>>>>>>>
>>>>>>>>>>     void test() {
>>>>>>>>>>         __WhereVal(E) {
>>>>>>>>>>             Object o = (Object)get();
>>>>>>>>>>         }
>>>>>>>>>>     }
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> Maurizio
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>


From maurizio.cimadamore at oracle.com  Sun Feb  8 22:10:09 2015
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Sun, 08 Feb 2015 22:10:09 +0000
Subject: experiences with prototype
In-Reply-To: <54D74D8B.7030200@gmail.com>
References: <54D113DA.9060509@gmail.com>	<54D11792.3090504@oracle.com>	<54D11CEC.6080009@oracle.com>	<54D1380A.5070503@gmail.com>	<54D13EDB.5050704@oracle.com>	<54D1DC5D.7060408@gmail.com>
	<54D1F1E3.10209@oracle.com> <54D213C2.3020603@oracle.com>
	<54D4D200.9090906@oracle.com> <54D73A1E.9020102@gmail.com>
	<54D73DC8.6000900@gmail.com> <54D74D8B.7030200@gmail.com>
Message-ID: <54D7DEC1.7060807@oracle.com>


On 08/02/15 11:50, Peter Levart wrote:
> Are lambdas/method references also not compatible with specialization 
> currently?
Hi Peter,
yes, lambdas are like inner classes - they are dealt with by javac after 
erasure takes place, meaning that most of the code generated by javac 
won't have any generic type info associated with it. It is of course 
annoying, but fixable (though not in the short term).

Thanks for your persistence with the prototype - I believe you have 
encountered all the major flaws related to javac's synthetic code not 
supported by specialization:

* non-static inner classes
* enhanced for loops
* private members accessors
* lambdas/method references

This info could also be useful for other people trying out the prototype.

Maurizio

From maurizio.cimadamore at oracle.com  Sun Feb  8 22:17:37 2015
From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore)
Date: Sun, 08 Feb 2015 22:17:37 +0000
Subject: experiences with prototype
In-Reply-To: <54D73A1E.9020102@gmail.com>
References: <54D113DA.9060509@gmail.com>	<54D11792.3090504@oracle.com>	<54D11CEC.6080009@oracle.com>	<54D1380A.5070503@gmail.com>	<54D13EDB.5050704@oracle.com>	<54D1DC5D.7060408@gmail.com>
	<54D1F1E3.10209@oracle.com> <54D213C2.3020603@oracle.com>
	<54D4D200.9090906@oracle.com> <54D73A1E.9020102@gmail.com>
Message-ID: <54D7E081.7020107@oracle.com>


On 08/02/15 10:27, Peter Levart wrote:
> This is great. I replaced all new-style for loops on <any T> T[] 
> arrays with classic on-index iteration and applied your patch 
> selectively without the redundant cast in javany/util/Arrays.java:823:
>
>     a = (E[])Objects.requireNonNull(array)
Yep - that cast is not needed as per very latest compiler fix.

Maurizio

From nipa at codefx.org  Sat Feb 14 09:37:21 2015
From: nipa at codefx.org (Nicolai Parlog)
Date: Sat, 14 Feb 2015 10:37:21 +0100
Subject: Value-Based Classes
Message-ID: <54DF1751.7030300@codefx.org>

 Hi!

I have two questions regarding value-based classes[1]. I considered
posting them to other lists (lambda-dev?) but thought that their
relation to value types makes this list a good place to ask them. If
not, I will happily post them elsewhere.


#1: Serialization

[N.B.: I'm a little fearful bringing up serialization, knowing how
much of a favorite feature it is for some of the people on this list.]

How come value-based classes are not allowed to be used for
serialization (being an "identity sensitive operation") but still many
such classes are serializable (e.g. [2])? I can understand both sides,
i.e. why the limitation is necessary and why value-based classes
should "of course" be serializable.

Is there any insight into that tension or into plans on how to break it?


#2: Implementation of equals, hashCode, toString

How am I to understand the following sentence?

"[Instances of a value-based class] have implementations of equals,
hashCode, and toString which are computed solely from the instance's
state and not from its identity or the state of any other object or
variable;" (from [1])

What exactly is "any other object"? Would, e.g., the value wrapped by
an Optional count as one? If so, isn't calling equals on that value a
computation based on its state?


Besides general curiosity I am interested in this because I will soon
be working on FindBugs rules which check these declaration site
limitations and warn the user if they overstep them in their own
value-based classes. (I will start after being done with enforcing the
use site limitations[3].)

 Greetings
 Nicolai Parlog


[1]
https://docs.oracle.com/javase/8/docs/api/java/lang/doc-files/ValueBased.html
[2] http://docs.oracle.com/javase/8/docs/api/java/time/LocalTime.html
[3] https://github.com/CodeFX-org/FindBugs-Fork/pull/1



-- 

PGP Key:
    http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509

Web:
    http://codefx.org
        a blog about software development
    http://do-foss.de
        Free and Open Source Software for the City of Dortmund

Twitter:
    https://twitter.com/nipafx


From brian.goetz at oracle.com  Sat Feb 14 16:42:24 2015
From: brian.goetz at oracle.com (Brian Goetz)
Date: Sat, 14 Feb 2015 11:42:24 -0500
Subject: Value-Based Classes
In-Reply-To: <54DF1751.7030300@codefx.org>
References: <54DF1751.7030300@codefx.org>
Message-ID: <54DF7AF0.2040507@oracle.com>

> How come value-based classes are not allowed to be used for
> serialization (being an "identity sensitive operation") but still many
> such classes are serializable (e.g. [2])? I can understand both sides,
> i.e. why the limitation is necessary and why value-based classes
> should "of course" be serializable.
>
> Is there any insight into that tension or into plans on how to break it?

The primary tension is that serialization depends on identity to ensure 
a topology-preserving reproduction of the original object graph.  For 
example, imagine Foo is a value-based class:

Foo[] arr = new Foo[2];
arr[0] = new Foo(0);
arr[1] = new Foo(0);

Serialization promises that on deserialization of arr, elements 0 and 1 
will not be aliased.  Similarly, in:

Foo[] arr = new Foo[2];
arr[0] = new Foo(0);
arr[1] = arr[0];

Serialization promises that on deserialization of arr, elements 0 and 1 
*will* be aliased.

True value types cannot -- and don't want to -- preserve these promises. 
  So serializability is one of those things that might inhibit migrating 
a value-based class to a true value type.  Hence the warning that this 
falls outside the notion of value-based.

The intent of the value-based description is to give people guidance of 
how to write classes that could be transformed into value types in the 
future.  (Still, there's one big hole that even this attempt can't plug 
-- nullity.  Value types are not going to be nullable.  The fact that we 
can't proscribe value-based classes from being nullable is going to 
create some really nasty migration compatibility constraints.)

> #2: Implementation of equals, hashCode, toString
>
> How am I to understand the following sentence?
>
> "[Instances of a value-based class] have implementations of equals,
> hashCode, and toString which are computed solely from the instance's
> state and not from its identity or the state of any other object or
> variable;" (from [1])
>
> What exactly is "any other object"? Would, e.g., the value wrapped by
> an Optional count as one? If so, isn't calling equals on that value a
> computation based on its state?

The phrasing is admittedly murky; in our defense, the problem is 
intrinsically murky.  Clearly you want a class like:

final class StringHolder {
     final String s;

     // obvious ctor, factory
     // equals/hashCode that delegate to equality/hashCode of s
}

to be value-based.  And yet equals/hashCode depends on the state of 
"another object" (a string), at least by some interpretations.  Am I 
right that your question is "so, what's up with that?"

Precisely describing value-ness in a mutable-by-default language is 
indeed pretty tricky; just figuring out what to do with String (which is 
clearly intended to be value-like, but doesn't even meet the criteria 
for value-based) is pretty messy.  The stab we took in [1] tried to 
stake out the spirit of the idea, but probably needs some refinement.


From nipa at codefx.org  Sat Feb 14 19:29:26 2015
From: nipa at codefx.org (Nicolai Parlog)
Date: Sat, 14 Feb 2015 20:29:26 +0100
Subject: Value-Based Classes - Serialization
In-Reply-To: <54DF7AF0.2040507@oracle.com>
References: <54DF1751.7030300@codefx.org> <54DF7AF0.2040507@oracle.com>
Message-ID: <54DFA216.7080308@codefx.org>

 Hi Brian,

thank you for taking the time to answer. For the second time, as I
must say, because in parts you already did so on StackOverflow some
time ago[1]. So I understand why serializing value-based instances is
troublesome.

OTOH: Isn't it "obvious" (I mean the "obvious but ..." you so often
refer to) that value-based classes should be serializable? Maybe minus
all identity promises. But I can see that this might not be the right
time to discuss this as the design might not be that far along.

So my question was asking why still some value-based classes *are*
serializable. That is rather confusing. Does the warning overrule the
interface implementation? I'm currently implementing FindBugs rules
for this. Should it create warnings for all serializations of
value-based instances?

> The intent of the value-based description is to give people
> guidance of how to write classes that could be transformed into
> value types in the future.
I call this part of the documentation the "declaration site
limitations". So developer provided value-based classes should not
implement Serializable, right?

> (Still, there's one big hole that even this attempt can't plug --
> nullity.  Value types are not going to be nullable. The fact that
> we can't proscribe value-based classes from being nullable is going
> to create some really nasty migration compatibility constraints.)
Damn. Please make it so that Optional becomes a value type (or box
thereof)! All else would suck. (I guess this is no news, though.)

Tools like FindBugs could help developers treat vbc as effectively
not-nullable. But I guess this can not be taken into account to loosen
those constraints.

 so long ... Nicolai


[1]
http://stackoverflow.com/questions/26451590/why-should-javas-value-based-classes-not-be-serialized



PS: I'll write a separate mail for the second part.



On 14.02.2015 17:42, Brian Goetz wrote:
>> How come value-based classes are not allowed to be used for 
>> serialization (being an "identity sensitive operation") but still
>> many such classes are serializable (e.g. [2])? I can understand
>> both sides, i.e. why the limitation is necessary and why
>> value-based classes should "of course" be serializable.
>> 
>> Is there any insight into that tension or into plans on how to
>> break it?
> 
> The primary tension is that serialization depends on identity to
> ensure a topology-preserving reproduction of the original object
> graph.  For example, imagine Foo is a value-based class:
> 
> Foo[] arr = new Foo[2]; arr[0] = new Foo(0); arr[1] = new Foo(0);
> 
> Serialization promises that on deserialization of arr, elements 0
> and 1 will not be aliased.  Similarly, in:
> 
> Foo[] arr = new Foo[2]; arr[0] = new Foo(0); arr[1] = arr[0];
> 
> Serialization promises that on deserialization of arr, elements 0
> and 1 *will* be aliased.
> 
> True value types cannot -- and don't want to -- preserve these
> promises. So serializability is one of those things that might
> inhibit migrating a value-based class to a true value type.  Hence
> the warning that this falls outside the notion of value-based.
> 
> The intent of the value-based description is to give people
> guidance of how to write classes that could be transformed into
> value types in the future.  (Still, there's one big hole that even
> this attempt can't plug -- nullity.  Value types are not going to
> be nullable.  The fact that we can't proscribe value-based classes
> from being nullable is going to create some really nasty migration
> compatibility constraints.)
> 
>> #2: Implementation of equals, hashCode, toString
>> 
>> How am I to understand the following sentence?
>> 
>> "[Instances of a value-based class] have implementations of
>> equals, hashCode, and toString which are computed solely from the
>> instance's state and not from its identity or the state of any
>> other object or variable;" (from [1])
>> 
>> What exactly is "any other object"? Would, e.g., the value
>> wrapped by an Optional count as one? If so, isn't calling equals
>> on that value a computation based on its state?
> 
> The phrasing is admittedly murky; in our defense, the problem is 
> intrinsically murky.  Clearly you want a class like:
> 
> final class StringHolder { final String s;
> 
> // obvious ctor, factory // equals/hashCode that delegate to
> equality/hashCode of s }
> 
> to be value-based.  And yet equals/hashCode depends on the state
> of "another object" (a string), at least by some interpretations.
> Am I right that your question is "so, what's up with that?"
> 
> Precisely describing value-ness in a mutable-by-default language
> is indeed pretty tricky; just figuring out what to do with String
> (which is clearly intended to be value-like, but doesn't even meet
> the criteria for value-based) is pretty messy.  The stab we took in
> [1] tried to stake out the spirit of the idea, but probably needs
> some refinement.
> 

-- 

PGP Key:
    http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509

Web:
    http://codefx.org
        a blog about software development
    http://do-foss.de
        Free and Open Source Software for the City of Dortmund

Twitter:
    https://twitter.com/nipafx


From nipa at codefx.org  Sat Feb 14 20:03:01 2015
From: nipa at codefx.org (Nicolai Parlog)
Date: Sat, 14 Feb 2015 21:03:01 +0100
Subject: Value-Based Classes - equals/hashCode
In-Reply-To: <54DF7AF0.2040507@oracle.com>
References: <54DF1751.7030300@codefx.org> <54DF7AF0.2040507@oracle.com>
Message-ID: <54DFA9F5.3080207@codefx.org>

 Hi again,

coming back to "implementations of equals, hashCode, and toString
which are [not] computed from the state of any other object"...

> And yet equals/hashCode depends on the state of "another object"
> (a string), at least by some interpretations.  Am I right that
> your question is "so, what's up with that?"
Currently, I'm still trying to understand the reason for this
limitation and I hoped to get there by understanding the precise
meaning. Maybe I should have asked the other way around...

But yes, also, what's up with that? One interpretation is surely that
the StringHolder's string (to stick with that example) can not be
accessed as it is "any other object". On the other hand the string is
surely part of the holder's state which is expressively allowed to be
used for that computation. So what is it?

> The phrasing is admittedly murky; in our defense, the problem is 
> intrinsically murky.
No need to defend anyone. I'm just trying to understand this so I know
what to do and what not to do. :)

 so long ... Nicolai



On 14.02.2015 17:42, Brian Goetz wrote:
>> How come value-based classes are not allowed to be used for 
>> serialization (being an "identity sensitive operation") but still
>> many such classes are serializable (e.g. [2])? I can understand
>> both sides, i.e. why the limitation is necessary and why
>> value-based classes should "of course" be serializable.
>> 
>> Is there any insight into that tension or into plans on how to
>> break it?
> 
> The primary tension is that serialization depends on identity to
> ensure a topology-preserving reproduction of the original object
> graph.  For example, imagine Foo is a value-based class:
> 
> Foo[] arr = new Foo[2]; arr[0] = new Foo(0); arr[1] = new Foo(0);
> 
> Serialization promises that on deserialization of arr, elements 0
> and 1 will not be aliased.  Similarly, in:
> 
> Foo[] arr = new Foo[2]; arr[0] = new Foo(0); arr[1] = arr[0];
> 
> Serialization promises that on deserialization of arr, elements 0
> and 1 *will* be aliased.
> 
> True value types cannot -- and don't want to -- preserve these
> promises. So serializability is one of those things that might
> inhibit migrating a value-based class to a true value type.  Hence
> the warning that this falls outside the notion of value-based.
> 
> The intent of the value-based description is to give people
> guidance of how to write classes that could be transformed into
> value types in the future.  (Still, there's one big hole that even
> this attempt can't plug -- nullity.  Value types are not going to
> be nullable.  The fact that we can't proscribe value-based classes
> from being nullable is going to create some really nasty migration
> compatibility constraints.)
> 
>> #2: Implementation of equals, hashCode, toString
>> 
>> How am I to understand the following sentence?
>> 
>> "[Instances of a value-based class] have implementations of
>> equals, hashCode, and toString which are computed solely from the
>> instance's state and not from its identity or the state of any
>> other object or variable;" (from [1])
>> 
>> What exactly is "any other object"? Would, e.g., the value
>> wrapped by an Optional count as one? If so, isn't calling equals
>> on that value a computation based on its state?
> 
> The phrasing is admittedly murky; in our defense, the problem is 
> intrinsically murky.  Clearly you want a class like:
> 
> final class StringHolder { final String s;
> 
> // obvious ctor, factory // equals/hashCode that delegate to
> equality/hashCode of s }
> 
> to be value-based.  And yet equals/hashCode depends on the state
> of "another object" (a string), at least by some interpretations.
> Am I right that your question is "so, what's up with that?"
> 
> Precisely describing value-ness in a mutable-by-default language
> is indeed pretty tricky; just figuring out what to do with String
> (which is clearly intended to be value-like, but doesn't even meet
> the criteria for value-based) is pretty messy.  The stab we took in
> [1] tried to stake out the spirit of the idea, but probably needs
> some refinement.
> 

-- 

PGP Key:
    http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509

Web:
    http://codefx.org
        a blog about software development
    http://do-foss.de
        Free and Open Source Software for the City of Dortmund

Twitter:
    https://twitter.com/nipafx


From brian.goetz at oracle.com  Sat Feb 14 20:38:00 2015
From: brian.goetz at oracle.com (Brian Goetz)
Date: Sat, 14 Feb 2015 15:38:00 -0500
Subject: Value-Based Classes - Serialization
In-Reply-To: <54DFA216.7080308@codefx.org>
References: <54DF1751.7030300@codefx.org> <54DF7AF0.2040507@oracle.com>
	<54DFA216.7080308@codefx.org>
Message-ID: <54DFB228.7070403@oracle.com>

> So my question was asking why still some value-based classes *are*
> serializable.

I think you have to ask the authors of those classes.  None of the ones 
delivered by JSR-335, to my knowledge, are.  Arguably, the stricture 
against serializability in the value-based definition is too strict; 
there is a reasonable semantics for serialization of *true* value types. 
  But, because serialization, as currently defined, makes promises about 
object identity, and value-based classes do have an identity (albeit one 
that should be ignored), the definition erred on the side of caution.

> That is rather confusing. Does the warning overrule the
> interface implementation? I'm currently implementing FindBugs rules
> for this. Should it create warnings for all serializations of
> value-based instances?

Probably not; I think the more significant error is using == to compare 
value-based instances (except perhaps to null.)

>> (Still, there's one big hole that even this attempt can't plug --
>> nullity.  Value types are not going to be nullable. The fact that
>> we can't proscribe value-based classes from being nullable is going
>> to create some really nasty migration compatibility constraints.)
> Damn. Please make it so that Optional becomes a value type (or box
> thereof)! All else would suck. (I guess this is no news, though.)

Yeah, this is pretty bad.  On the one hand, Optional is the canonical 
example of a value type.  And, assigning null to an Optional is a pretty 
questionable thing, given that the whole point of Optional is to make 
the "present/absent" distinction more explicit.  But switching Optional 
to a value type would definitely be incompatible.  There's an argument 
we could make that this sort of incompatibility is OK, but it's going to 
be pretty hard to make that same argument for value-based classes like 
LocalDateTime.

I haven't given up on this, but the ideas we've come up with so far in 
this area aren't super-attractive.


From brian.goetz at oracle.com  Sat Feb 14 21:03:41 2015
From: brian.goetz at oracle.com (Brian Goetz)
Date: Sat, 14 Feb 2015 16:03:41 -0500
Subject: Value-Based Classes - equals/hashCode
In-Reply-To: <54DFA9F5.3080207@codefx.org>
References: <54DF1751.7030300@codefx.org> <54DF7AF0.2040507@oracle.com>
	<54DFA9F5.3080207@codefx.org>
Message-ID: <54DFB82D.3020800@oracle.com>

> But yes, also, what's up with that? One interpretation is surely that
> the StringHolder's string (to stick with that example) can not be
> accessed as it is "any other object". On the other hand the string is
> surely part of the holder's state which is expressively allowed to be
> used for that computation. So what is it?

Which brings us back to the fundamentally murky part -- what comprises 
the state of an object?

In languages like Lime (a Java-like language designed for GPU 
computation), a value type must be "values all the way down".  (Their 
primitive data type is "bit"; everything else is defined as composites 
of bit.)  This is a beautiful and clean way to define things, but not 
applicable to the practical problem of adding true values to a 
mutation-happy language like Java; even simple types like String require 
some mental gymnastics to see them as value-like (String stores 
characters in an array, which is mutable, but just not mutated after 
initialization, and even has a lazily-initialized, mutable field for the 
hash code (if that's not bad enough, this field is updated using data 
races!))  But not being able to have String as a component of a value 
would be silly, and no one disputes that strings are "value-ish".  But 
the clean definitions don't apply.

We found ourselves with a related problem when trying to define the 
constraints on behavioral parameters passed to stream methods (e.g., the 
Predicate passed to Stream.filter).  The "obvious" restrictions were 
"side-effect free" or "pure function".  But neither of these is the 
right answer; both are unnecessarily restrictive.  (Printing a debug 
message or updating a history buffer are acceptable side effects; 
looking up data in a mutable but not-mutated-during-the-query HashMap is 
an acceptable compromise of purity.)  We settled on a murky definition 
of "non-interfering" and "stateless", knowing full well that our 
definitions are built on sand, but tried to capture the true spirit of 
the restriction, which is "don't depend on stuff that might change 
during the calculation."

Said sand includes notions like "what is the state of an object".

Here's a stab at thinking about this, which I'm sure is wrong, but might 
be a helpful start.  Say a value-ish class C has fields c_1..c_n.  Now 
let's try and define S(C), the variables that comprise the "state" of C. 
  S(C) includes:
  - \forall_i c_i, if c_i is a primitive
  - \forall_i S(c_i), if c_i is a reference to a value-ish class
  - \forall_i c_i, if c_i is a reference to a non-value-ish class

In other words, if a value contains other values, we recursively include 
the dependent value in the state; if a value contains references to 
non-values, we can only include the that reference (references are 
values!), but not any state reachable through that reference.  (We wave 
our hands and magically define certain otherwise-problematic classes 
(like String) to be value-ish.)

So, in the following class:

final class Urk {
     private final MutableStuff mc;

     private Urk(MutableStuff mc) { this.mc = mc; }
     public Urk make(MutableStuff mc) { return new Urk(mc); }

     public boolean equals(Object other) {
         return (other instanceof Urk)
             && ((Urk) other).mc == mc;
     }

     // similar for hashCode
}

I would argue this is following the rules; Urk's state includes the 
*reference to* mc, but not the *state of* mc.  If Urk were to dive into 
the state of mc, then it would have crossed the line.

Hope this helps.


From peter.levart at gmail.com  Sat Feb 14 23:46:33 2015
From: peter.levart at gmail.com (Peter Levart)
Date: Sun, 15 Feb 2015 00:46:33 +0100
Subject: Value-Based Classes - equals/hashCode
In-Reply-To: <54DFB82D.3020800@oracle.com>
References: <54DF1751.7030300@codefx.org>
	<54DF7AF0.2040507@oracle.com>	<54DFA9F5.3080207@codefx.org>
	<54DFB82D.3020800@oracle.com>
Message-ID: <54DFDE59.8050505@gmail.com>


On 02/14/2015 10:03 PM, Brian Goetz wrote:
> Here's a stab at thinking about this, which I'm sure is wrong, but 
> might be a helpful start.  Say a value-ish class C has fields 
> c_1..c_n. Now let's try and define S(C), the variables that comprise 
> the "state" of C.  S(C) includes:
>  - \forall_i c_i, if c_i is a primitive
>  - \forall_i S(c_i), if c_i is a reference to a value-ish class
>  - \forall_i c_i, if c_i is a reference to a non-value-ish class
>
> In other words, if a value contains other values, we recursively 
> include the dependent value in the state; if a value contains 
> references to non-values, we can only include the that reference 
> (references are values!), but not any state reachable through that 
> reference.  (We wave our hands and magically define certain 
> otherwise-problematic classes (like String) to be value-ish.) 

Here's another constraint for value-ish classes: they must not form 
circles composed from only value-ish objects in object graphs.

Peter


From nipa at codefx.org  Sun Feb 15 16:52:33 2015
From: nipa at codefx.org (Nicolai Parlog)
Date: Sun, 15 Feb 2015 17:52:33 +0100
Subject: Value-Based Classes vs Value Types
Message-ID: <54E0CED1.3060304@codefx.org>

 Hi!

I'm currently writing a post about value-based classes warning people
about what not do with them and why those limitations exist. Now that
the article is more or less done, I realize that it heavily relies on
a concrete idea of how the relation of value-based classes and value
types looks like.

The State of the Values from 04/2014 [1] says "In fact, it seems
likely that the boxed form of every value type will be a value-based
class." If I understand this correctly, it's like we're implementing
'Integer' now, waiting for 'int' to be implementable in a couple of
years.

Or is it more like we are already writing the value types, waiting for
the language to catch up so we can then add the right qualifiers and
effectively transform value-based classes into value types? To
continue the example from above, do we currently write Integer but by
replacing "public class Integer" with "public value integer" (syntax
is irrelevant) we effectively create 'int', leaving all other code
untouched (because we followed all the rules)?

Do even more options exist? Btw, I'm not asking this to get a detailed
picture of how it's gonna be. I'm rather looking for an explanation
which I can repeat (and is not already known to be wrong) for how
value-based classes relate to value types and why this creates the
documented limitations.

 so long ... Nicolai


[1] http://cr.openjdk.java.net/~jrose/values/values.html



-- 

PGP Key:
    http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509

Web:
    http://codefx.org
        a blog about software development
    http://do-foss.de
        Free and Open Source Software for the City of Dortmund

Twitter:
    https://twitter.com/nipafx


From brian.goetz at oracle.com  Sun Feb 15 18:16:55 2015
From: brian.goetz at oracle.com (Brian Goetz)
Date: Sun, 15 Feb 2015 13:16:55 -0500
Subject: Value-Based Classes vs Value Types
In-Reply-To: <54E0CED1.3060304@codefx.org>
References: <54E0CED1.3060304@codefx.org>
Message-ID: <54E0E297.8070703@oracle.com>

> The State of the Values from 04/2014 [1] says "In fact, it seems
> likely that the boxed form of every value type will be a value-based
> class." If I understand this correctly, it's like we're implementing
> 'Integer' now, waiting for 'int' to be implementable in a couple of
> years.

That's a good mental model.  We can't write 'int' yet, but we can design 
its API, and if we are careful, we may be rewarded with better 
performance in the future.

> Or is it more like we are already writing the value types, waiting for
> the language to catch up so we can then add the right qualifiers and
> effectively transform value-based classes into value types?

I don't think this is as good a model.  Right now, one has to be quite 
careful to not make identity-centric assumptions.  Value-based classes 
have other advantages besides their possible migration to values (e.g., 
the many benefits of immutability) but value types is definitely "give 
up something to get something", where that something is our reliance on 
identity and its consequences.


From nipa at codefx.org  Sun Feb 15 19:53:35 2015
From: nipa at codefx.org (Nicolai Parlog)
Date: Sun, 15 Feb 2015 20:53:35 +0100
Subject: Value-Based Classes - equals/hashCode
In-Reply-To: <54DFB82D.3020800@oracle.com>
References: <54DF1751.7030300@codefx.org> <54DF7AF0.2040507@oracle.com>
	<54DFA9F5.3080207@codefx.org> <54DFB82D.3020800@oracle.com>
Message-ID: <54E0F93F.4080703@codefx.org>

> Hope this helps.
Yes it does. Ignoring the sand, I'd say I got it.

And I assume this is important because extending equality to the state
of (instead of the reference to) non-value-ish instances threatens the
interchangeability* of values?

* "Instances of a value-based class are freely substitutable when
equal, meaning that interchanging any two instances x and y that are
equal according to equals() in any computation or method invocation
should produce no visible change in behavior."



On 14.02.2015 22:03, Brian Goetz wrote:
>> But yes, also, what's up with that? One interpretation is surely
>> that the StringHolder's string (to stick with that example) can
>> not be accessed as it is "any other object". On the other hand
>> the string is surely part of the holder's state which is
>> expressively allowed to be used for that computation. So what is
>> it?
> 
> Which brings us back to the fundamentally murky part -- what
> comprises the state of an object?
> 
> In languages like Lime (a Java-like language designed for GPU 
> computation), a value type must be "values all the way down".
> (Their primitive data type is "bit"; everything else is defined as
> composites of bit.)  This is a beautiful and clean way to define
> things, but not applicable to the practical problem of adding true
> values to a mutation-happy language like Java; even simple types
> like String require some mental gymnastics to see them as
> value-like (String stores characters in an array, which is mutable,
> but just not mutated after initialization, and even has a
> lazily-initialized, mutable field for the hash code (if that's not
> bad enough, this field is updated using data races!))  But not
> being able to have String as a component of a value would be silly,
> and no one disputes that strings are "value-ish".  But the clean
> definitions don't apply.
> 
> We found ourselves with a related problem when trying to define
> the constraints on behavioral parameters passed to stream methods
> (e.g., the Predicate passed to Stream.filter).  The "obvious"
> restrictions were "side-effect free" or "pure function".  But
> neither of these is the right answer; both are unnecessarily
> restrictive.  (Printing a debug message or updating a history
> buffer are acceptable side effects; looking up data in a mutable
> but not-mutated-during-the-query HashMap is an acceptable
> compromise of purity.)  We settled on a murky definition of
> "non-interfering" and "stateless", knowing full well that our 
> definitions are built on sand, but tried to capture the true spirit
> of the restriction, which is "don't depend on stuff that might
> change during the calculation."
> 
> Said sand includes notions like "what is the state of an object".
> 
> Here's a stab at thinking about this, which I'm sure is wrong, but
> might be a helpful start.  Say a value-ish class C has fields
> c_1..c_n.  Now let's try and define S(C), the variables that
> comprise the "state" of C. S(C) includes: - \forall_i c_i, if c_i
> is a primitive - \forall_i S(c_i), if c_i is a reference to a
> value-ish class - \forall_i c_i, if c_i is a reference to a
> non-value-ish class
> 
> In other words, if a value contains other values, we recursively
> include the dependent value in the state; if a value contains
> references to non-values, we can only include the that reference
> (references are values!), but not any state reachable through that
> reference.  (We wave our hands and magically define certain
> otherwise-problematic classes (like String) to be value-ish.)
> 
> So, in the following class:
> 
> final class Urk { private final MutableStuff mc;
> 
> private Urk(MutableStuff mc) { this.mc = mc; } public Urk
> make(MutableStuff mc) { return new Urk(mc); }
> 
> public boolean equals(Object other) { return (other instanceof
> Urk) && ((Urk) other).mc == mc; }
> 
> // similar for hashCode }
> 
> I would argue this is following the rules; Urk's state includes
> the *reference to* mc, but not the *state of* mc.  If Urk were to
> dive into the state of mc, then it would have crossed the line.
> 
> Hope this helps.
> 

-- 

PGP Key:
    http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509

Web:
    http://codefx.org
        a blog about software development
    http://do-foss.de
        Free and Open Source Software for the City of Dortmund

Twitter:
    https://twitter.com/nipafx


From brian.goetz at oracle.com  Sun Feb 15 20:14:09 2015
From: brian.goetz at oracle.com (Brian Goetz)
Date: Sun, 15 Feb 2015 15:14:09 -0500
Subject: Value-Based Classes - equals/hashCode
In-Reply-To: <54E0F93F.4080703@codefx.org>
References: <54DF1751.7030300@codefx.org> <54DF7AF0.2040507@oracle.com>
	<54DFA9F5.3080207@codefx.org> <54DFB82D.3020800@oracle.com>
	<54E0F93F.4080703@codefx.org>
Message-ID: <54E0FE11.4070402@oracle.com>

> And I assume this is important because extending equality to the state
> of (instead of the reference to) non-value-ish instances threatens the
> interchangeability* of values?

Right.  Without identity, the only way to compare two instances is via 
state.  All instances of the number "3" are the same value...

From twhitmore.nz at gmail.com  Mon Feb 16 03:57:58 2015
From: twhitmore.nz at gmail.com (Thomas W)
Date: Mon, 16 Feb 2015 16:57:58 +1300
Subject: valhalla-dev Digest, Vol 8, Issue 14
In-Reply-To: <mailman.44518.1423946292.10816.valhalla-dev@openjdk.java.net>
References: <mailman.44518.1423946292.10816.valhalla-dev@openjdk.java.net>
Message-ID: <CAE6vqCm1RwAvhtEg8Nirp1t7-z-OHV41YB48yLz7SroSJsRA7A@mail.gmail.com>

I don't really see such difficulty?

1) Serialization preserving identity aliases

Since values do not have a significant identity, "preserving same identity"
must be meaningless for de/serialization. It's not required, it can't be
achieved, and it doesn't mean anything for value types! I would therefore
say that guarantees as to deserialized identity for Value types just
becomes dropped/ meaningless, rather than anything that would break/
prevent serialization.

Considering that, I don't see the reason for value & value-based types to
not be serializable/ or for there to be difficulties in this area.
Value-like types should indeed logically be the types most easiest & most
able to be serialized.

2) Equals/ Hash on composites

Theoretically, a Value-like type should be fully able to compose immutable
values; StringHolder{String s} is obviously fine, once we can recognize
String as immutable. And these "child" value fields should logically be
fully usable in equals()/ hashCode(). So that part of the specification
seems slightly amiss -- are we are lacking an annotation to mark classes as
immutable/ value-like?

We could also potentially consider allowing Value-like types to compose
mutable objects, if they could be proved or designated to be all of the
following:   A) exclusively owned by the holder,  B) be encapsulated by the
holder (no exposed reference), and C) have no mutators exposed via the
holder.  Meeting all of these conditions yields -> D) an object whose value
is held, but cannot be mutated.


Regards
Thomas Whitmore



<brian.goetz at oracle.com <valhalla-dev-request at openjdk.java.net>> wrote:

> The primary tension is that serialization depends on identity to ensure
> a topology-preserving reproduction of the original object graph.  For
> example, imagine Foo is a value-based class:
>
> Foo[] arr = new Foo[2];
> arr[0] = new Foo(0);
> arr[1] = new Foo(0);
>
> Serialization promises that on deserialization of arr, elements 0 and 1
> will not be aliased.  Similarly, in:
>
> Foo[] arr = new Foo[2];
> arr[0] = new Foo(0);
> arr[1] = arr[0];
>
> Serialization promises that on deserialization of arr, elements 0 and 1
> *will* be aliased.
>
> True value types cannot -- and don't want to -- preserve these promises.
>   So serializability is one of those things that might inhibit migrating
> a value-based class to a true value type.  Hence the warning that this
> falls outside the notion of value-based.
>

From nipa at codefx.org  Mon Feb 16 21:09:47 2015
From: nipa at codefx.org (Nicolai Parlog)
Date: Mon, 16 Feb 2015 22:09:47 +0100
Subject: valhalla-dev Digest, Vol 8, Issue 14
In-Reply-To: <CAE6vqCm1RwAvhtEg8Nirp1t7-z-OHV41YB48yLz7SroSJsRA7A@mail.gmail.com>
References: <mailman.44518.1423946292.10816.valhalla-dev@openjdk.java.net>
	<CAE6vqCm1RwAvhtEg8Nirp1t7-z-OHV41YB48yLz7SroSJsRA7A@mail.gmail.com>
Message-ID: <54E25C9B.60801@codefx.org>

 Hi Thomas.

> 1) Serialization preserving identity aliases
I totally agree with everything you write. But we were not discussing
serialization of value types in Java X but of value-based classes in
Java 8. And here the fact of the matter is that serialization makes
guarantees which might not be upheld in the future, which is why such
uses must be prevented.

> 2) Equals/ Hash on composites
I like your rules A, B, C -> D a lot. I actually came up with pretty
much the same set (sure I can say that now) when I created examples
for this whole state thing.

Now I wonder whether this would be a generally advisable way to deal
with this scenario. As far as I see it, (A, B, C) could be considered
the gate into the value's state. Everything that passes is game for
equals/hashCode.

 so long ... Nicolai



On 16.02.2015 04:57, Thomas W wrote:
> I don't really see such difficulty?
> 
> 1) Serialization preserving identity aliases
> 
> Since values do not have a significant identity, "preserving same
> identity" must be meaningless for de/serialization. It's not
> required, it can't be achieved, and it doesn't mean anything for
> value types! I would therefore say that guarantees as to
> deserialized identity for Value types just becomes dropped/
> meaningless, rather than anything that would break/ prevent
> serialization.
> 
> Considering that, I don't see the reason for value & value-based
> types to not be serializable/ or for there to be difficulties in
> this area. Value-like types should indeed logically be the types
> most easiest & most able to be serialized.
> 
> 2) Equals/ Hash on composites
> 
> Theoretically, a Value-like type should be fully able to compose
> immutable values; StringHolder{String s} is obviously fine, once we
> can recognize String as immutable. And these "child" value fields
> should logically be fully usable in equals()/ hashCode(). So that
> part of the specification seems slightly amiss -- are we are
> lacking an annotation to mark classes as immutable/ value-like?
> 
> We could also potentially consider allowing Value-like types to
> compose mutable objects, if they could be proved or designated to
> be all of the following:   A) exclusively owned by the holder,  B)
> be encapsulated by the holder (no exposed reference), and C) have
> no mutators exposed via the holder.  Meeting all of these
> conditions yields -> D) an object whose value is held, but cannot
> be mutated.
> 
> 
> Regards Thomas Whitmore
> 
> 
> 
> <brian.goetz at oracle.com <valhalla-dev-request at openjdk.java.net>>
> wrote:
> 
>> The primary tension is that serialization depends on identity to
>> ensure a topology-preserving reproduction of the original object
>> graph.  For example, imagine Foo is a value-based class:
>> 
>> Foo[] arr = new Foo[2]; arr[0] = new Foo(0); arr[1] = new
>> Foo(0);
>> 
>> Serialization promises that on deserialization of arr, elements 0
>> and 1 will not be aliased.  Similarly, in:
>> 
>> Foo[] arr = new Foo[2]; arr[0] = new Foo(0); arr[1] = arr[0];
>> 
>> Serialization promises that on deserialization of arr, elements 0
>> and 1 *will* be aliased.
>> 
>> True value types cannot -- and don't want to -- preserve these
>> promises. So serializability is one of those things that might
>> inhibit migrating a value-based class to a true value type.
>> Hence the warning that this falls outside the notion of
>> value-based.
>> 
> 

-- 

PGP Key:
    http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509

Web:
    http://codefx.org
        a blog about software development
    http://do-foss.de
        Free and Open Source Software for the City of Dortmund

Twitter:
    https://twitter.com/nipafx


From twhitmore.nz at gmail.com  Tue Feb 17 03:46:17 2015
From: twhitmore.nz at gmail.com (Thomas W)
Date: Tue, 17 Feb 2015 16:46:17 +1300
Subject: valhalla-dev Digest, Vol 8, Issue 14
In-Reply-To: <54E25C9B.60801@codefx.org>
References: <mailman.44518.1423946292.10816.valhalla-dev@openjdk.java.net>
	<CAE6vqCm1RwAvhtEg8Nirp1t7-z-OHV41YB48yLz7SroSJsRA7A@mail.gmail.com>
	<54E25C9B.60801@codefx.org>
Message-ID: <CAE6vqCkyqeKEd2fj0LQ8ovt=sOidLNaFwT4uuKWzSEyH0NDBjA@mail.gmail.com>

Hi Nicolai.

[1) Serialization preserving identity aliases]
> We were not discussing serialization of value types in Java X but of
> value-based classes in Java 8. And here the fact of the matter is that
> serialization makes guarantees which might not be upheld in the future,
> which is why such uses must be prevented.

Thanks for your comments!  What I was trying to capture was the "logical
insight" into what a Value is, whether that value is implemented as a value
type (Java X) or a "value-like", "value-based" etc class sooner.

The "guarantees" about serialization & aliasing, may effectively be more
about being able to handle (serialize & deserialize) _reference cycles_
than about preserving aliases. Cycles crop up early on in rich data
structures, and need to be handled -- this would probably be the first
driver of the requirement for serialization to handle identity. Spec about
"aliasing" may just be a poor way of wording this.

This would lead to the suggestion that (hopefully) cycles aren't going to
be relevant in Value-based or Value-like classes, either. If we could
decide that the spec is more about handling cycles than about preserving
aliases, this might clear the path forward to some degree.

Regardless, Java should offer a clear and planned path to remove such
guarantees for Value-based types.

Value-like types (String, Integer etc) are a more difficult kettle of fish;
 we should annotate these, but there are probably limitations:
- we cannot in the case of String, get rid of local identity;   since
intern() explicitly makes string identity an optimization & an issue.
- I've seen code using String as a lock.
and OTOH:
- I've never seen anyone care about the identity of Integer.
- nor have I ever seen (nor do I want to) anyone lock on Integer.


[2) Equals/ Hash on composites]
> I like your rules A, B, C -> D a lot. I actually came up with pretty
> much the same set (sure I can say that now) when I created examples
> for this whole state thing.

These extra rules for "mutable objects being effectively encapsulated &
immutable" are quite complex;  almost certainly too complex for a compiler
or language spec, in determining fields to be used/ or which may be used
for equals()/ hashCode().

However, they might be quite good for an analyzer :)


Regards,
Thomas