From maurizio.cimadamore at oracle.com Tue Jan 5 12:56:43 2016 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Tue, 05 Jan 2016 12:56:43 +0000 Subject: hg: valhalla/valhalla/langtools: Fix: add hash to desugared generic method inner classes Message-ID: <201601051256.u05CuiOP010118@aojmv0008.oracle.com> Changeset: 3ae1321e6516 Author: mcimadamore Date: 2016-01-05 12:56 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/3ae1321e6516 Fix: add hash to desugared generic method inner classes ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/SpecializeTypes.java ! test/tools/javac/valhalla/typespec/GenericMethod01.java + test/tools/javac/valhalla/typespec/GenericMethod02.java From maurizio.cimadamore at oracle.com Tue Jan 5 12:57:47 2016 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Tue, 05 Jan 2016 12:57:47 +0000 Subject: hg: valhalla/valhalla/jdk: Fix: add hash to desugared generic method inner classes Message-ID: <201601051257.u05Cvl1L010397@aojmv0008.oracle.com> Changeset: d93cdb9135bd Author: mcimadamore Date: 2016-01-05 12:57 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/d93cdb9135bd Fix: add hash to desugared generic method inner classes ! src/java.base/share/classes/java/lang/invoke/DispatchContext.java From maurizio.cimadamore at oracle.com Mon Jan 11 14:14:59 2016 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Mon, 11 Jan 2016 14:14:59 +0000 Subject: hg: valhalla/valhalla/langtools: Alt generic method translation fixes: Message-ID: <201601111414.u0BEExo2002350@aojmv0008.oracle.com> Changeset: 7af97c773606 Author: mcimadamore Date: 2016-01-11 14:14 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/7af97c773606 Alt generic method translation fixes: * inferred calls are not patched correctly * local variable declaration in specializable calls are not patched correctly * method calls/var access using simple names are not patched correctly * disable lambda alt metafactory static arg tunnelling for alt generic method translation ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/SpecializeTypes.java From maurizio.cimadamore at oracle.com Mon Jan 11 14:17:55 2016 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Mon, 11 Jan 2016 14:17:55 +0000 Subject: hg: valhalla/valhalla/jdk: Alt generic method translation fixes: Message-ID: <201601111417.u0BEHuuw003761@aojmv0008.oracle.com> Changeset: 59dbcafb2cfa Author: mcimadamore Date: 2016-01-11 14:17 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/59dbcafb2cfa Alt generic method translation fixes: * use IMPL_LOOKUP to perform instance method lookup w/o incurring into access woes * reflectively select target generic specializable method to avoid issues with bridging * add visitor to retrieve correct any tvar arity from specialized receiver in case of partial specialization ! src/java.base/share/classes/java/lang/invoke/DispatchContext.java ! src/java.base/share/classes/java/lang/invoke/GenericInstanceDispatch.java ! src/java.base/share/classes/java/lang/invoke/GenericStaticDispatch.java ! src/java.base/share/classes/java/lang/invoke/LambdaMetafactory.java ! src/java.base/share/classes/valhalla/specializer/Specializer.java From brian.goetz at oracle.com Mon Jan 11 21:08:35 2016 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 11 Jan 2016 16:08:35 -0500 Subject: Announcing: valhalla-spec-{observers,comments} In-Reply-To: <566F2195.2030100@oracle.com> References: <566F2195.2030100@oracle.com> Message-ID: <569419D3.80706@oracle.com> As promised, we've now set up the additional mailing lists, valhalla-spec-observers and valhalla-spec-comments. Here are the details. valhalla-spec-observers at openjdk.java.net is open to anyone to subscribe, and subscribers may post to it. Traffic from valhalla-spec-experts is automatically forwarded to valhalla-spec-observers. This list is primarily intended so that people can follow the EG list traffic easily, and also for people to have discussions within the -observers community. The primary means for the community to provide feedback into the design and specification discussion is valhalla-spec-comments. Periodically, comments from the -comments list will be "read into the record" on the -experts list, where the EG can discuss the issue raised if it wants. But with this access comes responsibility! Specifically, a post to -comments should be well-reasoned, self-contained, and focused on a single issue (as well as clear and polite, of course). Think of it as a suggestion box; we guarantee that it will be seen by the EG, but if you want it to be taken seriously by the EG, put some time and effort into making your point clear argument. Note that -comments *cannot* be subscribed to; it can only be posted to. (The reason for this is simple; having discussions on this list, which is inevitable were it subscribable, would defeat the purpose.) The postings are archived, and their content (if appropriate) will eventually show up on the EG list. Postings are archived, and moderated to manage spam and otherwise inappropriate content. On 12/14/2015 3:07 PM, Brian Goetz wrote: > It's time to take the next step in Valhalla's evolution. Accordingly, > I've created the valhalla-spec-experts list, which will be the > precursor to an eventual JCP Expert Group, and will operate under the > JCP-friendly ToU. This list is publicly archived in the obvious > place, and the initial membership includes both corporate > participation (Oracle, IBM, Red Hat, Jetbrains, Google) and > individuals from the OpenJDK community. > > As was the case when we formed the Lambda experts list, the role of > valhalla-dev should now revert to its intended purpose, which is to > discuss the *implementation* of Valhalla in OpenJDK. Examples of > appropriate discussion topics for valhalla-dev include: > > - Bug reports > - Code review comments > - Test cases > - Build or porting problems > - Migration experiences for your own code > > What we are trying to avoid is: having design discussions play out > concurrently in multiple places with differing sets of players. So > please, we ask that you exercise restraint in commenting on active EG > discussions -- as design decisions start to converge, we'll announce > them here, and of course the primary place for implementation will > remain the valhalla repositories. > > Please remember that EG discussions often happen at a different pace > than in other open-source projects. Days can go by without a > response, but this doesn't mean that the EG is not thinking about it, > doesn't care, or is "stumped". Often it simply means the EG members > are working privately (or with their teams) to figure out what their > position is, and will respond when they've thought about it more. So, > please let those discussions play out. > > From brian.goetz at oracle.com Tue Jan 12 20:25:17 2016 From: brian.goetz at oracle.com (brian.goetz at oracle.com) Date: Tue, 12 Jan 2016 20:25:17 +0000 Subject: hg: valhalla/valhalla/jdk: More tests and bugfixes for anyfied collections Message-ID: <201601122025.u0CKPHll026445@aojmv0008.oracle.com> Changeset: 38882d3a83e9 Author: briangoetz Date: 2016-01-12 15:25 -0500 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/38882d3a83e9 More tests and bugfixes for anyfied collections ! src/java.base/share/classes/java/anyutil/AbstractList.java ! src/java.base/share/classes/java/anyutil/ArrayList.java ! test/valhalla/test/valhalla/anyutil/CollectionsTest.java From richard.warburton at gmail.com Wed Jan 13 12:22:21 2016 From: richard.warburton at gmail.com (Richard Warburton) Date: Wed, 13 Jan 2016 12:22:21 +0000 Subject: Announcing: valhalla-spec-{observers,comments} In-Reply-To: <569419D3.80706@oracle.com> References: <566F2195.2030100@oracle.com> <569419D3.80706@oracle.com> Message-ID: Hi, As promised, we've now set up the additional mailing lists, > valhalla-spec-observers and valhalla-spec-comments. Here are the details. > > valhalla-spec-observers at openjdk.java.net is open to anyone to subscribe, > and subscribers may post to it. Traffic from valhalla-spec-experts is > automatically forwarded to valhalla-spec-observers. This list is primarily > intended so that people can follow the EG list traffic easily, and also for > people to have discussions within the -observers community. > > The primary means for the community to provide feedback into the design > and specification discussion is valhalla-spec-comments. Periodically, > comments from the -comments list will be "read into the record" on the > -experts list, where the EG can discuss the issue raised if it wants. But > with this access comes responsibility! Specifically, a post to -comments > should be well-reasoned, self-contained, and focused on a single issue (as > well as clear and polite, of course). Think of it as a suggestion box; we > guarantee that it will be seen by the EG, but if you want it to be taken > seriously by the EG, put some time and effort into making your point clear > argument. > > Note that -comments *cannot* be subscribed to; it can only be posted to. > (The reason for this is simple; having discussions on this list, which is > inevitable were it subscribable, would defeat the purpose.) The postings > are archived, and their content (if appropriate) will eventually show up on > the EG list. Postings are archived, and moderated to manage spam and > otherwise inappropriate content. > > On 12/14/2015 3:07 PM, Brian Goetz wrote: > >> It's time to take the next step in Valhalla's evolution. Accordingly, >> I've created the valhalla-spec-experts list, which will be the precursor to >> an eventual JCP Expert Group, and will operate under the JCP-friendly ToU. >> This list is publicly archived in the obvious place, and the initial >> membership includes both corporate participation (Oracle, IBM, Red Hat, >> Jetbrains, Google) and individuals from the OpenJDK community. >> >> As was the case when we formed the Lambda experts list, the role of >> valhalla-dev should now revert to its intended purpose, which is to discuss >> the *implementation* of Valhalla in OpenJDK. Examples of appropriate >> discussion topics for valhalla-dev include: >> >> - Bug reports >> - Code review comments >> - Test cases >> - Build or porting problems >> - Migration experiences for your own code >> >> What we are trying to avoid is: having design discussions play out >> concurrently in multiple places with differing sets of players. So please, >> we ask that you exercise restraint in commenting on active EG discussions >> -- as design decisions start to converge, we'll announce them here, and of >> course the primary place for implementation will remain the valhalla >> repositories. >> >> Please remember that EG discussions often happen at a different pace than >> in other open-source projects. Days can go by without a response, but this >> doesn't mean that the EG is not thinking about it, doesn't care, or is >> "stumped". Often it simply means the EG members are working privately (or >> with their teams) to figure out what their position is, and will respond >> when they've thought about it more. So, please let those discussions play >> out. > > Thanks for taking the time to setup these mailing lists. I think it would help people who are beginning to look at project Valhalla to have links to both spec-observers and spec-comments on the main valhalla project page. regards, Richard Warburton http://insightfullogic.com @RichardWarburto From martijnverburg at gmail.com Thu Jan 14 00:45:55 2016 From: martijnverburg at gmail.com (Martijn Verburg) Date: Thu, 14 Jan 2016 13:45:55 +1300 Subject: Announcing: valhalla-spec-{observers,comments} In-Reply-To: References: <566F2195.2030100@oracle.com> <569419D3.80706@oracle.com> Message-ID: Hi Richard, I'll get an updated page sent to Brian shortly - it needs someone with access to the webserver to change the pages there. Cheers, Martijn On 14 January 2016 at 01:22, Richard Warburton wrote: > Hi, > > As promised, we've now set up the additional mailing lists, > > valhalla-spec-observers and valhalla-spec-comments. Here are the > details. > > > > valhalla-spec-observers at openjdk.java.net is open to anyone to subscribe, > > and subscribers may post to it. Traffic from valhalla-spec-experts is > > automatically forwarded to valhalla-spec-observers. This list is > primarily > > intended so that people can follow the EG list traffic easily, and also > for > > people to have discussions within the -observers community. > > > > The primary means for the community to provide feedback into the design > > and specification discussion is valhalla-spec-comments. Periodically, > > comments from the -comments list will be "read into the record" on the > > -experts list, where the EG can discuss the issue raised if it wants. > But > > with this access comes responsibility! Specifically, a post to -comments > > should be well-reasoned, self-contained, and focused on a single issue > (as > > well as clear and polite, of course). Think of it as a suggestion box; > we > > guarantee that it will be seen by the EG, but if you want it to be taken > > seriously by the EG, put some time and effort into making your point > clear > > argument. > > > > Note that -comments *cannot* be subscribed to; it can only be posted to. > > (The reason for this is simple; having discussions on this list, which is > > inevitable were it subscribable, would defeat the purpose.) The postings > > are archived, and their content (if appropriate) will eventually show up > on > > the EG list. Postings are archived, and moderated to manage spam and > > otherwise inappropriate content. > > > > On 12/14/2015 3:07 PM, Brian Goetz wrote: > > > >> It's time to take the next step in Valhalla's evolution. Accordingly, > >> I've created the valhalla-spec-experts list, which will be the > precursor to > >> an eventual JCP Expert Group, and will operate under the JCP-friendly > ToU. > >> This list is publicly archived in the obvious place, and the initial > >> membership includes both corporate participation (Oracle, IBM, Red Hat, > >> Jetbrains, Google) and individuals from the OpenJDK community. > >> > >> As was the case when we formed the Lambda experts list, the role of > >> valhalla-dev should now revert to its intended purpose, which is to > discuss > >> the *implementation* of Valhalla in OpenJDK. Examples of appropriate > >> discussion topics for valhalla-dev include: > >> > >> - Bug reports > >> - Code review comments > >> - Test cases > >> - Build or porting problems > >> - Migration experiences for your own code > >> > >> What we are trying to avoid is: having design discussions play out > >> concurrently in multiple places with differing sets of players. So > please, > >> we ask that you exercise restraint in commenting on active EG > discussions > >> -- as design decisions start to converge, we'll announce them here, and > of > >> course the primary place for implementation will remain the valhalla > >> repositories. > >> > >> Please remember that EG discussions often happen at a different pace > than > >> in other open-source projects. Days can go by without a response, but > this > >> doesn't mean that the EG is not thinking about it, doesn't care, or is > >> "stumped". Often it simply means the EG members are working privately > (or > >> with their teams) to figure out what their position is, and will respond > >> when they've thought about it more. So, please let those discussions > play > >> out. > > > > > Thanks for taking the time to setup these mailing lists. I think it would > help people who are beginning to look at project Valhalla to have links to > both spec-observers and spec-comments on the main valhalla project page. > > regards, > > Richard Warburton > > http://insightfullogic.com > @RichardWarburto > From aleksey.shipilev at oracle.com Thu Jan 14 22:39:18 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Fri, 15 Jan 2016 01:39:18 +0300 Subject: Aligned long views over byte arrays and heap ByteBuffers Message-ID: <56982396.3020802@oracle.com> Hi, In VarHandles, there is a suggested API that can provide wider operations on some raw and unsuspecting entities, like byte[] and ByteBuffers, pretty much like Unsafe.getX does. For example, you can read "long" out of byte[] array: static final VarHandle VH = MethodHandles.byteArrayViewVarHandle( long[].class, /* unaligned = */ false, ByteOrder.nativeOrder() == ByteOrder.BIG_ENDIAN); byte[] storage = ...; long v = (long)VH.get(bytes, 0); // reads 8 bytes at index = 0 Note there are two flavors: aligned views and unaligned views. Aligned view is when we access the underlying storage aligned by "reinterpreted" element size. E.g. accessing 0-th long is accessing [0..7] bytes, 1-st long is [8..15] bytes, etc. Aligned views provide two important advantages: a) we can do atomic operations (reads, writes, CASes) for larger widths; and b) it maps well on hardware instructions, especially when hardware cannot make misaligned accesses (but, Unsafe.getXUnaligned works decently well). However, aligned views come with an (obvious in hindsight) problem: it is not guaranteed that array base for byte[] is aligned at 8. In fact, byte[] array base is coincidentally aligned to 8 for 64-bit HotSpot [1], but not for 32-bit HotSpot [2], where it is aligned by 4. This means, we are not able to universally guarantee heap ByteBuffer and byte[] views are aligned for *long* accesses. We still can do this for direct ByteBuffers. So, there are two obvious ways out of this conundrum: A) Give up on heap BBs and byte[] views, leave only direct BB. It is still an improvement over current state of affairs where even direct BB do not allow atomic ops. This breaks the symmetry between heap and direct BBs. A variant of this solution is to disallow only 8-byte views, long[] and double[], leaving others intact. B) Force VM to align all array bases to 8. Luckily, 64-bit HotSpot, which hopefully is the most ubiquitous VM nowadays, is unaffected. This penalizes current 32-bit HotSpot, and has unknown impact on other VMs that would try to run VarHandles. Any VM vendors on this list to chime in? There are completely unexplored alternatives: C) Work out an API for heap BBs that allow to "align" their storage, without getting VM into the picture. John suggested something like "BB.align(int unit) => returns this if aligned access of size unit already possible, else restricts the BB to an offset (and size) where that alignment is possible, using wrap(byte[],int,int)". D) Go the other way around, and define the byte views over already aligned heap LongBuffers, so that you would get both aligned byte-wide and long-wide accesses. There is no perfect solution that jumps out immediately, so I'm trolling for ideas. Thoughts? Thanks, -Aleksey ------------------------------------------------------------------- [1] $ java -cp ~/projects/jol/jol-samples/target/jol-samples.jar org.openjdk.jol.samples.JOLSample_25_ArrayAlignment Running 64-bit HotSpot VM. Using compressed references with 3-bit shift. Objects are 8 bytes aligned. Field sizes by type: 4, 1, 1, 2, 2, 4, 4, 8, 8 [bytes] Array element sizes: 4, 1, 1, 2, 2, 4, 4, 8, 8 [bytes] [J object internals: OFFSET SIZE TYPE DESCRIPTION VALUE 0 4 (object header) ... 4 4 (object header) ... 8 4 (object header) ... 12 4 (object header) ... 16 0 long [J. N/A Instance size: 16 bytes (reported by Instrumentation API) Space losses: 0 bytes internal + 0 bytes external = 0 bytes total [B object internals: OFFSET SIZE TYPE DESCRIPTION VALUE 0 4 (object header) ... 4 4 (object header) ... 8 4 (object header) ... 12 4 (object header) ... 16 0 byte [B. N/A Instance size: 16 bytes (reported by Instrumentation API) Space losses: 0 bytes internal + 0 bytes external = 0 bytes total ------------------------------------------------------------------- [2] $ java -cp ~/projects/jol/jol-samples/target/jol-samples.jar org.openjdk.jol.samples.JOLSample_25_ArrayAlignment Running 32-bit HotSpot VM. Objects are 8 bytes aligned. Field sizes by type: 4, 1, 1, 2, 2, 4, 4, 8, 8 [bytes] Array element sizes: 4, 1, 1, 2, 2, 4, 4, 8, 8 [bytes] [J object internals: OFFSET SIZE TYPE DESCRIPTION VALUE 0 4 (object header) ... 4 4 (object header) ... 8 4 (object header) ... 12 4 (alignment/padding gap) N/A 16 0 long [J. N/A Instance size: 16 bytes (reported by Instrumentation API) Space losses: 4 bytes internal + 0 bytes external = 4 bytes total [B object internals: OFFSET SIZE TYPE DESCRIPTION VALUE 0 4 (object header) ... 4 4 (object header) ... 8 4 (object header) ... 12 0 byte [B. N/A 12 4 (loss due to the next object alignment) Instance size: 16 bytes (reported by Instrumentation API) Space losses: 0 bytes internal + 4 bytes external = 4 bytes total From vitalyd at gmail.com Thu Jan 14 23:58:05 2016 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 14 Jan 2016 18:58:05 -0500 Subject: Aligned long views over byte arrays and heap ByteBuffers In-Reply-To: <56982396.3020802@oracle.com> References: <56982396.3020802@oracle.com> Message-ID: Is it possible to detect at runtime whether arrays are 8-byte aligned (i.e. 64 or 32 bit VM) and then change how long accesses via VarHandle operate? I suppose on 32-bit you'd have to do the access in software vs hardware instructions, but as you say, 32bit JVMs (at least Hotspot) aren't common these days. On Thu, Jan 14, 2016 at 5:39 PM, Aleksey Shipilev < aleksey.shipilev at oracle.com> wrote: > Hi, > > In VarHandles, there is a suggested API that can provide wider > operations on some raw and unsuspecting entities, like byte[] and > ByteBuffers, pretty much like Unsafe.getX does. For example, you can > read "long" out of byte[] array: > > static final VarHandle VH = > MethodHandles.byteArrayViewVarHandle( > long[].class, > /* unaligned = */ false, > ByteOrder.nativeOrder() == ByteOrder.BIG_ENDIAN); > > byte[] storage = ...; > long v = (long)VH.get(bytes, 0); // reads 8 bytes at index = 0 > > Note there are two flavors: aligned views and unaligned views. Aligned > view is when we access the underlying storage aligned by "reinterpreted" > element size. E.g. accessing 0-th long is accessing [0..7] bytes, 1-st > long is [8..15] bytes, etc. > > Aligned views provide two important advantages: a) we can do atomic > operations (reads, writes, CASes) for larger widths; and b) it maps well > on hardware instructions, especially when hardware cannot make > misaligned accesses (but, Unsafe.getXUnaligned works decently well). > > However, aligned views come with an (obvious in hindsight) problem: it > is not guaranteed that array base for byte[] is aligned at 8. In fact, > byte[] array base is coincidentally aligned to 8 for 64-bit HotSpot [1], > but not for 32-bit HotSpot [2], where it is aligned by 4. > > This means, we are not able to universally guarantee heap ByteBuffer and > byte[] views are aligned for *long* accesses. We still can do this for > direct ByteBuffers. > > So, there are two obvious ways out of this conundrum: > > A) Give up on heap BBs and byte[] views, leave only direct BB. It is > still an improvement over current state of affairs where even direct BB > do not allow atomic ops. This breaks the symmetry between heap and > direct BBs. A variant of this solution is to disallow only 8-byte views, > long[] and double[], leaving others intact. > > B) Force VM to align all array bases to 8. Luckily, 64-bit HotSpot, > which hopefully is the most ubiquitous VM nowadays, is unaffected. This > penalizes current 32-bit HotSpot, and has unknown impact on other VMs > that would try to run VarHandles. Any VM vendors on this list to chime in? > > There are completely unexplored alternatives: > > C) Work out an API for heap BBs that allow to "align" their storage, > without getting VM into the picture. John suggested something like > "BB.align(int unit) => returns this if aligned access of size unit > already possible, else restricts the BB to an offset (and size) where > that alignment is possible, using wrap(byte[],int,int)". > > D) Go the other way around, and define the byte views over already > aligned heap LongBuffers, so that you would get both aligned byte-wide > and long-wide accesses. > > There is no perfect solution that jumps out immediately, so I'm trolling > for ideas. Thoughts? > > Thanks, > -Aleksey > > ------------------------------------------------------------------- > > [1] $ java -cp ~/projects/jol/jol-samples/target/jol-samples.jar > org.openjdk.jol.samples.JOLSample_25_ArrayAlignment > > Running 64-bit HotSpot VM. > Using compressed references with 3-bit shift. > Objects are 8 bytes aligned. > Field sizes by type: 4, 1, 1, 2, 2, 4, 4, 8, 8 [bytes] > Array element sizes: 4, 1, 1, 2, 2, 4, 4, 8, 8 [bytes] > > [J object internals: > OFFSET SIZE TYPE DESCRIPTION VALUE > 0 4 (object header) ... > 4 4 (object header) ... > 8 4 (object header) ... > 12 4 (object header) ... > 16 0 long [J. N/A > Instance size: 16 bytes (reported by Instrumentation API) > Space losses: 0 bytes internal + 0 bytes external = 0 bytes total > > [B object internals: > OFFSET SIZE TYPE DESCRIPTION VALUE > 0 4 (object header) ... > 4 4 (object header) ... > 8 4 (object header) ... > 12 4 (object header) ... > 16 0 byte [B. N/A > Instance size: 16 bytes (reported by Instrumentation API) > Space losses: 0 bytes internal + 0 bytes external = 0 bytes total > > ------------------------------------------------------------------- > > [2] $ java -cp ~/projects/jol/jol-samples/target/jol-samples.jar > org.openjdk.jol.samples.JOLSample_25_ArrayAlignment > > Running 32-bit HotSpot VM. > Objects are 8 bytes aligned. > Field sizes by type: 4, 1, 1, 2, 2, 4, 4, 8, 8 [bytes] > Array element sizes: 4, 1, 1, 2, 2, 4, 4, 8, 8 [bytes] > > [J object internals: > OFFSET SIZE TYPE DESCRIPTION VALUE > 0 4 (object header) ... > 4 4 (object header) ... > 8 4 (object header) ... > 12 4 (alignment/padding gap) N/A > 16 0 long [J. N/A > Instance size: 16 bytes (reported by Instrumentation API) > Space losses: 4 bytes internal + 0 bytes external = 4 bytes total > > [B object internals: > OFFSET SIZE TYPE DESCRIPTION VALUE > 0 4 (object header) ... > 4 4 (object header) ... > 8 4 (object header) ... > 12 0 byte [B. N/A > 12 4 (loss due to the next object alignment) > Instance size: 16 bytes (reported by Instrumentation API) > Space losses: 0 bytes internal + 4 bytes external = 4 bytes total > > > From paul.sandoz at oracle.com Fri Jan 15 09:28:25 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 15 Jan 2016 10:28:25 +0100 Subject: Aligned long views over byte arrays and heap ByteBuffers In-Reply-To: References: <56982396.3020802@oracle.com> Message-ID: Hi Vitaly, That should be possible if alignment behaviour is always guaranteed on hardware supporting platforms. It would mean that all atomic accesses should go through the handle such that on software supported platforms a lock can be used, which IIUC is kind of similar to Unsafe behaviour. Thanks, Paul. > On 15 Jan 2016, at 00:58, Vitaly Davidovich wrote: > > Is it possible to detect at runtime whether arrays are 8-byte aligned (i.e. > 64 or 32 bit VM) and then change how long accesses via VarHandle operate? I > suppose on 32-bit you'd have to do the access in software vs hardware > instructions, but as you say, 32bit JVMs (at least Hotspot) aren't common > these days. > From paul.sandoz at oracle.com Fri Jan 15 09:54:46 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 15 Jan 2016 10:54:46 +0100 Subject: Aligned long views over byte arrays and heap ByteBuffers In-Reply-To: <56982396.3020802@oracle.com> References: <56982396.3020802@oracle.com> Message-ID: <78CE437C-C7A8-49C5-A386-D565BA81E485@oracle.com> Hi Aleksey, Thanks for calling this out. The current aligned implementation for Buffers throws an ISE if the base address is not aligned [*], but this was neglected for aligned array views. It might be tricky to create a cross-platform and portable aligned solution for anything other than direct Buffers. Vitaly?s suggestion could be a way out, it seems feasible. A fall back position might be: 1) Aligned direct byte buffer views 2) Unaligned array views 2.1) Optionally fenced accessors (with potential non-atomic behaviour depending on alignment) 3) Optionally unaligned heap and direct byte views, with perhaps 2.1) behaviour, although that is questionable. My main motivation for that was to bypass the Buffer class hierarchy, but it might be of questionable value. Paul. [*] Access to heap and direct buffers is unified, with some tweaks of the Buffer code, so that the Unsafe double address mode is always used. > On 14 Jan 2016, at 23:39, Aleksey Shipilev wrote: > > Hi, > > In VarHandles, there is a suggested API that can provide wider > operations on some raw and unsuspecting entities, like byte[] and > ByteBuffers, pretty much like Unsafe.getX does. For example, you can > read "long" out of byte[] array: > > static final VarHandle VH = > MethodHandles.byteArrayViewVarHandle( > long[].class, > /* unaligned = */ false, > ByteOrder.nativeOrder() == ByteOrder.BIG_ENDIAN); > > byte[] storage = ...; > long v = (long)VH.get(bytes, 0); // reads 8 bytes at index = 0 > > Note there are two flavors: aligned views and unaligned views. Aligned > view is when we access the underlying storage aligned by "reinterpreted" > element size. E.g. accessing 0-th long is accessing [0..7] bytes, 1-st > long is [8..15] bytes, etc. > > Aligned views provide two important advantages: a) we can do atomic > operations (reads, writes, CASes) for larger widths; and b) it maps well > on hardware instructions, especially when hardware cannot make > misaligned accesses (but, Unsafe.getXUnaligned works decently well). > > However, aligned views come with an (obvious in hindsight) problem: it > is not guaranteed that array base for byte[] is aligned at 8. In fact, > byte[] array base is coincidentally aligned to 8 for 64-bit HotSpot [1], > but not for 32-bit HotSpot [2], where it is aligned by 4. > > This means, we are not able to universally guarantee heap ByteBuffer and > byte[] views are aligned for *long* accesses. We still can do this for > direct ByteBuffers. > > So, there are two obvious ways out of this conundrum: > > A) Give up on heap BBs and byte[] views, leave only direct BB. It is > still an improvement over current state of affairs where even direct BB > do not allow atomic ops. This breaks the symmetry between heap and > direct BBs. A variant of this solution is to disallow only 8-byte views, > long[] and double[], leaving others intact. > > B) Force VM to align all array bases to 8. Luckily, 64-bit HotSpot, > which hopefully is the most ubiquitous VM nowadays, is unaffected. This > penalizes current 32-bit HotSpot, and has unknown impact on other VMs > that would try to run VarHandles. Any VM vendors on this list to chime in? > > There are completely unexplored alternatives: > > C) Work out an API for heap BBs that allow to "align" their storage, > without getting VM into the picture. John suggested something like > "BB.align(int unit) => returns this if aligned access of size unit > already possible, else restricts the BB to an offset (and size) where > that alignment is possible, using wrap(byte[],int,int)". > > D) Go the other way around, and define the byte views over already > aligned heap LongBuffers, so that you would get both aligned byte-wide > and long-wide accesses. > > There is no perfect solution that jumps out immediately, so I'm trolling > for ideas. Thoughts? > > Thanks, > -Aleksey > > ------------------------------------------------------------------- > > [1] $ java -cp ~/projects/jol/jol-samples/target/jol-samples.jar > org.openjdk.jol.samples.JOLSample_25_ArrayAlignment > > Running 64-bit HotSpot VM. > Using compressed references with 3-bit shift. > Objects are 8 bytes aligned. > Field sizes by type: 4, 1, 1, 2, 2, 4, 4, 8, 8 [bytes] > Array element sizes: 4, 1, 1, 2, 2, 4, 4, 8, 8 [bytes] > > [J object internals: > OFFSET SIZE TYPE DESCRIPTION VALUE > 0 4 (object header) ... > 4 4 (object header) ... > 8 4 (object header) ... > 12 4 (object header) ... > 16 0 long [J. N/A > Instance size: 16 bytes (reported by Instrumentation API) > Space losses: 0 bytes internal + 0 bytes external = 0 bytes total > > [B object internals: > OFFSET SIZE TYPE DESCRIPTION VALUE > 0 4 (object header) ... > 4 4 (object header) ... > 8 4 (object header) ... > 12 4 (object header) ... > 16 0 byte [B. N/A > Instance size: 16 bytes (reported by Instrumentation API) > Space losses: 0 bytes internal + 0 bytes external = 0 bytes total > > ------------------------------------------------------------------- > > [2] $ java -cp ~/projects/jol/jol-samples/target/jol-samples.jar > org.openjdk.jol.samples.JOLSample_25_ArrayAlignment > > Running 32-bit HotSpot VM. > Objects are 8 bytes aligned. > Field sizes by type: 4, 1, 1, 2, 2, 4, 4, 8, 8 [bytes] > Array element sizes: 4, 1, 1, 2, 2, 4, 4, 8, 8 [bytes] > > [J object internals: > OFFSET SIZE TYPE DESCRIPTION VALUE > 0 4 (object header) ... > 4 4 (object header) ... > 8 4 (object header) ... > 12 4 (alignment/padding gap) N/A > 16 0 long [J. N/A > Instance size: 16 bytes (reported by Instrumentation API) > Space losses: 4 bytes internal + 0 bytes external = 4 bytes total > > [B object internals: > OFFSET SIZE TYPE DESCRIPTION VALUE > 0 4 (object header) ... > 4 4 (object header) ... > 8 4 (object header) ... > 12 0 byte [B. N/A > 12 4 (loss due to the next object alignment) > Instance size: 16 bytes (reported by Instrumentation API) > Space losses: 0 bytes internal + 4 bytes external = 4 bytes total > > From duncan.macgregor at ge.com Fri Jan 15 11:34:19 2016 From: duncan.macgregor at ge.com (MacGregor, Duncan (GE Energy Management)) Date: Fri, 15 Jan 2016 11:34:19 +0000 Subject: Aligned long views over byte arrays and heap ByteBuffers In-Reply-To: <56982396.3020802@oracle.com> References: <56982396.3020802@oracle.com> Message-ID: Can we guarantee that even direct byte buffers are aligned? While we can control the creation of them by the JVM itself is it reasonable to force users of JNI to ensure that the address passed into NewDirectByteBuffer meets the VM?s alignment rules? There?s probably also some hideous interaction with views onto an array byte buffer and how the aligned access should interact with those. Duncan. On 14/01/2016, 22:39, "valhalla-dev on behalf of Aleksey Shipilev" wrote: >Hi, > >In VarHandles, there is a suggested API that can provide wider >operations on some raw and unsuspecting entities, like byte[] and >ByteBuffers, pretty much like Unsafe.getX does. For example, you can >read "long" out of byte[] array: > > static final VarHandle VH = > MethodHandles.byteArrayViewVarHandle( > long[].class, > /* unaligned = */ false, > ByteOrder.nativeOrder() == ByteOrder.BIG_ENDIAN); > > byte[] storage = ...; > long v = (long)VH.get(bytes, 0); // reads 8 bytes at index = 0 > >Note there are two flavors: aligned views and unaligned views. Aligned >view is when we access the underlying storage aligned by "reinterpreted" >element size. E.g. accessing 0-th long is accessing [0..7] bytes, 1-st >long is [8..15] bytes, etc. > >Aligned views provide two important advantages: a) we can do atomic >operations (reads, writes, CASes) for larger widths; and b) it maps well >on hardware instructions, especially when hardware cannot make >misaligned accesses (but, Unsafe.getXUnaligned works decently well). > >However, aligned views come with an (obvious in hindsight) problem: it >is not guaranteed that array base for byte[] is aligned at 8. In fact, >byte[] array base is coincidentally aligned to 8 for 64-bit HotSpot [1], >but not for 32-bit HotSpot [2], where it is aligned by 4. > >This means, we are not able to universally guarantee heap ByteBuffer and >byte[] views are aligned for *long* accesses. We still can do this for >direct ByteBuffers. > >So, there are two obvious ways out of this conundrum: > > A) Give up on heap BBs and byte[] views, leave only direct BB. It is >still an improvement over current state of affairs where even direct BB >do not allow atomic ops. This breaks the symmetry between heap and >direct BBs. A variant of this solution is to disallow only 8-byte views, >long[] and double[], leaving others intact. > > B) Force VM to align all array bases to 8. Luckily, 64-bit HotSpot, >which hopefully is the most ubiquitous VM nowadays, is unaffected. This >penalizes current 32-bit HotSpot, and has unknown impact on other VMs >that would try to run VarHandles. Any VM vendors on this list to chime in? > >There are completely unexplored alternatives: > > C) Work out an API for heap BBs that allow to "align" their storage, >without getting VM into the picture. John suggested something like >"BB.align(int unit) => returns this if aligned access of size unit >already possible, else restricts the BB to an offset (and size) where >that alignment is possible, using wrap(byte[],int,int)". > > D) Go the other way around, and define the byte views over already >aligned heap LongBuffers, so that you would get both aligned byte-wide >and long-wide accesses. > >There is no perfect solution that jumps out immediately, so I'm trolling >for ideas. Thoughts? > >Thanks, >-Aleksey > >------------------------------------------------------------------- > >[1] $ java -cp ~/projects/jol/jol-samples/target/jol-samples.jar >org.openjdk.jol.samples.JOLSample_25_ArrayAlignment > >Running 64-bit HotSpot VM. >Using compressed references with 3-bit shift. >Objects are 8 bytes aligned. >Field sizes by type: 4, 1, 1, 2, 2, 4, 4, 8, 8 [bytes] >Array element sizes: 4, 1, 1, 2, 2, 4, 4, 8, 8 [bytes] > >[J object internals: > OFFSET SIZE TYPE DESCRIPTION VALUE > 0 4 (object header) ... > 4 4 (object header) ... > 8 4 (object header) ... > 12 4 (object header) ... > 16 0 long [J. N/A >Instance size: 16 bytes (reported by Instrumentation API) >Space losses: 0 bytes internal + 0 bytes external = 0 bytes total > >[B object internals: > OFFSET SIZE TYPE DESCRIPTION VALUE > 0 4 (object header) ... > 4 4 (object header) ... > 8 4 (object header) ... > 12 4 (object header) ... > 16 0 byte [B. N/A >Instance size: 16 bytes (reported by Instrumentation API) >Space losses: 0 bytes internal + 0 bytes external = 0 bytes total > >------------------------------------------------------------------- > >[2] $ java -cp ~/projects/jol/jol-samples/target/jol-samples.jar >org.openjdk.jol.samples.JOLSample_25_ArrayAlignment > >Running 32-bit HotSpot VM. >Objects are 8 bytes aligned. >Field sizes by type: 4, 1, 1, 2, 2, 4, 4, 8, 8 [bytes] >Array element sizes: 4, 1, 1, 2, 2, 4, 4, 8, 8 [bytes] > >[J object internals: > OFFSET SIZE TYPE DESCRIPTION VALUE > 0 4 (object header) ... > 4 4 (object header) ... > 8 4 (object header) ... > 12 4 (alignment/padding gap) N/A > 16 0 long [J. N/A >Instance size: 16 bytes (reported by Instrumentation API) >Space losses: 4 bytes internal + 0 bytes external = 4 bytes total > >[B object internals: > OFFSET SIZE TYPE DESCRIPTION VALUE > 0 4 (object header) ... > 4 4 (object header) ... > 8 4 (object header) ... > 12 0 byte [B. N/A > 12 4 (loss due to the next object alignment) >Instance size: 16 bytes (reported by Instrumentation API) >Space losses: 0 bytes internal + 4 bytes external = 4 bytes total > > From paul.sandoz at oracle.com Fri Jan 15 12:47:12 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 15 Jan 2016 13:47:12 +0100 Subject: Aligned long views over byte arrays and heap ByteBuffers In-Reply-To: References: <56982396.3020802@oracle.com> Message-ID: > On 15 Jan 2016, at 12:34, MacGregor, Duncan (GE Energy Management) wrote: > > Can we guarantee that even direct byte buffers are aligned? No, the address is not always guaranteed to be aligned, for example: // Address not aligned ByteBuffer bb = ByteBuffer.allocateDirect(?).position(1).slice(); For the JDK implementation when a new DDB is allocated the address is guaranteed to be aligned (see Unsafe.allocateMemory), and the implementation takes advantage of that when viewing a DBB as a LongBuffer. The VH aligned view implementations perform a runtime alignment check and throw an exception on failure. Paul. > While we can > control the creation of them by the JVM itself is it reasonable to force > users of JNI to ensure that the address passed into NewDirectByteBuffer > meets the VM?s alignment rules? > > There?s probably also some hideous interaction with views onto an array > byte buffer and how the aligned access should interact with those. > > Duncan. > From vitalyd at gmail.com Fri Jan 15 13:38:57 2016 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 15 Jan 2016 08:38:57 -0500 Subject: Aligned long views over byte arrays and heap ByteBuffers In-Reply-To: References: <56982396.3020802@oracle.com> Message-ID: Hey Paul, That should be possible if alignment behaviour is always guaranteed on > hardware supporting platforms. The check could be (a) are arrays 8-byte aligned and (b) if not, does the platform support unaligned accesses. I'm not fully certain what type of "atomic" accesses you guys are referring to, but given Aleksey mentioned CAS, I'm assuming it's atomic instructions rather than CPU atomic accesses (e.g. Intel uses the term "atomic" to also talk about ordinary accesses to memory, which boils down to "does the read stay within a single cacheline"). In that case, some hardware does support unaligned atomic instructions (e.g. Intel modern cpus' LOCK prefix and cmpxchg do allow unaligned addresses), albeit they'll likely execute slower but probably faster than software emulation. I suppose part (b) may not be that important, at least for Hotspot purposes, since 32-bit Hotspot is likely a small fraction of deployments. But once you have the software emulation fallback, you (or other vendors) could extend the checks to cover (b) or whatever else is relevant. On Fri, Jan 15, 2016 at 4:28 AM, Paul Sandoz wrote: > Hi Vitaly, > > That should be possible if alignment behaviour is always guaranteed on > hardware supporting platforms. > > It would mean that all atomic accesses should go through the handle such > that on software supported platforms a lock can be used, which IIUC is kind > of similar to Unsafe behaviour. > > Thanks, > Paul. > > > On 15 Jan 2016, at 00:58, Vitaly Davidovich wrote: > > > > Is it possible to detect at runtime whether arrays are 8-byte aligned > (i.e. > > 64 or 32 bit VM) and then change how long accesses via VarHandle > operate? I > > suppose on 32-bit you'd have to do the access in software vs hardware > > instructions, but as you say, 32bit JVMs (at least Hotspot) aren't common > > these days. > > > > From john.r.rose at oracle.com Fri Jan 15 16:34:33 2016 From: john.r.rose at oracle.com (John Rose) Date: Fri, 15 Jan 2016 08:34:33 -0800 Subject: Aligned long views over byte arrays and heap ByteBuffers In-Reply-To: References: <56982396.3020802@oracle.com> Message-ID: <382DB2EC-3943-4791-8FD1-CF8FB23A41C4@oracle.com> On Jan 15, 2016, at 4:47 AM, Paul Sandoz wrote: > > For the JDK implementation when a new DDB is allocated the address is guaranteed to be aligned (see Unsafe.allocateMemory), and the implementation takes advantage of that when viewing a DBB as a LongBuffer. ?Also, the JNI entry point NewDirectByteBuffer uses the address as-is, which is a feature not a bug. // Invoked only by JNI: NewDirectByteBuffer(void*, long) // private Direct$Type$Buffer(long addr, int cap) { super(-1, 0, cap, cap); address = addr; cleaner = null; att = null; } From john.r.rose at oracle.com Fri Jan 15 16:41:39 2016 From: john.r.rose at oracle.com (John Rose) Date: Fri, 15 Jan 2016 08:41:39 -0800 Subject: Aligned long views over byte arrays and heap ByteBuffers In-Reply-To: <78CE437C-C7A8-49C5-A386-D565BA81E485@oracle.com> References: <56982396.3020802@oracle.com> <78CE437C-C7A8-49C5-A386-D565BA81E485@oracle.com> Message-ID: <11038A0B-FF7E-436A-9628-91F00E609AA6@oracle.com> On Jan 15, 2016, at 1:54 AM, Paul Sandoz wrote: > > [*] Access to heap and direct buffers is unified, with some tweaks of the Buffer code, so that the Unsafe double address mode is always used. FTR: This is a highly technical point, and I'm glad to see it done right. We can now remove polymorphism (bi-morphism) in inner loops which operate on a mix of buffer types. It's important for location-agnostic algorithms. From paul.sandoz at oracle.com Fri Jan 15 16:41:43 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 15 Jan 2016 17:41:43 +0100 Subject: Aligned long views over byte arrays and heap ByteBuffers In-Reply-To: <382DB2EC-3943-4791-8FD1-CF8FB23A41C4@oracle.com> References: <56982396.3020802@oracle.com> <382DB2EC-3943-4791-8FD1-CF8FB23A41C4@oracle.com> Message-ID: <7AD5CB18-DD9C-4EB6-A685-D6F9B61281DC@oracle.com> > On 15 Jan 2016, at 17:34, John Rose wrote: > > On Jan 15, 2016, at 4:47 AM, Paul Sandoz > wrote: >> >> For the JDK implementation when a new DDB is allocated the address is guaranteed to be aligned (see Unsafe.allocateMemory), and the implementation takes advantage of that when viewing a DBB as a LongBuffer. > > ?Also, the JNI entry point NewDirectByteBuffer uses the address as-is, which is a feature not a bug. > > // Invoked only by JNI: NewDirectByteBuffer(void*, long) > // > private Direct$Type$Buffer(long addr, int cap) { > super(-1, 0, cap, cap); > address = addr; > cleaner = null; > att = null; > } > Thanks (and to Duncan too), i was not aware of that constructor. Paul. From maurizio.cimadamore at oracle.com Fri Jan 15 18:16:08 2016 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Fri, 15 Jan 2016 18:16:08 +0000 Subject: hg: valhalla/valhalla/jdk: Enhancement: use Class static arg instead of String for unspecialized receiver in alt generic method bootstrap Message-ID: <201601151816.u0FIG8LK022192@aojmv0008.oracle.com> Changeset: db9a11a4397c Author: mcimadamore Date: 2016-01-15 18:15 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/db9a11a4397c Enhancement: use Class static arg instead of String for unspecialized receiver in alt generic method bootstrap ! src/java.base/share/classes/java/lang/invoke/DispatchContext.java ! src/java.base/share/classes/java/lang/invoke/GenericStaticDispatch.java From maurizio.cimadamore at oracle.com Fri Jan 15 18:20:07 2016 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Fri, 15 Jan 2016 18:20:07 +0000 Subject: hg: valhalla/valhalla/langtools: Enhancement: use Class static arg instead of String for unspecialized receiver in alt generic method bootstrap Message-ID: <201601151820.u0FIK7K0023907@aojmv0008.oracle.com> Changeset: de66408690d1 Author: mcimadamore Date: 2016-01-15 18:19 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/de66408690d1 Enhancement: use Class static arg instead of String for unspecialized receiver in alt generic method bootstrap ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/SpecializeTypes.java From paul.sandoz at oracle.com Mon Jan 18 11:38:40 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 18 Jan 2016 12:38:40 +0100 Subject: Aligned long views over byte arrays and heap ByteBuffers In-Reply-To: References: <56982396.3020802@oracle.com> Message-ID: <5555501D-B4BC-4A0C-BCEC-CE9C159EA3C1@oracle.com> Hi Vitaly, I had a quick chat with John and we think may be able to avoid the aligned vs. unaligned distinction in the API and any split in terms of platform implementations. We can add some new alignment methods to ByteBuffer: 1) query if an index is aligned 2) slice to aligned Then we can provide a singular view over a ByteBuffer for source-based indexes, where for enhanced atomic operations an ISE is thrown if the index is not aligned. Thus a user can set up the alignment with 1)/2) to avoid ISEs. If a user wants enhanced atomic operations on byte[] they could first wrap that in a ByteBuffer, although we could provide some static methods on ByteBuffer to avoid such wrapping. Paul. > On 15 Jan 2016, at 14:38, Vitaly Davidovich wrote: > > Hey Paul, > > That should be possible if alignment behaviour is always guaranteed on hardware supporting platforms. > > The check could be (a) are arrays 8-byte aligned and (b) if not, does the platform support unaligned accesses. I'm not fully certain what type of "atomic" accesses you guys are referring to, but given Aleksey mentioned CAS, I'm assuming it's atomic instructions rather than CPU atomic accesses (e.g. Intel uses the term "atomic" to also talk about ordinary accesses to memory, which boils down to "does the read stay within a single cacheline"). In that case, some hardware does support unaligned atomic instructions (e.g. Intel modern cpus' LOCK prefix and cmpxchg do allow unaligned addresses), albeit they'll likely execute slower but probably faster than software emulation. > > I suppose part (b) may not be that important, at least for Hotspot purposes, since 32-bit Hotspot is likely a small fraction of deployments. But once you have the software emulation fallback, you (or other vendors) could extend the checks to cover (b) or whatever else is relevant. > > > On Fri, Jan 15, 2016 at 4:28 AM, Paul Sandoz > wrote: > Hi Vitaly, > > That should be possible if alignment behaviour is always guaranteed on hardware supporting platforms. > > It would mean that all atomic accesses should go through the handle such that on software supported platforms a lock can be used, which IIUC is kind of similar to Unsafe behaviour. > > Thanks, > Paul. > > > On 15 Jan 2016, at 00:58, Vitaly Davidovich > wrote: > > > > Is it possible to detect at runtime whether arrays are 8-byte aligned (i.e. > > 64 or 32 bit VM) and then change how long accesses via VarHandle operate? I > > suppose on 32-bit you'd have to do the access in software vs hardware > > instructions, but as you say, 32bit JVMs (at least Hotspot) aren't common > > these days. > > > > From paul.sandoz at oracle.com Mon Jan 18 13:44:51 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 18 Jan 2016 14:44:51 +0100 Subject: Aligned long views over byte arrays and heap ByteBuffers In-Reply-To: <5555501D-B4BC-4A0C-BCEC-CE9C159EA3C1@oracle.com> References: <56982396.3020802@oracle.com> <5555501D-B4BC-4A0C-BCEC-CE9C159EA3C1@oracle.com> Message-ID: > On 18 Jan 2016, at 12:38, Paul Sandoz wrote: > > Hi Vitaly, > > I had a quick chat with John and we think may be able to avoid the aligned vs. unaligned distinction in the API and any split in terms of platform implementations. > > We can add some new alignment methods to ByteBuffer: > > 1) query if an index is aligned > 2) slice to aligned > > Then we can provide a singular view over a ByteBuffer for source-based indexes, where for enhanced atomic operations an ISE is thrown if the index is not aligned. Thus a user can set up the alignment with 1)/2) to avoid ISEs. > > If a user wants enhanced atomic operations on byte[] they could first wrap that in a ByteBuffer, although we could provide some static methods on ByteBuffer to avoid such wrapping. > Critiquing myself? Since GC can move the byte[] i presume we cannot guarantee alignment stability e.g. on a 32-bit system a query at one point might report true for alignment of index 0 with a long-based unit value, and later on might report false if the underlying byte[] has been moved between the two queries. Thus such proposed methods would reliably work only for DBBs and as such they would be rather awkward on BB. Paul. From aleksey.shipilev at oracle.com Mon Jan 18 14:17:01 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Mon, 18 Jan 2016 17:17:01 +0300 Subject: Aligned long views over byte arrays and heap ByteBuffers In-Reply-To: References: <56982396.3020802@oracle.com> <5555501D-B4BC-4A0C-BCEC-CE9C159EA3C1@oracle.com> Message-ID: <569CF3DD.30005@oracle.com> On 18.01.2016 16:44, Paul Sandoz wrote: > Critiquing myself? > > Since GC can move the byte[] i presume we cannot guarantee alignment > stability e.g. on a 32-bit system a query at one point might report > true for alignment of index 0 with a long-based unit value, and later > on might report false if the underlying byte[] has been moved between > the two queries. Hotspot uses 8-byte alignments for Java objects universally, so the absolute 8-byte alignment of byte[] base (and consequently, any i-th element) stays the same: it it failed/passed the alignment check once, it will fail/pass always. That being said, there probably are VMs that implement multiple alignments, like I proposed here: https://bugs.openjdk.java.net/browse/JDK-8025677 -- where this invariant indeed is not stable. Thanks, -Aleksey From brian.goetz at oracle.com Mon Jan 18 15:29:46 2016 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 18 Jan 2016 10:29:46 -0500 Subject: Aligned long views over byte arrays and heap ByteBuffers In-Reply-To: <5555501D-B4BC-4A0C-BCEC-CE9C159EA3C1@oracle.com> References: <56982396.3020802@oracle.com> <5555501D-B4BC-4A0C-BCEC-CE9C159EA3C1@oracle.com> Message-ID: <569D04EA.9070901@oracle.com> Agree, this seems a better way to solve the problem. On 1/18/2016 6:38 AM, Paul Sandoz wrote: > Hi Vitaly, > > I had a quick chat with John and we think may be able to avoid the aligned vs. unaligned distinction in the API and any split in terms of platform implementations. > > We can add some new alignment methods to ByteBuffer: > > 1) query if an index is aligned > 2) slice to aligned > > Then we can provide a singular view over a ByteBuffer for source-based indexes, where for enhanced atomic operations an ISE is thrown if the index is not aligned. Thus a user can set up the alignment with 1)/2) to avoid ISEs. > > If a user wants enhanced atomic operations on byte[] they could first wrap that in a ByteBuffer, although we could provide some static methods on ByteBuffer to avoid such wrapping. > > Paul. > > >> On 15 Jan 2016, at 14:38, Vitaly Davidovich wrote: >> >> Hey Paul, >> >> That should be possible if alignment behaviour is always guaranteed on hardware supporting platforms. >> >> The check could be (a) are arrays 8-byte aligned and (b) if not, does the platform support unaligned accesses. I'm not fully certain what type of "atomic" accesses you guys are referring to, but given Aleksey mentioned CAS, I'm assuming it's atomic instructions rather than CPU atomic accesses (e.g. Intel uses the term "atomic" to also talk about ordinary accesses to memory, which boils down to "does the read stay within a single cacheline"). In that case, some hardware does support unaligned atomic instructions (e.g. Intel modern cpus' LOCK prefix and cmpxchg do allow unaligned addresses), albeit they'll likely execute slower but probably faster than software emulation. >> >> I suppose part (b) may not be that important, at least for Hotspot purposes, since 32-bit Hotspot is likely a small fraction of deployments. But once you have the software emulation fallback, you (or other vendors) could extend the checks to cover (b) or whatever else is relevant. >> >> >> On Fri, Jan 15, 2016 at 4:28 AM, Paul Sandoz > wrote: >> Hi Vitaly, >> >> That should be possible if alignment behaviour is always guaranteed on hardware supporting platforms. >> >> It would mean that all atomic accesses should go through the handle such that on software supported platforms a lock can be used, which IIUC is kind of similar to Unsafe behaviour. >> >> Thanks, >> Paul. >> >>> On 15 Jan 2016, at 00:58, Vitaly Davidovich > wrote: >>> >>> Is it possible to detect at runtime whether arrays are 8-byte aligned (i.e. >>> 64 or 32 bit VM) and then change how long accesses via VarHandle operate? I >>> suppose on 32-bit you'd have to do the access in software vs hardware >>> instructions, but as you say, 32bit JVMs (at least Hotspot) aren't common >>> these days. >>> >> From john.r.rose at oracle.com Mon Jan 18 18:08:08 2016 From: john.r.rose at oracle.com (John Rose) Date: Mon, 18 Jan 2016 10:08:08 -0800 Subject: Aligned long views over byte arrays and heap ByteBuffers In-Reply-To: <569CF3DD.30005@oracle.com> References: <56982396.3020802@oracle.com> <5555501D-B4BC-4A0C-BCEC-CE9C159EA3C1@oracle.com> <569CF3DD.30005@oracle.com> Message-ID: <39B35639-D213-4832-A988-F35E029E1CDF@oracle.com> > On Jan 18, 2016, at 6:17 AM, Aleksey Shipilev wrote: > > That being said, there probably are VMs Such exotic VMs (for tiny platforms, perhaps) can throw appropriate UOEs. Most widespread 32-bit system ABIs align long variables. We can expect nearly all JVMs to loosely follow ABI practices. If you don't like the UOEs and want bullet proof GC-able buffers, simply use long arrays. Later, we will get into odd territory if we introduce jumbo alignments for fast vector work. (E.g. AlignedLong4[] or Aligned[] or even Aligned<256,Long4>[].) Such arrays will need special GC processing to enforce alignment, and byte arrays will probably never be conformable to such requirements. We may want to have a factory method for building maximally aligned buffers. ? John From maurizio.cimadamore at oracle.com Tue Jan 19 15:23:55 2016 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Tue, 19 Jan 2016 15:23:55 +0000 Subject: hg: valhalla/valhalla/langtools: Enhancement: consolidate options to generate generic pool forms into a single flag Message-ID: <201601191523.u0JFNtXh002576@aojmv0008.oracle.com> Changeset: ce45cc7d0b52 Author: mcimadamore Date: 2016-01-19 15:23 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/ce45cc7d0b52 Enhancement: consolidate options to generate generic pool forms into a single flag The new flag -XDgenericClassFile superseded the old -XDpoolModes flag. The new flag also enables alternate generic method translation. ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Attr.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/SpecializeTypes.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/ClassWriter.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Gen.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Items.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/PoolWriter.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/sym/CreateSymbols.java + test/tools/javac/valhalla/typespec/GenericPool01.java From paul.sandoz at oracle.com Wed Jan 20 13:12:06 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 20 Jan 2016 14:12:06 +0100 Subject: Aligned long views over byte arrays and heap ByteBuffers In-Reply-To: <39B35639-D213-4832-A988-F35E029E1CDF@oracle.com> References: <56982396.3020802@oracle.com> <5555501D-B4BC-4A0C-BCEC-CE9C159EA3C1@oracle.com> <569CF3DD.30005@oracle.com> <39B35639-D213-4832-A988-F35E029E1CDF@oracle.com> Message-ID: I propose the following methods on ByteBuffer: /** * Returns the memory address, pointing to the byte at the given index, * modulus the given unit size. * *

A return value of zero indicates the byte at the index is aligned to * the unit size, otherwise the value indicates how much the index should be * rounded down or up to locate the nearest aligned byte. * *

If the unit size is a power of two then this method may be utilized * to determine if unit size bytes from the index can be accessed * atomically, if supported by the native platform. * * @param index * The index to query for alignment, must be non-negative, no upper * bounds check is performed * * @param unitSize * The unit size, must be positive * * @return The indexed byte's memory address modulus the unit size * * @throws IllegalArgumentException * If the index is negative or the unit size is non-positive * * @see #alignedSlice(int) * @since 9 */ public final int isAligned(int index, int unitSize) /** * Creates a new byte buffer whose content is a shared and aligned * subsequence of this buffer's content. * *

The content of the new buffer will start at this buffer's current * position rounded up to the index of the nearest aligned byte for the * given unit size, and end at this buffer's limit rounded down to the index * of the nearest aligned byte for the given unit size. * The new buffer's current position and limit will equal this buffer's * position if rounding results in out-of-bound values. If rounding is * within bounds the following expressions will be true for a new buffer * {@code nb} and unit size {@code unitSize}: * 

{@code
 * nb.isAligned(nb.position(), unitSize) == 0
 * nb.isAligned(nb.limit(), unitSize) == 0
 * (nb.limit() - nb.position()) % unitSize == 0
 * }
* *

Changes to this buffer's content will be visible in the new * buffer, and vice versa; the two buffers' position, limit, and mark * values will be independent. * *

The new buffer's position will be zero, its mark will be undefined, * and its byte order will be {@link ByteOrder#BIG_ENDIAN BIG_ENDIAN}. * * The new buffer will be direct if, and only if, this buffer is direct, and * it will be read-only if, and only if, this buffer is read-only.

* * @param unitSize * The unit size, must be positive * * @return The new byte buffer * * @throws IllegalArgumentException * If the unit size is non-positive * * @see #isAligned(int, int) * @see #slice() * @since 9 */ public final ByteBuffer alignedSlice(int unitSize) Paul. > On 18 Jan 2016, at 19:08, John Rose wrote: > > >> On Jan 18, 2016, at 6:17 AM, Aleksey Shipilev wrote: >> >> That being said, there probably are VMs > > > Such exotic VMs (for tiny platforms, perhaps) can throw appropriate UOEs. > > Most widespread 32-bit system ABIs align long variables. We can expect nearly all JVMs to loosely follow ABI practices. > > If you don't like the UOEs and want bullet proof GC-able buffers, simply use long arrays. > > Later, we will get into odd territory if we introduce jumbo alignments for fast vector work. (E.g. AlignedLong4[] or Aligned[] or even Aligned<256,Long4>[].) Such arrays will need special GC processing to enforce alignment, and byte arrays will probably never be conformable to such requirements. We may want to have a factory method for building maximally aligned buffers. > > ? John > From vitalyd at gmail.com Wed Jan 20 13:27:22 2016 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 20 Jan 2016 08:27:22 -0500 Subject: Aligned long views over byte arrays and heap ByteBuffers In-Reply-To: References: <56982396.3020802@oracle.com> <5555501D-B4BC-4A0C-BCEC-CE9C159EA3C1@oracle.com> <569CF3DD.30005@oracle.com> <39B35639-D213-4832-A988-F35E029E1CDF@oracle.com> Message-ID: Hi Paul, isAligned(int,int) could probably use a better name since it's not returning a boolean. How about alignIndex(int,int)? By the way, what's the rationale behind adding new APIs vs the runtime detection option we discussed earlier? Thanks On Wed, Jan 20, 2016 at 8:12 AM, Paul Sandoz wrote: > I propose the following methods on ByteBuffer: > > /** > * Returns the memory address, pointing to the byte at the given index, > * modulus the given unit size. > * > *

A return value of zero indicates the byte at the index is aligned to > * the unit size, otherwise the value indicates how much the index should > be > * rounded down or up to locate the nearest aligned byte. > * > *

If the unit size is a power of two then this method may be utilized > * to determine if unit size bytes from the index can be accessed > * atomically, if supported by the native platform. > * > * @param index > * The index to query for alignment, must be non-negative, no upper > * bounds check is performed > * > * @param unitSize > * The unit size, must be positive > * > * @return The indexed byte's memory address modulus the unit size > * > * @throws IllegalArgumentException > * If the index is negative or the unit size is non-positive > * > * @see #alignedSlice(int) > * @since 9 > */ > public final int isAligned(int index, int unitSize) > > /** > * Creates a new byte buffer whose content is a shared and aligned > * subsequence of this buffer's content. > * > *

The content of the new buffer will start at this buffer's current > * position rounded up to the index of the nearest aligned byte for the > * given unit size, and end at this buffer's limit rounded down to the > index > * of the nearest aligned byte for the given unit size. > * The new buffer's current position and limit will equal this buffer's > * position if rounding results in out-of-bound values. If rounding is > * within bounds the following expressions will be true for a new buffer > * {@code nb} and unit size {@code unitSize}: > * 

{@code
>  * nb.isAligned(nb.position(), unitSize) == 0
>  * nb.isAligned(nb.limit(), unitSize) == 0
>  * (nb.limit() - nb.position()) % unitSize == 0
>  * }
> * > *

Changes to this buffer's content will be visible in the new > * buffer, and vice versa; the two buffers' position, limit, and mark > * values will be independent. > * > *

The new buffer's position will be zero, its mark will be undefined, > * and its byte order will be {@link ByteOrder#BIG_ENDIAN BIG_ENDIAN}. > * > * The new buffer will be direct if, and only if, this buffer is direct, > and > * it will be read-only if, and only if, this buffer is read-only.

> * > * @param unitSize > * The unit size, must be positive > * > * @return The new byte buffer > * > * @throws IllegalArgumentException > * If the unit size is non-positive > * > * @see #isAligned(int, int) > * @see #slice() > * @since 9 > */ > public final ByteBuffer alignedSlice(int unitSize) > > Paul. > > > On 18 Jan 2016, at 19:08, John Rose wrote: > > > > > >> On Jan 18, 2016, at 6:17 AM, Aleksey Shipilev < > aleksey.shipilev at oracle.com> wrote: > >> > >> That being said, there probably are VMs > > > > > > Such exotic VMs (for tiny platforms, perhaps) can throw appropriate UOEs. > > > > Most widespread 32-bit system ABIs align long variables. We can expect > nearly all JVMs to loosely follow ABI practices. > > > > If you don't like the UOEs and want bullet proof GC-able buffers, simply > use long arrays. > > > > Later, we will get into odd territory if we introduce jumbo alignments > for fast vector work. (E.g. AlignedLong4[] or Aligned[] or even > Aligned<256,Long4>[].) Such arrays will need special GC processing to > enforce alignment, and byte arrays will probably never be conformable to > such requirements. We may want to have a factory method for building > maximally aligned buffers. > > > > ? John > > > > From paul.sandoz at oracle.com Wed Jan 20 14:15:06 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 20 Jan 2016 15:15:06 +0100 Subject: Aligned long views over byte arrays and heap ByteBuffers In-Reply-To: References: <56982396.3020802@oracle.com> <5555501D-B4BC-4A0C-BCEC-CE9C159EA3C1@oracle.com> <569CF3DD.30005@oracle.com> <39B35639-D213-4832-A988-F35E029E1CDF@oracle.com> Message-ID: <9625381C-6DEE-4A39-B61C-D6453B4711D7@oracle.com> > On 20 Jan 2016, at 14:27, Vitaly Davidovich wrote: > > Hi Paul, > > isAligned(int,int) could probably use a better name since it's not returning a boolean. Yeah, "isAligned(?) == 0? is the boolean producing expression. > How about alignIndex(int,int)? > That might work, although it implies returning an aligned index rather than a modulus associated with the current index and unit size. I think that would be more precise if we include boolean argument for rounding up/down to the nearest aligned index: alignedIndex(int index, int unitSize, boolean roundUp) For rounding down a negative value might be returned. I still prefer a modulus returning method. > By the way, what's the rationale behind adding new APIs vs the runtime detection option we discussed earlier? > So we don?t have to deal with two separate software and hardware implementations, with a runtime switch, since alignment cannot always be determined statically for buffers. Requiring explicit alignment is simpler, and arguably the performance characteristics are clearer. Paul. From duncan.macgregor at ge.com Wed Jan 20 15:01:31 2016 From: duncan.macgregor at ge.com (MacGregor, Duncan (GE Energy Management)) Date: Wed, 20 Jan 2016 15:01:31 +0000 Subject: Aligned long views over byte arrays and heap ByteBuffers In-Reply-To: <9625381C-6DEE-4A39-B61C-D6453B4711D7@oracle.com> References: <56982396.3020802@oracle.com> <5555501D-B4BC-4A0C-BCEC-CE9C159EA3C1@oracle.com> <569CF3DD.30005@oracle.com> <39B35639-D213-4832-A988-F35E029E1CDF@oracle.com> <9625381C-6DEE-4A39-B61C-D6453B4711D7@oracle.com> Message-ID: On 20/01/2016, 14:15, "valhalla-dev on behalf of Paul Sandoz" wrote: > >> On 20 Jan 2016, at 14:27, Vitaly Davidovich wrote: >> >>How about alignIndex(int,int)? >> > >That might work, although it implies returning an aligned index rather >than a modulus associated with the current index and unit size. I think >that would be more precise if we include boolean argument for rounding >up/down to the nearest aligned index: > > alignedIndex(int index, int unitSize, boolean roundUp) > >For rounding down a negative value might be returned. I still prefer a >modulus returning method. I kind of agree that alignedIndex suggests an actual index is returned. How about alignedOffset, or alignedDistance maybe? Duncan. From vitalyd at gmail.com Wed Jan 20 15:04:06 2016 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 20 Jan 2016 10:04:06 -0500 Subject: Aligned long views over byte arrays and heap ByteBuffers In-Reply-To: References: <56982396.3020802@oracle.com> <5555501D-B4BC-4A0C-BCEC-CE9C159EA3C1@oracle.com> <569CF3DD.30005@oracle.com> <39B35639-D213-4832-A988-F35E029E1CDF@oracle.com> <9625381C-6DEE-4A39-B61C-D6453B4711D7@oracle.com> Message-ID: I like alignOffset and thought about that too. Main issue is that isAligned is not idiomatic given it returns an offset adjustment. On Wednesday, January 20, 2016, MacGregor, Duncan (GE Energy Management) < duncan.macgregor at ge.com> wrote: > On 20/01/2016, 14:15, "valhalla-dev on behalf of Paul Sandoz" > on behalf of > paul.sandoz at oracle.com > wrote: > > > > >> On 20 Jan 2016, at 14:27, Vitaly Davidovich > wrote: > >> > >>How about alignIndex(int,int)? > >> > > > >That might work, although it implies returning an aligned index rather > >than a modulus associated with the current index and unit size. I think > >that would be more precise if we include boolean argument for rounding > >up/down to the nearest aligned index: > > > > alignedIndex(int index, int unitSize, boolean roundUp) > > > >For rounding down a negative value might be returned. I still prefer a > >modulus returning method. > > I kind of agree that alignedIndex suggests an actual index is returned. > How about alignedOffset, or alignedDistance maybe? > > Duncan. > > -- Sent from my phone From maurizio.cimadamore at oracle.com Thu Jan 21 16:30:42 2016 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Thu, 21 Jan 2016 16:30:42 +0000 Subject: hg: valhalla/valhalla/langtools: Enhancement: Split CompoundType_info into ArrayType_info and MethodDescriptor_info Message-ID: <201601211630.u0LGUuIm021172@aojmv0008.oracle.com> Changeset: e48d890d4788 Author: mcimadamore Date: 2016-01-21 16:30 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/e48d890d4788 Enhancement: Split CompoundType_info into ArrayType_info and MethodDescriptor_info ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/ClassFile.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Pool.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/PoolReader.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/PoolWriter.java ! src/jdk.jdeps/share/classes/com/sun/tools/classfile/ClassTranslator.java ! src/jdk.jdeps/share/classes/com/sun/tools/classfile/ClassWriter.java ! src/jdk.jdeps/share/classes/com/sun/tools/classfile/ConstantPool.java ! src/jdk.jdeps/share/classes/com/sun/tools/classfile/Dependencies.java ! src/jdk.jdeps/share/classes/com/sun/tools/classfile/ReferenceFinder.java ! src/jdk.jdeps/share/classes/com/sun/tools/javap/ConstantWriter.java ! test/tools/javac/lambda/ByteCodeTest.java From brian.goetz at oracle.com Thu Jan 21 16:36:40 2016 From: brian.goetz at oracle.com (brian.goetz at oracle.com) Date: Thu, 21 Jan 2016 16:36:40 +0000 Subject: hg: valhalla/valhalla/jdk: Continued porting of streams library; bugfix in Model3Converter classloader support Message-ID: <201601211636.u0LGaemQ023888@aojmv0008.oracle.com> Changeset: 6574a02e528c Author: briangoetz Date: 2016-01-21 11:34 -0500 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/6574a02e528c Continued porting of streams library; bugfix in Model3Converter classloader support ! src/java.base/share/classes/java/anyutil/stream/DoubleStream.java ! src/java.base/share/classes/java/anyutil/stream/IntStream.java ! src/java.base/share/classes/java/anyutil/stream/LongStream.java ! src/java.base/share/classes/java/anyutil/stream/Streams.java ! src/java.base/share/classes/java/net/URLClassLoader.java ! src/java.base/share/classes/valhalla/model3/Model3Converter.java From maurizio.cimadamore at oracle.com Thu Jan 21 17:13:09 2016 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Thu, 21 Jan 2016 17:13:09 +0000 Subject: hg: valhalla/valhalla/jdk: Enhancement: Split CompoundType_info into ArrayType_info and MethodDescriptor_info Message-ID: <201601211713.u0LHDAF3007237@aojmv0008.oracle.com> Changeset: d700d240b7ff Author: mcimadamore Date: 2016-01-21 17:13 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/d700d240b7ff Enhancement: Split CompoundType_info into ArrayType_info and MethodDescriptor_info ! src/java.base/share/classes/valhalla/model3/Model3Converter.java ! src/java.base/share/classes/valhalla/model3/classfile/ClassTranslator.java ! src/java.base/share/classes/valhalla/model3/classfile/ClassWriter.java ! src/java.base/share/classes/valhalla/model3/classfile/ConstantPool.java ! src/java.base/share/classes/valhalla/model3/classfile/Dependencies.java ! src/java.base/share/classes/valhalla/model3/classfile/ReferenceFinder.java From brian.goetz at oracle.com Thu Jan 21 21:31:56 2016 From: brian.goetz at oracle.com (brian.goetz at oracle.com) Date: Thu, 21 Jan 2016 21:31:56 +0000 Subject: hg: valhalla/valhalla/jdk: Add sorting, distinct support to streams prototype Message-ID: <201601212131.u0LLVuJd020132@aojmv0008.oracle.com> Changeset: 220711e41f14 Author: briangoetz Date: 2016-01-21 16:31 -0500 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/220711e41f14 Add sorting, distinct support to streams prototype ! src/java.base/share/classes/java/anyutil/Any.java ! src/java.base/share/classes/java/anyutil/Arrays.java ! src/java.base/share/classes/java/anyutil/Deque.java ! src/java.base/share/classes/java/anyutil/TimSort.java ! src/java.base/share/classes/java/anyutil/stream/DistinctOps.java ! src/java.base/share/classes/java/anyutil/stream/Nodes.java ! src/java.base/share/classes/java/anyutil/stream/Pipeline.java + src/java.base/share/classes/java/anyutil/stream/SortedOps.java ! test/valhalla/test/valhalla/anyutil/SimplePipelineTest.java From paul.sandoz at oracle.com Thu Jan 21 22:42:47 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 21 Jan 2016 23:42:47 +0100 Subject: API review of VarHandles Message-ID: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> Hi This is a request to review the VarHandles API. The code reviews and pushes will occur separately, and flow through the hs-comp repo, most likely from the bottom up first with Unsafe changes. The specdiff can be found here: http://cr.openjdk.java.net/~psandoz/jdk9/varhandles/specdiff/overview-summary.html (Note that specdiff renders some aspects of JavaDoc incorrectly, so just ignore any such quirks.) A consensus on the set of access mode methods proposed by Doug was previously discussed and reached. For the moment please ignore the following methods on MethodHandles: byteArrayViewVarHandle; and byteBufferViewVarHandle. It is necessary to revisit that functionality w.r.t. alignment and proposed enhancements to ByteBuffer (in discussion on valhalla-dev). Paul. From david.lloyd at redhat.com Thu Jan 21 23:05:53 2016 From: david.lloyd at redhat.com (David M. Lloyd) Date: Thu, 21 Jan 2016 17:05:53 -0600 Subject: API review of VarHandles In-Reply-To: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> Message-ID: <56A16451.6010709@redhat.com> On 01/21/2016 04:42 PM, Paul Sandoz wrote: > Hi > > This is a request to review the VarHandles API. The code reviews and pushes will occur separately, and flow through the hs-comp repo, most likely from the bottom up first with Unsafe changes. > > The specdiff can be found here: > > http://cr.openjdk.java.net/~psandoz/jdk9/varhandles/specdiff/overview-summary.html > > (Note that specdiff renders some aspects of JavaDoc incorrectly, so just ignore any such quirks.) > > A consensus on the set of access mode methods proposed by Doug was previously discussed and reached. > > For the moment please ignore the following methods on MethodHandles: byteArrayViewVarHandle; and byteBufferViewVarHandle. It is necessary to revisit that functionality w.r.t. alignment and proposed enhancements to ByteBuffer (in discussion on valhalla-dev). Language like that used in the VarHandle documentation is very precise and relevant to a JVM maintainer. But from a use perspective, this is terrible! Are we really expecting users to make sense of this API? I can tell you that as a user, I'd be looking for methods that look and act just like what I see on the existing Atomic*FieldUpdater style classes. Understanding the dissertation that is the description of VarHandles as a prerequisite for using this feature is pretty rough for users. In summary, this API might be expedient implementation-wise (I mean, it must be, right?), but in terms of style, future comprehensibility, ease of use, understandability, consistency with existing constructs, and (I can only imagine) implementation complexity, this monster cannot possibly be the best we can do. I am baffled as to how the original language syntax proposal of using some trick like "xx.volatile.imaginaryMethod()" plus maybe one or two new bytecodes was considered unacceptable; looking at this API, I know that none of the aforementioned metrics were considered as acceptance criteria. How did we get to this strange place? Is it just that MethodHandles are the latest shiny golden hammer? -- - DML From vitalyd at gmail.com Fri Jan 22 00:01:47 2016 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 21 Jan 2016 19:01:47 -0500 Subject: API review of VarHandles In-Reply-To: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> Message-ID: I think the get/setOpaque methods need a bit more explanation ("opaque" is an odd naming choice, IMO). Specifically, it says the operations are done in program order but have no effect on inter-thread ordering/visibility. Is this spec verbiage for a full compiler-only fence? On Thursday, January 21, 2016, Paul Sandoz wrote: > Hi > > This is a request to review the VarHandles API. The code reviews and > pushes will occur separately, and flow through the hs-comp repo, most > likely from the bottom up first with Unsafe changes. > > The specdiff can be found here: > > > http://cr.openjdk.java.net/~psandoz/jdk9/varhandles/specdiff/overview-summary.html > > (Note that specdiff renders some aspects of JavaDoc incorrectly, so just > ignore any such quirks.) > > A consensus on the set of access mode methods proposed by Doug was > previously discussed and reached. > > For the moment please ignore the following methods on MethodHandles: > byteArrayViewVarHandle; and byteBufferViewVarHandle. It is necessary to > revisit that functionality w.r.t. alignment and proposed enhancements to > ByteBuffer (in discussion on valhalla-dev). > > Paul. > > > -- Sent from my phone From brian.goetz at oracle.com Fri Jan 22 01:32:29 2016 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 21 Jan 2016 20:32:29 -0500 Subject: API review of VarHandles In-Reply-To: <56A16451.6010709@redhat.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A16451.6010709@redhat.com> Message-ID: <56A186AD.8090100@oracle.com> > I am baffled as to how the original language syntax proposal of using > some trick like "xx.volatile.imaginaryMethod()" plus maybe one or two > new bytecodes was considered unacceptable; looking at this API, I know > that none of the aforementioned metrics were considered as acceptance > criteria. How did we get to this strange place? The path was actually quite obvious. The first question was, "OK, if the language had this magic syntax, what bytecode would get generated? And clearly it couldn't be Unsafe. So either a pile of new bytecodes, or an API, was needed that the front-end compiler could generate against. Building APIs is generally preferable to new bytecodes, if an API can do the job, and it turned out that it could do it just as well. Once we had an API that met the requirements, it was pretty clear that language syntax was not only unnecessary, but likely undesirable -- these exotic access modes are power tools for power users (do you really want typical users to reason about mixed volatile and relaxed access to variables?), and they meet the needs of such users just fine (those users have been using Unsafe all along, and this is clearly better and safer.) So it was not the either-or of "either an API *or* magic language syntax", it was "either an API *or* syntax plus an API". At which point it was clear that the syntax didn't carry its own weight. I think this is a fine example of how the obvious idea was wrong, and that most of the value of the initial syntax idea was framing the problem space. I'm glad we saw fit to jettison that when it ceased to provide value. From peter.levart at gmail.com Fri Jan 22 08:09:19 2016 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 22 Jan 2016 09:09:19 +0100 Subject: API review of VarHandles In-Reply-To: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> Message-ID: <56A1E3AF.4010201@gmail.com> Hi Paul, I found a typo (which was probably caused by unconscious influence from jigsaw ;-). Search for "Java Memory Module"... Regards, Peter On 01/21/2016 11:42 PM, Paul Sandoz wrote: > Hi > > This is a request to review the VarHandles API. The code reviews and pushes will occur separately, and flow through the hs-comp repo, most likely from the bottom up first with Unsafe changes. > > The specdiff can be found here: > > http://cr.openjdk.java.net/~psandoz/jdk9/varhandles/specdiff/overview-summary.html > > (Note that specdiff renders some aspects of JavaDoc incorrectly, so just ignore any such quirks.) > > A consensus on the set of access mode methods proposed by Doug was previously discussed and reached. > > For the moment please ignore the following methods on MethodHandles: byteArrayViewVarHandle; and byteBufferViewVarHandle. It is necessary to revisit that functionality w.r.t. alignment and proposed enhancements to ByteBuffer (in discussion on valhalla-dev). > > Paul. > > From martijnverburg at gmail.com Fri Jan 22 09:58:14 2016 From: martijnverburg at gmail.com (Martijn Verburg) Date: Fri, 22 Jan 2016 09:58:14 +0000 Subject: Announcing: valhalla-spec-{observers,comments} In-Reply-To: References: <566F2195.2030100@oracle.com> <569419D3.80706@oracle.com> Message-ID: Apologies for the delay - the page has now been sent, I imagine it will be up early next week. Cheers, Martijn On 14 January 2016 at 00:45, Martijn Verburg wrote: > Hi Richard, > > I'll get an updated page sent to Brian shortly - it needs someone with > access to the webserver to change the pages there. > > Cheers, > Martijn > > On 14 January 2016 at 01:22, Richard Warburton < > richard.warburton at gmail.com> wrote: > >> Hi, >> >> As promised, we've now set up the additional mailing lists, >> > valhalla-spec-observers and valhalla-spec-comments. Here are the >> details. >> > >> > valhalla-spec-observers at openjdk.java.net is open to anyone to >> subscribe, >> > and subscribers may post to it. Traffic from valhalla-spec-experts is >> > automatically forwarded to valhalla-spec-observers. This list is >> primarily >> > intended so that people can follow the EG list traffic easily, and also >> for >> > people to have discussions within the -observers community. >> > >> > The primary means for the community to provide feedback into the design >> > and specification discussion is valhalla-spec-comments. Periodically, >> > comments from the -comments list will be "read into the record" on the >> > -experts list, where the EG can discuss the issue raised if it wants. >> But >> > with this access comes responsibility! Specifically, a post to >> -comments >> > should be well-reasoned, self-contained, and focused on a single issue >> (as >> > well as clear and polite, of course). Think of it as a suggestion box; >> we >> > guarantee that it will be seen by the EG, but if you want it to be taken >> > seriously by the EG, put some time and effort into making your point >> clear >> > argument. >> > >> > Note that -comments *cannot* be subscribed to; it can only be posted to. >> > (The reason for this is simple; having discussions on this list, which >> is >> > inevitable were it subscribable, would defeat the purpose.) The >> postings >> > are archived, and their content (if appropriate) will eventually show >> up on >> > the EG list. Postings are archived, and moderated to manage spam and >> > otherwise inappropriate content. >> > >> > On 12/14/2015 3:07 PM, Brian Goetz wrote: >> > >> >> It's time to take the next step in Valhalla's evolution. Accordingly, >> >> I've created the valhalla-spec-experts list, which will be the >> precursor to >> >> an eventual JCP Expert Group, and will operate under the JCP-friendly >> ToU. >> >> This list is publicly archived in the obvious place, and the initial >> >> membership includes both corporate participation (Oracle, IBM, Red Hat, >> >> Jetbrains, Google) and individuals from the OpenJDK community. >> >> >> >> As was the case when we formed the Lambda experts list, the role of >> >> valhalla-dev should now revert to its intended purpose, which is to >> discuss >> >> the *implementation* of Valhalla in OpenJDK. Examples of appropriate >> >> discussion topics for valhalla-dev include: >> >> >> >> - Bug reports >> >> - Code review comments >> >> - Test cases >> >> - Build or porting problems >> >> - Migration experiences for your own code >> >> >> >> What we are trying to avoid is: having design discussions play out >> >> concurrently in multiple places with differing sets of players. So >> please, >> >> we ask that you exercise restraint in commenting on active EG >> discussions >> >> -- as design decisions start to converge, we'll announce them here, >> and of >> >> course the primary place for implementation will remain the valhalla >> >> repositories. >> >> >> >> Please remember that EG discussions often happen at a different pace >> than >> >> in other open-source projects. Days can go by without a response, but >> this >> >> doesn't mean that the EG is not thinking about it, doesn't care, or is >> >> "stumped". Often it simply means the EG members are working privately >> (or >> >> with their teams) to figure out what their position is, and will >> respond >> >> when they've thought about it more. So, please let those discussions >> play >> >> out. >> > >> > >> Thanks for taking the time to setup these mailing lists. I think it would >> help people who are beginning to look at project Valhalla to have links to >> both spec-observers and spec-comments on the main valhalla project page. >> >> regards, >> >> Richard Warburton >> >> http://insightfullogic.com >> @RichardWarburto >> > > From srikanth.adayapalam at oracle.com Fri Jan 22 11:01:27 2016 From: srikanth.adayapalam at oracle.com (srikanth.adayapalam at oracle.com) Date: Fri, 22 Jan 2016 11:01:27 +0000 Subject: hg: valhalla/valhalla/langtools: Summary: Add an option -XDGenerateValueAsReference under which javac would treat value types as reference types for code generation purposes while treating them as value types for all other purposes (to allow for early user testing.) Message-ID: <201601221101.u0MB1SrV007752@aojmv0008.oracle.com> Changeset: 659f44b99b6c Author: sadayapalam Date: 2016-01-22 16:25 +0530 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/659f44b99b6c Summary: Add an option -XDGenerateValueAsReference under which javac would treat value types as reference types for code generation purposes while treating them as value types for all other purposes (to allow for early user testing.) Reviewed-by: mcimadamore ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Symbol.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Symtab.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Type.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Types.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Attr.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Check.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Resolve.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TypeEnter.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/ClassWriter.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Gen.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Items.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/main/JavaCompiler.java + test/tools/javac/valhalla/values/GetFieldTest.java + test/tools/javac/valhalla/values/NestingTest.java + test/tools/javac/valhalla/values/NestingTest2.java + test/tools/javac/valhalla/values/SimpleTest.java From vitalyd at gmail.com Fri Jan 22 11:15:28 2016 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 22 Jan 2016 06:15:28 -0500 Subject: API review of VarHandles In-Reply-To: <56A1FBA1.9010108@redhat.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A1FBA1.9010108@redhat.com> Message-ID: On Friday, January 22, 2016, Andrew Haley wrote: > On 22/01/16 00:01, Vitaly Davidovich wrote: > > I think the get/setOpaque methods need a bit more explanation ("opaque" > is > > an odd naming choice, IMO). Specifically, it says the operations are > done > > in program order but have no effect on inter-thread ordering/visibility. > > Is this spec verbiage for a full compiler-only fence? > > It's like C++ memory_order_relaxed, I guessed. I understand that but > not "opaque". I thought so too before reading opaque javadoc (no pun intended). C++ relaxed says nothing about order though, only atomicity. Opaque doesn't mention atomicity at all. I'm guessing opaque is compiler only fence because a method that a compiler doesn't inline (as an example) is considered "opaque" to the compiler and serves like a fence. Perhaps that's how opaque ended up here. At any rate, I think it's a poor name. > > Andrew. > -- Sent from my phone From vitalyd at gmail.com Fri Jan 22 12:08:40 2016 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 22 Jan 2016 07:08:40 -0500 Subject: API review of VarHandles In-Reply-To: <56A186AD.8090100@oracle.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A16451.6010709@redhat.com> <56A186AD.8090100@oracle.com> Message-ID: On Thursday, January 21, 2016, Brian Goetz wrote: > > I am baffled as to how the original language syntax proposal of using some >> trick like "xx.volatile.imaginaryMethod()" plus maybe one or two new >> bytecodes was considered unacceptable; looking at this API, I know that >> none of the aforementioned metrics were considered as acceptance criteria. >> How did we get to this strange place? >> > > The path was actually quite obvious. > > The first question was, "OK, if the language had this magic syntax, what > bytecode would get generated? And clearly it couldn't be Unsafe. So > either a pile of new bytecodes, or an API, was needed that the front-end > compiler could generate against. > > Building APIs is generally preferable to new bytecodes, if an API can do > the job, and it turned out that it could do it just as well. Once we had an > API that met the requirements, it was pretty clear that language syntax was > not only unnecessary, but likely undesirable -- these exotic access modes > are power tools for power users (do you really want typical users to reason > about mixed volatile and relaxed access to variables?), and they meet the > needs of such users just fine (those users have been using Unsafe all > along, and this is clearly better and safer.) The VarHandle API isn't ergonomic like Unsafe; this being for power users is irrelevant to the ergonomics of the API. Instead, it's fairly verbose with setup ceremony. Now, I personally don't mind that too much and more interested in the features it provides but I'm not surprised by David's reaction. People already provide nicer APIs on top of Unsafe in their own projects, and I suspect this will be even more so with VH. > > So it was not the either-or of "either an API *or* magic language syntax", > it was "either an API *or* syntax plus an API". At which point it was > clear that the syntax didn't carry its own weight. > > I think this is a fine example of how the obvious idea was wrong, and that > most of the value of the initial syntax idea was framing the problem > space. I'm glad we saw fit to jettison that when it ceased to provide > value. > > > -- Sent from my phone From maurizio.cimadamore at oracle.com Fri Jan 22 12:57:28 2016 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Fri, 22 Jan 2016 12:57:28 +0000 Subject: hg: valhalla/valhalla/jdk: Convert back lambdas into method references in DistinctOps.java Message-ID: <201601221257.u0MCvS7l022164@aojmv0008.oracle.com> Changeset: 4431b5694b25 Author: mcimadamore Date: 2016-01-22 12:57 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/4431b5694b25 Convert back lambdas into method references in DistinctOps.java ! src/java.base/share/classes/java/anyutil/stream/DistinctOps.java From aleksey.shipilev at oracle.com Fri Jan 22 13:06:09 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Fri, 22 Jan 2016 16:06:09 +0300 Subject: API review of VarHandles In-Reply-To: References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A16451.6010709@redhat.com> <56A186AD.8090100@oracle.com> Message-ID: <56A22941.1050702@oracle.com> On 01/22/2016 03:08 PM, Vitaly Davidovich wrote: > The VarHandle API isn't ergonomic like Unsafe; this being for power users > is irrelevant to the ergonomics of the API. Instead, it's fairly verbose > with setup ceremony. Now, I personally don't mind that too much and more > interested in the features it provides but I'm not surprised by David's > reaction. Um, I don't get the "fairly verbose" and "setup ceremony" parts, especially in comparison with Unsafe. Unsafe also comes with a setup ceremony, don't you think? See: public class C { static class CU { static final Unsafe U; static final long OFFSET; static { try { Field f = Unsafe.class.getDeclaredField("theUnsafe"); f.setAccessible(true); U = (Unsafe) f.get(null); OFFSET = U.objectFieldOffset(CU.class.getDeclaredField("x")); } catch (ReflectiveOperationException eh) { throw new IllegalStateException(eh); } } private int x; public boolean doCas(int expected, int value) { return U.compareAndSwapInt(this, OFFSET, expected, value); } } static class VU { static final VarHandle VH; static { try { VH = MethodHandles.lookup().findVarHandle( VU.class, "x", int.class); } catch (ReflectiveOperationException eh) { throw new IllegalStateException(eh); } } private int x; public boolean doCas(int expected, int value) { return VH.compareAndSet(this, expected, value); } } } Even if you pull off Unsafe lookup into separate "holder" class, you'd still need to lookup OFFSET. Granted, VH lookup requires learning new stuff, but so does Unsafe, A*FU, Reflection, or any other special API/syntax. The mind trick is that you already *know* all those APIs, and so they appear conceptually simple to you. Once you internalize that VarHandle is a "handle", which you have to acquire before use, it clicks. > People already provide nicer APIs on top of Unsafe in their own projects, > and I suspect this will be even more so with VH. Yes, but lacking the syntax support, that's the best you can do: static class MyAwesomeWrapper { public static VarHandle iAmPrettySureThatFieldExists( MethodHandles.Lookup lookup, Class recv, String name, Class type) { try { return lookup.findVarHandle(recv, name, type); } catch (ReflectiveOperationException eh) { throw new IllegalStateException(eh); } } } static class VUH { static final VarHandle VH = MyAwesomeWrapper.iAmPrettySureThatFieldExists( MethodHandles.lookup(), VUH.class, "x", int.class); private int x; public boolean doCas(int expected, int value) { return VH.compareAndSet(this, expected, value); } } ...which is not that bad? I am not discounting David's comments about the conceptual complexity of the documentation. It looks like a normative text that 99% users would not read, but stroll straight into code examples. -Aleksey From vitalyd at gmail.com Fri Jan 22 13:09:29 2016 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 22 Jan 2016 08:09:29 -0500 Subject: API review of VarHandles In-Reply-To: <56A2266D.5060202@redhat.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A1FBA1.9010108@redhat.com> <56A2266D.5060202@redhat.com> Message-ID: On Friday, January 22, 2016, Andrew Haley wrote: > On 01/22/2016 11:15 AM, Vitaly Davidovich wrote: > > > I'm guessing opaque is compiler only fence because a method that a > > compiler doesn't inline (as an example) is considered "opaque" to > > the compiler and serves like a fence. Perhaps that's how opaque > > ended up here. At any rate, I think it's a poor name. > > Humm. I guess so, but I'm not sure why you'd want a compiler fence in > Java. It's not like C where you can access memory-mapped I/O. I > suppose it disallows roach-motel code motion. Well that's part of my confusion on what this is :). C++ relaxed makes sense to me - don't care about anything other than atomicity, which could be useful for Java too so that one could reliably/portably do atomic stores to, e.g, longs and have that compile to plain mov on 64 bit machines and something possibly more expensive on 32 bit. Maybe that's what this actually is but "program order" is too vague here. > > Andrew. > -- Sent from my phone From david.lloyd at redhat.com Fri Jan 22 13:11:01 2016 From: david.lloyd at redhat.com (David M. Lloyd) Date: Fri, 22 Jan 2016 07:11:01 -0600 Subject: API review of VarHandles In-Reply-To: <56A186AD.8090100@oracle.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A16451.6010709@redhat.com> <56A186AD.8090100@oracle.com> Message-ID: <56A22A65.4040200@redhat.com> On 01/21/2016 07:32 PM, Brian Goetz wrote: > >> I am baffled as to how the original language syntax proposal of using >> some trick like "xx.volatile.imaginaryMethod()" plus maybe one or two >> new bytecodes was considered unacceptable; looking at this API, I know >> that none of the aforementioned metrics were considered as acceptance >> criteria. How did we get to this strange place? > > The path was actually quite obvious. > > The first question was, "OK, if the language had this magic syntax, what > bytecode would get generated? And clearly it couldn't be Unsafe. So > either a pile of new bytecodes, or an API, was needed that the front-end > compiler could generate against. > > Building APIs is generally preferable to new bytecodes, if an API can do > the job, and it turned out that it could do it just as well. Once we had > an API that met the requirements, it was pretty clear that language > syntax was not only unnecessary, but likely undesirable -- these exotic > access modes are power tools for power users (do you really want typical > users to reason about mixed volatile and relaxed access to variables?), > and they meet the needs of such users just fine (those users have been > using Unsafe all along, and this is clearly better and safer.) > > So it was not the either-or of "either an API *or* magic language > syntax", it was "either an API *or* syntax plus an API". At which point > it was clear that the syntax didn't carry its own weight. > > I think this is a fine example of how the obvious idea was wrong, and > that most of the value of the initial syntax idea was framing the > problem space. I'm glad we saw fit to jettison that when it ceased to > provide value. I will contest the "APIs is generally preferable to new bytecodes, if an API can do the job" tack because it clearly doesn't track logically: most of the bytecodes in the JVM today could have been intrinsic API methods instead, e.g. Integer.add(int, int) and that sort of thing, but they clearly benefit from being bytecodes just from a simplicity standpoint. Having new bytecodes would certainly be simpler for javac; it would be simpler for the user also, especially the user of class generators and transformers (who by this point will now have to do some pretty bananas things to transform field references); I can only speculate that it would also be simpler for the interpreter/JIT itself as well because really you're just generating variations on getfield, *aload, etc. But that is all besides the point. Even the API itself is really heavy and weird. Experts need simple APIs more than anyone - especially those writing complex concurrent algorithms. They're the ones who aren't content to copy and paste blobs off of StackOverflow; rather they need to be able to prove that their program is correct, and the more complexity there is, the more difficult a task that becomes. But even the premise that this API is experts-only is self-fulfilling: atomics and fences are quite useful to less advanced users too, especially simple get-and-set and CAS operations. But there is little chance that this API is going to be used (or used correctly) by such users. I have no problem with the methods present on the final product, in terms of the types of fences and operations. But the setup procedures and concepts are far more complex than necessary, and I don't see this being useful to most users in its current shape as most people will flee from it. -- - DML From vitalyd at gmail.com Fri Jan 22 13:22:04 2016 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 22 Jan 2016 08:22:04 -0500 Subject: API review of VarHandles In-Reply-To: <56A22941.1050702@oracle.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A16451.6010709@redhat.com> <56A186AD.8090100@oracle.com> <56A22941.1050702@oracle.com> Message-ID: On Friday, January 22, 2016, Aleksey Shipilev wrote: > On 01/22/2016 03:08 PM, Vitaly Davidovich wrote: > > The VarHandle API isn't ergonomic like Unsafe; this being for power users > > is irrelevant to the ergonomics of the API. Instead, it's fairly verbose > > with setup ceremony. Now, I personally don't mind that too much and more > > interested in the features it provides but I'm not surprised by David's > > reaction. > > Um, I don't get the "fairly verbose" and "setup ceremony" parts, > especially in comparison with Unsafe. Unsafe also comes with a setup > ceremony, don't you think? Yes unsafe also has some ceremony, but it seems like VH is a bit more so. As I mentioned, I'm not too bothered by that aspect given wrappers will be built anyway (just like Unsafe). But, if this were in the language it would likely be more ergonomic. Not saying ergonomics trumps everything else, but it's something David alluded to (lang vs API). > > See: > > public class C { > static class CU { > static final Unsafe U; > static final long OFFSET; > > static { > try { > Field f = Unsafe.class.getDeclaredField("theUnsafe"); > f.setAccessible(true); > U = (Unsafe) f.get(null); > OFFSET = U.objectFieldOffset(CU.class.getDeclaredField("x")); > } catch (ReflectiveOperationException eh) { > throw new IllegalStateException(eh); > } > } > > private int x; > > public boolean doCas(int expected, int value) { > return U.compareAndSwapInt(this, OFFSET, expected, value); > } > } > > static class VU { > static final VarHandle VH; > > static { > try { > VH = MethodHandles.lookup().findVarHandle( > VU.class, "x", int.class); > } catch (ReflectiveOperationException eh) { > throw new IllegalStateException(eh); > } > } > > private int x; > > public boolean doCas(int expected, int value) { > return VH.compareAndSet(this, expected, value); > } > } > } > > Even if you pull off Unsafe lookup into separate "holder" class, you'd > still need to lookup OFFSET. > > Granted, VH lookup requires learning new stuff, but so does Unsafe, > A*FU, Reflection, or any other special API/syntax. The mind trick is > that you already *know* all those APIs, and so they appear conceptually > simple to you. Once you internalize that VarHandle is a "handle", which > you have to acquire before use, it clicks. I get the handle part and don't have a problem with the concept. I do agree that one will get used to it. Again, for me, features matter more than syntax (for the most part). > > > > People already provide nicer APIs on top of Unsafe in their own projects, > > and I suspect this will be even more so with VH. > > Yes, but lacking the syntax support, that's the best you can do: > > static class MyAwesomeWrapper { > public static VarHandle iAmPrettySureThatFieldExists( > MethodHandles.Lookup lookup, Class recv, String name, > Class type) { > try { > return lookup.findVarHandle(recv, name, type); > } catch (ReflectiveOperationException eh) { > throw new IllegalStateException(eh); > } > } > } > > static class VUH { > static final VarHandle VH = > MyAwesomeWrapper.iAmPrettySureThatFieldExists( > MethodHandles.lookup(), > VUH.class, "x", int.class); > > private int x; > > public boolean doCas(int expected, int value) { > return VH.compareAndSet(this, expected, value); > } > } > > ...which is not that bad? > > I am not discounting David's comments about the conceptual complexity of > the documentation. It looks like a normative text that 99% users would > not read, but stroll straight into code examples. I'm actually not bothered by the spec-like docs; in fact I like the extra detail in the same place as the API. Perhaps it can be rearranged so the impl details are not the first thing a reader sees. This is, by definition, a low level API with misuse leading to very subtle bugs - over documenting is much better than under documenting. > > -Aleksey > > -- Sent from my phone From dl at cs.oswego.edu Fri Jan 22 13:23:53 2016 From: dl at cs.oswego.edu (Doug Lea) Date: Fri, 22 Jan 2016 08:23:53 -0500 Subject: API review of VarHandles In-Reply-To: <56A1FBA1.9010108@redhat.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A1FBA1.9010108@redhat.com> Message-ID: <56A22D69.7040404@cs.oswego.edu> On 01/22/2016 04:51 AM, Andrew Haley wrote: > On 22/01/16 00:01, Vitaly Davidovich wrote: >> I think the get/setOpaque methods need a bit more explanation ("opaque" is >> an odd naming choice, IMO). Specifically, it says the operations are done >> in program order but have no effect on inter-thread ordering/visibility. >> Is this spec verbiage for a full compiler-only fence? Basically yes. It would be the same as C "volatile" (not C++/C11 "atomic"), if C volatile had a memory model spec. But in any case we can't use "volatile" for this in Java because it already means something else. > > It's like C++ memory_order_relaxed, I guessed. I understand that but > not "opaque". > The new name is required because opaque is only "like" C++ memory_order_relaxed. See the jmm-dev list discussions last year for details, but annoyingly, Java normal variables and C++ memory_order_relaxed are not quite the same, and neither are the same as C-volatile/opaque. -Doug From vitalyd at gmail.com Fri Jan 22 13:29:09 2016 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 22 Jan 2016 08:29:09 -0500 Subject: API review of VarHandles In-Reply-To: <56A22A65.4040200@redhat.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A16451.6010709@redhat.com> <56A186AD.8090100@oracle.com> <56A22A65.4040200@redhat.com> Message-ID: On Friday, January 22, 2016, David M. Lloyd wrote: > On 01/21/2016 07:32 PM, Brian Goetz wrote: > >> >> I am baffled as to how the original language syntax proposal of using >>> some trick like "xx.volatile.imaginaryMethod()" plus maybe one or two >>> new bytecodes was considered unacceptable; looking at this API, I know >>> that none of the aforementioned metrics were considered as acceptance >>> criteria. How did we get to this strange place? >>> >> >> The path was actually quite obvious. >> >> The first question was, "OK, if the language had this magic syntax, what >> bytecode would get generated? And clearly it couldn't be Unsafe. So >> either a pile of new bytecodes, or an API, was needed that the front-end >> compiler could generate against. >> >> Building APIs is generally preferable to new bytecodes, if an API can do >> the job, and it turned out that it could do it just as well. Once we had >> an API that met the requirements, it was pretty clear that language >> syntax was not only unnecessary, but likely undesirable -- these exotic >> access modes are power tools for power users (do you really want typical >> users to reason about mixed volatile and relaxed access to variables?), >> and they meet the needs of such users just fine (those users have been >> using Unsafe all along, and this is clearly better and safer.) >> >> So it was not the either-or of "either an API *or* magic language >> syntax", it was "either an API *or* syntax plus an API". At which point >> it was clear that the syntax didn't carry its own weight. >> >> I think this is a fine example of how the obvious idea was wrong, and >> that most of the value of the initial syntax idea was framing the >> problem space. I'm glad we saw fit to jettison that when it ceased to >> provide value. >> > > I will contest the "APIs is generally preferable to new bytecodes, if an > API can do the job" tack because it clearly doesn't track logically: most > of the bytecodes in the JVM today could have been intrinsic API methods > instead, e.g. Integer.add(int, int) and that sort of thing, but they > clearly benefit from being bytecodes just from a simplicity standpoint. > Having new bytecodes would certainly be simpler for javac; it would be > simpler for the user also, especially the user of class generators and > transformers (who by this point will now have to do some pretty bananas > things to transform field references); I can only speculate that it would > also be simpler for the interpreter/JIT itself as well because really > you're just generating variations on getfield, *aload, etc. But that is > all besides the point. > > Even the API itself is really heavy and weird. Experts need simple APIs > more than anyone - especially those writing complex concurrent algorithms. > They're the ones who aren't content to copy and paste blobs off of > StackOverflow; rather they need to be able to prove that their program is > correct, and the more complexity there is, the more difficult a task that > becomes. But even the premise that this API is experts-only is > self-fulfilling: atomics and fences are quite useful to less advanced users > too, especially simple get-and-set and CAS operations. But there is little > chance that this API is going to be used (or used correctly) by such users. Experts need control more than anything else. This is not to say they'll be happy with a subpar API but control typically means lots of details so there's a limit on how abstracted the API can be. As mentioned, given the API provides low level control one can build a higher level one for their own needs. The topic of ordering is complex by its nature, there's a danger in hiding details in more abstract API. > > I have no problem with the methods present on the final product, in terms > of the types of fences and operations. But the setup procedures and > concepts are far more complex than necessary, and I don't see this being > useful to most users in its current shape as most people will flee from it. > > -- > - DML > -- Sent from my phone From david.lloyd at redhat.com Fri Jan 22 13:34:53 2016 From: david.lloyd at redhat.com (David M. Lloyd) Date: Fri, 22 Jan 2016 07:34:53 -0600 Subject: API review of VarHandles In-Reply-To: References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A16451.6010709@redhat.com> <56A186AD.8090100@oracle.com> <56A22A65.4040200@redhat.com> Message-ID: <56A22FFD.3090104@redhat.com> On 01/22/2016 07:29 AM, Vitaly Davidovich wrote: > Experts need control more than anything else. This is not to say they'll > be happy with a subpar API but control typically means lots of details so > there's a limit on how abstracted the API can be. As mentioned, given the > API provides low level control one can build a higher level one for their > own needs. The topic of ordering is complex by its nature, there's a > danger in hiding details in more abstract API. I understand that, and that is reasonable. However, will it be possible to build higher level APIs on this that do not suffer the same costs that make the existing Atomic* classes unattractive? -- - DML From vitalyd at gmail.com Fri Jan 22 13:40:51 2016 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 22 Jan 2016 08:40:51 -0500 Subject: API review of VarHandles In-Reply-To: <56A22FFD.3090104@redhat.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A16451.6010709@redhat.com> <56A186AD.8090100@oracle.com> <56A22A65.4040200@redhat.com> <56A22FFD.3090104@redhat.com> Message-ID: Which existing Atomic* classes? I take it you mean the field updaters and not AtomicInteger and friends. The biggest issue with field updaters is their performance, for me. I take it you mean something else by "costs" though? Having to specify the target (field) as a string is annoying, and I do wish there was a way javac could assist there. VH is no worse in this respect, but also no better. On Fri, Jan 22, 2016 at 8:34 AM, David M. Lloyd wrote: > On 01/22/2016 07:29 AM, Vitaly Davidovich wrote: > >> Experts need control more than anything else. This is not to say they'll >> be happy with a subpar API but control typically means lots of details so >> there's a limit on how abstracted the API can be. As mentioned, given the >> API provides low level control one can build a higher level one for their >> own needs. The topic of ordering is complex by its nature, there's a >> danger in hiding details in more abstract API. >> > > I understand that, and that is reasonable. However, will it be possible > to build higher level APIs on this that do not suffer the same costs that > make the existing Atomic* classes unattractive? > > -- > - DML > From david.lloyd at redhat.com Fri Jan 22 13:54:26 2016 From: david.lloyd at redhat.com (David M. Lloyd) Date: Fri, 22 Jan 2016 07:54:26 -0600 Subject: API review of VarHandles In-Reply-To: References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A16451.6010709@redhat.com> <56A186AD.8090100@oracle.com> <56A22A65.4040200@redhat.com> <56A22FFD.3090104@redhat.com> Message-ID: <56A23492.4030006@redhat.com> The costs are both performance and memory overhead. The Atomic*FieldUpdaters are good for memory usage and reasonably usable, but suffer from performance problems and generics issues. The "regular" Atomic* classes have good usability (including with generic types) and generally very good performance on the actual operations, but generally unacceptable memory overhead. The only way to do atomic array element access is via the Atomic*Array though, which has pretty good performance but again has memory overhead that can be unacceptable. Overall this whole thing is a lot of gymnastics for something that's basically a fancy way of accessing a plain field or array element - one of the most basic concepts in Java. On 01/22/2016 07:40 AM, Vitaly Davidovich wrote: > Which existing Atomic* classes? I take it you mean the field updaters and > not AtomicInteger and friends. The biggest issue with field updaters is > their performance, for me. I take it you mean something else by "costs" > though? Having to specify the target (field) as a string is annoying, and I > do wish there was a way javac could assist there. VH is no worse in this > respect, but also no better. > > On Fri, Jan 22, 2016 at 8:34 AM, David M. Lloyd > wrote: > >> On 01/22/2016 07:29 AM, Vitaly Davidovich wrote: >> >>> Experts need control more than anything else. This is not to say they'll >>> be happy with a subpar API but control typically means lots of details so >>> there's a limit on how abstracted the API can be. As mentioned, given the >>> API provides low level control one can build a higher level one for their >>> own needs. The topic of ordering is complex by its nature, there's a >>> danger in hiding details in more abstract API. >>> >> >> I understand that, and that is reasonable. However, will it be possible >> to build higher level APIs on this that do not suffer the same costs that >> make the existing Atomic* classes unattractive? >> >> -- >> - DML >> > -- - DML From aleksey.shipilev at oracle.com Fri Jan 22 13:57:13 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Fri, 22 Jan 2016 16:57:13 +0300 Subject: API review of VarHandles In-Reply-To: <56A23492.4030006@redhat.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A16451.6010709@redhat.com> <56A186AD.8090100@oracle.com> <56A22A65.4040200@redhat.com> <56A22FFD.3090104@redhat.com> <56A23492.4030006@redhat.com> Message-ID: <56A23539.4000702@oracle.com> On 01/22/2016 04:54 PM, David M. Lloyd wrote: > The costs are both performance and memory overhead. The > Atomic*FieldUpdaters are good for memory usage and reasonably usable, > but suffer from performance problems and generics issues. Hopefully, not anymore: http://shipilev.net/blog/2015/faster-atomic-fu/ BTW, those are the same techniques we use to compile VarHandles down to the naked memory accesses. Cheers, -Aleksey From vitalyd at gmail.com Fri Jan 22 13:58:48 2016 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 22 Jan 2016 08:58:48 -0500 Subject: API review of VarHandles In-Reply-To: <56A23492.4030006@redhat.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A16451.6010709@redhat.com> <56A186AD.8090100@oracle.com> <56A22A65.4040200@redhat.com> <56A22FFD.3090104@redhat.com> <56A23492.4030006@redhat.com> Message-ID: So in this regard, VH ought to be better since it will have performance like Unsafe (rather than field updaters - is that accurate by the way? Is VH on par with Unsafe for issuing atomic ops?) and not have the memory bloat of the instance atomic classes. Generics is a non-issue for VH. I do agree with the "this is all just hoop jumping to issue basic memory ops", but that's more a reflection on Java (memory footprint of atomic instance classes, generics issues) rather than VH. On Fri, Jan 22, 2016 at 8:54 AM, David M. Lloyd wrote: > The costs are both performance and memory overhead. The > Atomic*FieldUpdaters are good for memory usage and reasonably usable, but > suffer from performance problems and generics issues. The "regular" > Atomic* classes have good usability (including with generic types) and > generally very good performance on the actual operations, but generally > unacceptable memory overhead. The only way to do atomic array element > access is via the Atomic*Array though, which has pretty good performance > but again has memory overhead that can be unacceptable. > > Overall this whole thing is a lot of gymnastics for something that's > basically a fancy way of accessing a plain field or array element - one of > the most basic concepts in Java. > > > On 01/22/2016 07:40 AM, Vitaly Davidovich wrote: > >> Which existing Atomic* classes? I take it you mean the field updaters and >> not AtomicInteger and friends. The biggest issue with field updaters is >> their performance, for me. I take it you mean something else by "costs" >> though? Having to specify the target (field) as a string is annoying, and >> I >> do wish there was a way javac could assist there. VH is no worse in this >> respect, but also no better. >> >> On Fri, Jan 22, 2016 at 8:34 AM, David M. Lloyd >> wrote: >> >> On 01/22/2016 07:29 AM, Vitaly Davidovich wrote: >>> >>> Experts need control more than anything else. This is not to say they'll >>>> be happy with a subpar API but control typically means lots of details >>>> so >>>> there's a limit on how abstracted the API can be. As mentioned, given >>>> the >>>> API provides low level control one can build a higher level one for >>>> their >>>> own needs. The topic of ordering is complex by its nature, there's a >>>> danger in hiding details in more abstract API. >>>> >>>> >>> I understand that, and that is reasonable. However, will it be possible >>> to build higher level APIs on this that do not suffer the same costs that >>> make the existing Atomic* classes unattractive? >>> >>> -- >>> - DML >>> >>> >> > -- > - DML > From david.lloyd at redhat.com Fri Jan 22 14:04:23 2016 From: david.lloyd at redhat.com (David M. Lloyd) Date: Fri, 22 Jan 2016 08:04:23 -0600 Subject: API review of VarHandles In-Reply-To: <56A23539.4000702@oracle.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A16451.6010709@redhat.com> <56A186AD.8090100@oracle.com> <56A22A65.4040200@redhat.com> <56A22FFD.3090104@redhat.com> <56A23492.4030006@redhat.com> <56A23539.4000702@oracle.com> Message-ID: <56A236E7.8070601@redhat.com> On 01/22/2016 07:57 AM, Aleksey Shipilev wrote: > On 01/22/2016 04:54 PM, David M. Lloyd wrote: >> The costs are both performance and memory overhead. The >> Atomic*FieldUpdaters are good for memory usage and reasonably usable, >> but suffer from performance problems and generics issues. > > Hopefully, not anymore: > http://shipilev.net/blog/2015/faster-atomic-fu/ > > BTW, those are the same techniques we use to compile VarHandles down to > the naked memory accesses. Great, nice work, and nice article! -- - DML From vitalyd at gmail.com Fri Jan 22 14:14:43 2016 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 22 Jan 2016 09:14:43 -0500 Subject: API review of VarHandles In-Reply-To: <56A236E7.8070601@redhat.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A16451.6010709@redhat.com> <56A186AD.8090100@oracle.com> <56A22A65.4040200@redhat.com> <56A22FFD.3090104@redhat.com> <56A23492.4030006@redhat.com> <56A23539.4000702@oracle.com> <56A236E7.8070601@redhat.com> Message-ID: +1, good stuff Aleksey (I had actually seen chatter about this change on one of these openjdk lists, but people have already moved away from the field updaters), except every time I think about final instance fields not being treated as constants I get sad :). The other wrinkle is the "am I lucky today to get proper inlining" in order to peel away costs. I'm not sure whether VH machinery will make this work reliably, irrespective of callsite frequency, inlining depth, etc (that's another advantage of bytecode -- there's no need to crack method chains). On Fri, Jan 22, 2016 at 9:04 AM, David M. Lloyd wrote: > On 01/22/2016 07:57 AM, Aleksey Shipilev wrote: > >> On 01/22/2016 04:54 PM, David M. Lloyd wrote: >> >>> The costs are both performance and memory overhead. The >>> Atomic*FieldUpdaters are good for memory usage and reasonably usable, >>> but suffer from performance problems and generics issues. >>> >> >> Hopefully, not anymore: >> http://shipilev.net/blog/2015/faster-atomic-fu/ >> >> BTW, those are the same techniques we use to compile VarHandles down to >> the naked memory accesses. >> > > Great, nice work, and nice article! > -- > - DML > From aph at redhat.com Fri Jan 22 09:51:29 2016 From: aph at redhat.com (Andrew Haley) Date: Fri, 22 Jan 2016 09:51:29 +0000 Subject: API review of VarHandles In-Reply-To: References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> Message-ID: <56A1FBA1.9010108@redhat.com> On 22/01/16 00:01, Vitaly Davidovich wrote: > I think the get/setOpaque methods need a bit more explanation ("opaque" is > an odd naming choice, IMO). Specifically, it says the operations are done > in program order but have no effect on inter-thread ordering/visibility. > Is this spec verbiage for a full compiler-only fence? It's like C++ memory_order_relaxed, I guessed. I understand that but not "opaque". Andrew. From aph at redhat.com Fri Jan 22 12:54:05 2016 From: aph at redhat.com (Andrew Haley) Date: Fri, 22 Jan 2016 12:54:05 +0000 Subject: API review of VarHandles In-Reply-To: References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A1FBA1.9010108@redhat.com> Message-ID: <56A2266D.5060202@redhat.com> On 01/22/2016 11:15 AM, Vitaly Davidovich wrote: > I'm guessing opaque is compiler only fence because a method that a > compiler doesn't inline (as an example) is considered "opaque" to > the compiler and serves like a fence. Perhaps that's how opaque > ended up here. At any rate, I think it's a poor name. Humm. I guess so, but I'm not sure why you'd want a compiler fence in Java. It's not like C where you can access memory-mapped I/O. I suppose it disallows roach-motel code motion. Andrew. From brian.goetz at oracle.com Fri Jan 22 14:46:08 2016 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 22 Jan 2016 09:46:08 -0500 Subject: API review of VarHandles In-Reply-To: References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A16451.6010709@redhat.com> <56A186AD.8090100@oracle.com> Message-ID: <56A240B0.5040108@oracle.com> > The VarHandle API isn't ergonomic like Unsafe; this being for power > users is irrelevant to the ergonomics of the API. Instead, it's fairly > verbose with setup ceremony. Now, I personally don't mind that too > much and more interested in the features it provides but I'm not > surprised by David's reaction. > > People already provide nicer APIs on top of Unsafe in their own > projects, and I suspect this will be even more so with VH. And that's absolutely fine with us. We wanted to provide a secure, safe, fast substrate that code generators and brave library writers alike will use; if people want to wrap it, we view that as success, not failure. From vitalyd at gmail.com Fri Jan 22 14:52:26 2016 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 22 Jan 2016 09:52:26 -0500 Subject: API review of VarHandles In-Reply-To: <56A240B0.5040108@oracle.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A16451.6010709@redhat.com> <56A186AD.8090100@oracle.com> <56A240B0.5040108@oracle.com> Message-ID: Agreed; it's a building block, not an assembled piece. On Friday, January 22, 2016, Brian Goetz wrote: > > The VarHandle API isn't ergonomic like Unsafe; this being for power users >> is irrelevant to the ergonomics of the API. Instead, it's fairly verbose >> with setup ceremony. Now, I personally don't mind that too much and more >> interested in the features it provides but I'm not surprised by David's >> reaction. >> >> People already provide nicer APIs on top of Unsafe in their own projects, >> and I suspect this will be even more so with VH. >> > > And that's absolutely fine with us. We wanted to provide a secure, safe, > fast substrate that code generators and brave library writers alike will > use; if people want to wrap it, we view that as success, not failure. > > > -- Sent from my phone From maurizio.cimadamore at oracle.com Fri Jan 22 16:32:45 2016 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Fri, 22 Jan 2016 16:32:45 +0000 Subject: hg: valhalla/valhalla/langtools: Enhancement: Refine ParameterizedType_info to include enclosing class info and value vs. class flag Message-ID: <201601221632.u0MGWjM2014677@aojmv0008.oracle.com> Changeset: b345d7339ab3 Author: mcimadamore Date: 2016-01-22 16:32 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/b345d7339ab3 Enhancement: Refine ParameterizedType_info to include enclosing class info and value vs. class flag ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Pool.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/PoolReader.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/PoolWriter.java ! src/jdk.jdeps/share/classes/com/sun/tools/classfile/ClassTranslator.java ! src/jdk.jdeps/share/classes/com/sun/tools/classfile/ConstantPool.java ! src/jdk.jdeps/share/classes/com/sun/tools/javap/ConstantWriter.java From maurizio.cimadamore at oracle.com Fri Jan 22 16:33:31 2016 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Fri, 22 Jan 2016 16:33:31 +0000 Subject: hg: valhalla/valhalla/jdk: Enhancement: Refine ParameterizedType_info to include enclosing class info and $ Message-ID: <201601221633.u0MGXWos014902@aojmv0008.oracle.com> Changeset: aa8050cd1f1a Author: mcimadamore Date: 2016-01-22 16:33 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/aa8050cd1f1a Enhancement: Refine ParameterizedType_info to include enclosing class info and $ ! src/java.base/share/classes/valhalla/model3/Model3Converter.java ! src/java.base/share/classes/valhalla/model3/classfile/ClassTranslator.java ! src/java.base/share/classes/valhalla/model3/classfile/ConstantPool.java From maurizio.cimadamore at oracle.com Fri Jan 22 17:14:11 2016 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Fri, 22 Jan 2016 17:14:11 +0000 Subject: hg: valhalla/valhalla/langtools: Fix: adjust offset for ParameterizedType_info in PoolReader Message-ID: <201601221714.u0MHEB5g024245@aojmv0008.oracle.com> Changeset: 7943e5b37578 Author: mcimadamore Date: 2016-01-22 17:13 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/7943e5b37578 Fix: adjust offset for ParameterizedType_info in PoolReader ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/PoolReader.java From maurizio.cimadamore at oracle.com Fri Jan 22 17:19:30 2016 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Fri, 22 Jan 2016 17:19:30 +0000 Subject: hg: valhalla/valhalla/langtools: Fix: adjust offset for ParameterizedType_info in PoolReader (take two) Message-ID: <201601221719.u0MHJU6n026680@aojmv0008.oracle.com> Changeset: 7a9f046780db Author: mcimadamore Date: 2016-01-22 17:19 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/7a9f046780db Fix: adjust offset for ParameterizedType_info in PoolReader (take two) ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/PoolReader.java From jeremymanson at google.com Fri Jan 22 18:03:10 2016 From: jeremymanson at google.com (Jeremy Manson) Date: Fri, 22 Jan 2016 10:03:10 -0800 Subject: API review of VarHandles In-Reply-To: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> Message-ID: Couple of thoughts: > The Java Language Specification permits operations to be executed in > orders different than are apparent in program source code, subject to > constraints arising, for example, from the use of locks, volatile fields or > VarHandles. The static methods, fullFence > > , acquireFence > > , releaseFence > > , loadLoadFence > > andstoreStoreFence > , > can also be used to impose constraints. Their specifications are phrased in > terms of the lack of "reorderings" -- observable ordering effects that > might otherwise occur if the fence were not present. > There is no notion of reordering (per se) in the JLS; the fact that reorderings can occur is just implied by the way programs can be observed to behave. So, these fence operations don't map to anything non-implementation dependent. I don't think it would be impossible to fix the spec to define something that works the way you want (similar to the final field semantics), but it isn't in there already. You might want to call that out (either by saying this behavior is best effort implementation-dependent or by fixing the spec). If a VarHandle references a read-only variable (for example a final field) > then write, atomic update and numeric atomic update access modes are not > supported and corresponding methods throw UnsupportedOperationException. Are you sure you want to limit it in this way? There are an awful lot of calls to setAccessible in the world of reflection. And writing to final fields *does* map to something sensible in the spec. In fact, it would be great to have something that wasn't quite such a blunt instrument as setAccessible. getVolatile > public final Object > > getVolatile(Object > > ... args) > Returns the value of a variable, with memory semantics of reading a > volatile variable. Reading *which* volatile variable? You probably mean that all getVolatiles and setVolatiles provide semantics that behave as if the variable being written / read was declared volatile in the first place, but it is worth calling out. Nits: As is usual with virtual methods, source-level calls to access mode methods > compile to an invokevirtual instruction. More unusually, the compiler > must record the actual argument types, and may not perform method > invocation conversions on the arguments. Instead, it must push them on the > stack according to their own unconverted types. The VarHandle object itself > is pushed on the stack before the arguments. The compiler then calls the > VarHandle with a symbolic type descriptor which describes the argument and > return types. This is somewhat oddly worded. The compiler doesn't push arguments on a stack or call anything - it generates instructions that do that. The first time a invokevirtual instruction is executed it is linked, by > symbolically resolving the names in the instruction and verifying that the > method call is statically legal I keep trying not to call this out as a nit, but there should be no comma between "linked" and "by". Jeremy On Thu, Jan 21, 2016 at 2:42 PM, Paul Sandoz wrote: > Hi > > This is a request to review the VarHandles API. The code reviews and > pushes will occur separately, and flow through the hs-comp repo, most > likely from the bottom up first with Unsafe changes. > > The specdiff can be found here: > > > http://cr.openjdk.java.net/~psandoz/jdk9/varhandles/specdiff/overview-summary.html > > (Note that specdiff renders some aspects of JavaDoc incorrectly, so just > ignore any such quirks.) > > A consensus on the set of access mode methods proposed by Doug was > previously discussed and reached. > > For the moment please ignore the following methods on MethodHandles: > byteArrayViewVarHandle; and byteBufferViewVarHandle. It is necessary to > revisit that functionality w.r.t. alignment and proposed enhancements to > ByteBuffer (in discussion on valhalla-dev). > > Paul. > > > From maurizio.cimadamore at oracle.com Mon Jan 25 16:46:25 2016 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Mon, 25 Jan 2016 16:46:25 +0000 Subject: hg: valhalla/valhalla/langtools: Enhancement: Update TypeVar_info to match latest draft Message-ID: <201601251646.u0PGkQfY028785@aojmv0008.oracle.com> Changeset: 6a97b8d3ea09 Author: mcimadamore Date: 2016-01-25 16:46 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/6a97b8d3ea09 Enhancement: Update TypeVar_info to match latest draft ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/ClassReader.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Gen.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/Pool.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/PoolReader.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/PoolWriter.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/sym/CreateSymbols.java ! src/jdk.jdeps/share/classes/com/sun/tools/classfile/ClassTranslator.java ! src/jdk.jdeps/share/classes/com/sun/tools/classfile/ClassWriter.java ! src/jdk.jdeps/share/classes/com/sun/tools/classfile/ConstantPool.java ! src/jdk.jdeps/share/classes/com/sun/tools/classfile/Dependencies.java ! src/jdk.jdeps/share/classes/com/sun/tools/classfile/TypeVariablesMap_attribute.java ! src/jdk.jdeps/share/classes/com/sun/tools/javap/ConstantWriter.java From maurizio.cimadamore at oracle.com Mon Jan 25 16:47:59 2016 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Mon, 25 Jan 2016 16:47:59 +0000 Subject: hg: valhalla/valhalla/jdk: Enhancement: Update TypeVar_info to match latest draft Message-ID: <201601251647.u0PGlxiH029537@aojmv0008.oracle.com> Changeset: f63f16975a2d Author: mcimadamore Date: 2016-01-25 16:47 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/f63f16975a2d Enhancement: Update TypeVar_info to match latest draft ! src/java.base/share/classes/valhalla/model3/Model3Converter.java ! src/java.base/share/classes/valhalla/model3/classfile/ClassTranslator.java ! src/java.base/share/classes/valhalla/model3/classfile/ClassWriter.java ! src/java.base/share/classes/valhalla/model3/classfile/ConstantPool.java ! src/java.base/share/classes/valhalla/model3/classfile/Dependencies.java ! src/java.base/share/classes/valhalla/model3/classfile/TypeVariablesMap_attribute.java From maurizio.cimadamore at oracle.com Mon Jan 25 18:28:39 2016 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Mon, 25 Jan 2016 18:28:39 +0000 Subject: hg: valhalla/valhalla/langtools: Add separate compilaton test for new CP forms. Message-ID: <201601251828.u0PISdfo023240@aojmv0008.oracle.com> Changeset: 9e83e2551f52 Author: mcimadamore Date: 2016-01-25 18:28 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/9e83e2551f52 Add separate compilaton test for new CP forms. ! test/tools/javac/valhalla/typespec/separate06/Separate06.java + test/tools/javac/valhalla/typespec/separate07/GenericPool.java + test/tools/javac/valhalla/typespec/separate07/Separate07.java From maurizio.cimadamore at oracle.com Mon Jan 25 18:37:01 2016 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Mon, 25 Jan 2016 18:37:01 +0000 Subject: hg: valhalla/valhalla/langtools: Enhancement: TypeVariableMap should emit structural entries for bounds if -XDgenericClassFile is enabled Message-ID: <201601251837.u0PIb1lc026879@aojmv0008.oracle.com> Changeset: 34140d95412b Author: mcimadamore Date: 2016-01-25 18:36 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/34140d95412b Enhancement: TypeVariableMap should emit structural entries for bounds if -XDgenericClassFile is enabled ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/ClassWriter.java From maurizio.cimadamore at oracle.com Mon Jan 25 18:37:56 2016 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Mon, 25 Jan 2016 18:37:56 +0000 Subject: hg: valhalla/valhalla/jdk: Enhancement: Tweak Model3Converter to handle structural bounds in TypeVariableMap Message-ID: <201601251837.u0PIbuW4027360@aojmv0008.oracle.com> Changeset: f07cef6effa4 Author: mcimadamore Date: 2016-01-25 18:37 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/f07cef6effa4 Enhancement: Tweak Model3Converter to handle structural bounds in TypeVariableMap ! src/java.base/share/classes/valhalla/model3/Model3Converter.java From brian.goetz at oracle.com Mon Jan 25 20:07:53 2016 From: brian.goetz at oracle.com (brian.goetz at oracle.com) Date: Mon, 25 Jan 2016 20:07:53 +0000 Subject: hg: valhalla/valhalla/jdk: Bugfixes in M3 specializer Message-ID: <201601252007.u0PK7rd1027268@aojmv0008.oracle.com> Changeset: f744800c70b0 Author: briangoetz Date: 2016-01-25 15:07 -0500 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/f744800c70b0 Bugfixes in M3 specializer ! src/java.base/share/classes/java/net/URLClassLoader.java ! src/java.base/share/classes/valhalla/model3/Model3Converter.java From martinrb at google.com Mon Jan 25 21:42:35 2016 From: martinrb at google.com (Martin Buchholz) Date: Mon, 25 Jan 2016 13:42:35 -0800 Subject: API review of VarHandles In-Reply-To: <56A22D69.7040404@cs.oswego.edu> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A1FBA1.9010108@redhat.com> <56A22D69.7040404@cs.oswego.edu> Message-ID: On Fri, Jan 22, 2016 at 5:23 AM, Doug Lea
wrote: > On 01/22/2016 04:51 AM, Andrew Haley wrote: >> >> On 22/01/16 00:01, Vitaly Davidovich wrote: >>> >>> I think the get/setOpaque methods need a bit more explanation ("opaque" >>> is >>> an odd naming choice, IMO). Specifically, it says the operations are >>> done >>> in program order but have no effect on inter-thread ordering/visibility. >>> Is this spec verbiage for a full compiler-only fence? > > > > Basically yes. It would be the same as C "volatile" (not C++/C11 > "atomic"), if C volatile had a memory model spec. But in any case > we can't use "volatile" for this in Java because it already means > something else. It would be nice if we could actually say this in the (non-normative part of the) javadoc. >> >> It's like C++ memory_order_relaxed, I guessed. I understand that but >> not "opaque". >> > > The new name is required because opaque is only "like" C++ > memory_order_relaxed. > See the jmm-dev list discussions last year for details, but annoyingly, > Java normal variables and C++ memory_order_relaxed are not quite the > same, and neither are the same as C-volatile/opaque. Could we make atomicity guarantees clear for every method? When can we be sure not to get OOTA values? From vitalyd at gmail.com Mon Jan 25 21:52:25 2016 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Mon, 25 Jan 2016 16:52:25 -0500 Subject: API review of VarHandles In-Reply-To: <56A22D69.7040404@cs.oswego.edu> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A1FBA1.9010108@redhat.com> <56A22D69.7040404@cs.oswego.edu> Message-ID: Hi Doug, I somehow missed your reply and just now noticed it after Martin's last email quoted it -- thanks for elaborating. The new name is required because opaque is only "like" C++ > memory_order_relaxed. > See the jmm-dev list discussions last year for details, but annoyingly, > Java normal variables and C++ memory_order_relaxed are not quite the > same, and neither are the same as C-volatile/opaque. AFAICT, difference between java normal load/store and memory_order_relaxed is the latter guaranteeing atomicity. If I'm understanding you correctly, "opaque" is *just* compiler barrier and does not guarantee atomicity? If that's so, what is/are the use case(s) for opaque? Andrew mentioned preventing roach motel movement of loads/stores into the motel. Is there something more than that? Assuming my above statement is correct, it would seem like having memory_order_relaxed would be more useful than simply "no code motion around this load/store", or if not more useful, just useful on its own; there's plenty of code that doesn't care about ordering but would not tolerate torn reads/stores. On Fri, Jan 22, 2016 at 8:23 AM, Doug Lea
wrote: > On 01/22/2016 04:51 AM, Andrew Haley wrote: > >> On 22/01/16 00:01, Vitaly Davidovich wrote: >> >>> I think the get/setOpaque methods need a bit more explanation ("opaque" >>> is >>> an odd naming choice, IMO). Specifically, it says the operations are >>> done >>> in program order but have no effect on inter-thread ordering/visibility. >>> Is this spec verbiage for a full compiler-only fence? >>> >> > > Basically yes. It would be the same as C "volatile" (not C++/C11 > "atomic"), if C volatile had a memory model spec. But in any case > we can't use "volatile" for this in Java because it already means > something else. > > >> It's like C++ memory_order_relaxed, I guessed. I understand that but >> not "opaque". >> >> > The new name is required because opaque is only "like" C++ > memory_order_relaxed. > See the jmm-dev list discussions last year for details, but annoyingly, > Java normal variables and C++ memory_order_relaxed are not quite the > same, and neither are the same as C-volatile/opaque. > > -Doug > > > > From martinrb at google.com Mon Jan 25 22:08:41 2016 From: martinrb at google.com (Martin Buchholz) Date: Mon, 25 Jan 2016 14:08:41 -0800 Subject: API review of VarHandles In-Reply-To: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> Message-ID: This is my first attempt to understand java.lang.invoke. It seems like a new alien world, non-java-like. I'll need to rebuild my internal performance model of java code, perhaps by staring at jit compiled code. I expect new APIs for atomic variable access to appear in j.u.c.atomic, not j.l.invoke. Aleksey has made atomic updaters fast. Can we continue on that road, providing more updaters or more methods on current updaters? Sequential consistency is much easier than release/acquire, which is in turn much easier than load/load store/store. (using the latter is almost always a mistake?) Memory ordering combined with variables is easier than independent fences. I continue to be uneasy about handing out the sharpest knives in public APIs. We're not providing much in the way of scare text to get users to think twice. From brian.goetz at oracle.com Tue Jan 26 00:21:48 2016 From: brian.goetz at oracle.com (brian.goetz at oracle.com) Date: Tue, 26 Jan 2016 00:21:48 +0000 Subject: hg: valhalla/valhalla/jdk: Overhaul M3 converter for updated constant pool forms; use simpler name-encoding scheme Message-ID: <201601260021.u0Q0LmbW004009@aojmv0008.oracle.com> Changeset: d2399ce48ce7 Author: briangoetz Date: 2016-01-25 19:21 -0500 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/d2399ce48ce7 Overhaul M3 converter for updated constant pool forms; use simpler name-encoding scheme ! src/java.base/share/classes/java/net/URLClassLoader.java ! src/java.base/share/classes/valhalla/model3/Model3Converter.java - src/java.base/share/classes/valhalla/model3/SpeciesKey.java ! src/java.base/share/classes/valhalla/specializer/SignatureSpecializer.java From maurizio.cimadamore at oracle.com Tue Jan 26 13:58:33 2016 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Tue, 26 Jan 2016 13:58:33 +0000 Subject: hg: valhalla/valhalla/jdk: Fix: javap generates spurious errors due to bogus descriptor parsing logic Message-ID: <201601261358.u0QDwXFu001460@aojmv0008.oracle.com> Changeset: 6b6ef08b3b2e Author: mcimadamore Date: 2016-01-26 13:58 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/6b6ef08b3b2e Fix: javap generates spurious errors due to bogus descriptor parsing logic ! src/java.base/share/classes/valhalla/model3/classfile/ConstantPool.java ! src/java.base/share/classes/valhalla/model3/classfile/Descriptor.java From maurizio.cimadamore at oracle.com Tue Jan 26 13:59:20 2016 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Tue, 26 Jan 2016 13:59:20 +0000 Subject: hg: valhalla/valhalla/langtools: Fix: javap generates spurious errors due to bogus descriptor parsing logic Message-ID: <201601261359.u0QDxK6m001999@aojmv0008.oracle.com> Changeset: d1015b6cf7f4 Author: mcimadamore Date: 2016-01-26 13:59 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/d1015b6cf7f4 Fix: javap generates spurious errors due to bogus descriptor parsing logic ! src/jdk.jdeps/share/classes/com/sun/tools/classfile/ConstantPool.java ! src/jdk.jdeps/share/classes/com/sun/tools/classfile/Descriptor.java From paul.sandoz at oracle.com Tue Jan 26 15:27:30 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 26 Jan 2016 16:27:30 +0100 Subject: API review of VarHandles In-Reply-To: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> Message-ID: <282FAEC5-CF12-43C8-A58B-E16FDAC3C9B7@oracle.com> Hi Many thanks for the feedback so far. Some high-level responses: 1) I don?t think there is much we can do right now to reduce the verbosity of reflective lookup. We have discussed this at least once before in the past. We need field literals, as mentioned in the JEP, to really knock this on the head (which might lead to constant pool forms, or even a ?constant-dynamic?, and perhaps local variables to VHs that constant fold). But such support of literals is just too much work to take on at the moment. 2) Examples are needed in api notes much like there are examples for MHs. It might be possible to rearrange things so it?s more user-focused up front with suitable "Government Health Warnings". I wanted to focus on the normative aspects first. 3) Yes, it?s a sharp knife, a low-level building block, consolidating many forms of access and shape under one interface, designed for power users to avail of directly or wrap in higher-level forms as required, and leveraging the low-level method handle invocation mechanism that has been heavily optimized to constant fold and inline. Paul. > On 21 Jan 2016, at 23:42, Paul Sandoz wrote: > > Hi > > This is a request to review the VarHandles API. The code reviews and pushes will occur separately, and flow through the hs-comp repo, most likely from the bottom up first with Unsafe changes. > > The specdiff can be found here: > > http://cr.openjdk.java.net/~psandoz/jdk9/varhandles/specdiff/overview-summary.html > > (Note that specdiff renders some aspects of JavaDoc incorrectly, so just ignore any such quirks.) > > A consensus on the set of access mode methods proposed by Doug was previously discussed and reached. > > For the moment please ignore the following methods on MethodHandles: byteArrayViewVarHandle; and byteBufferViewVarHandle. It is necessary to revisit that functionality w.r.t. alignment and proposed enhancements to ByteBuffer (in discussion on valhalla-dev). > > Paul. > > From brian.goetz at oracle.com Tue Jan 26 15:37:17 2016 From: brian.goetz at oracle.com (brian.goetz at oracle.com) Date: Tue, 26 Jan 2016 15:37:17 +0000 Subject: hg: valhalla/valhalla/jdk: Two minor bugfixes surrounding handling of _ in specializations Message-ID: <201601261537.u0QFbHf5017082@aojmv0008.oracle.com> Changeset: 79bbca61e984 Author: briangoetz Date: 2016-01-26 10:36 -0500 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/79bbca61e984 Two minor bugfixes surrounding handling of _ in specializations ! src/java.base/share/classes/valhalla/model3/Model3Converter.java ! src/java.base/share/classes/valhalla/specializer/SignatureSpecializer.java From maurizio.cimadamore at oracle.com Tue Jan 26 15:41:59 2016 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Tue, 26 Jan 2016 15:41:59 +0000 Subject: hg: valhalla/valhalla/langtools: Fix: reference type-arguments should be turned into '_' in ParameterizedType entries Message-ID: <201601261541.u0QFfx3o019387@aojmv0008.oracle.com> Changeset: ade1263cab01 Author: mcimadamore Date: 2016-01-26 15:41 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/ade1263cab01 Fix: reference type-arguments should be turned into '_' in ParameterizedType entries ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/PoolWriter.java From martinrb at google.com Tue Jan 26 20:31:26 2016 From: martinrb at google.com (Martin Buchholz) Date: Tue, 26 Jan 2016 12:31:26 -0800 Subject: API review of VarHandles In-Reply-To: <282FAEC5-CF12-43C8-A58B-E16FDAC3C9B7@oracle.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <282FAEC5-CF12-43C8-A58B-E16FDAC3C9B7@oracle.com> Message-ID: There's a big "expectations" effect here. j.l.invoke is "supposed to be" for making dynamic languages less slow, not for making low-level, ultra-non-dynamic operations faster. Asking the Unsafe users of the world to switch to dynamic VarHandle is like asking C programmers to rewrite their code in perl 6 ... for performance! It's the same "srsly?" feeling one gets reading """We can currently use RPerl to speed up low-magic Perl 5 code with over 300x performance gain.""" From brian.goetz at oracle.com Tue Jan 26 21:46:15 2016 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 26 Jan 2016 16:46:15 -0500 Subject: API review of VarHandles In-Reply-To: References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <282FAEC5-CF12-43C8-A58B-E16FDAC3C9B7@oracle.com> Message-ID: <56A7E927.2070909@oracle.com> I think that expectation is just out of date (if not outright mistaken.) Yes, j.l.i was originally called "java.dyn", but prior to shipping *7* we renamed it to j.l.i precisely because it had turned into a general customizable linkage mechanism that was usable far beyond dynamic languages. (To wit, Java 8 lambdas got a significant linkage/capture boost from using invokedynamic.) So to claim it's "supposed to be" only for dynamic languages represents a significant "missing of the memo." On 1/26/2016 3:31 PM, Martin Buchholz wrote: > There's a big "expectations" effect here. j.l.invoke is "supposed to > be" for making dynamic languages less slow, not for making low-level, > ultra-non-dynamic operations faster. Asking the Unsafe users of the > world to switch to dynamic VarHandle is like asking C programmers to > rewrite their code in perl 6 ... for performance! It's the same > "srsly?" feeling one gets reading """We can currently use RPerl to > speed up low-magic Perl 5 code with over 300x performance gain.""" From michael.haupt at oracle.com Tue Jan 26 22:11:14 2016 From: michael.haupt at oracle.com (Michael Haupt) Date: Tue, 26 Jan 2016 23:11:14 +0100 Subject: API review of VarHandles In-Reply-To: References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <282FAEC5-CF12-43C8-A58B-E16FDAC3C9B7@oracle.com> Message-ID: <95DCB2A7-D773-4601-BAAC-282AE7DC1D15@oracle.com> Hi Martin, how about seeing j.l.i as a framework for lightweight reflection on the one hand, and for method handle-based meta-programming on the other? That's clearly usable beyond the domain of dynamic language implementation. Granted, the latter remains an important application area, but there are others. For example, JEP 280 uses invokedynamic for string concatenation. In that vein, how about seeing VarHandles as a safer replacement for some of the Unsafe API? It's going to be public API, readily usable without having to enable access to the carefully tucked-away Unsafe. Best, Michael > Am 26.01.2016 um 21:31 schrieb Martin Buchholz : > > There's a big "expectations" effect here. j.l.invoke is "supposed to > be" for making dynamic languages less slow, not for making low-level, > ultra-non-dynamic operations faster. Asking the Unsafe users of the > world to switch to dynamic VarHandle is like asking C programmers to > rewrite their code in perl 6 ... for performance! It's the same > "srsly?" feeling one gets reading """We can currently use RPerl to > speed up low-magic Perl 5 code with over 300x performance gain.""" -- Dr. Michael Haupt | Principal Member of Technical Staff Phone: +49 331 200 7277 | Fax: +49 331 200 7561 Oracle Java Platform Group | LangTools Team | Nashorn Oracle Deutschland B.V. & Co. KG | Schiffbauergasse 14 | 14467 Potsdam, Germany ORACLE Deutschland B.V. & Co. KG | Hauptverwaltung: Riesstra?e 25, D-80992 M?nchen Registergericht: Amtsgericht M?nchen, HRA 95603 Komplement?rin: ORACLE Deutschland Verwaltung B.V. | Hertogswetering 163/167, 3543 AS Utrecht, Niederlande Handelsregister der Handelskammer Midden-Nederland, Nr. 30143697 Gesch?ftsf?hrer: Alexander van der Ven, Jan Schultheiss, Val Maher Oracle is committed to developing practices and products that help protect the environment From paul.sandoz at oracle.com Wed Jan 27 08:44:36 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 27 Jan 2016 09:44:36 +0100 Subject: API review of VarHandles In-Reply-To: <56A7E927.2070909@oracle.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <282FAEC5-CF12-43C8-A58B-E16FDAC3C9B7@oracle.com> <56A7E927.2070909@oracle.com> Message-ID: <39841106-6BC8-4CA3-9748-8DFB6F56C08E@oracle.com> > On 26 Jan 2016, at 22:46, Brian Goetz wrote: > > I think that expectation is just out of date (if not outright mistaken.) Yes, j.l.i was originally called "java.dyn", but prior to shipping *7* we renamed it to j.l.i precisely because it had turned into a general customizable linkage mechanism that was usable far beyond dynamic languages. (To wit, Java 8 lambdas got a significant linkage/capture boost from using invokedynamic.) So to claim it's "supposed to be" only for dynamic languages represents a significant "missing of the memo.? > Furthermore, Java has always been quite dynamic under the covers of the language. See also Aleksey?s recent indy string concat enhancements. In future this area is likely to be leveraged further. And i know Remi?s is always on the look out to expose indy up to into the language :-) and with good reason. Paul. > > On 1/26/2016 3:31 PM, Martin Buchholz wrote: >> There's a big "expectations" effect here. j.l.invoke is "supposed to >> be" for making dynamic languages less slow, not for making low-level, >> ultra-non-dynamic operations faster. Asking the Unsafe users of the >> world to switch to dynamic VarHandle is like asking C programmers to >> rewrite their code in perl 6 ... for performance! It's the same >> "srsly?" feeling one gets reading """We can currently use RPerl to >> speed up low-magic Perl 5 code with over 300x performance gain.""" > From brian.goetz at oracle.com Wed Jan 27 22:34:37 2016 From: brian.goetz at oracle.com (brian.goetz at oracle.com) Date: Wed, 27 Jan 2016 22:34:37 +0000 Subject: hg: valhalla/valhalla/jdk: Add support for nested classes to Model 3 converter Message-ID: <201601272234.u0RMYbbH012802@aojmv0008.oracle.com> Changeset: 00268e07f1cd Author: briangoetz Date: 2016-01-27 17:34 -0500 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/00268e07f1cd Add support for nested classes to Model 3 converter ! src/java.base/share/classes/java/net/URLClassLoader.java ! src/java.base/share/classes/java/util/stream/Stream.java ! src/java.base/share/classes/valhalla/model3/Model3Converter.java ! src/java.base/share/classes/valhalla/model3/classfile/TypeVariablesMap_attribute.java ! src/java.base/share/classes/valhalla/specializer/SignatureSpecializer.java From paul.sandoz at oracle.com Thu Jan 28 09:18:09 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 28 Jan 2016 10:18:09 +0100 Subject: Aligned long views over byte arrays and heap ByteBuffers In-Reply-To: References: <56982396.3020802@oracle.com> <5555501D-B4BC-4A0C-BCEC-CE9C159EA3C1@oracle.com> <569CF3DD.30005@oracle.com> <39B35639-D213-4832-A988-F35E029E1CDF@oracle.com> <9625381C-6DEE-4A39-B61C-D6453B4711D7@oracle.com> Message-ID: <3AA8A44A-98D1-4588-B645-90AE355C02ED@oracle.com> John and I tweaked things a little, here is an update pushed to the VHs sandbox: http://hg.openjdk.java.net/jdk9/sandbox/jdk/file/JEP-193-VarHandles-branch/src/java.base/share/classes/java/nio/X-Buffer.java.template#l1618 http://hg.openjdk.java.net/jdk9/sandbox/jdk/file/JEP-193-VarHandles-branch/src/java.base/share/classes/java/nio/X-Buffer.java.template#l1660 It?s tempting to use short as the unit size to reduce the potentially for incorrect usages but resulting casts are painful. New tests added pass on 64 and 32-bit platforms. I will try and roll in such changes separately to VHs stuff. Paul. > On 20 Jan 2016, at 16:04, Vitaly Davidovich wrote: > > I like alignOffset and thought about that too. Main issue is that isAligned is not idiomatic given it returns an offset adjustment. > > On Wednesday, January 20, 2016, MacGregor, Duncan (GE Energy Management) > wrote: > On 20/01/2016, 14:15, "valhalla-dev on behalf of Paul Sandoz" > on behalf of > paul.sandoz at oracle.com > wrote: > > > > >> On 20 Jan 2016, at 14:27, Vitaly Davidovich > wrote: > >> > >>How about alignIndex(int,int)? > >> > > > >That might work, although it implies returning an aligned index rather > >than a modulus associated with the current index and unit size. I think > >that would be more precise if we include boolean argument for rounding > >up/down to the nearest aligned index: > > > > alignedIndex(int index, int unitSize, boolean roundUp) > > > >For rounding down a negative value might be returned. I still prefer a > >modulus returning method. > > I kind of agree that alignedIndex suggests an actual index is returned. > How about alignedOffset, or alignedDistance maybe? > > Duncan. > > > > -- > Sent from my phone From maurizio.cimadamore at oracle.com Thu Jan 28 17:46:58 2016 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Thu, 28 Jan 2016 17:46:58 +0000 Subject: hg: valhalla/valhalla/langtools: Fix: Infinite loop when generating ParamType entry with raw enclosing Message-ID: <201601281746.u0SHkwaE027478@aojmv0008.oracle.com> Changeset: ee3dd386c16b Author: mcimadamore Date: 2016-01-28 17:46 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/ee3dd386c16b Fix: Infinite loop when generating ParamType entry with raw enclosing Fix: Missing call to ClassFile.externalize on ParamType Utf8 name. ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/PoolWriter.java ! test/tools/javac/valhalla/typespec/GenericMethod01.java ! test/tools/javac/valhalla/typespec/GenericMethod02.java From maurizio.cimadamore at oracle.com Fri Jan 29 15:57:28 2016 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Fri, 29 Jan 2016 15:57:28 +0000 Subject: hg: valhalla/valhalla/langtools: Fix: descriptors should not mention type-variables that are not in scope when using -XDgenericClassFile Message-ID: <201601291557.u0TFvTWR008360@aojmv0008.oracle.com> Changeset: 3700d512c76d Author: mcimadamore Date: 2016-01-29 15:57 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/3700d512c76d Fix: descriptors should not mention type-variables that are not in scope when using -XDgenericClassFile ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Flags.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Check.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/SpecializeTypes.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TransTypes.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/ClassWriter.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/PoolWriter.java ! test/tools/javac/valhalla/typespec/GenericMethod01.java ! test/tools/javac/valhalla/typespec/GenericMethod02.java From maurizio.cimadamore at oracle.com Fri Jan 29 19:04:19 2016 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Fri, 29 Jan 2016 19:04:19 +0000 Subject: hg: valhalla/valhalla/jdk: Enhancement: add generic method support for Model 3 Message-ID: <201601291904.u0TJ4JbY021329@aojmv0008.oracle.com> Changeset: fb187afc8ece Author: mcimadamore Date: 2016-01-29 19:04 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/jdk/rev/fb187afc8ece Enhancement: add generic method support for Model 3 Fix: misc fixes for Model 3 Converter when handling raw types ! src/java.base/share/classes/java/lang/invoke/DispatchContext.java ! src/java.base/share/classes/java/lang/invoke/GenericInstanceDispatch.java ! src/java.base/share/classes/java/lang/invoke/GenericStaticDispatch.java ! src/java.base/share/classes/java/net/URLClassLoader.java ! src/java.base/share/classes/valhalla/model3/Model3Converter.java From maurizio.cimadamore at oracle.com Fri Jan 29 19:05:03 2016 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Fri, 29 Jan 2016 19:05:03 +0000 Subject: hg: valhalla/valhalla/langtools: Fix: BootstrapMethodKeys.equals() ignores differences in BSM handle Message-ID: <201601291905.u0TJ530I021953@aojmv0008.oracle.com> Changeset: 776432d52150 Author: mcimadamore Date: 2016-01-29 19:04 +0000 URL: http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/776432d52150 Fix: BootstrapMethodKeys.equals() ignores differences in BSM handle Enhancement: simpler synthetic class names for translated generic methods Test: added stress tests for generic methods and Model 3 ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/SpecializeTypes.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/PoolWriter.java ! test/tools/javac/valhalla/typespec/GenericMethod01.java ! test/tools/javac/valhalla/typespec/GenericMethod02.java + test/tools/javac/valhalla/typespec/GenericMethod03.java From srikanth.adayapalam at oracle.com Fri Jan 29 23:48:25 2016 From: srikanth.adayapalam at oracle.com (Srikanth) Date: Sat, 30 Jan 2016 05:18:25 +0530 Subject: Unable to run _ByValue example In-Reply-To: <5680C208.8080208@oracle.com> References: <5680C208.8080208@oracle.com> Message-ID: <56ABFA49.3080403@oracle.com> On Monday 28 December 2015 10:30 AM, Srikanth wrote: > Hello Boaz, > > This is a known issue - ATM, the VM does not yet handle the changes to > signatures > that embed 'Q' in them to denote value types and it may be some time > before the > VM support for this is fully there. > > We are exploring some interim options to allow interested early access > testers to > experiment with the usability aspects of the value type model (knowing > that attendant > performance improvements are yet to come) Hello Boaz, As of http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/659f44b99b6c, pushed on Jan 22nd, by using the javac option -XDGenerateValueAsReference, you can produce class files that can be run - the new syntax for value types will be accepted, you will not yet see any performance improvements - This is iput in place to enable early testers to get a feel for the usability model and share feedback. Please try this out. Thanks! Srikanth > > Stay tuned. > Thanks! > Srikanth > > On Friday 25 December 2015 05:56 PM, Boaz Nahum wrote: >> (last pull valhalla today) >> Trying this: >> >> public final __ByValue class Point { >> final int x; >> final int y; >> Point(int x, int y) { >> this.x = x; >> this.y = y; >> } >> >> public static void main(String[] args) { >> Point p = __Make Point(5, 6); >> } >> } >> >> (Compile file directly with bin/javac.exe not using IntelliJ) >> >> When trying to run: >> >> F:\Dev\JDKBuild\valhalla\build\windows-x86_64-normal-server-release\images\jdk\bin\java >> >> -cp F:\Dev\IntelliJProjectsOut\LearnJDK10\production\LearnJDK10 >> by_val.Point >> java.lang.ClassFormatError: Method "" in class by_val/Point >> has illegal signature "(II)Qby_val/Point;" >> at java.lang.ClassLoader.defineClass1(Native Method) >> at java.lang.ClassLoader.defineClass(ClassLoader.java:759) >> at java.lang.ClassLoader.defineClass(ClassLoader.java:835) >> at >> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:184) >> at java.net.URLClassLoader.defineClass(URLClassLoader.java:547) >> at java.net.URLClassLoader.access$100(URLClassLoader.java:87) >> at java.net.URLClassLoader$1.run(URLClassLoader.java:413) >> at java.net.URLClassLoader$1.run(URLClassLoader.java:397) >> at java.security.AccessController.doPrivileged(Native Method) >> at java.net.URLClassLoader.findClass(URLClassLoader.java:396) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:423) >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:262) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:356) >> at >> sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:503) >> Error: A JNI error has occurred, please check your installation and >> try again >> Exception in thread "main" >> Process finished with exit code 1 >> >> Thx >> Boaz > From brian.goetz at oracle.com Fri Jan 29 23:52:04 2016 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 29 Jan 2016 18:52:04 -0500 Subject: Unable to run _ByValue example In-Reply-To: <56ABFA49.3080403@oracle.com> References: <5680C208.8080208@oracle.com> <56ABFA49.3080403@oracle.com> Message-ID: <82B96491-C6A3-4402-805B-163A080602C3@oracle.com> Just a periodic reminder that performance of all the prototypes is going to be pathologically awful for quite a while; attempting to measure performance at this early stage counterproductive. We?re still in the ?validating that the model works? stage. Prototype accordingly! On Jan 29, 2016, at 6:48 PM, Srikanth wrote: > > > On Monday 28 December 2015 10:30 AM, Srikanth wrote: >> Hello Boaz, >> >> This is a known issue - ATM, the VM does not yet handle the changes to signatures >> that embed 'Q' in them to denote value types and it may be some time before the >> VM support for this is fully there. >> >> We are exploring some interim options to allow interested early access testers to >> experiment with the usability aspects of the value type model (knowing that attendant >> performance improvements are yet to come) > > Hello Boaz, > > As of http://hg.openjdk.java.net/valhalla/valhalla/langtools/rev/659f44b99b6c, pushed on > Jan 22nd, by using the javac option -XDGenerateValueAsReference, you can produce class > files that can be run - the new syntax for value types will be accepted, you will not yet > see any performance improvements - This is iput in place to enable early testers to > get a feel for the usability model and share feedback. Please try this out. > > Thanks! > Srikanth > >> >> Stay tuned. >> Thanks! >> Srikanth >> >> On Friday 25 December 2015 05:56 PM, Boaz Nahum wrote: >>> (last pull valhalla today) >>> Trying this: >>> >>> public final __ByValue class Point { >>> final int x; >>> final int y; >>> Point(int x, int y) { >>> this.x = x; >>> this.y = y; >>> } >>> >>> public static void main(String[] args) { >>> Point p = __Make Point(5, 6); >>> } >>> } >>> >>> (Compile file directly with bin/javac.exe not using IntelliJ) >>> >>> When trying to run: >>> >>> F:\Dev\JDKBuild\valhalla\build\windows-x86_64-normal-server-release\images\jdk\bin\java >>> -cp F:\Dev\IntelliJProjectsOut\LearnJDK10\production\LearnJDK10 >>> by_val.Point >>> java.lang.ClassFormatError: Method "" in class by_val/Point >>> has illegal signature "(II)Qby_val/Point;" >>> at java.lang.ClassLoader.defineClass1(Native Method) >>> at java.lang.ClassLoader.defineClass(ClassLoader.java:759) >>> at java.lang.ClassLoader.defineClass(ClassLoader.java:835) >>> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:184) >>> at java.net.URLClassLoader.defineClass(URLClassLoader.java:547) >>> at java.net.URLClassLoader.access$100(URLClassLoader.java:87) >>> at java.net.URLClassLoader$1.run(URLClassLoader.java:413) >>> at java.net.URLClassLoader$1.run(URLClassLoader.java:397) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at java.net.URLClassLoader.findClass(URLClassLoader.java:396) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:423) >>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:262) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:356) >>> at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:503) >>> Error: A JNI error has occurred, please check your installation and try again >>> Exception in thread "main" >>> Process finished with exit code 1 >>> >>> Thx >>> Boaz >> >