Hi, while working and trying the new JDK9 MethodHandles features like MethodHandles#countedLoop, I recognized a API inconsistency problem with it. We also found this while implementing the "Painless" scripting language for Elasticsearch (see https://goo.gl/DbOzjC)! Painless is a very limited scripting language with a very limited subset of classes to access and close integration into Elasticsearch (similar to Lucene's Expressions Module). Both compile script code to anonymous classes and Elasticsearch uses invokedynamic for the untyped stuff. It is much faster than Groovy or Nashorn, because it compiles most of the code completely static with known types (like Lucene expressions), but also supports a Groovy-like "def" type. The latter one uses invokedynamic, everything else is exactly the same speed as native Java code. During implementing array getters and setters for untyped stuff with invokedynamic we recognized the following problem (Java 8 code, see https://github.com/elastic/elasticsearch/pull/18232): There is MethodHandles.arrayElementGetter and MethodHandles.arrayElementSetter, so we can have full speed and full dynamic flexibility for array loads and stores - the speed is great, same as native array access! But there is one factory method missing in Java 8: MethodHandles.arrayLengthGetter (or similar name). As "length" is not a field on the array class, it is impossible to create a "simple" MethodHandle as field access to "array.class#length" to access it (it must invoke the "arraylength" bytecode, there is no field). I think the "length" field is just syntactic sugar of javac compiler. I know that Nashorn used to use java.lang.reflect.Array#getLength (not sure if this is still the case), but this makes usage inconsistent. You have to explicitly create a MethodHandle using the reflective array class to use it: And it is not strongly typed (it accepts Object). A real MethodHandle would be created for a specific type of array class and would return its length without runtime type checks. I know that at least Array.getLength() is optimized by Hotspot (in contrast to the other get/set methods, which are like 20 times slower than a method access, so MethodHandles.arrayElementGetter/Setter is very useful), but I would really like to suggest to fix this! With Java 9 this gets a bit worse: There is no "easy way" with the MethodHanldes class to generate a MethodHandles.countedLoop() on all elements of an array: public static MethodHandle countedLoop(MethodHandle iterations, MethodHandle init, MethodHandle body) [new in Java 9] With a full-featured API one could write: Class<?> type = arraytype, e.g. long[].class or String[].class MethodHandle body = some code that uses MethodHandles.arrayElementGetter(type) for further processing MethodHandles.countedLoop(MethodHandles.arrayLengthGetter(type), MethodHandles.constant(int.class, 0), body); As you see, for the first parameter (count), you would need to use the reflective part in java.lang.reflect.Array if the method is still missing in Java 9. This is not bad here, because it is not called all the time, but for our scripting language, the reflective class was slower. We implemented our own version of "arrayLengthGetter": public class ArrayLengthHelper { private ArrayLengthHelper() {} private static final Lookup PRIV_LOOKUP = MethodHandles.lookup(); private static final Map<Class<?>,MethodHandle> ARRAY_TYPE_MH_MAPPING = Collections.unmodifiableMap( Stream.of(boolean[].class, byte[].class, short[].class, int[].class, long[].class, char[].class, float[].class, double[].class, Object[].class) .collect(Collectors.toMap(Function.identity(), type -> { try { return PRIV_LOOKUP.findStatic(PRIV_LOOKUP.lookupClass(), "getArrayLength", MethodType.methodType(int.class, type)); } catch (ReflectiveOperationException e) { throw new AssertionError(e); } })) ); private static final MethodHandle OBJECT_ARRAY_MH = ARRAY_TYPE_MH_MAPPING.get(Object[].class); static int getArrayLength(boolean[] array) { return array.length; } static int getArrayLength(byte[] array) { return array.length; } static int getArrayLength(short[] array) { return array.length; } static int getArrayLength(int[] array) { return array.length; } static int getArrayLength(long[] array) { return array.length; } static int getArrayLength(char[] array) { return array.length; } static int getArrayLength(float[] array) { return array.length; } static int getArrayLength(double[] array) { return array.length; } static int getArrayLength(Object[] array) { return array.length; } public static MethodHandle arrayLengthGetter(Class<?> arrayType) { if (!arrayType.isArray()) { throw new IllegalArgumentException("type must be an array"); } return (ARRAY_TYPE_MH_MAPPING.containsKey(arrayType)) ? ARRAY_TYPE_MH_MAPPING.get(arrayType) : OBJECT_ARRAY_MH.asType(OBJECT_ARRAY_MH.type().changeParameterType(0, arrayType)); } } Interestingly I later found out that MethodHandles.arrayElementGetter/Setter uses the same "trick" behind the scenes! See MethodHandleImpl.ArrayAccessor: http://goo.gl/94f6OB I would suggest to add the missing Method to MethodHandles class and implement it together with the getters and setters in ArrayAccessor, similar to our example code (it is just a few lines more). In addition this one could then also use the extra intrinsic improvement that our class cannot use: makeIntrinsic(getAccessor(Object[].class, false), Intrinsic.ARRAY_LOAD); [with another intrinsic added: Intrinsic.ARRAY_LENGTH for the arraylength bytecode] What would be the process to propose such a change? Bug/Issue/JEP? I could quickly create a patch, but fixing the Intrinsic/LambdaForm stuff is way too complicated for me, so I would suggest to take this as an inspiration how to do it Uwe ----- Uwe Schindler uschindler@apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany http://lucene.apache.org/
----- Mail original -----
De: "Uwe Schindler" <uschindler@apache.org> À: "Core-Libs-Dev" <core-libs-dev@openjdk.java.net> Envoyé: Mercredi 11 Mai 2016 21:35:32 Objet: MethodHandle for array length
Hi,
Hi Uwe,
while working and trying the new JDK9 MethodHandles features like MethodHandles#countedLoop, I recognized a API inconsistency problem with it.
We also found this while implementing the "Painless" scripting language for Elasticsearch (see https://goo.gl/DbOzjC)! Painless is a very limited scripting language with a very limited subset of classes to access and close integration into Elasticsearch (similar to Lucene's Expressions Module). Both compile script code to anonymous classes and Elasticsearch uses invokedynamic for the untyped stuff. It is much faster than Groovy or Nashorn, because it compiles most of the code completely static with known types (like Lucene expressions), but also supports a Groovy-like "def" type. The latter one uses invokedynamic, everything else is exactly the same speed as native Java code.
I think you can do something similar in Groovy with the annotation @TypeChecl and @StaticCompilation, see [1].
During implementing array getters and setters for untyped stuff with invokedynamic we recognized the following problem (Java 8 code, see https://github.com/elastic/elasticsearch/pull/18232):
There is MethodHandles.arrayElementGetter and MethodHandles.arrayElementSetter, so we can have full speed and full dynamic flexibility for array loads and stores - the speed is great, same as native array access! But there is one factory method missing in Java 8: MethodHandles.arrayLengthGetter (or similar name). As "length" is not a field on the array class, it is impossible to create a "simple" MethodHandle as field access to "array.class#length" to access it (it must invoke the "arraylength" bytecode, there is no field). I think the "length" field is just syntactic sugar of javac compiler.
I know that Nashorn used to use java.lang.reflect.Array#getLength (not sure if this is still the case), but this makes usage inconsistent. You have to explicitly create a MethodHandle using the reflective array class to use it: And it is not strongly typed (it accepts Object). A real MethodHandle would be created for a specific type of array class and would return its length without runtime type checks. I know that at least Array.getLength() is optimized by Hotspot (in contrast to the other get/set methods, which are like 20 times slower than a method access, so MethodHandles.arrayElementGetter/Setter is very useful), but I would really like to suggest to fix this!
With Java 9 this gets a bit worse: There is no "easy way" with the MethodHanldes class to generate a MethodHandles.countedLoop() on all elements of an array:
public static MethodHandle countedLoop(MethodHandle iterations, MethodHandle init, MethodHandle body) [new in Java 9]
With a full-featured API one could write:
Class<?> type = arraytype, e.g. long[].class or String[].class MethodHandle body = some code that uses MethodHandles.arrayElementGetter(type) for further processing MethodHandles.countedLoop(MethodHandles.arrayLengthGetter(type), MethodHandles.constant(int.class, 0), body);
As you see, for the first parameter (count), you would need to use the reflective part in java.lang.reflect.Array if the method is still missing in Java 9. This is not bad here, because it is not called all the time, but for our scripting language, the reflective class was slower.
We implemented our own version of "arrayLengthGetter":
public class ArrayLengthHelper { private ArrayLengthHelper() {}
private static final Lookup PRIV_LOOKUP = MethodHandles.lookup(); private static final Map<Class<?>,MethodHandle> ARRAY_TYPE_MH_MAPPING = Collections.unmodifiableMap( Stream.of(boolean[].class, byte[].class, short[].class, int[].class, long[].class, char[].class, float[].class, double[].class, Object[].class) .collect(Collectors.toMap(Function.identity(), type -> { try { return PRIV_LOOKUP.findStatic(PRIV_LOOKUP.lookupClass(), "getArrayLength", MethodType.methodType(int.class, type)); } catch (ReflectiveOperationException e) { throw new AssertionError(e); } })) ); private static final MethodHandle OBJECT_ARRAY_MH = ARRAY_TYPE_MH_MAPPING.get(Object[].class);
static int getArrayLength(boolean[] array) { return array.length; } static int getArrayLength(byte[] array) { return array.length; } static int getArrayLength(short[] array) { return array.length; } static int getArrayLength(int[] array) { return array.length; } static int getArrayLength(long[] array) { return array.length; } static int getArrayLength(char[] array) { return array.length; } static int getArrayLength(float[] array) { return array.length; } static int getArrayLength(double[] array) { return array.length; } static int getArrayLength(Object[] array) { return array.length; }
public static MethodHandle arrayLengthGetter(Class<?> arrayType) { if (!arrayType.isArray()) { throw new IllegalArgumentException("type must be an array"); } return (ARRAY_TYPE_MH_MAPPING.containsKey(arrayType)) ? ARRAY_TYPE_MH_MAPPING.get(arrayType) : OBJECT_ARRAY_MH.asType(OBJECT_ARRAY_MH.type().changeParameterType(0, arrayType)); } }
Interestingly I later found out that MethodHandles.arrayElementGetter/Setter uses the same "trick" behind the scenes! See MethodHandleImpl.ArrayAccessor: http://goo.gl/94f6OB
I would suggest to add the missing Method to MethodHandles class and implement it together with the getters and setters in ArrayAccessor, similar to our example code (it is just a few lines more). In addition this one could then also use the extra intrinsic improvement that our class cannot use: makeIntrinsic(getAccessor(Object[].class, false), Intrinsic.ARRAY_LOAD); [with another intrinsic added: Intrinsic.ARRAY_LENGTH for the arraylength bytecode]
What would be the process to propose such a change? Bug/Issue/JEP? I could quickly create a patch, but fixing the Intrinsic/LambdaForm stuff is way too complicated for me, so I would suggest to take this as an inspiration how to do it
Uwe
as you said, Array.getLength is already optimized by the JIT, it's because the length of an array is at the same offset for any kind of array in Hotspot, so unlike array access that depends on the kind of array, getting the length of an array is the same code for any kind of array. So you can safely, performance wise, create a method handle on Array.getLength, that's why there is no arrayLengthGetter in MethodHandles. regards, Rémi [1] http://www.infoq.com/articles/new-groovy-20
----- Uwe Schindler uschindler@apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany http://lucene.apache.org/
Hi Remi,
while working and trying the new JDK9 MethodHandles features like MethodHandles#countedLoop, I recognized a API inconsistency problem with it.
We also found this while implementing the "Painless" scripting language for Elasticsearch (see https://goo.gl/DbOzjC)! Painless is a very limited scripting language with a very limited subset of classes to access and close integration into Elasticsearch (similar to Lucene's Expressions Module). Both compile script code to anonymous classes and Elasticsearch uses invokedynamic for the untyped stuff. It is much faster than Groovy or Nashorn, because it compiles most of the code completely static with known types (like Lucene expressions), but also supports a Groovy-like "def" type. The latter one uses invokedynamic, everything else is exactly the same speed as native Java code.
I think you can do something similar in Groovy with the annotation @TypeChecl and @StaticCompilation, see [1].
The reason why we not use Groovy is security-related. It is impossible to limit access to only a very limited subset of Java classes and methods with Groovy. Because of many security issues Groovy is no longer an option. But that is unrelated to this issue, of course :-) The script snippets in Painless or Lucene expressions are scripts that are executed multiple times on millions of search results during collecting fulltext search hits. E.g. when you want to customize the score. For that to work it is important that the compiled script is using primitive types for score and so on.
During implementing array getters and setters for untyped stuff with invokedynamic we recognized the following problem (Java 8 code, see https://github.com/elastic/elasticsearch/pull/18232):
There is MethodHandles.arrayElementGetter and MethodHandles.arrayElementSetter, so we can have full speed and full dynamic flexibility for array loads and stores - the speed is great, same as native array access! But there is one factory method missing in Java 8: MethodHandles.arrayLengthGetter (or similar name). As "length" is not a field on the array class, it is impossible to create a "simple" MethodHandle as field access to "array.class#length" to access it (it must invoke the "arraylength" bytecode, there is no field). I think the "length" field is just syntactic sugar of javac compiler.
I know that Nashorn used to use java.lang.reflect.Array#getLength (not sure if this is still the case), but this makes usage inconsistent. You have to explicitly create a MethodHandle using the reflective array class to use it: And it is not strongly typed (it accepts Object). A real MethodHandle would be created for a specific type of array class and would return its length without runtime type checks. I know that at least Array.getLength() is optimized by Hotspot (in contrast to the other get/set methods, which are like 20 times slower than a method access, so MethodHandles.arrayElementGetter/Setter is very useful), but I would really like to suggest to fix this!
With Java 9 this gets a bit worse: There is no "easy way" with the MethodHanldes class to generate a MethodHandles.countedLoop() on all elements of an array:
public static MethodHandle countedLoop(MethodHandle iterations, MethodHandle init, MethodHandle body) [new in Java 9]
With a full-featured API one could write:
Class<?> type = arraytype, e.g. long[].class or String[].class MethodHandle body = some code that uses MethodHandles.arrayElementGetter(type) for further processing MethodHandles.countedLoop(MethodHandles.arrayLengthGetter(type), MethodHandles.constant(int.class, 0), body);
As you see, for the first parameter (count), you would need to use the reflective part in java.lang.reflect.Array if the method is still missing in Java 9. This is not bad here, because it is not called all the time, but for our scripting language, the reflective class was slower.
We implemented our own version of "arrayLengthGetter":
public class ArrayLengthHelper { private ArrayLengthHelper() {}
private static final Lookup PRIV_LOOKUP = MethodHandles.lookup(); private static final Map<Class<?>,MethodHandle> ARRAY_TYPE_MH_MAPPING = Collections.unmodifiableMap( Stream.of(boolean[].class, byte[].class, short[].class, int[].class, long[].class, char[].class, float[].class, double[].class, Object[].class) .collect(Collectors.toMap(Function.identity(), type -> { try { return PRIV_LOOKUP.findStatic(PRIV_LOOKUP.lookupClass(), "getArrayLength", MethodType.methodType(int.class, type)); } catch (ReflectiveOperationException e) { throw new AssertionError(e); } })) ); private static final MethodHandle OBJECT_ARRAY_MH = ARRAY_TYPE_MH_MAPPING.get(Object[].class);
static int getArrayLength(boolean[] array) { return array.length; } static int getArrayLength(byte[] array) { return array.length; } static int getArrayLength(short[] array) { return array.length; } static int getArrayLength(int[] array) { return array.length; } static int getArrayLength(long[] array) { return array.length; } static int getArrayLength(char[] array) { return array.length; } static int getArrayLength(float[] array) { return array.length; } static int getArrayLength(double[] array) { return array.length; } static int getArrayLength(Object[] array) { return array.length; }
public static MethodHandle arrayLengthGetter(Class<?> arrayType) { if (!arrayType.isArray()) { throw new IllegalArgumentException("type must be an array"); } return (ARRAY_TYPE_MH_MAPPING.containsKey(arrayType)) ? ARRAY_TYPE_MH_MAPPING.get(arrayType) :
OBJECT_ARRAY_MH.asType(OBJECT_ARRAY_MH.type().changeParameterTy pe(0,
arrayType)); } }
Interestingly I later found out that
MethodHandles.arrayElementGetter/Setter
uses the same "trick" behind the scenes! See MethodHandleImpl.ArrayAccessor: http://goo.gl/94f6OB
I would suggest to add the missing Method to MethodHandles class and implement it together with the getters and setters in ArrayAccessor, similar to our example code (it is just a few lines more). In addition this one could then also use the extra intrinsic improvement that our class cannot use: makeIntrinsic(getAccessor(Object[].class, false), Intrinsic.ARRAY_LOAD); [with another intrinsic added: Intrinsic.ARRAY_LENGTH for the arraylength bytecode]
What would be the process to propose such a change? Bug/Issue/JEP? I could quickly create a patch, but fixing the Intrinsic/LambdaForm stuff is way too complicated for me, so I would suggest to take this as an inspiration how to do it
Uwe
as you said, Array.getLength is already optimized by the JIT, it's because the length of an array is at the same offset for any kind of array in Hotspot, so unlike array access that depends on the kind of array, getting the length of an array is the same code for any kind of array. So you can safely, performance wise, create a method handle on Array.getLength, that's why there is no arrayLengthGetter in MethodHandles.
That might be fine, but it is not user friendly. You have to spend a lot of time to get this running. If the API would be consistent the above countedLoop would be a no-brainer to implement. I don’t care if the MethodHandles.arrayLengthGetter is implemented with java.lang.reflect.Array or the proposed code in MethodHandlesImpl.ArrayAccessor. It should just be abstracted and an implementation detail. Because the array length is something very special, so there should be an "official and easy to understand way" to get a MethodHandle for it, that is consistent with the other API. MethodHandles has accessor to array elements but not to length - that seems wrong - really wrong, sorry! If you think that java.lang.reflect.Array is fine because of the intrinsic, then just implement MethodHandles.arrayLengthGetter like: private static final MH_ARRAY_LENGTH = publicLookup().findStatic(Array.class, "getLength", methodType(int.class, Object.class); // add trycatch public static MethodHandle arrayLengthGetter(Class<?> arrayType) { if (!arrayType.isArray()) { throw new IllegalArgumentException("arrayType must be an array"); } return MH_ARRAY_LENGTH.asType(MH_ARRAY_LENGTH.type().changeParameterType(0, arrayType)); } It should just be there! The change is simple (both cases), so why not add it? What do others think? Uwe
Hi Uwe, If you’re targetting JDK 9 only, you could also consider basing your language’s dynamic operations on Dynalink; it’s integrated into JDK 9 and is specifically meant as a way to ease implementation of languages that have dynamic types. It happens to have a predefined GET_LENGTH[1] operation that does exactly what you want when BeansLinker[2] applies it to Java arrays. The documentation can be found at <http://download.java.net/java/jdk9/docs/jdk/api/dynalink <http://download.java.net/java/jdk9/docs/jdk/api/dynalink//jdk/dynalink/StandardOperation.html#GET_LENGTH>>; the jdk.dynalink package description is a good starting point (for some reason the main package is listed in a separate “Other Packages” section on the overview page… probably a Javadoc glitch w/regard to Jigsaw) Attila. [1] http://download.java.net/java/jdk9/docs/jdk/api/dynalink//jdk/dynalink/Stand... <http://download.java.net/java/jdk9/docs/jdk/api/dynalink//jdk/dynalink/StandardOperation.html#GET_LENGTH> [2] http://download.java.net/java/jdk9/docs/jdk/api/dynalink/jdk/dynalink/beans/...
On 11 May 2016, at 21:35, Uwe Schindler <uschindler@apache.org> wrote:
Hi,
while working and trying the new JDK9 MethodHandles features like MethodHandles#countedLoop, I recognized a API inconsistency problem with it.
We also found this while implementing the "Painless" scripting language for Elasticsearch (see https://goo.gl/DbOzjC)! Painless is a very limited scripting language with a very limited subset of classes to access and close integration into Elasticsearch (similar to Lucene's Expressions Module). Both compile script code to anonymous classes and Elasticsearch uses invokedynamic for the untyped stuff. It is much faster than Groovy or Nashorn, because it compiles most of the code completely static with known types (like Lucene expressions), but also supports a Groovy-like "def" type. The latter one uses invokedynamic, everything else is exactly the same speed as native Java code.
During implementing array getters and setters for untyped stuff with invokedynamic we recognized the following problem (Java 8 code, see https://github.com/elastic/elasticsearch/pull/18232):
There is MethodHandles.arrayElementGetter and MethodHandles.arrayElementSetter, so we can have full speed and full dynamic flexibility for array loads and stores - the speed is great, same as native array access! But there is one factory method missing in Java 8: MethodHandles.arrayLengthGetter (or similar name). As "length" is not a field on the array class, it is impossible to create a "simple" MethodHandle as field access to "array.class#length" to access it (it must invoke the "arraylength" bytecode, there is no field). I think the "length" field is just syntactic sugar of javac compiler.
I know that Nashorn used to use java.lang.reflect.Array#getLength (not sure if this is still the case), but this makes usage inconsistent. You have to explicitly create a MethodHandle using the reflective array class to use it: And it is not strongly typed (it accepts Object). A real MethodHandle would be created for a specific type of array class and would return its length without runtime type checks. I know that at least Array.getLength() is optimized by Hotspot (in contrast to the other get/set methods, which are like 20 times slower than a method access, so MethodHandles.arrayElementGetter/Setter is very useful), but I would really like to suggest to fix this!
With Java 9 this gets a bit worse: There is no "easy way" with the MethodHanldes class to generate a MethodHandles.countedLoop() on all elements of an array:
public static MethodHandle countedLoop(MethodHandle iterations, MethodHandle init, MethodHandle body) [new in Java 9]
With a full-featured API one could write:
Class<?> type = arraytype, e.g. long[].class or String[].class MethodHandle body = some code that uses MethodHandles.arrayElementGetter(type) for further processing MethodHandles.countedLoop(MethodHandles.arrayLengthGetter(type), MethodHandles.constant(int.class, 0), body);
As you see, for the first parameter (count), you would need to use the reflective part in java.lang.reflect.Array if the method is still missing in Java 9. This is not bad here, because it is not called all the time, but for our scripting language, the reflective class was slower.
We implemented our own version of "arrayLengthGetter":
public class ArrayLengthHelper { private ArrayLengthHelper() {}
private static final Lookup PRIV_LOOKUP = MethodHandles.lookup(); private static final Map<Class<?>,MethodHandle> ARRAY_TYPE_MH_MAPPING = Collections.unmodifiableMap( Stream.of(boolean[].class, byte[].class, short[].class, int[].class, long[].class, char[].class, float[].class, double[].class, Object[].class) .collect(Collectors.toMap(Function.identity(), type -> { try { return PRIV_LOOKUP.findStatic(PRIV_LOOKUP.lookupClass(), "getArrayLength", MethodType.methodType(int.class, type)); } catch (ReflectiveOperationException e) { throw new AssertionError(e); } })) ); private static final MethodHandle OBJECT_ARRAY_MH = ARRAY_TYPE_MH_MAPPING.get(Object[].class);
static int getArrayLength(boolean[] array) { return array.length; } static int getArrayLength(byte[] array) { return array.length; } static int getArrayLength(short[] array) { return array.length; } static int getArrayLength(int[] array) { return array.length; } static int getArrayLength(long[] array) { return array.length; } static int getArrayLength(char[] array) { return array.length; } static int getArrayLength(float[] array) { return array.length; } static int getArrayLength(double[] array) { return array.length; } static int getArrayLength(Object[] array) { return array.length; }
public static MethodHandle arrayLengthGetter(Class<?> arrayType) { if (!arrayType.isArray()) { throw new IllegalArgumentException("type must be an array"); } return (ARRAY_TYPE_MH_MAPPING.containsKey(arrayType)) ? ARRAY_TYPE_MH_MAPPING.get(arrayType) : OBJECT_ARRAY_MH.asType(OBJECT_ARRAY_MH.type().changeParameterType(0, arrayType)); } }
Interestingly I later found out that MethodHandles.arrayElementGetter/Setter uses the same "trick" behind the scenes! See MethodHandleImpl.ArrayAccessor: http://goo.gl/94f6OB
I would suggest to add the missing Method to MethodHandles class and implement it together with the getters and setters in ArrayAccessor, similar to our example code (it is just a few lines more). In addition this one could then also use the extra intrinsic improvement that our class cannot use: makeIntrinsic(getAccessor(Object[].class, false), Intrinsic.ARRAY_LOAD); [with another intrinsic added: Intrinsic.ARRAY_LENGTH for the arraylength bytecode]
What would be the process to propose such a change? Bug/Issue/JEP? I could quickly create a patch, but fixing the Intrinsic/LambdaForm stuff is way too complicated for me, so I would suggest to take this as an inspiration how to do it
Uwe
----- Uwe Schindler uschindler@apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany http://lucene.apache.org/
Hi Attila, thanks for the info. We are currently targeting Java 8. The proposal made here was more about making the API in the MethodHandles class “feature complete”, because you have the array element getters/setters there, but not the length. This is pure inconsistent, because for countedLoop (which is new in Java 9), you coulduse it. From a user perspective this is also easier to understand. In our code we stay for now with the code as described. I was just hoping that Java 9 would make the API more consistent and easy to use. The new MethodHandles.countedLoop really cries for support to natively and straigtforward way to get the array length – because this is the “normal way to process arrays with for-loops. And this is not natively possible unless you use my hack or java.lang.reflect.Array#getLength. For now, I added this as a “hack some private inner class like ArrayAccessor in the JDK”, but I would like to get rid of it once we are targetting Java 9. This is why I started the thread here. My request is just to make the public MethodHandles API easy to use and consistent, which it isn’t. And addition of that should be easy and could be done in the same JEP like the new loop stuff in MethodHandles. I’d make a proposal, but I have no idea what the process would be. FYI, we also have a DynamicCallSite in Painless that uses polymorphic caching, up to 5 different types using MethodHandles.guardWithTest: https://goo.gl/GhWmTS Uwe ----- Uwe Schindler uschindler@apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany http://lucene.apache.org/ From: Attila Szegedi [mailto:szegedia@gmail.com] Sent: Thursday, May 12, 2016 11:37 AM To: Uwe Schindler <uschindler@apache.org> Cc: Core-Libs-Dev <core-libs-dev@openjdk.java.net> Subject: Re: MethodHandle for array length Hi Uwe, If you’re targetting JDK 9 only, you could also consider basing your language’s dynamic operations on Dynalink; it’s integrated into JDK 9 and is specifically meant as a way to ease implementation of languages that have dynamic types. It happens to have a predefined GET_LENGTH[1] operation that does exactly what you want when BeansLinker[2] applies it to Java arrays. The documentation can be found at <http://download.java.net/java/jdk9/docs/jdk/api/dynalink <http://download.java.net/java/jdk9/docs/jdk/api/dynalink/jdk/dynalink/StandardOperation.html#GET_LENGTH> >; the jdk.dynalink package description is a good starting point (for some reason the main package is listed in a separate “Other Packages” section on the overview page… probably a Javadoc glitch w/regard to Jigsaw) Attila. [1] http://download.java.net/java/jdk9/docs/jdk/api/dynalink//jdk/dynalink/Stand... <http://download.java.net/java/jdk9/docs/jdk/api/dynalink/jdk/dynalink/StandardOperation.html#GET_LENGTH> [2] http://download.java.net/java/jdk9/docs/jdk/api/dynalink/jdk/dynalink/beans/... On 11 May 2016, at 21:35, Uwe Schindler <uschindler@apache.org <mailto:uschindler@apache.org> > wrote: Hi, while working and trying the new JDK9 MethodHandles features like MethodHandles#countedLoop, I recognized a API inconsistency problem with it. We also found this while implementing the "Painless" scripting language for Elasticsearch (see https://goo.gl/DbOzjC)! Painless is a very limited scripting language with a very limited subset of classes to access and close integration into Elasticsearch (similar to Lucene's Expressions Module). Both compile script code to anonymous classes and Elasticsearch uses invokedynamic for the untyped stuff. It is much faster than Groovy or Nashorn, because it compiles most of the code completely static with known types (like Lucene expressions), but also supports a Groovy-like "def" type. The latter one uses invokedynamic, everything else is exactly the same speed as native Java code. During implementing array getters and setters for untyped stuff with invokedynamic we recognized the following problem (Java 8 code, see https://github.com/elastic/elasticsearch/pull/18232): There is MethodHandles.arrayElementGetter and MethodHandles.arrayElementSetter, so we can have full speed and full dynamic flexibility for array loads and stores - the speed is great, same as native array access! But there is one factory method missing in Java 8: MethodHandles.arrayLengthGetter (or similar name). As "length" is not a field on the array class, it is impossible to create a "simple" MethodHandle as field access to "array.class#length" to access it (it must invoke the "arraylength" bytecode, there is no field). I think the "length" field is just syntactic sugar of javac compiler. I know that Nashorn used to use java.lang.reflect.Array#getLength (not sure if this is still the case), but this makes usage inconsistent. You have to explicitly create a MethodHandle using the reflective array class to use it: And it is not strongly typed (it accepts Object). A real MethodHandle would be created for a specific type of array class and would return its length without runtime type checks. I know that at least Array.getLength() is optimized by Hotspot (in contrast to the other get/set methods, which are like 20 times slower than a method access, so MethodHandles.arrayElementGetter/Setter is very useful), but I would really like to suggest to fix this! With Java 9 this gets a bit worse: There is no "easy way" with the MethodHanldes class to generate a MethodHandles.countedLoop() on all elements of an array: public static MethodHandle countedLoop(MethodHandle iterations, MethodHandle init, MethodHandle body) [new in Java 9] With a full-featured API one could write: Class<?> type = arraytype, e.g. long[].class or String[].class MethodHandle body = some code that uses MethodHandles.arrayElementGetter(type) for further processing MethodHandles.countedLoop(MethodHandles.arrayLengthGetter(type), MethodHandles.constant(int.class, 0), body); As you see, for the first parameter (count), you would need to use the reflective part in java.lang.reflect.Array if the method is still missing in Java 9. This is not bad here, because it is not called all the time, but for our scripting language, the reflective class was slower. We implemented our own version of "arrayLengthGetter": public class ArrayLengthHelper { private ArrayLengthHelper() {} private static final Lookup PRIV_LOOKUP = MethodHandles.lookup(); private static final Map<Class<?>,MethodHandle> ARRAY_TYPE_MH_MAPPING = Collections.unmodifiableMap( Stream.of(boolean[].class, byte[].class, short[].class, int[].class, long[].class, char[].class, float[].class, double[].class, Object[].class) .collect(Collectors.toMap(Function.identity(), type -> { try { return PRIV_LOOKUP.findStatic(PRIV_LOOKUP.lookupClass(), "getArrayLength", MethodType.methodType(int.class, type)); } catch (ReflectiveOperationException e) { throw new AssertionError(e); } })) ); private static final MethodHandle OBJECT_ARRAY_MH = ARRAY_TYPE_MH_MAPPING.get(Object[].class); static int getArrayLength(boolean[] array) { return array.length; } static int getArrayLength(byte[] array) { return array.length; } static int getArrayLength(short[] array) { return array.length; } static int getArrayLength(int[] array) { return array.length; } static int getArrayLength(long[] array) { return array.length; } static int getArrayLength(char[] array) { return array.length; } static int getArrayLength(float[] array) { return array.length; } static int getArrayLength(double[] array) { return array.length; } static int getArrayLength(Object[] array) { return array.length; } public static MethodHandle arrayLengthGetter(Class<?> arrayType) { if (!arrayType.isArray()) { throw new IllegalArgumentException("type must be an array"); } return (ARRAY_TYPE_MH_MAPPING.containsKey(arrayType)) ? ARRAY_TYPE_MH_MAPPING.get(arrayType) : OBJECT_ARRAY_MH.asType(OBJECT_ARRAY_MH.type().changeParameterType(0, arrayType)); } } Interestingly I later found out that MethodHandles.arrayElementGetter/Setter uses the same "trick" behind the scenes! See MethodHandleImpl.ArrayAccessor: http://goo.gl/94f6OB I would suggest to add the missing Method to MethodHandles class and implement it together with the getters and setters in ArrayAccessor, similar to our example code (it is just a few lines more). In addition this one could then also use the extra intrinsic improvement that our class cannot use: makeIntrinsic(getAccessor(Object[].class, false), Intrinsic.ARRAY_LOAD); [with another intrinsic added: Intrinsic.ARRAY_LENGTH for the arraylength bytecode] What would be the process to propose such a change? Bug/Issue/JEP? I could quickly create a patch, but fixing the Intrinsic/LambdaForm stuff is way too complicated for me, so I would suggest to take this as an inspiration how to do it Uwe ----- Uwe Schindler uschindler@apache.org <mailto:uschindler@apache.org> ASF Member, Apache Lucene PMC / Committer Bremen, Germany http://lucene.apache.org/
Hi Uwe,
Am 11.05.2016 um 21:35 schrieb Uwe Schindler <uschindler@apache.org>: With Java 9 this gets a bit worse: There is no "easy way" with the MethodHanldes class to generate a MethodHandles.countedLoop() on all elements of an array:
public static MethodHandle countedLoop(MethodHandle iterations, MethodHandle init, MethodHandle body) [new in Java 9]
this isn't a remedy when you need the index in each iteration, but you can generate a loop over all elements of an array using iteratedLoop(), as illustrated by this test from LoopCombinatorTest: @Test public static void testIterateSum() throws Throwable { // Integer[] a = new Integer[]{1,2,3,4,5,6}; int sum = 0; for (int e : a) { sum += e; } return sum; => 21 MethodHandle loop = MethodHandles.iteratedLoop(Iterate.MH_sumIterator, Iterate.MH_sumInit, Iterate.MH_sumStep); assertEquals(Iterate.MT_sum, loop.type()); assertEquals(21, loop.invoke(new Integer[]{1, 2, 3, 4, 5, 6})); } ... where MH_sumIterator is a handle to this method: static Iterator<Integer> sumIterator(Integer[] a) { return Arrays.asList(a).iterator(); } Best, Michael -- <http://www.oracle.com/> Dr. Michael Haupt | Principal Member of Technical Staff Phone: +49 331 200 7277 | Fax: +49 331 200 7561 Oracle Java Platform Group | LangTools Team | Nashorn Oracle Deutschland B.V. & Co. KG | Schiffbauergasse 14 | 14467 Potsdam, Germany ORACLE Deutschland B.V. & Co. KG | Hauptverwaltung: Riesstraße 25, D-80992 München Registergericht: Amtsgericht München, HRA 95603 Komplementärin: ORACLE Deutschland Verwaltung B.V. | Hertogswetering 163/167, 3543 AS Utrecht, Niederlande Handelsregister der Handelskammer Midden-Nederland, Nr. 30143697 Geschäftsführer: Alexander van der Ven, Jan Schultheiss, Val Maher <http://www.oracle.com/commitment> Oracle is committed to developing practices and products that help protect the environment
Hi Michael,
Am 11.05.2016 um 21:35 schrieb Uwe Schindler <uschindler@apache.org>: With Java 9 this gets a bit worse: There is no "easy way" with the MethodHanldes class to generate a MethodHandles.countedLoop() on all elements of an array:
public static MethodHandle countedLoop(MethodHandle iterations, MethodHandle init, MethodHandle body) [new in Java 9]
this isn't a remedy when you need the index in each iteration, but you can generate a loop over all elements of an array using iteratedLoop(), as illustrated by this test from LoopCombinatorTest:
@Test public static void testIterateSum() throws Throwable { // Integer[] a = new Integer[]{1,2,3,4,5,6}; int sum = 0; for (int e : a) { sum += e; } return sum; => 21 MethodHandle loop = MethodHandles.iteratedLoop(Iterate.MH_sumIterator, Iterate.MH_sumInit, Iterate.MH_sumStep); assertEquals(Iterate.MT_sum, loop.type()); assertEquals(21, loop.invoke(new Integer[]{1, 2, 3, 4, 5, 6})); }
... where MH_sumIterator is a handle to this method:
static Iterator<Integer> sumIterator(Integer[] a) { return Arrays.asList(a).iterator(); }
Of course this works, too. My proposal or question here was more about the API inconsistency in general. You can do a lot here and there, you can implement effective or non-effective loops with MethodHandles, but still the following holds true: java.lang.invoke.MethodHandles is missing an accessor for "array.length", which is not understandable. The "countedLoop" here was just an example use case (if that one is great or not does not matter), it was just posted to actually show the API and how it *could* be used. The real example from our script engine was not related to loops, it was an actual invokedynamic callsite implementation where it was not possible to figure out how to get a method handle to the very special arraylength byte code, aka "array.length javac sugar". Everything else to handle arrays is there at one single place, but the array length is missing and you get very annoyed. If you are not so familiar to bytecode and the mechanics behind, you have a very hard time to find out how to do this: the intuitive solutions does not work: "array.length" -> aha it’s a field, so let's try Lookup.findGetter(long[].class, "length", int.class) -> does not work! What is so hard to accept the proposal to have MethodHandles.arrayLengthGetter, implemented in the most effective way (as a methodhandle that solely calls the special bytecode "arraylength")? In fact thais is like a 10 liner to implement + Javadocs. No need to delay a release, there are still changes in JDK 9 going on that are far more risky than this. Best, Uwe
Best,
Michael
--
<http://www.oracle.com/> Dr. Michael Haupt | Principal Member of Technical Staff Phone: +49 331 200 7277 | Fax: +49 331 200 7561 Oracle Java Platform Group | LangTools Team | Nashorn Oracle Deutschland B.V. & Co. KG | Schiffbauergasse 14 | 14467 Potsdam, Germany
ORACLE Deutschland B.V. & Co. KG | Hauptverwaltung: Riesstraße 25, D- 80992 München Registergericht: Amtsgericht München, HRA 95603
Komplementärin: ORACLE Deutschland Verwaltung B.V. | Hertogswetering 163/167, 3543 AS Utrecht, Niederlande Handelsregister der Handelskammer Midden-Nederland, Nr. 30143697 Geschäftsführer: Alexander van der Ven, Jan Schultheiss, Val Maher <http://www.oracle.com/commitment> Oracle is committed to developing practices and products that help protect the environment
Hi Uwe, I was planning to add a bug for this feature but it seems that Michael was faster than me, https://bugs.openjdk.java.net/browse/JDK-8156915 regards, Rémi ----- Mail original -----
De: "Uwe Schindler" <uschindler@apache.org> À: "Michael Haupt" <michael.haupt@oracle.com>, "Core-Libs-Dev" <core-libs-dev@openjdk.java.net> Envoyé: Jeudi 12 Mai 2016 14:34:34 Objet: RE: MethodHandle for array length
Hi Michael,
Am 11.05.2016 um 21:35 schrieb Uwe Schindler <uschindler@apache.org>: With Java 9 this gets a bit worse: There is no "easy way" with the MethodHanldes class to generate a MethodHandles.countedLoop() on all elements of an array:
public static MethodHandle countedLoop(MethodHandle iterations, MethodHandle init, MethodHandle body) [new in Java 9]
this isn't a remedy when you need the index in each iteration, but you can generate a loop over all elements of an array using iteratedLoop(), as illustrated by this test from LoopCombinatorTest:
@Test public static void testIterateSum() throws Throwable { // Integer[] a = new Integer[]{1,2,3,4,5,6}; int sum = 0; for (int e : a) { sum += e; } return sum; => 21 MethodHandle loop = MethodHandles.iteratedLoop(Iterate.MH_sumIterator, Iterate.MH_sumInit, Iterate.MH_sumStep); assertEquals(Iterate.MT_sum, loop.type()); assertEquals(21, loop.invoke(new Integer[]{1, 2, 3, 4, 5, 6})); }
... where MH_sumIterator is a handle to this method:
static Iterator<Integer> sumIterator(Integer[] a) { return Arrays.asList(a).iterator(); }
Of course this works, too.
My proposal or question here was more about the API inconsistency in general. You can do a lot here and there, you can implement effective or non-effective loops with MethodHandles, but still the following holds true: java.lang.invoke.MethodHandles is missing an accessor for "array.length", which is not understandable. The "countedLoop" here was just an example use case (if that one is great or not does not matter), it was just posted to actually show the API and how it *could* be used.
The real example from our script engine was not related to loops, it was an actual invokedynamic callsite implementation where it was not possible to figure out how to get a method handle to the very special arraylength byte code, aka "array.length javac sugar". Everything else to handle arrays is there at one single place, but the array length is missing and you get very annoyed. If you are not so familiar to bytecode and the mechanics behind, you have a very hard time to find out how to do this: the intuitive solutions does not work: "array.length" -> aha it’s a field, so let's try Lookup.findGetter(long[].class, "length", int.class) -> does not work!
What is so hard to accept the proposal to have MethodHandles.arrayLengthGetter, implemented in the most effective way (as a methodhandle that solely calls the special bytecode "arraylength")? In fact thais is like a 10 liner to implement + Javadocs. No need to delay a release, there are still changes in JDK 9 going on that are far more risky than this.
Best, Uwe
Best,
Michael
--
<http://www.oracle.com/> Dr. Michael Haupt | Principal Member of Technical Staff Phone: +49 331 200 7277 | Fax: +49 331 200 7561 Oracle Java Platform Group | LangTools Team | Nashorn Oracle Deutschland B.V. & Co. KG | Schiffbauergasse 14 | 14467 Potsdam, Germany
ORACLE Deutschland B.V. & Co. KG | Hauptverwaltung: Riesstraße 25, D- 80992 München Registergericht: Amtsgericht München, HRA 95603
Komplementärin: ORACLE Deutschland Verwaltung B.V. | Hertogswetering 163/167, 3543 AS Utrecht, Niederlande Handelsregister der Handelskammer Midden-Nederland, Nr. 30143697 Geschäftsführer: Alexander van der Ven, Jan Schultheiss, Val Maher <http://www.oracle.com/commitment> Oracle is committed to developing practices and products that help protect the environment
Hi, OK, thanks for creating an issue! Uwe ----- Uwe Schindler uschindler@apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany http://lucene.apache.org/
-----Original Message----- From: Remi Forax [mailto:forax@univ-mlv.fr] Sent: Friday, May 13, 2016 10:37 AM To: Uwe Schindler <uschindler@apache.org> Cc: Michael Haupt <michael.haupt@oracle.com>; Core-Libs-Dev <core-libs- dev@openjdk.java.net> Subject: Re: MethodHandle for array length
Hi Uwe, I was planning to add a bug for this feature but it seems that Michael was faster than me, https://bugs.openjdk.java.net/browse/JDK-8156915
regards, Rémi
----- Mail original -----
De: "Uwe Schindler" <uschindler@apache.org> À: "Michael Haupt" <michael.haupt@oracle.com>, "Core-Libs-Dev" <core- libs-dev@openjdk.java.net> Envoyé: Jeudi 12 Mai 2016 14:34:34 Objet: RE: MethodHandle for array length
Hi Michael,
Am 11.05.2016 um 21:35 schrieb Uwe Schindler <uschindler@apache.org>: With Java 9 this gets a bit worse: There is no "easy way" with the MethodHanldes class to generate a MethodHandles.countedLoop() on all elements of an array:
public static MethodHandle countedLoop(MethodHandle iterations, MethodHandle init, MethodHandle body) [new in Java 9]
this isn't a remedy when you need the index in each iteration, but you can generate a loop over all elements of an array using iteratedLoop(), as illustrated by this test from LoopCombinatorTest:
@Test public static void testIterateSum() throws Throwable { // Integer[] a = new Integer[]{1,2,3,4,5,6}; int sum = 0; for (int e : a) { sum += e; } return sum; => 21 MethodHandle loop = MethodHandles.iteratedLoop(Iterate.MH_sumIterator, Iterate.MH_sumInit, Iterate.MH_sumStep); assertEquals(Iterate.MT_sum, loop.type()); assertEquals(21, loop.invoke(new Integer[]{1, 2, 3, 4, 5, 6})); }
... where MH_sumIterator is a handle to this method:
static Iterator<Integer> sumIterator(Integer[] a) { return Arrays.asList(a).iterator(); }
Of course this works, too.
My proposal or question here was more about the API inconsistency in general. You can do a lot here and there, you can implement effective or non-effective loops with MethodHandles, but still the following holds true: java.lang.invoke.MethodHandles is missing an accessor for "array.length", which is not understandable. The "countedLoop" here was just an example use case (if that one is great or not does not matter), it was just posted to actually show the API and how it *could* be used.
The real example from our script engine was not related to loops, it was an actual invokedynamic callsite implementation where it was not possible to figure out how to get a method handle to the very special arraylength byte code, aka "array.length javac sugar". Everything else to handle arrays is there at one single place, but the array length is missing and you get very annoyed. If you are not so familiar to bytecode and the mechanics behind, you have a very hard time to find out how to do this: the intuitive solutions does not work: "array.length" -> aha it’s a field, so let's try Lookup.findGetter(long[].class, "length", int.class) -> does not work!
What is so hard to accept the proposal to have MethodHandles.arrayLengthGetter, implemented in the most effective way (as a methodhandle that solely calls the special bytecode "arraylength")? In fact thais is like a 10 liner to implement + Javadocs. No need to delay a release, there are still changes in JDK 9 going on that are far more risky than this.
Best, Uwe
Best,
Michael
--
<http://www.oracle.com/> Dr. Michael Haupt | Principal Member of Technical Staff Phone: +49 331 200 7277 | Fax: +49 331 200 7561 Oracle Java Platform Group | LangTools Team | Nashorn Oracle Deutschland B.V. & Co. KG | Schiffbauergasse 14 | 14467 Potsdam, Germany
ORACLE Deutschland B.V. & Co. KG | Hauptverwaltung: Riesstraße 25, D- 80992 München Registergericht: Amtsgericht München, HRA 95603
Komplementärin: ORACLE Deutschland Verwaltung B.V. | Hertogswetering 163/167, 3543 AS Utrecht, Niederlande Handelsregister der Handelskammer Midden-Nederland, Nr. 30143697 Geschäftsführer: Alexander van der Ven, Jan Schultheiss, Val Maher <http://www.oracle.com/commitment> Oracle is committed to developing practices and products that help protect the environment
Hi Uwe, you're welcome. Note it's up for review now: http://mail.openjdk.java.net/pipermail/core-libs-dev/2016-May/041211.html Best, Michael
Am 13.05.2016 um 11:54 schrieb Uwe Schindler <uschindler@apache.org>:
Hi,
OK, thanks for creating an issue!
Uwe
----- Uwe Schindler uschindler@apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany http://lucene.apache.org/
-----Original Message----- From: Remi Forax [mailto:forax@univ-mlv.fr] Sent: Friday, May 13, 2016 10:37 AM To: Uwe Schindler <uschindler@apache.org> Cc: Michael Haupt <michael.haupt@oracle.com>; Core-Libs-Dev <core-libs- dev@openjdk.java.net> Subject: Re: MethodHandle for array length
Hi Uwe, I was planning to add a bug for this feature but it seems that Michael was faster than me, https://bugs.openjdk.java.net/browse/JDK-8156915
regards, Rémi
----- Mail original -----
De: "Uwe Schindler" <uschindler@apache.org> À: "Michael Haupt" <michael.haupt@oracle.com>, "Core-Libs-Dev" <core- libs-dev@openjdk.java.net> Envoyé: Jeudi 12 Mai 2016 14:34:34 Objet: RE: MethodHandle for array length
Hi Michael,
Am 11.05.2016 um 21:35 schrieb Uwe Schindler <uschindler@apache.org>: With Java 9 this gets a bit worse: There is no "easy way" with the MethodHanldes class to generate a MethodHandles.countedLoop() on all elements of an array:
public static MethodHandle countedLoop(MethodHandle iterations, MethodHandle init, MethodHandle body) [new in Java 9]
this isn't a remedy when you need the index in each iteration, but you can generate a loop over all elements of an array using iteratedLoop(), as illustrated by this test from LoopCombinatorTest:
@Test public static void testIterateSum() throws Throwable { // Integer[] a = new Integer[]{1,2,3,4,5,6}; int sum = 0; for (int e : a) { sum += e; } return sum; => 21 MethodHandle loop = MethodHandles.iteratedLoop(Iterate.MH_sumIterator, Iterate.MH_sumInit, Iterate.MH_sumStep); assertEquals(Iterate.MT_sum, loop.type()); assertEquals(21, loop.invoke(new Integer[]{1, 2, 3, 4, 5, 6})); }
... where MH_sumIterator is a handle to this method:
static Iterator<Integer> sumIterator(Integer[] a) { return Arrays.asList(a).iterator(); }
Of course this works, too.
My proposal or question here was more about the API inconsistency in general. You can do a lot here and there, you can implement effective or non-effective loops with MethodHandles, but still the following holds true: java.lang.invoke.MethodHandles is missing an accessor for "array.length", which is not understandable. The "countedLoop" here was just an example use case (if that one is great or not does not matter), it was just posted to actually show the API and how it *could* be used.
The real example from our script engine was not related to loops, it was an actual invokedynamic callsite implementation where it was not possible to figure out how to get a method handle to the very special arraylength byte code, aka "array.length javac sugar". Everything else to handle arrays is there at one single place, but the array length is missing and you get very annoyed. If you are not so familiar to bytecode and the mechanics behind, you have a very hard time to find out how to do this: the intuitive solutions does not work: "array.length" -> aha it’s a field, so let's try Lookup.findGetter(long[].class, "length", int.class) -> does not work!
What is so hard to accept the proposal to have MethodHandles.arrayLengthGetter, implemented in the most effective way (as a methodhandle that solely calls the special bytecode "arraylength")? In fact thais is like a 10 liner to implement + Javadocs. No need to delay a release, there are still changes in JDK 9 going on that are far more risky than this.
Best, Uwe
-- <http://www.oracle.com/> Dr. Michael Haupt | Principal Member of Technical Staff Phone: +49 331 200 7277 | Fax: +49 331 200 7561 Oracle Java Platform Group | LangTools Team | Nashorn Oracle Deutschland B.V. & Co. KG | Schiffbauergasse 14 | 14467 Potsdam, Germany ORACLE Deutschland B.V. & Co. KG | Hauptverwaltung: Riesstraße 25, D-80992 München Registergericht: Amtsgericht München, HRA 95603 Komplementärin: ORACLE Deutschland Verwaltung B.V. | Hertogswetering 163/167, 3543 AS Utrecht, Niederlande Handelsregister der Handelskammer Midden-Nederland, Nr. 30143697 Geschäftsführer: Alexander van der Ven, Jan Schultheiss, Val Maher <http://www.oracle.com/commitment> Oracle is committed to developing practices and products that help protect the environment
Hi, One addition, maybe add to issue: If this was added, jdk.dynalink module could use it, too - this was mentioned by Attila already: http://hg.openjdk.java.net/jdk9/dev/nashorn/file/4b118e012ac4/src/jdk.dynali... Uwe ----- Uwe Schindler uschindler@apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany http://lucene.apache.org/
-----Original Message----- From: Uwe Schindler [mailto:uschindler@apache.org] Sent: Friday, May 13, 2016 11:55 AM To: 'Remi Forax' <forax@univ-mlv.fr> Cc: 'Michael Haupt' <michael.haupt@oracle.com>; 'Core-Libs-Dev' <core-libs- dev@openjdk.java.net> Subject: RE: MethodHandle for array length
Hi,
OK, thanks for creating an issue!
Uwe
----- Uwe Schindler uschindler@apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany http://lucene.apache.org/
-----Original Message----- From: Remi Forax [mailto:forax@univ-mlv.fr] Sent: Friday, May 13, 2016 10:37 AM To: Uwe Schindler <uschindler@apache.org> Cc: Michael Haupt <michael.haupt@oracle.com>; Core-Libs-Dev <core-libs- dev@openjdk.java.net> Subject: Re: MethodHandle for array length
Hi Uwe, I was planning to add a bug for this feature but it seems that Michael was faster than me, https://bugs.openjdk.java.net/browse/JDK-8156915
regards, Rémi
----- Mail original -----
De: "Uwe Schindler" <uschindler@apache.org> À: "Michael Haupt" <michael.haupt@oracle.com>, "Core-Libs-Dev" <core- libs-dev@openjdk.java.net> Envoyé: Jeudi 12 Mai 2016 14:34:34 Objet: RE: MethodHandle for array length
Hi Michael,
Am 11.05.2016 um 21:35 schrieb Uwe Schindler <uschindler@apache.org>: With Java 9 this gets a bit worse: There is no "easy way" with the MethodHanldes class to generate a MethodHandles.countedLoop() on all elements of an array:
public static MethodHandle countedLoop(MethodHandle iterations, MethodHandle init, MethodHandle body) [new in Java 9]
this isn't a remedy when you need the index in each iteration, but you can generate a loop over all elements of an array using iteratedLoop(), as illustrated by this test from LoopCombinatorTest:
@Test public static void testIterateSum() throws Throwable { // Integer[] a = new Integer[]{1,2,3,4,5,6}; int sum = 0; for (int e : a) { sum += e; } return sum; => 21 MethodHandle loop = MethodHandles.iteratedLoop(Iterate.MH_sumIterator, Iterate.MH_sumInit, Iterate.MH_sumStep); assertEquals(Iterate.MT_sum, loop.type()); assertEquals(21, loop.invoke(new Integer[]{1, 2, 3, 4, 5, 6})); }
... where MH_sumIterator is a handle to this method:
static Iterator<Integer> sumIterator(Integer[] a) { return Arrays.asList(a).iterator(); }
Of course this works, too.
My proposal or question here was more about the API inconsistency in general. You can do a lot here and there, you can implement effective or non-effective loops with MethodHandles, but still the following holds true: java.lang.invoke.MethodHandles is missing an accessor for "array.length", which is not understandable. The "countedLoop" here was just an example use case (if that one is great or not does not matter), it was just posted to actually show the API and how it *could* be used.
The real example from our script engine was not related to loops, it was an actual invokedynamic callsite implementation where it was not possible to figure out how to get a method handle to the very special arraylength byte code, aka "array.length javac sugar". Everything else to handle arrays is there at one single place, but the array length is missing and you get very annoyed. If you are not so familiar to bytecode and the mechanics behind, you have a very hard time to find out how to do this: the intuitive solutions does not work: "array.length" -> aha it’s a field, so let's try Lookup.findGetter(long[].class, "length", int.class) -> does not work!
What is so hard to accept the proposal to have MethodHandles.arrayLengthGetter, implemented in the most effective way (as a methodhandle that solely calls the special bytecode "arraylength")? In fact thais is like a 10 liner to implement + Javadocs. No need to delay a release, there are still changes in JDK 9 going on that are far more risky than this.
Best, Uwe
Best,
Michael
--
<http://www.oracle.com/> Dr. Michael Haupt | Principal Member of Technical Staff Phone: +49 331 200 7277 | Fax: +49 331 200 7561 Oracle Java Platform Group | LangTools Team | Nashorn Oracle Deutschland B.V. & Co. KG | Schiffbauergasse 14 | 14467 Potsdam, Germany
ORACLE Deutschland B.V. & Co. KG | Hauptverwaltung: Riesstraße 25, D- 80992 München Registergericht: Amtsgericht München, HRA 95603
Komplementärin: ORACLE Deutschland Verwaltung B.V. | Hertogswetering 163/167, 3543 AS Utrecht, Niederlande Handelsregister der Handelskammer Midden-Nederland, Nr. 30143697 Geschäftsführer: Alexander van der Ven, Jan Schultheiss, Val Maher <http://www.oracle.com/commitment> Oracle is committed to developing practices and products that help protect the environment
Hi Uwe, it's a different issue, but still: https://bugs.openjdk.java.net/browse/JDK-8157225 Best, Michael
Am 13.05.2016 um 12:14 schrieb Uwe Schindler <uschindler@apache.org>:
Hi,
One addition, maybe add to issue:
If this was added, jdk.dynalink module could use it, too - this was mentioned by Attila already: http://hg.openjdk.java.net/jdk9/dev/nashorn/file/4b118e012ac4/src/jdk.dynali...
Uwe
----- Uwe Schindler uschindler@apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany http://lucene.apache.org/
-----Original Message----- From: Uwe Schindler [mailto:uschindler@apache.org] Sent: Friday, May 13, 2016 11:55 AM To: 'Remi Forax' <forax@univ-mlv.fr> Cc: 'Michael Haupt' <michael.haupt@oracle.com>; 'Core-Libs-Dev' <core-libs- dev@openjdk.java.net> Subject: RE: MethodHandle for array length
Hi,
OK, thanks for creating an issue!
Uwe
----- Uwe Schindler uschindler@apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany http://lucene.apache.org/
-----Original Message----- From: Remi Forax [mailto:forax@univ-mlv.fr] Sent: Friday, May 13, 2016 10:37 AM To: Uwe Schindler <uschindler@apache.org> Cc: Michael Haupt <michael.haupt@oracle.com>; Core-Libs-Dev <core-libs- dev@openjdk.java.net> Subject: Re: MethodHandle for array length
Hi Uwe, I was planning to add a bug for this feature but it seems that Michael was faster than me, https://bugs.openjdk.java.net/browse/JDK-8156915
regards, Rémi
----- Mail original -----
De: "Uwe Schindler" <uschindler@apache.org> À: "Michael Haupt" <michael.haupt@oracle.com>, "Core-Libs-Dev" <core- libs-dev@openjdk.java.net> Envoyé: Jeudi 12 Mai 2016 14:34:34 Objet: RE: MethodHandle for array length
Hi Michael,
Am 11.05.2016 um 21:35 schrieb Uwe Schindler <uschindler@apache.org>: With Java 9 this gets a bit worse: There is no "easy way" with the MethodHanldes class to generate a MethodHandles.countedLoop() on all elements of an array:
public static MethodHandle countedLoop(MethodHandle iterations, MethodHandle init, MethodHandle body) [new in Java 9]
this isn't a remedy when you need the index in each iteration, but you can generate a loop over all elements of an array using iteratedLoop(), as illustrated by this test from LoopCombinatorTest:
@Test public static void testIterateSum() throws Throwable { // Integer[] a = new Integer[]{1,2,3,4,5,6}; int sum = 0; for (int e : a) { sum += e; } return sum; => 21 MethodHandle loop = MethodHandles.iteratedLoop(Iterate.MH_sumIterator, Iterate.MH_sumInit, Iterate.MH_sumStep); assertEquals(Iterate.MT_sum, loop.type()); assertEquals(21, loop.invoke(new Integer[]{1, 2, 3, 4, 5, 6})); }
... where MH_sumIterator is a handle to this method:
static Iterator<Integer> sumIterator(Integer[] a) { return Arrays.asList(a).iterator(); }
Of course this works, too.
My proposal or question here was more about the API inconsistency in general. You can do a lot here and there, you can implement effective or non-effective loops with MethodHandles, but still the following holds true: java.lang.invoke.MethodHandles is missing an accessor for "array.length", which is not understandable. The "countedLoop" here was just an example use case (if that one is great or not does not matter), it was just posted to actually show the API and how it *could* be used.
The real example from our script engine was not related to loops, it was an actual invokedynamic callsite implementation where it was not possible to figure out how to get a method handle to the very special arraylength byte code, aka "array.length javac sugar". Everything else to handle arrays is there at one single place, but the array length is missing and you get very annoyed. If you are not so familiar to bytecode and the mechanics behind, you have a very hard time to find out how to do this: the intuitive solutions does not work: "array.length" -> aha it’s a field, so let's try Lookup.findGetter(long[].class, "length", int.class) -> does not work!
What is so hard to accept the proposal to have MethodHandles.arrayLengthGetter, implemented in the most effective way (as a methodhandle that solely calls the special bytecode "arraylength")? In fact thais is like a 10 liner to implement + Javadocs. No need to delay a release, there are still changes in JDK 9 going on that are far more risky than this.
Best, Uwe
-- <http://www.oracle.com/> Dr. Michael Haupt | Principal Member of Technical Staff Phone: +49 331 200 7277 | Fax: +49 331 200 7561 Oracle Java Platform Group | LangTools Team | Nashorn Oracle Deutschland B.V. & Co. KG | Schiffbauergasse 14 | 14467 Potsdam, Germany ORACLE Deutschland B.V. & Co. KG | Hauptverwaltung: Riesstraße 25, D-80992 München Registergericht: Amtsgericht München, HRA 95603 Komplementärin: ORACLE Deutschland Verwaltung B.V. | Hertogswetering 163/167, 3543 AS Utrecht, Niederlande Handelsregister der Handelskammer Midden-Nederland, Nr. 30143697 Geschäftsführer: Alexander van der Ven, Jan Schultheiss, Val Maher <http://www.oracle.com/commitment> Oracle is committed to developing practices and products that help protect the environment
participants (4)
-
Attila Szegedi
-
Michael Haupt
-
Remi Forax
-
Uwe Schindler