From tomasz.kowalczewski at gmail.com Wed Jan 2 09:34:16 2019 From: tomasz.kowalczewski at gmail.com (Tomasz Kowalczewski) Date: Wed, 2 Jan 2019 10:34:16 +0100 Subject: [vector api] Operations on Mask-s Message-ID: Hi, I was working on implementing a simple sorting algorithm[1] using VectorAPI and stumbled on something that might be a gap in Mask API. It might be intentional for this stage of API maturity - please advise. I need to modify bits of the mask I get from _mm512_cmp_X_mask (e.g. vector.lessThan(input)) by doing and-s and or-s which can be simulated by preparing appropriate Mask objects. What I found missing is shift operations. Example: ( compMask & 0x55 ) | ( ( compMask & 0x55 ) << 1) I cannot do this using Mask object. Most natural alternative would be to convert the mask into long, perform these operations and convert it back. Unfortunately I was unable to find how to convert long back to Mask. Of course this can be done (via boolean[] array) but that does not seem efficient :). Excuse me if I missed some API call. 1. https://hal.inria.fr/hal-01512970v1/document (page 14, Code 1). -- Thanks, Tomasz Kowalczewski From zhuoren.wz at alibaba-inc.com Wed Jan 2 09:47:04 2019 From: zhuoren.wz at alibaba-inc.com (=?UTF-8?B?546L5Y2TKOWNk+S7gSk=?=) Date: Wed, 02 Jan 2019 17:47:04 +0800 Subject: =?UTF-8?B?W1ZlY3RvckFQSV1hbnlUcnVlL2FsbFRydWUgZ290IHdyb25nIHJlc3VsdHMgd2hlbiB1c2lu?= =?UTF-8?B?ZyBVc2VBVlg9Mw==?= Message-ID: Hello, I found anyTrue/allTrue got wrong results when using UseAVX=3. Let me describe this bug. Under UseAVX=3, following assembly code may be generated for anyTrue/allTrue. 0x00002ad5dca584f6: vpxord %xmm22,%xmm22,%xmm22 0x00002ad5dca584fc: vpsubb %xmm0,%xmm22,%xmm22 0x00002ad5dca58502: vpmovsxbd %xmm22,%ymm22 0x00002ad5dca58508: vptest %ymm1,%ymm6 0x00002ad5dca5850d: setb %r10b The second operand of vptest should be ymm22. But since there is no EVEX version for vptest, it is encoded as ymm6 and gives us wrong results. Currently I am using the following fix for this bug. Move source operands into ymm14/ymm15/xmm14/xmm15, and then they are used as operands of vptest. Please give advise on this fix. diff -r 636478e1ee75 src/hotspot/cpu/x86/x86.ad --- a/src/hotspot/cpu/x86/x86.ad Thu Dec 13 16:40:28 2018 -0800 +++ b/src/hotspot/cpu/x86/x86.ad Sat Dec 29 11:16:55 2018 +0800 @@ -21930,7 +21930,7 @@ %} instruct vptest4ieq(rRegI dst, vecX src1, vecX src2) %{ - predicate(UseAVX > 0 && static_cast(n)->get_predicate() == Assembler::notZero); + predicate(UseAVX < 3 && UseAVX > 0 && static_cast(n)->get_predicate() == Assembler::notZero); match(Set dst (VectorTest src1 src2 )); format %{ "vptest $src1,$src2\n\t" "setb $dst\t!" %} @@ -21943,8 +21943,54 @@ ins_pipe( pipe_slow ); %} +instruct vptest4inaeavx3(rRegI dst, vecX src1, vecX src2, rxmm14 tmp14, rxmm15 tmp15) %{ + predicate(UseAVX > 2 && static_cast(n)->get_predicate() == Assembler::carrySet); + match(Set dst (VectorTest src1 src2 )); + effect(TEMP tmp14, TEMP tmp15); + format %{ "movdqu $tmp14,$src1\n\t" + "movdqu $tmp15,$src2\n\t" + "vptest $tmp14,$tmp15\n\t" + "setb $dst\t!" %} + ins_encode %{ + int vector_len = 0; + if ($tmp14$$XMMRegister != $src1$$XMMRegister) { + __ movdqu($tmp14$$XMMRegister, $src1$$XMMRegister); + } + if ($tmp15$$XMMRegister != $src2$$XMMRegister) { + __ movdqu($tmp15$$XMMRegister, $src2$$XMMRegister); + } + __ vptest($tmp14$$XMMRegister, $tmp15$$XMMRegister, vector_len); + __ setb(Assembler::carrySet, $dst$$Register); + __ movzbl($dst$$Register, $dst$$Register); + %} + ins_pipe( pipe_slow ); +%} + +instruct vptest4ieqavx3(rRegI dst, vecX src1, vecX src2, rxmm14 tmp14, rxmm15 tmp15) %{ + predicate(UseAVX > 0 && static_cast(n)->get_predicate() == Assembler::notZero); + match(Set dst (VectorTest src1 src2 )); + effect(TEMP tmp14, TEMP tmp15); + format %{ "movdqu $tmp14,$src1\n\t" + "movdqu $tmp15,$src2\n\t" + "vptest $tmp14,$tmp15\n\t" + "setb $dst\t!" %} + ins_encode %{ + int vector_len = 0; + if ($tmp14$$XMMRegister != $src1$$XMMRegister) { + __ movdqu($tmp14$$XMMRegister, $src1$$XMMRegister); + } + if ($tmp15$$XMMRegister != $src2$$XMMRegister) { + __ movdqu($tmp15$$XMMRegister, $src2$$XMMRegister); + } + __ vptest($tmp14$$XMMRegister, $tmp15$$XMMRegister, vector_len); + __ setb(Assembler::notZero, $dst$$Register); + __ movzbl($dst$$Register, $dst$$Register); + %} + ins_pipe( pipe_slow ); +%} + instruct vptest8inae(rRegI dst, vecY src1, vecY src2) %{ - predicate(UseAVX > 0 && static_cast(n)->get_predicate() == Assembler::carrySet); + predicate(UseAVX < 3 && UseAVX > 0 && static_cast(n)->get_predicate() == Assembler::carrySet); match(Set dst (VectorTest src1 src2 )); format %{ "vptest $src1,$src2\n\t" "setb $dst\t!" %} @@ -21958,7 +22004,7 @@ %} instruct vptest8ieq(rRegI dst, vecY src1, vecY src2) %{ - predicate(UseAVX > 0 && static_cast(n)->get_predicate() == Assembler::notZero); + predicate(UseAVX < 3 && UseAVX > 0 && static_cast(n)->get_predicate() == Assembler::notZero); match(Set dst (VectorTest src1 src2 )); format %{ "vptest $src1,$src2\n\t" "setb $dst\t!" %} @@ -21971,6 +22017,52 @@ ins_pipe( pipe_slow ); %} +instruct vptest8inaeavx3(rRegI dst, vecY src1, vecY src2, rymm14 tmp14, rymm15 tmp15) %{ + predicate(UseAVX > 2 && static_cast(n)->get_predicate() == Assembler::carrySet); + match(Set dst (VectorTest src1 src2 )); + effect(TEMP tmp14, TEMP tmp15); + format %{ "movdqu $tmp14,$src1\n\t" + "movdqu $tmp15,$src2\n\t" + "vptest $tmp14,$tmp15\n\t" + "setb $dst\t!" %} + ins_encode %{ + int vector_len = 1; + if ($tmp14$$XMMRegister != $src1$$XMMRegister) { + __ vmovdqu($tmp14$$XMMRegister, $src1$$XMMRegister); + } + if ($tmp15$$XMMRegister != $src2$$XMMRegister) { + __ vmovdqu($tmp15$$XMMRegister, $src2$$XMMRegister); + } + __ vptest($tmp14$$XMMRegister, $tmp15$$XMMRegister, vector_len); + __ setb(Assembler::carrySet, $dst$$Register); + __ movzbl($dst$$Register, $dst$$Register); + %} + ins_pipe( pipe_slow ); +%} + +instruct vptest8ieqavx3(rRegI dst, vecY src1, vecY src2, rymm14 tmp14, rymm15 tmp15) %{ + predicate(UseAVX > 2 && static_cast(n)->get_predicate() == Assembler::notZero); + match(Set dst (VectorTest src1 src2 )); + effect(TEMP tmp14, TEMP tmp15); + format %{ "movdqu $tmp14,$src1\n\t" + "movdqu $tmp15,$src2\n\t" + "vptest $tmp14,$tmp15\n\t" + "setb $dst\t!" %} + ins_encode %{ + int vector_len = 1; + if ($tmp14$$XMMRegister != $src1$$XMMRegister) { + __ vmovdqu($tmp14$$XMMRegister, $src1$$XMMRegister); + } + if ($tmp15$$XMMRegister != $src2$$XMMRegister) { + __ vmovdqu($tmp15$$XMMRegister, $src2$$XMMRegister); + } + __ vptest($tmp14$$XMMRegister, $tmp15$$XMMRegister, vector_len); + __ setb(Assembler::notZero, $dst$$Register); + __ movzbl($dst$$Register, $dst$$Register); + %} + ins_pipe( pipe_slow ); +%} + instruct loadmask8b(vecD dst, vecD src) %{ predicate(UseSSE >= 2 && n->as_Vector()->length() == 8 && n->bottom_type()->is_vect()->element_basic_type() == T_BYTE); match(Set dst (VectorLoadMask src)); diff -r 636478e1ee75 src/hotspot/cpu/x86/x86_64.ad --- a/src/hotspot/cpu/x86/x86_64.ad Thu Dec 13 16:40:28 2018 -0800 +++ b/src/hotspot/cpu/x86/x86_64.ad Sat Dec 29 11:16:55 2018 +0800 @@ -4426,6 +4426,17 @@ predicate(UseAVX == 3); format%{%} interface(REG_INTER); %} + +operand rymm14() %{ + constraint(ALLOC_IN_RC(ymm14_reg)); match(VecY); + predicate((UseSSE > 0) && (UseAVX <= 3)); format%{%} interface(REG_INTER); +%} +operand rymm15() %{ + constraint(ALLOC_IN_RC(ymm15_reg)); match(VecY); + predicate((UseSSE > 0) && (UseAVX <= 3)); format%{%} interface(REG_INTER); +%} + + //----------OPERAND CLASSES---------------------------------------------------- // Operand Classes are groups of operands that are used as to simplify // instruction definitions by not requiring the AD writer to specify separate The test to reproduce this bug. Please DO set -XX:UseAVX=3 import jdk.incubator.vector.*; import java.util.Arrays; import java.util.Random; import java.lang.reflect.Field; import java.io.IOException; import jdk.incubator.vector.Vector.Mask; import jdk.incubator.vector.Vector.Shape; public class VectorTrueTest { static Random random = new Random(); static final IntVector.IntSpecies Species = IntVector.species(Vector.Shape.S_256_BIT); public static int size = 1024 * 16; public static int length = Species.length(); public static int resultSize = size / length; static boolean[] anyResultV = new boolean[resultSize]; static boolean[] allResultV = new boolean[resultSize]; static boolean[] anyInput = new boolean[size]; static boolean[] allInput = new boolean[size]; public static void main(String[] args) throws NoSuchFieldException, SecurityException, IllegalArgumentException, IllegalAccessException, InstantiationException { long start0 = System.currentTimeMillis(); long startv = System.currentTimeMillis(); long normalTime = 0; long vecTime = 0; int i = 0; for (i = 0; i < size; i++) { anyInput[i] = false; allInput[i] = false; } for (i = 0; i < 20000; i++) { vecTest(Species); } for (i = 0; i < resultSize; i++) { anyResultV[i] = true; allResultV[i] = true; } vecTest(Species); for (i = 0; i < (resultSize - 1); i++) { if (anyResultV[i] != false) throw new RuntimeException("Wrong anyTrue result! Should be all false, index " + i); if (allResultV[i] != false) throw new RuntimeException("Wrong allTrue result! Should be all false, index " + i); } } static void vecTest(IntVector.IntSpecies Speciesint ) { IntVector v0; int i = 0; int j = 0; Mask maskAny = Speciesint.maskFromArray(anyInput, i); Mask maskAll = Speciesint.maskFromArray(allInput, i); for (i = 0; i + (Speciesint.length()) <= size; i += Speciesint.length()) { allResultV[j] = maskAll.allTrue(); anyResultV[j] = maskAny.anyTrue(); j++; } return; } } Regards, Zhuoren From maurizio.cimadamore at oracle.com Wed Jan 2 20:58:17 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 02 Jan 2019 20:58:17 +0000 Subject: hg: panama/dev: 25 new changesets Message-ID: <201901022058.x02KwK8e013961@aojmv0008.oracle.com> Changeset: 35530ca3e0b2 Author: kbarrett Date: 2018-12-26 19:24 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/35530ca3e0b2 8214201: Make PtrQueueSet completed buffer list private Summary: Merge and make private in PtrQueueSet all completed buffer list handling Reviewed-by: tschatzl, sjohanss ! src/hotspot/share/gc/g1/dirtyCardQueue.cpp ! src/hotspot/share/gc/g1/dirtyCardQueue.hpp ! src/hotspot/share/gc/g1/g1CollectedHeap.cpp ! src/hotspot/share/gc/g1/g1ConcurrentRefineThread.cpp ! src/hotspot/share/gc/shared/ptrQueue.cpp ! src/hotspot/share/gc/shared/ptrQueue.hpp ! src/hotspot/share/gc/shared/satbMarkQueue.cpp Changeset: 67e3a8b3449c Author: dholmes Date: 2018-12-27 21:17 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/67e3a8b3449c 8214097: Rework thread initialization and teardown logic Reviewed-by: rehn, mgronlun, dcubed, kbarrett ! src/hotspot/os/linux/os_linux.cpp ! src/hotspot/share/gc/parallel/gcTaskThread.cpp ! src/hotspot/share/gc/shared/concurrentGCThread.cpp ! src/hotspot/share/gc/shared/workgroup.cpp ! src/hotspot/share/jfr/periodic/sampling/jfrThreadSampler.cpp ! src/hotspot/share/runtime/thread.cpp ! src/hotspot/share/runtime/thread.hpp ! src/hotspot/share/runtime/vmThread.cpp ! src/hotspot/share/services/management.cpp ! test/hotspot/gtest/threadHelper.inline.hpp Changeset: 95937fc2a05e Author: mbaesken Date: 2018-12-21 14:42 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/95937fc2a05e 8215707: [macosx] fix pthread_getschedparam and pthread_setschedparam calls Reviewed-by: clanger, dholmes ! src/hotspot/os/bsd/os_bsd.cpp Changeset: 6ac04cfbefdf Author: dholmes Date: 2018-12-28 16:31 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/6ac04cfbefdf 8215954: [testbug] Remove unnecessary casts in test/hotspot/gtest/threadHelper.inline.hpp Reviewed-by: dcubed ! test/hotspot/gtest/threadHelper.inline.hpp Changeset: 6c3407eee455 Author: phh Date: 2018-12-28 15:19 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/6c3407eee455 8215966: GeneratePropertyPassword.sh uses bash syntax Summary: Use "case" instead of "if" for the NT check to make sh happy. Reviewed-by: dholmes Contributed-by: merkel05 at gmail.com ! test/jdk/sun/management/jmxremote/bootstrap/GeneratePropertyPassword.sh Changeset: cfceb4df2499 Author: tvaleev Date: 2018-12-30 08:57 +0700 URL: http://hg.openjdk.java.net/panama/dev/rev/cfceb4df2499 8214687: Optimize Collections.nCopies().hashCode() and equals() Reviewed-by: igerasim, smarks ! src/java.base/share/classes/java/util/Collections.java ! test/jdk/java/util/Collections/NCopies.java Changeset: e90315ae8aa9 Author: kbarrett Date: 2018-12-31 15:40 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/e90315ae8aa9 8213481: [REDO] Fix incorrect copy constructors in hotspot Summary: Fix and use ResourceObj copy constructor. Reviewed-by: coleenp, dholmes, kvn ! src/hotspot/share/classfile/stackMapFrame.hpp ! src/hotspot/share/libadt/dict.cpp ! src/hotspot/share/libadt/set.hpp ! src/hotspot/share/memory/allocation.cpp ! src/hotspot/share/memory/allocation.hpp Changeset: b99b41325d89 Author: dholmes Date: 2019-01-01 20:09 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/b99b41325d89 8215977: hsdis installation documentation update Reviewed-by: dholmes Contributed-by: Sergei Ustimenko ! src/utils/hsdis/README Changeset: 50677f43ac3d Author: erikj Date: 2019-01-02 12:59 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/50677f43ac3d 8215445: Enable building for Windows in WSL Reviewed-by: ihse Contributed-by: andrewluotechnologies at outlook.com, erik.joelsson at oracle.com ! doc/building.html ! doc/building.md ! make/Images.gmk ! make/autoconf/basics.m4 ! make/autoconf/basics_windows.m4 ! make/autoconf/boot-jdk.m4 ! make/autoconf/build-aux/config.guess ! make/autoconf/build-aux/config.sub ! make/autoconf/compare.sh.in ! make/autoconf/platform.m4 ! make/autoconf/spec.gmk.in ! make/autoconf/toolchain.m4 ! make/autoconf/toolchain_windows.m4 ! make/gendata/Gendata-java.base.gmk ! make/gensrc/GensrcBuffer.gmk ! make/gensrc/GensrcCharsetCoder.gmk ! make/gensrc/GensrcVarHandles.gmk ! make/jdk/src/classes/build/tools/generatecurrencydata/GenerateCurrencyData.java ! make/jdk/src/classes/build/tools/spp/Spp.java + make/scripts/windowsShortName.bat ! make/src/native/fixpath.c Changeset: 315f53a48199 Author: gadams Date: 2019-01-02 07:19 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/315f53a48199 8211343: nsk_jvmti_parseoptions should handle multiple suboptions Reviewed-by: sspitsyn, cjplummer ! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetNativeMethodPrefix/SetNativeMethodPrefix001/TestDescription.java ! test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/jvmti_tools.cpp Changeset: 6e8c8d16ecb4 Author: pbansal Date: 2018-12-21 11:43 +0530 URL: http://hg.openjdk.java.net/panama/dev/rev/6e8c8d16ecb4 8215364: JavaFX crashes on Ubuntu 18.04 with Wayland while using Swing-FX interop Reviewed-by: prr, kcr ! src/java.desktop/unix/native/libawt_xawt/xawt/XToolkit.c Changeset: a92cd6585f60 Author: hannesw Date: 2018-12-21 11:02 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/a92cd6585f60 8215291: Broken links when generating from project without modules Reviewed-by: jjg ! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/resources/search.js ! test/langtools/jdk/javadoc/doclet/testSearch/TestSearch.java Changeset: b0686d0be73f Author: aph Date: 2018-12-21 17:14 +0000 URL: http://hg.openjdk.java.net/panama/dev/rev/b0686d0be73f 8215202: AArch64: jtreg test test/jdk/sun/nio/cs/FindEncoderBugs.java fails Reviewed-by: aph Contributed-by: nick.gasson at arm.com ! src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp ! test/jdk/sun/nio/cs/FindEncoderBugs.java Changeset: b5c41404f2d1 Author: aph Date: 2018-12-21 18:26 +0000 URL: http://hg.openjdk.java.net/panama/dev/rev/b5c41404f2d1 8215879: AArch64: ReservedStackAccess may leave stack guard in inconsistent state Reviewed-by: aph, dholmes Contributed-by: Andrey Petushkov ! src/hotspot/os_cpu/linux_aarch64/os_linux_aarch64.cpp ! src/hotspot/share/runtime/thread.cpp Changeset: bb03098c4dde Author: sangheki Date: 2018-12-21 08:18 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/bb03098c4dde 8211425: Allocation of old generation of java heap on alternate memory devices - G1 GC 8202286: Allocation of old generation of Java heap on alternate memory devices Summary: Enable an experimental feature in HotSpot JVM to allocate old generation of G1 GC on an alternative memory device, such as NV-DIMMs. Reviewed-by: sangheki, sjohanss Contributed-by: kishor.kharbas at intel.com ! src/hotspot/os/linux/os_linux.cpp ! src/hotspot/share/gc/g1/g1Allocator.inline.hpp ! src/hotspot/share/gc/g1/g1Arguments.cpp ! src/hotspot/share/gc/g1/g1CardCounts.cpp ! src/hotspot/share/gc/g1/g1CollectedHeap.cpp ! src/hotspot/share/gc/g1/g1CollectedHeap.hpp ! src/hotspot/share/gc/g1/g1CollectedHeap.inline.hpp ! src/hotspot/share/gc/g1/g1CollectorPolicy.cpp ! src/hotspot/share/gc/g1/g1CollectorPolicy.hpp ! src/hotspot/share/gc/g1/g1HeapVerifier.cpp + src/hotspot/share/gc/g1/g1HeterogeneousCollectorPolicy.cpp + src/hotspot/share/gc/g1/g1HeterogeneousCollectorPolicy.hpp + src/hotspot/share/gc/g1/g1HeterogeneousHeapPolicy.cpp + src/hotspot/share/gc/g1/g1HeterogeneousHeapPolicy.hpp + src/hotspot/share/gc/g1/g1HeterogeneousHeapYoungGenSizer.cpp + src/hotspot/share/gc/g1/g1HeterogeneousHeapYoungGenSizer.hpp ! src/hotspot/share/gc/g1/g1PageBasedVirtualSpace.cpp ! src/hotspot/share/gc/g1/g1PageBasedVirtualSpace.hpp ! src/hotspot/share/gc/g1/g1Policy.cpp ! src/hotspot/share/gc/g1/g1Policy.hpp ! src/hotspot/share/gc/g1/g1RegionToSpaceMapper.cpp ! src/hotspot/share/gc/g1/g1RegionToSpaceMapper.hpp ! src/hotspot/share/gc/g1/g1VMOperations.cpp ! src/hotspot/share/gc/g1/g1YoungGenSizer.cpp ! src/hotspot/share/gc/g1/g1YoungGenSizer.hpp ! src/hotspot/share/gc/g1/g1_globals.hpp ! src/hotspot/share/gc/g1/heapRegionManager.cpp ! src/hotspot/share/gc/g1/heapRegionManager.hpp ! src/hotspot/share/gc/g1/heapRegionSet.cpp ! src/hotspot/share/gc/g1/heapRegionSet.hpp ! src/hotspot/share/gc/g1/heapRegionType.cpp ! src/hotspot/share/gc/g1/heapRegionType.hpp + src/hotspot/share/gc/g1/heterogeneousHeapRegionManager.cpp + src/hotspot/share/gc/g1/heterogeneousHeapRegionManager.hpp ! src/hotspot/share/gc/g1/vmStructs_g1.hpp ! src/hotspot/share/gc/shared/gcArguments.cpp ! src/hotspot/share/gc/shared/gcArguments.hpp ! src/hotspot/share/prims/whitebox.cpp ! src/hotspot/share/runtime/arguments.cpp ! src/hotspot/share/runtime/globals.hpp ! src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/g1/G1CollectedHeap.java ! test/hotspot/jtreg/TEST.groups + test/hotspot/jtreg/gc/nvdimm/TestAllocateOldGenAt.java + test/hotspot/jtreg/gc/nvdimm/TestAllocateOldGenAtError.java + test/hotspot/jtreg/gc/nvdimm/TestAllocateOldGenAtMultiple.java + test/hotspot/jtreg/gc/nvdimm/TestHumongousObjectsOnNvdimm.java + test/hotspot/jtreg/gc/nvdimm/TestOldObjectsOnNvdimm.java + test/hotspot/jtreg/gc/nvdimm/TestYoungObjectsOnDram.java ! test/lib/sun/hotspot/WhiteBox.java Changeset: 37930c6ba6d7 Author: sangheki Date: 2018-12-21 08:23 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/37930c6ba6d7 8211424: Allocation of old generation of java heap on alternate memory devices - Parallel GC 8202286: Allocation of old generation of Java heap on alternate memory devices Summary: Enable an experimental feature in HotSpot JVM to allocate old generation of Parallel GC on an alternative memory device, such as NV-DIMMs. Reviewed-by: sangheki, sjohanss Contributed-by: kishor.kharbas at intel.com ! src/hotspot/share/gc/parallel/adjoiningGenerations.cpp ! src/hotspot/share/gc/parallel/adjoiningGenerations.hpp + src/hotspot/share/gc/parallel/adjoiningGenerationsForHeteroHeap.cpp + src/hotspot/share/gc/parallel/adjoiningGenerationsForHeteroHeap.hpp ! src/hotspot/share/gc/parallel/adjoiningVirtualSpaces.hpp ! src/hotspot/share/gc/parallel/generationSizer.cpp ! src/hotspot/share/gc/parallel/generationSizer.hpp + src/hotspot/share/gc/parallel/heterogeneousGenerationSizer.cpp + src/hotspot/share/gc/parallel/heterogeneousGenerationSizer.hpp ! src/hotspot/share/gc/parallel/parallelArguments.cpp ! src/hotspot/share/gc/parallel/parallelScavengeHeap.cpp ! src/hotspot/share/gc/parallel/parallelScavengeHeap.hpp + src/hotspot/share/gc/parallel/psFileBackedVirtualspace.cpp + src/hotspot/share/gc/parallel/psFileBackedVirtualspace.hpp ! src/hotspot/share/gc/parallel/psOldGen.cpp ! src/hotspot/share/gc/parallel/psParallelCompact.cpp ! src/hotspot/share/prims/whitebox.cpp ! test/hotspot/jtreg/gc/nvdimm/TestAllocateOldGenAt.java ! test/hotspot/jtreg/gc/nvdimm/TestAllocateOldGenAtError.java ! test/hotspot/jtreg/gc/nvdimm/TestOldObjectsOnNvdimm.java ! test/hotspot/jtreg/gc/nvdimm/TestYoungObjectsOnDram.java Changeset: 93a5f4b4c67d Author: sjohanss Date: 2018-12-22 15:46 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/93a5f4b4c67d 8215897: Build broken on zero after JDK-8211424 Reviewed-by: tschatzl ! src/hotspot/share/prims/whitebox.cpp Changeset: 375b10185c40 Author: sjohanss Date: 2018-12-22 15:47 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/375b10185c40 8215898: Build broken on 32-bit after JDK-8211425 Reviewed-by: tschatzl ! src/hotspot/share/gc/shared/gcArguments.cpp Changeset: de9fd809bb47 Author: rpatil Date: 2018-12-26 17:09 +0530 URL: http://hg.openjdk.java.net/panama/dev/rev/de9fd809bb47 8214567: Use {@systemProperty} for definitions of system properties 8214569: Use {@systemProperty} for definitions of system properties Reviewed-by: lancea, mchung, alanb, naoto Contributed-by: Deepak kejriwal ! src/java.base/share/classes/java/lang/ClassLoader.java ! src/java.base/share/classes/java/net/URL.java ! src/java.base/share/classes/java/time/zone/ZoneRulesProvider.java ! src/java.base/share/classes/java/util/Currency.java ! src/java.base/share/classes/java/util/PropertyResourceBundle.java ! src/java.base/share/classes/java/util/jar/Pack200.java ! src/java.base/share/classes/java/util/spi/LocaleServiceProvider.java ! src/java.logging/share/classes/java/util/logging/LogManager.java ! src/java.logging/share/classes/java/util/logging/SimpleFormatter.java ! src/java.rmi/share/classes/java/rmi/server/ObjID.java ! src/java.rmi/share/classes/java/rmi/server/RMIClassLoader.java ! src/java.rmi/share/classes/java/rmi/server/RMISocketFactory.java ! src/java.rmi/share/classes/javax/rmi/ssl/SslRMIClientSocketFactory.java Changeset: 4c4651aba203 Author: jwilhelm Date: 2019-01-02 16:05 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/4c4651aba203 Merge ! src/hotspot/os/linux/os_linux.cpp ! src/hotspot/share/gc/g1/g1CardCounts.cpp ! src/hotspot/share/gc/g1/g1CollectedHeap.cpp ! src/hotspot/share/gc/g1/g1HeapVerifier.cpp ! src/hotspot/share/runtime/arguments.cpp ! src/hotspot/share/runtime/thread.cpp ! test/langtools/jdk/javadoc/doclet/testSearch/TestSearch.java Changeset: 883a1a80a6dc Author: hseigel Date: 2019-01-02 10:35 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/883a1a80a6dc 8215398: -Xlog option usage => Invalid decorator '\temp\app_cds.log'. Summary: On Windows, do not treat ':' as a delimeter when it's in a string such as "C:..." Reviewed-by: dholmes, sspitsyn ! src/hotspot/share/logging/logConfiguration.cpp + test/hotspot/jtreg/serviceability/logging/TestFullNames.java Changeset: 96ce82319e82 Author: ghaug Date: 2018-12-21 10:19 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/96ce82319e82 8215791: Tiny bug in VM monitoring/management Reviewed-by: dholmes, simonis ! src/hotspot/share/services/threadService.hpp Changeset: a22e41395bfa Author: apetcher Date: 2019-01-02 13:06 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/a22e41395bfa 8215643: Microbenchmarks for KeyAgreement and Cipher Summary: adding some missing microbenchmarks for crypto algorithms Reviewed-by: jnimeh + test/micro/org/openjdk/bench/javax/crypto/full/CipherBench.java + test/micro/org/openjdk/bench/javax/crypto/full/KeyAgreementBench.java + test/micro/org/openjdk/bench/javax/crypto/small/CipherBench.java + test/micro/org/openjdk/bench/javax/crypto/small/KeyAgreementBench.java Changeset: dee9426ef417 Author: redestad Date: 2019-01-02 19:06 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/dee9426ef417 8215990: Avoid using reflection to create common default URLStreamHandlers Reviewed-by: alanb ! src/java.base/share/classes/java/net/URL.java Changeset: 3d0f6ef91216 Author: ecaspole Date: 2019-01-02 13:37 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/3d0f6ef91216 8215572: Add new Arrays micros Summary: New micros for mismatch and fill Reviewed-by: kvn, vlivanov + test/micro/org/openjdk/bench/java/util/ArraysFill.java + test/micro/org/openjdk/bench/java/util/ArraysMismatch.java From maurizio.cimadamore at oracle.com Wed Jan 2 21:03:31 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 02 Jan 2019 21:03:31 +0000 Subject: hg: panama/dev: Automatic merge with default Message-ID: <201901022103.x02L3WhO015558@aojmv0008.oracle.com> Changeset: d29e26df89d4 Author: mcimadamore Date: 2019-01-02 22:08 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/d29e26df89d4 Automatic merge with default From maurizio.cimadamore at oracle.com Wed Jan 2 21:03:52 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 02 Jan 2019 21:03:52 +0000 Subject: hg: panama/dev: Automatic merge with default Message-ID: <201901022103.x02L3rqq015902@aojmv0008.oracle.com> Changeset: 2d161775b994 Author: mcimadamore Date: 2019-01-02 22:09 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/2d161775b994 Automatic merge with default ! make/autoconf/basics.m4 ! make/autoconf/spec.gmk.in ! src/java.base/share/classes/java/lang/ClassLoader.java From maurizio.cimadamore at oracle.com Wed Jan 2 21:04:13 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 02 Jan 2019 21:04:13 +0000 Subject: hg: panama/dev: Automatic merge with default Message-ID: <201901022104.x02L4E5U016325@aojmv0008.oracle.com> Changeset: 00c9d1db9335 Author: mcimadamore Date: 2019-01-02 22:09 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/00c9d1db9335 Automatic merge with default ! make/autoconf/toolchain.m4 ! make/jdk/src/classes/build/tools/spp/Spp.java ! src/hotspot/share/runtime/globals.hpp From maurizio.cimadamore at oracle.com Wed Jan 2 21:04:33 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 02 Jan 2019 21:04:33 +0000 Subject: hg: panama/dev: Automatic merge with default Message-ID: <201901022104.x02L4Ys0016664@aojmv0008.oracle.com> Changeset: fc01e2fa9fbb Author: mcimadamore Date: 2019-01-02 22:09 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/fc01e2fa9fbb Automatic merge with default From maurizio.cimadamore at oracle.com Wed Jan 2 21:04:53 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 02 Jan 2019 21:04:53 +0000 Subject: hg: panama/dev: Automatic merge with default Message-ID: <201901022104.x02L4s6T016950@aojmv0008.oracle.com> Changeset: 050b60aa8531 Author: mcimadamore Date: 2019-01-02 22:10 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/050b60aa8531 Automatic merge with default ! src/hotspot/share/runtime/globals.hpp From maurizio.cimadamore at oracle.com Wed Jan 2 21:05:13 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 02 Jan 2019 21:05:13 +0000 Subject: hg: panama/dev: Automatic merge with foreign Message-ID: <201901022105.x02L5DrN017477@aojmv0008.oracle.com> Changeset: 436d895a0c9e Author: mcimadamore Date: 2019-01-02 22:10 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/436d895a0c9e Automatic merge with foreign ! src/hotspot/share/runtime/thread.hpp From maurizio.cimadamore at oracle.com Wed Jan 2 21:05:31 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 02 Jan 2019 21:05:31 +0000 Subject: hg: panama/dev: Automatic merge with linkToNative Message-ID: <201901022105.x02L5VLk017752@aojmv0008.oracle.com> Changeset: 070af3b10b83 Author: mcimadamore Date: 2019-01-02 22:10 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/070af3b10b83 Automatic merge with linkToNative From maurizio.cimadamore at oracle.com Wed Jan 2 21:05:51 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 02 Jan 2019 21:05:51 +0000 Subject: hg: panama/dev: Automatic merge with foreign Message-ID: <201901022105.x02L5p9Q018166@aojmv0008.oracle.com> Changeset: 2081fcfb7b46 Author: mcimadamore Date: 2019-01-02 22:11 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/2081fcfb7b46 Automatic merge with foreign ! make/autoconf/toolchain.m4 ! make/jdk/src/classes/build/tools/spp/Spp.java ! src/hotspot/share/runtime/globals.hpp From maurizio.cimadamore at oracle.com Wed Jan 2 21:06:09 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 02 Jan 2019 21:06:09 +0000 Subject: hg: panama/dev: Automatic merge with vectorIntrinsics Message-ID: <201901022106.x02L6Ain018494@aojmv0008.oracle.com> Changeset: 5c93d4cf6619 Author: mcimadamore Date: 2019-01-02 22:11 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/5c93d4cf6619 Automatic merge with vectorIntrinsics From vladimir.x.ivanov at oracle.com Thu Jan 3 21:24:40 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Thu, 3 Jan 2019 13:24:40 -0800 Subject: [VectorAPI]anyTrue/allTrue got wrong results when using UseAVX=3 In-Reply-To: References: Message-ID: <9f8fcace-dc11-e74d-cd39-9018eda0e4d3@oracle.com> Good catch, Zhuoren! The problem looks very similar to the one fixed by JDK-8210764 [1]. Switching to restricted register classes (legVec*) introduced as part of JDK-8210764 looks like the optimal fix in this case. Best regards, Vladimir Ivanov [1] https://bugs.openjdk.java.net/browse/JDK-8210764 On 02/01/2019 01:47, ??(??) wrote: > > Hello, I found anyTrue/allTrue got wrong results when using UseAVX=3. > > Let me describe this bug. Under UseAVX=3, following assembly code may be generated for anyTrue/allTrue. > > 0x00002ad5dca584f6: vpxord %xmm22,%xmm22,%xmm22 > 0x00002ad5dca584fc: vpsubb %xmm0,%xmm22,%xmm22 > 0x00002ad5dca58502: vpmovsxbd %xmm22,%ymm22 > 0x00002ad5dca58508: vptest %ymm1,%ymm6 > 0x00002ad5dca5850d: setb %r10b > > The second operand of vptest should be ymm22. But since there is no EVEX version for vptest, it is encoded as ymm6 and gives us wrong results. > > Currently I am using the following fix for this bug. Move source operands into ymm14/ymm15/xmm14/xmm15, and then they are used as operands of vptest. Please give advise on this fix. > diff -r 636478e1ee75 src/hotspot/cpu/x86/x86.ad > --- a/src/hotspot/cpu/x86/x86.ad Thu Dec 13 16:40:28 2018 -0800 > +++ b/src/hotspot/cpu/x86/x86.ad Sat Dec 29 11:16:55 2018 +0800 > @@ -21930,7 +21930,7 @@ > %} > > instruct vptest4ieq(rRegI dst, vecX src1, vecX src2) %{ > - predicate(UseAVX > 0 && static_cast(n)->get_predicate() == Assembler::notZero); > + predicate(UseAVX < 3 && UseAVX > 0 && static_cast(n)->get_predicate() == Assembler::notZero); > match(Set dst (VectorTest src1 src2 )); > format %{ "vptest $src1,$src2\n\t" > "setb $dst\t!" %} > @@ -21943,8 +21943,54 @@ > ins_pipe( pipe_slow ); > %} > > +instruct vptest4inaeavx3(rRegI dst, vecX src1, vecX src2, rxmm14 tmp14, rxmm15 tmp15) %{ > + predicate(UseAVX > 2 && static_cast(n)->get_predicate() == Assembler::carrySet); > + match(Set dst (VectorTest src1 src2 )); > + effect(TEMP tmp14, TEMP tmp15); > + format %{ "movdqu $tmp14,$src1\n\t" > + "movdqu $tmp15,$src2\n\t" > + "vptest $tmp14,$tmp15\n\t" > + "setb $dst\t!" %} > + ins_encode %{ > + int vector_len = 0; > + if ($tmp14$$XMMRegister != $src1$$XMMRegister) { > + __ movdqu($tmp14$$XMMRegister, $src1$$XMMRegister); > + } > + if ($tmp15$$XMMRegister != $src2$$XMMRegister) { > + __ movdqu($tmp15$$XMMRegister, $src2$$XMMRegister); > + } > + __ vptest($tmp14$$XMMRegister, $tmp15$$XMMRegister, vector_len); > + __ setb(Assembler::carrySet, $dst$$Register); > + __ movzbl($dst$$Register, $dst$$Register); > + %} > + ins_pipe( pipe_slow ); > +%} > + > +instruct vptest4ieqavx3(rRegI dst, vecX src1, vecX src2, rxmm14 tmp14, rxmm15 tmp15) %{ > + predicate(UseAVX > 0 && static_cast(n)->get_predicate() == Assembler::notZero); > + match(Set dst (VectorTest src1 src2 )); > + effect(TEMP tmp14, TEMP tmp15); > + format %{ "movdqu $tmp14,$src1\n\t" > + "movdqu $tmp15,$src2\n\t" > + "vptest $tmp14,$tmp15\n\t" > + "setb $dst\t!" %} > + ins_encode %{ > + int vector_len = 0; > + if ($tmp14$$XMMRegister != $src1$$XMMRegister) { > + __ movdqu($tmp14$$XMMRegister, $src1$$XMMRegister); > + } > + if ($tmp15$$XMMRegister != $src2$$XMMRegister) { > + __ movdqu($tmp15$$XMMRegister, $src2$$XMMRegister); > + } > + __ vptest($tmp14$$XMMRegister, $tmp15$$XMMRegister, vector_len); > + __ setb(Assembler::notZero, $dst$$Register); > + __ movzbl($dst$$Register, $dst$$Register); > + %} > + ins_pipe( pipe_slow ); > +%} > + > instruct vptest8inae(rRegI dst, vecY src1, vecY src2) %{ > - predicate(UseAVX > 0 && static_cast(n)->get_predicate() == Assembler::carrySet); > + predicate(UseAVX < 3 && UseAVX > 0 && static_cast(n)->get_predicate() == Assembler::carrySet); > match(Set dst (VectorTest src1 src2 )); > format %{ "vptest $src1,$src2\n\t" > "setb $dst\t!" %} > @@ -21958,7 +22004,7 @@ > %} > > instruct vptest8ieq(rRegI dst, vecY src1, vecY src2) %{ > - predicate(UseAVX > 0 && static_cast(n)->get_predicate() == Assembler::notZero); > + predicate(UseAVX < 3 && UseAVX > 0 && static_cast(n)->get_predicate() == Assembler::notZero); > match(Set dst (VectorTest src1 src2 )); > format %{ "vptest $src1,$src2\n\t" > "setb $dst\t!" %} > @@ -21971,6 +22017,52 @@ > ins_pipe( pipe_slow ); > %} > +instruct vptest8inaeavx3(rRegI dst, vecY src1, vecY src2, rymm14 tmp14, rymm15 tmp15) %{ > + predicate(UseAVX > 2 && static_cast(n)->get_predicate() == Assembler::carrySet); > + match(Set dst (VectorTest src1 src2 )); > + effect(TEMP tmp14, TEMP tmp15); > + format %{ "movdqu $tmp14,$src1\n\t" > + "movdqu $tmp15,$src2\n\t" > + "vptest $tmp14,$tmp15\n\t" > + "setb $dst\t!" %} > + ins_encode %{ > + int vector_len = 1; > + if ($tmp14$$XMMRegister != $src1$$XMMRegister) { > + __ vmovdqu($tmp14$$XMMRegister, $src1$$XMMRegister); > + } > + if ($tmp15$$XMMRegister != $src2$$XMMRegister) { > + __ vmovdqu($tmp15$$XMMRegister, $src2$$XMMRegister); > + } > + __ vptest($tmp14$$XMMRegister, $tmp15$$XMMRegister, vector_len); > + __ setb(Assembler::carrySet, $dst$$Register); > + __ movzbl($dst$$Register, $dst$$Register); > + %} > + ins_pipe( pipe_slow ); > +%} > + > +instruct vptest8ieqavx3(rRegI dst, vecY src1, vecY src2, rymm14 tmp14, rymm15 tmp15) %{ > + predicate(UseAVX > 2 && static_cast(n)->get_predicate() == Assembler::notZero); > + match(Set dst (VectorTest src1 src2 )); > + effect(TEMP tmp14, TEMP tmp15); > + format %{ "movdqu $tmp14,$src1\n\t" > + "movdqu $tmp15,$src2\n\t" > + "vptest $tmp14,$tmp15\n\t" > + "setb $dst\t!" %} > + ins_encode %{ > + int vector_len = 1; > + if ($tmp14$$XMMRegister != $src1$$XMMRegister) { > + __ vmovdqu($tmp14$$XMMRegister, $src1$$XMMRegister); > + } > + if ($tmp15$$XMMRegister != $src2$$XMMRegister) { > + __ vmovdqu($tmp15$$XMMRegister, $src2$$XMMRegister); > + } > + __ vptest($tmp14$$XMMRegister, $tmp15$$XMMRegister, vector_len); > + __ setb(Assembler::notZero, $dst$$Register); > + __ movzbl($dst$$Register, $dst$$Register); > + %} > + ins_pipe( pipe_slow ); > +%} > + > instruct loadmask8b(vecD dst, vecD src) %{ > predicate(UseSSE >= 2 && n->as_Vector()->length() == 8 && n->bottom_type()->is_vect()->element_basic_type() == T_BYTE); > match(Set dst (VectorLoadMask src)); > diff -r 636478e1ee75 src/hotspot/cpu/x86/x86_64.ad > --- a/src/hotspot/cpu/x86/x86_64.ad Thu Dec 13 16:40:28 2018 -0800 > +++ b/src/hotspot/cpu/x86/x86_64.ad Sat Dec 29 11:16:55 2018 +0800 > @@ -4426,6 +4426,17 @@ > predicate(UseAVX == 3); format%{%} interface(REG_INTER); > %} > > + > +operand rymm14() %{ > + constraint(ALLOC_IN_RC(ymm14_reg)); match(VecY); > + predicate((UseSSE > 0) && (UseAVX <= 3)); format%{%} interface(REG_INTER); > +%} > +operand rymm15() %{ > + constraint(ALLOC_IN_RC(ymm15_reg)); match(VecY); > + predicate((UseSSE > 0) && (UseAVX <= 3)); format%{%} interface(REG_INTER); > +%} > + > + > //----------OPERAND CLASSES---------------------------------------------------- > // Operand Classes are groups of operands that are used as to simplify > // instruction definitions by not requiring the AD writer to specify separate > > > The test to reproduce this bug. Please DO set -XX:UseAVX=3 > > import jdk.incubator.vector.*; > import java.util.Arrays; > import java.util.Random; > import java.lang.reflect.Field; > import java.io.IOException; > import jdk.incubator.vector.Vector.Mask; > import jdk.incubator.vector.Vector.Shape; > public class VectorTrueTest > { > > static Random random = new Random(); > static final IntVector.IntSpecies Species = IntVector.species(Vector.Shape.S_256_BIT); > public static int size = 1024 * 16; > public static int length = Species.length(); > public static int resultSize = size / length; > static boolean[] anyResultV = new boolean[resultSize]; > static boolean[] allResultV = new boolean[resultSize]; > static boolean[] anyInput = new boolean[size]; > static boolean[] allInput = new boolean[size]; > public static void main(String[] args) throws NoSuchFieldException, SecurityException, IllegalArgumentException, IllegalAccessException, InstantiationException { > long start0 = System.currentTimeMillis(); > long startv = System.currentTimeMillis(); > long normalTime = 0; > long vecTime = 0; > int i = 0; > for (i = 0; i < size; i++) { > anyInput[i] = false; > allInput[i] = false; > } > for (i = 0; i < 20000; i++) { > vecTest(Species); > } > for (i = 0; i < resultSize; i++) { > anyResultV[i] = true; > allResultV[i] = true; > } > vecTest(Species); > for (i = 0; i < (resultSize - 1); i++) { > if (anyResultV[i] != false) throw new RuntimeException("Wrong anyTrue result! Should be all false, index " + i); > if (allResultV[i] != false) throw new RuntimeException("Wrong allTrue result! Should be all false, index " + i); > } > } > static void vecTest(IntVector.IntSpecies Speciesint ) { > IntVector v0; > int i = 0; > int j = 0; > Mask maskAny = Speciesint.maskFromArray(anyInput, i); > Mask maskAll = Speciesint.maskFromArray(allInput, i); > for (i = 0; i + (Speciesint.length()) <= size; i += Speciesint.length()) { > allResultV[j] = maskAll.allTrue(); > anyResultV[j] = maskAny.anyTrue(); > j++; > } > return; > } > } > > Regards, > Zhuoren > > > > From vladimir.x.ivanov at oracle.com Fri Jan 4 01:02:02 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Thu, 3 Jan 2019 17:02:02 -0800 Subject: [vector api] Operations on Mask-s In-Reply-To: References: Message-ID: <245a485c-1cfc-0798-300f-e8c88dda65ba@oracle.com> Tomasz, Yes, I had similar experience with complex use cases involving masks. The API works well for lane-wise cases, but falls apart when cross-lane operations are needed, requiring user to go through intermediate representations which aren't optimized well. There are different ways to fix that gap on API level, e.g.: * declare more operations on Mask * operate on long: Mask <=> long * operate on Vector: Mask <=> Vector or Mask <: BoolVector <: Vector But IMO the main limiting factor to proceed is implementation considerations. On x86 there are 2 ways to represent masks: (1) as high bits in vector registers (pre-AVX512) and (2) as opmask registers (k0-k7). (I believe it's similar on ARM with NEON and SVE.) C2 doesn't have full AVX512 support yet (e.g., no opmask register support in register allocator) and it was decided to focus on pre-AVX512 model first. So, current implementation represents masks as vectors uniformly across pre-AVX512 and AVX512-capable CPUs. And that's probably the main reason why Mask hasn't got enough attention yet. Regarding workarounds to your immediate problem: with the API in its current state, either Mask.toLong() or Mask.toVector() can be used, but considering current state of the implementation (where both haven't been intrinsified yet), I'd suggest to avoid them altogether in hot code for now and try the following workaround instead (pseudo-code follows): mask0x55 = ... < fromLong(0x55 << 0) > ... // mask constant mask0xAA = ... < fromLong(0x55 << 1) > ... // mask constant compMask0 = vector.lessThan(input) compMask1 = vector.shiftEL(1).lessThan(input.shiftEL(1)) // lessThan preserves zero in 1st element // equivalent of "(m & 0x55) | ((m << 1) & (0x55 << 1))" compMask0.and(mask0x55).or(compMask1.and(mask0xAA)) The idea is to shift vectors and recompute masks instead. (Unfortunately, Vector.shiftEL() hasn't been intrinsified yet as well, but you can implement an equivalent using Vector.rearrange(Shuffle) which has enough support on JVM side.) Hope that helps. Best regards, Vladimir Ivanov On 02/01/2019 01:34, Tomasz Kowalczewski wrote: > Hi, > > I was working on implementing a simple sorting algorithm[1] using VectorAPI > and stumbled on something that might be a gap in Mask API. It might be > intentional for this stage of API maturity - please advise. > > I need to modify bits of the mask I get from _mm512_cmp_X_mask (e.g. > vector.lessThan(input)) by doing and-s and or-s which can be simulated by > preparing appropriate Mask objects. What I found missing is shift > operations. Example: > > ( compMask & 0x55 ) | ( ( compMask & 0x55 ) << 1) > > I cannot do this using Mask object. Most natural alternative would be to > convert the mask into long, perform these operations and convert it back. > Unfortunately I was unable to find how to convert long back to Mask. Of > course this can be done (via boolean[] array) but that does not seem > efficient :). > > Excuse me if I missed some API call. > > 1. https://hal.inria.fr/hal-01512970v1/document (page 14, Code 1). > From Ningsheng.Jian at arm.com Fri Jan 4 01:57:42 2019 From: Ningsheng.Jian at arm.com (Ningsheng Jian (Arm Technology China)) Date: Fri, 4 Jan 2019 01:57:42 +0000 Subject: [vector api] Operations on Mask-s In-Reply-To: <245a485c-1cfc-0798-300f-e8c88dda65ba@oracle.com> References: <245a485c-1cfc-0798-300f-e8c88dda65ba@oracle.com> Message-ID: Hi, > -----Original Message----- > From: panama-dev On Behalf Of > Vladimir Ivanov > Sent: Friday, January 4, 2019 9:02 AM > To: Tomasz Kowalczewski ; panama- > dev at openjdk.java.net > Subject: Re: [vector api] Operations on Mask-s > > Tomasz, > > Yes, I had similar experience with complex use cases involving masks. > The API works well for lane-wise cases, but falls apart when cross-lane > operations are needed, requiring user to go through intermediate > representations which aren't optimized well. > > There are different ways to fix that gap on API level, e.g.: > * declare more operations on Mask > * operate on long: Mask <=> long > * operate on Vector: Mask <=> Vector or Mask <: BoolVector <: Vector > > But IMO the main limiting factor to proceed is implementation considerations. > > On x86 there are 2 ways to represent masks: (1) as high bits in vector registers > (pre-AVX512) and (2) as opmask registers (k0-k7). (I believe it's similar on ARM > with NEON and SVE.) > Yes, and since SVE has fully predicate (mask) support, we would like to implement it in the mask register way, but need to keep compatible with non-mask Arches. > C2 doesn't have full AVX512 support yet (e.g., no opmask register support in > register allocator) and it was decided to focus on pre-AVX512 model first. So, > current implementation represents masks as vectors uniformly across pre- > AVX512 and AVX512-capable CPUs. And that's probably the main reason why > Mask hasn't got enough attention yet. > Do you (or anyone else) have any plan to work on opmask register support in register allocator? Thanks, Ningsheng IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. From zhuoren.wz at alibaba-inc.com Fri Jan 4 09:24:00 2019 From: zhuoren.wz at alibaba-inc.com (=?UTF-8?B?V2FuZyBaaHVvKFpodW9yZW4p?=) Date: Fri, 04 Jan 2019 17:24:00 +0800 Subject: =?UTF-8?B?UmU6IFtWZWN0b3JBUEldYW55VHJ1ZS9hbGxUcnVlIGdvdCB3cm9uZyByZXN1bHRzIHdoZW4g?= =?UTF-8?B?dXNpbmcgVXNlQVZYPTM=?= In-Reply-To: <9f8fcace-dc11-e74d-cd39-9018eda0e4d3@oracle.com> References: , <9f8fcace-dc11-e74d-cd39-9018eda0e4d3@oracle.com> Message-ID: <2a33a2b1-ae5d-4641-b772-52fd0575723c.zhuoren.wz@alibaba-inc.com> Thank you Ivanov, I used legVec* and it fixed this issue. Below is the new patch. Please review. diff -r f4f4ef0120ba src/hotspot/cpu/x86/x86.ad --- a/src/hotspot/cpu/x86/x86.ad Wed Dec 26 22:09:29 2018 +0100 +++ b/src/hotspot/cpu/x86/x86.ad Fri Jan 04 17:11:08 2019 +0800 @@ -21919,7 +21919,7 @@ ins_pipe( pipe_slow ); %} -instruct vptest4inae(rRegI dst, vecX src1, vecX src2) %{ +instruct vptest4inae(rRegI dst, legVecX src1, legVecX src2) %{ predicate(UseAVX > 0 && static_cast(n)->get_predicate() == Assembler::carrySet); match(Set dst (VectorTest src1 src2 )); format %{ "vptest $src1,$src2\n\t" @@ -21933,7 +21933,7 @@ ins_pipe( pipe_slow ); %} -instruct vptest4ieq(rRegI dst, vecX src1, vecX src2) %{ +instruct vptest4ieq(rRegI dst, legVecX src1, legVecX src2) %{ predicate(UseAVX > 0 && static_cast(n)->get_predicate() == Assembler::notZero); match(Set dst (VectorTest src1 src2 )); format %{ "vptest $src1,$src2\n\t" @@ -21947,7 +21947,7 @@ ins_pipe( pipe_slow ); %} -instruct vptest8inae(rRegI dst, vecY src1, vecY src2) %{ +instruct vptest8inae(rRegI dst, legVecY src1, legVecY src2) %{ predicate(UseAVX > 0 && static_cast(n)->get_predicate() == Assembler::carrySet); match(Set dst (VectorTest src1 src2 )); format %{ "vptest $src1,$src2\n\t" @@ -21961,7 +21961,7 @@ ins_pipe( pipe_slow ); %} -instruct vptest8ieq(rRegI dst, vecY src1, vecY src2) %{ +instruct vptest8ieq(rRegI dst, legVecY src1, legVecY src2) %{ predicate(UseAVX > 0 && static_cast(n)->get_predicate() == Assembler::notZero); match(Set dst (VectorTest src1 src2 )); format %{ "vptest $src1,$src2\n\t" Regards, Zhuoren ------------------------------------------------------------------ Sender:Vladimir Ivanov Sent At:2019 Jan. 4 (Fri.) 05:24 Recipient:Sandler ; panama-dev Cc:li sanhong ; Kingsum Chow Subject:Re: [VectorAPI]anyTrue/allTrue got wrong results when using UseAVX=3 Good catch, Zhuoren! The problem looks very similar to the one fixed by JDK-8210764 [1]. Switching to restricted register classes (legVec*) introduced as part of JDK-8210764 looks like the optimal fix in this case. Best regards, Vladimir Ivanov [1] https://bugs.openjdk.java.net/browse/JDK-8210764 On 02/01/2019 01:47, ??(??) wrote: > > Hello, I found anyTrue/allTrue got wrong results when using UseAVX=3. > > Let me describe this bug. Under UseAVX=3, following assembly code may be generated for anyTrue/allTrue. > > 0x00002ad5dca584f6: vpxord %xmm22,%xmm22,%xmm22 > 0x00002ad5dca584fc: vpsubb %xmm0,%xmm22,%xmm22 > 0x00002ad5dca58502: vpmovsxbd %xmm22,%ymm22 > 0x00002ad5dca58508: vptest %ymm1,%ymm6 > 0x00002ad5dca5850d: setb %r10b > > The second operand of vptest should be ymm22. But since there is no EVEX version for vptest, it is encoded as ymm6 and gives us wrong results. > > Currently I am using the following fix for this bug. Move source operands into ymm14/ymm15/xmm14/xmm15, and then they are used as operands of vptest. Please give advise on this fix. > diff -r 636478e1ee75 src/hotspot/cpu/x86/x86.ad > --- a/src/hotspot/cpu/x86/x86.ad Thu Dec 13 16:40:28 2018 -0800 > +++ b/src/hotspot/cpu/x86/x86.ad Sat Dec 29 11:16:55 2018 +0800 > @@ -21930,7 +21930,7 @@ > %} > > instruct vptest4ieq(rRegI dst, vecX src1, vecX src2) %{ > - predicate(UseAVX > 0 && static_cast(n)->get_predicate() == Assembler::notZero); > + predicate(UseAVX < 3 && UseAVX > 0 && static_cast(n)->get_predicate() == Assembler::notZero); > match(Set dst (VectorTest src1 src2 )); > format %{ "vptest $src1,$src2\n\t" > "setb $dst\t!" %} > @@ -21943,8 +21943,54 @@ > ins_pipe( pipe_slow ); > %} > > +instruct vptest4inaeavx3(rRegI dst, vecX src1, vecX src2, rxmm14 tmp14, rxmm15 tmp15) %{ > + predicate(UseAVX > 2 && static_cast(n)->get_predicate() == Assembler::carrySet); > + match(Set dst (VectorTest src1 src2 )); > + effect(TEMP tmp14, TEMP tmp15); > + format %{ "movdqu $tmp14,$src1\n\t" > + "movdqu $tmp15,$src2\n\t" > + "vptest $tmp14,$tmp15\n\t" > + "setb $dst\t!" %} > + ins_encode %{ > + int vector_len = 0; > + if ($tmp14$$XMMRegister != $src1$$XMMRegister) { > + __ movdqu($tmp14$$XMMRegister, $src1$$XMMRegister); > + } > + if ($tmp15$$XMMRegister != $src2$$XMMRegister) { > + __ movdqu($tmp15$$XMMRegister, $src2$$XMMRegister); > + } > + __ vptest($tmp14$$XMMRegister, $tmp15$$XMMRegister, vector_len); > + __ setb(Assembler::carrySet, $dst$$Register); > + __ movzbl($dst$$Register, $dst$$Register); > + %} > + ins_pipe( pipe_slow ); > +%} > + > +instruct vptest4ieqavx3(rRegI dst, vecX src1, vecX src2, rxmm14 tmp14, rxmm15 tmp15) %{ > + predicate(UseAVX > 0 && static_cast(n)->get_predicate() == Assembler::notZero); > + match(Set dst (VectorTest src1 src2 )); > + effect(TEMP tmp14, TEMP tmp15); > + format %{ "movdqu $tmp14,$src1\n\t" > + "movdqu $tmp15,$src2\n\t" > + "vptest $tmp14,$tmp15\n\t" > + "setb $dst\t!" %} > + ins_encode %{ > + int vector_len = 0; > + if ($tmp14$$XMMRegister != $src1$$XMMRegister) { > + __ movdqu($tmp14$$XMMRegister, $src1$$XMMRegister); > + } > + if ($tmp15$$XMMRegister != $src2$$XMMRegister) { > + __ movdqu($tmp15$$XMMRegister, $src2$$XMMRegister); > + } > + __ vptest($tmp14$$XMMRegister, $tmp15$$XMMRegister, vector_len); > + __ setb(Assembler::notZero, $dst$$Register); > + __ movzbl($dst$$Register, $dst$$Register); > + %} > + ins_pipe( pipe_slow ); > +%} > + > instruct vptest8inae(rRegI dst, vecY src1, vecY src2) %{ > - predicate(UseAVX > 0 && static_cast(n)->get_predicate() == Assembler::carrySet); > + predicate(UseAVX < 3 && UseAVX > 0 && static_cast(n)->get_predicate() == Assembler::carrySet); > match(Set dst (VectorTest src1 src2 )); > format %{ "vptest $src1,$src2\n\t" > "setb $dst\t!" %} > @@ -21958,7 +22004,7 @@ > %} > > instruct vptest8ieq(rRegI dst, vecY src1, vecY src2) %{ > - predicate(UseAVX > 0 && static_cast(n)->get_predicate() == Assembler::notZero); > + predicate(UseAVX < 3 && UseAVX > 0 && static_cast(n)->get_predicate() == Assembler::notZero); > match(Set dst (VectorTest src1 src2 )); > format %{ "vptest $src1,$src2\n\t" > "setb $dst\t!" %} > @@ -21971,6 +22017,52 @@ > ins_pipe( pipe_slow ); > %} > +instruct vptest8inaeavx3(rRegI dst, vecY src1, vecY src2, rymm14 tmp14, rymm15 tmp15) %{ > + predicate(UseAVX > 2 && static_cast(n)->get_predicate() == Assembler::carrySet); > + match(Set dst (VectorTest src1 src2 )); > + effect(TEMP tmp14, TEMP tmp15); > + format %{ "movdqu $tmp14,$src1\n\t" > + "movdqu $tmp15,$src2\n\t" > + "vptest $tmp14,$tmp15\n\t" > + "setb $dst\t!" %} > + ins_encode %{ > + int vector_len = 1; > + if ($tmp14$$XMMRegister != $src1$$XMMRegister) { > + __ vmovdqu($tmp14$$XMMRegister, $src1$$XMMRegister); > + } > + if ($tmp15$$XMMRegister != $src2$$XMMRegister) { > + __ vmovdqu($tmp15$$XMMRegister, $src2$$XMMRegister); > + } > + __ vptest($tmp14$$XMMRegister, $tmp15$$XMMRegister, vector_len); > + __ setb(Assembler::carrySet, $dst$$Register); > + __ movzbl($dst$$Register, $dst$$Register); > + %} > + ins_pipe( pipe_slow ); > +%} > + > +instruct vptest8ieqavx3(rRegI dst, vecY src1, vecY src2, rymm14 tmp14, rymm15 tmp15) %{ > + predicate(UseAVX > 2 && static_cast(n)->get_predicate() == Assembler::notZero); > + match(Set dst (VectorTest src1 src2 )); > + effect(TEMP tmp14, TEMP tmp15); > + format %{ "movdqu $tmp14,$src1\n\t" > + "movdqu $tmp15,$src2\n\t" > + "vptest $tmp14,$tmp15\n\t" > + "setb $dst\t!" %} > + ins_encode %{ > + int vector_len = 1; > + if ($tmp14$$XMMRegister != $src1$$XMMRegister) { > + __ vmovdqu($tmp14$$XMMRegister, $src1$$XMMRegister); > + } > + if ($tmp15$$XMMRegister != $src2$$XMMRegister) { > + __ vmovdqu($tmp15$$XMMRegister, $src2$$XMMRegister); > + } > + __ vptest($tmp14$$XMMRegister, $tmp15$$XMMRegister, vector_len); > + __ setb(Assembler::notZero, $dst$$Register); > + __ movzbl($dst$$Register, $dst$$Register); > + %} > + ins_pipe( pipe_slow ); > +%} > + > instruct loadmask8b(vecD dst, vecD src) %{ > predicate(UseSSE >= 2 && n->as_Vector()->length() == 8 && n->bottom_type()->is_vect()->element_basic_type() == T_BYTE); > match(Set dst (VectorLoadMask src)); > diff -r 636478e1ee75 src/hotspot/cpu/x86/x86_64.ad > --- a/src/hotspot/cpu/x86/x86_64.ad Thu Dec 13 16:40:28 2018 -0800 > +++ b/src/hotspot/cpu/x86/x86_64.ad Sat Dec 29 11:16:55 2018 +0800 > @@ -4426,6 +4426,17 @@ > predicate(UseAVX == 3); format%{%} interface(REG_INTER); > %} > > + > +operand rymm14() %{ > + constraint(ALLOC_IN_RC(ymm14_reg)); match(VecY); > + predicate((UseSSE > 0) && (UseAVX <= 3)); format%{%} interface(REG_INTER); > +%} > +operand rymm15() %{ > + constraint(ALLOC_IN_RC(ymm15_reg)); match(VecY); > + predicate((UseSSE > 0) && (UseAVX <= 3)); format%{%} interface(REG_INTER); > +%} > + > + > //----------OPERAND CLASSES---------------------------------------------------- > // Operand Classes are groups of operands that are used as to simplify > // instruction definitions by not requiring the AD writer to specify separate > > > The test to reproduce this bug. Please DO set -XX:UseAVX=3 > > import jdk.incubator.vector.*; > import java.util.Arrays; > import java.util.Random; > import java.lang.reflect.Field; > import java.io.IOException; > import jdk.incubator.vector.Vector.Mask; > import jdk.incubator.vector.Vector.Shape; > public class VectorTrueTest > { > > static Random random = new Random(); > static final IntVector.IntSpecies Species = IntVector.species(Vector.Shape.S_256_BIT); > public static int size = 1024 * 16; > public static int length = Species.length(); > public static int resultSize = size / length; > static boolean[] anyResultV = new boolean[resultSize]; > static boolean[] allResultV = new boolean[resultSize]; > static boolean[] anyInput = new boolean[size]; > static boolean[] allInput = new boolean[size]; > public static void main(String[] args) throws NoSuchFieldException, SecurityException, IllegalArgumentException, IllegalAccessException, InstantiationException { > long start0 = System.currentTimeMillis(); > long startv = System.currentTimeMillis(); > long normalTime = 0; > long vecTime = 0; > int i = 0; > for (i = 0; i < size; i++) { > anyInput[i] = false; > allInput[i] = false; > } > for (i = 0; i < 20000; i++) { > vecTest(Species); > } > for (i = 0; i < resultSize; i++) { > anyResultV[i] = true; > allResultV[i] = true; > } > vecTest(Species); > for (i = 0; i < (resultSize - 1); i++) { > if (anyResultV[i] != false) throw new RuntimeException("Wrong anyTrue result! Should be all false, index " + i); > if (allResultV[i] != false) throw new RuntimeException("Wrong allTrue result! Should be all false, index " + i); > } > } > static void vecTest(IntVector.IntSpecies Speciesint ) { > IntVector v0; > int i = 0; > int j = 0; > Mask maskAny = Speciesint.maskFromArray(anyInput, i); > Mask maskAll = Speciesint.maskFromArray(allInput, i); > for (i = 0; i + (Speciesint.length()) <= size; i += Speciesint.length()) { > allResultV[j] = maskAll.allTrue(); > anyResultV[j] = maskAny.anyTrue(); > j++; > } > return; > } > } > > Regards, > Zhuoren > > > > From vladimir.x.ivanov at oracle.com Fri Jan 4 18:10:18 2019 From: vladimir.x.ivanov at oracle.com (vladimir.x.ivanov at oracle.com) Date: Fri, 04 Jan 2019 18:10:18 +0000 Subject: hg: panama/dev: Fix register classes for inputs of vptest. Message-ID: <201901041810.x04IAIlV022486@aojmv0008.oracle.com> Changeset: dfe1c1d8a316 Author: vlivanov Date: 2019-01-04 10:08 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/dfe1c1d8a316 Fix register classes for inputs of vptest. Contributed-by: Wang Zhuo ! src/hotspot/cpu/x86/x86.ad From vladimir.x.ivanov at oracle.com Fri Jan 4 18:10:56 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 4 Jan 2019 10:10:56 -0800 Subject: [VectorAPI]anyTrue/allTrue got wrong results when using UseAVX=3 In-Reply-To: <2a33a2b1-ae5d-4641-b772-52fd0575723c.zhuoren.wz@alibaba-inc.com> References: <9f8fcace-dc11-e74d-cd39-9018eda0e4d3@oracle.com> <2a33a2b1-ae5d-4641-b772-52fd0575723c.zhuoren.wz@alibaba-inc.com> Message-ID: Thanks, pushed [1]. Best regards, Vladimir Ivanov [1] http://hg.openjdk.java.net/panama/dev/rev/dfe1c1d8a316 On 04/01/2019 01:24, Wang Zhuo(Zhuoren) wrote: > Thank you?Ivanov, > > I used?legVec* and it fixed this issue. > Below is the new patch. Please review. > > diff?-r?f4f4ef0120ba?src/hotspot/cpu/x86/x86.ad > ---?a/src/hotspot/cpu/x86/x86.ad????????Wed?Dec?26?22:09:29?2018?+0100 > +++?b/src/hotspot/cpu/x86/x86.ad????????Fri?Jan?04?17:11:08?2019?+0800 > @@?-21919,7?+21919,7?@@ > ???ins_pipe(?pipe_slow?); > ?%} > > -instruct?vptest4inae(rRegI?dst,?vecX?src1,?vecX?src2)?%{ > +instruct?vptest4inae(rRegI?dst,?legVecX?src1,?legVecX?src2)?%{ > ???predicate(UseAVX?>?0?&&?static_cast(n)->get_predicate()?==?Assembler::carrySet); > ???match(Set?dst?(VectorTest?src1?src2?)); > ???format?%{?"vptest??$src1,$src2\n\t" > @@?-21933,7?+21933,7?@@ > ???ins_pipe(?pipe_slow?); > ?%} > > -instruct?vptest4ieq(rRegI?dst,?vecX?src1,?vecX?src2)?%{ > +instruct?vptest4ieq(rRegI?dst,?legVecX?src1,?legVecX?src2)?%{ > ???predicate(UseAVX?>?0?&&?static_cast(n)->get_predicate()?==?Assembler::notZero); > ???match(Set?dst?(VectorTest?src1?src2?)); > ???format?%{?"vptest??$src1,$src2\n\t" > @@?-21947,7?+21947,7?@@ > ???ins_pipe(?pipe_slow?); > ?%} > > -instruct?vptest8inae(rRegI?dst,?vecY?src1,?vecY?src2)?%{ > +instruct?vptest8inae(rRegI?dst,?legVecY?src1,?legVecY?src2)?%{ > ???predicate(UseAVX?>?0?&&?static_cast(n)->get_predicate()?==?Assembler::carrySet); > ???match(Set?dst?(VectorTest?src1?src2?)); > ???format?%{?"vptest??$src1,$src2\n\t" > @@?-21961,7?+21961,7?@@ > ???ins_pipe(?pipe_slow?); > ?%} > > -instruct?vptest8ieq(rRegI?dst,?vecY?src1,?vecY?src2)?%{ > +instruct?vptest8ieq(rRegI?dst,?legVecY?src1,?legVecY?src2)?%{ > ???predicate(UseAVX?>?0?&&?static_cast(n)->get_predicate()?==?Assembler::notZero); > ???match(Set?dst?(VectorTest?src1?src2?)); > ???format?%{?"vptest??$src1,$src2\n\t" > > Regards, > Zhuoren > > ------------------------------------------------------------------ > Sender:Vladimir Ivanov > Sent At:2019 Jan. 4 (Fri.) 05:24 > Recipient:Sandler ; panama-dev > > Cc:li sanhong ; Kingsum Chow > > Subject:Re: [VectorAPI]anyTrue/allTrue got wrong results when using > UseAVX=3 > > Good?catch,?Zhuoren! > > The?problem?looks?very?similar?to?the?one?fixed?by?JDK-8210764?[1]. > > Switching?to?restricted?register?classes?(legVec*)?introduced?as?part?of > > JDK-8210764?looks?like?the?optimal?fix?in?this?case. > > Best?regards, > Vladimir?Ivanov > > [1]?https://bugs.openjdk.java.net/browse/JDK-8210764 > > On?02/01/2019?01:47,???(??)?wrote: > > > >?Hello,?I?found?anyTrue/allTrue?got?wrong?results?when?using?UseAVX=3. > > > >?Let?me?describe?this?bug.?Under?UseAVX=3,?following?assembly?code?may?be?generated?for?anyTrue/allTrue. > > > >????0x00002ad5dca584f6:?vpxord?%xmm22,%xmm22,%xmm22 > >????0x00002ad5dca584fc:?vpsubb?%xmm0,%xmm22,%xmm22 > >????0x00002ad5dca58502:?vpmovsxbd?%xmm22,%ymm22 > >????0x00002ad5dca58508:?vptest?%ymm1,%ymm6 > >????0x00002ad5dca5850d:?setb???%r10b > > > >?The?second?operand?of?vptest?should?be?ymm22.?But?since?there?is?no?EVEX?version?for?vptest,?it?is?encoded?as?ymm6?and?gives?us?wrong?results. > > > >?Currently?I?am?using?the?following?fix?for?this?bug.?Move?source?operands?into?ymm14/ymm15/xmm14/xmm15,??and?then?they?are?used?as?operands?of?vptest.?Please?give?advise?on?this?fix. > >?diff?-r?636478e1ee75?src/hotspot/cpu/x86/x86.ad > >?---?a/src/hotspot/cpu/x86/x86.ad????????Thu?Dec?13?16:40:28?2018?-0800 > >?+++?b/src/hotspot/cpu/x86/x86.ad????????Sat?Dec?29?11:16:55?2018?+0800 > >?@@?-21930,7?+21930,7?@@ > >???%} > > > >???instruct?vptest4ieq(rRegI?dst,?vecX?src1,?vecX?src2)?%{ > >?-??predicate(UseAVX?>?0?&&?static_cast(n)->get_predicate()?==?Assembler::notZero); > >?+??predicate(UseAVX??0?&&?static_cast(n)->get_predicate()?==?Assembler::notZero); > >?????match(Set?dst?(VectorTest?src1?src2?)); > >?????format?%{?"vptest??$src1,$src2\n\t" > >???????????????"setb??$dst\t!"?%} > >?@@?-21943,8?+21943,54?@@ > >?????ins_pipe(?pipe_slow?); > >???%} > > > >?+instruct?vptest4inaeavx3(rRegI?dst,?vecX?src1,?vecX?src2,?rxmm14?tmp14,?rxmm15?tmp15)?%{ > >?+??predicate(UseAVX?>?2?&&?static_cast(n)->get_predicate()?==?Assembler::carrySet); > >?+??match(Set?dst?(VectorTest?src1?src2?)); > >?+??effect(TEMP?tmp14,?TEMP?tmp15); > >?+??format?%{?"movdqu??????$tmp14,$src1\n\t" > >?+????????????"movdqu??????$tmp15,$src2\n\t" > >?+????????????"vptest??$tmp14,$tmp15\n\t" > >?+????????????"setb??$dst\t!"?%} > >?+??ins_encode?%{ > >?+????int?vector_len?=?0; > >?+????if?($tmp14$$XMMRegister?!=?$src1$$XMMRegister)?{ > >?+??????__?movdqu($tmp14$$XMMRegister,?$src1$$XMMRegister); > >?+????} > >?+????if?($tmp15$$XMMRegister?!=?$src2$$XMMRegister)?{ > >?+??????__?movdqu($tmp15$$XMMRegister,?$src2$$XMMRegister); > >?+????} > >?+????__?vptest($tmp14$$XMMRegister,?$tmp15$$XMMRegister,?vector_len); > >?+????__?setb(Assembler::carrySet,?$dst$$Register); > >?+????__?movzbl($dst$$Register,?$dst$$Register); > >?+??%} > >?+??ins_pipe(?pipe_slow?); > >?+%} > >?+ > >?+instruct?vptest4ieqavx3(rRegI?dst,?vecX?src1,?vecX?src2,?rxmm14?tmp14,?rxmm15?tmp15)?%{ > >?+??predicate(UseAVX?>?0?&&?static_cast(n)->get_predicate()?==?Assembler::notZero); > >?+??match(Set?dst?(VectorTest?src1?src2?)); > >?+??effect(TEMP?tmp14,?TEMP?tmp15); > >?+??format?%{?"movdqu??????$tmp14,$src1\n\t" > >?+????????????"movdqu??????$tmp15,$src2\n\t" > >?+????????????"vptest??$tmp14,$tmp15\n\t" > >?+????????????"setb??$dst\t!"?%} > >?+??ins_encode?%{ > >?+????int?vector_len?=?0; > >?+????if?($tmp14$$XMMRegister?!=?$src1$$XMMRegister)?{ > >?+??????__?movdqu($tmp14$$XMMRegister,?$src1$$XMMRegister); > >?+????} > >?+????if?($tmp15$$XMMRegister?!=?$src2$$XMMRegister)?{ > >?+??????__?movdqu($tmp15$$XMMRegister,?$src2$$XMMRegister); > >?+????} > >?+????__?vptest($tmp14$$XMMRegister,?$tmp15$$XMMRegister,?vector_len); > >?+????__?setb(Assembler::notZero,?$dst$$Register); > >?+????__?movzbl($dst$$Register,?$dst$$Register); > >?+??%} > >?+??ins_pipe(?pipe_slow?); > >?+%} > >?+ > >???instruct?vptest8inae(rRegI?dst,?vecY?src1,?vecY?src2)?%{ > >?-??predicate(UseAVX?>?0?&&?static_cast(n)->get_predicate()?==?Assembler::carrySet); > >?+??predicate(UseAVX??0?&&?static_cast(n)->get_predicate()?==?Assembler::carrySet); > >?????match(Set?dst?(VectorTest?src1?src2?)); > >?????format?%{?"vptest??$src1,$src2\n\t" > >???????????????"setb??$dst\t!"?%} > >?@@?-21958,7?+22004,7?@@ > >???%} > > > >???instruct?vptest8ieq(rRegI?dst,?vecY?src1,?vecY?src2)?%{ > >?-??predicate(UseAVX?>?0?&&?static_cast(n)->get_predicate()?==?Assembler::notZero); > >?+??predicate(UseAVX??0?&&?static_cast(n)->get_predicate()?==?Assembler::notZero); > >?????match(Set?dst?(VectorTest?src1?src2?)); > >?????format?%{?"vptest??$src1,$src2\n\t" > >???????????????"setb??$dst\t!"?%} > >?@@?-21971,6?+22017,52?@@ > >?????ins_pipe(?pipe_slow?); > >???%} > >?+instruct?vptest8inaeavx3(rRegI?dst,?vecY?src1,?vecY?src2,?rymm14?tmp14,?rymm15?tmp15)?%{ > >?+??predicate(UseAVX?>?2?&&?static_cast(n)->get_predicate()?==?Assembler::carrySet); > >?+??match(Set?dst?(VectorTest?src1?src2?)); > >?+??effect(TEMP?tmp14,?TEMP?tmp15); > >?+??format?%{?"movdqu??????$tmp14,$src1\n\t" > >?+????????????"movdqu??????$tmp15,$src2\n\t" > >?+????????????"vptest??$tmp14,$tmp15\n\t" > >?+????????????"setb??$dst\t!"?%} > >?+??ins_encode?%{ > >?+????int?vector_len?=?1; > >?+????if?($tmp14$$XMMRegister?!=?$src1$$XMMRegister)?{ > >?+??????__?vmovdqu($tmp14$$XMMRegister,?$src1$$XMMRegister); > >?+????} > >?+????if?($tmp15$$XMMRegister?!=?$src2$$XMMRegister)?{ > >?+??????__?vmovdqu($tmp15$$XMMRegister,?$src2$$XMMRegister); > >?+????} > >?+????__?vptest($tmp14$$XMMRegister,?$tmp15$$XMMRegister,?vector_len); > >?+????__?setb(Assembler::carrySet,?$dst$$Register); > >?+????__?movzbl($dst$$Register,?$dst$$Register); > >?+??%} > >?+??ins_pipe(?pipe_slow?); > >?+%} > >?+ > >?+instruct?vptest8ieqavx3(rRegI?dst,?vecY?src1,?vecY?src2,?rymm14?tmp14,?rymm15?tmp15)?%{ > >?+??predicate(UseAVX?>?2?&&?static_cast(n)->get_predicate()?==?Assembler::notZero); > >?+??match(Set?dst?(VectorTest?src1?src2?)); > >?+??effect(TEMP?tmp14,?TEMP?tmp15); > >?+??format?%{?"movdqu??????$tmp14,$src1\n\t" > >?+????????????"movdqu??????$tmp15,$src2\n\t" > >?+????????????"vptest??$tmp14,$tmp15\n\t" > >?+????????????"setb??$dst\t!"?%} > >?+??ins_encode?%{ > >?+????int?vector_len?=?1; > >?+????if?($tmp14$$XMMRegister?!=?$src1$$XMMRegister)?{ > >?+??????__?vmovdqu($tmp14$$XMMRegister,?$src1$$XMMRegister); > >?+????} > >?+????if?($tmp15$$XMMRegister?!=?$src2$$XMMRegister)?{ > >?+??????__?vmovdqu($tmp15$$XMMRegister,?$src2$$XMMRegister); > >?+????} > >?+????__?vptest($tmp14$$XMMRegister,?$tmp15$$XMMRegister,?vector_len); > >?+????__?setb(Assembler::notZero,?$dst$$Register); > >?+????__?movzbl($dst$$Register,?$dst$$Register); > >?+??%} > >?+??ins_pipe(?pipe_slow?); > >?+%} > >?+ > >???instruct?loadmask8b(vecD?dst,?vecD?src)?%{ > >?????predicate(UseSSE?>=?2?&&?n->as_Vector()->length()?==?8?&&?n->bottom_type()->is_vect()->element_basic_type()?==?T_BYTE); > >?????match(Set?dst?(VectorLoadMask?src)); > >?diff?-r?636478e1ee75?src/hotspot/cpu/x86/x86_64.ad > >?---?a/src/hotspot/cpu/x86/x86_64.ad?????Thu?Dec?13?16:40:28?2018?-0800 > >?+++?b/src/hotspot/cpu/x86/x86_64.ad?????Sat?Dec?29?11:16:55?2018?+0800 > >?@@?-4426,6?+4426,17?@@ > >?????predicate(UseAVX?==?3);??format%{%}??interface(REG_INTER); > >???%} > > > >?+ > >?+operand?rymm14()?%{ > >?+??constraint(ALLOC_IN_RC(ymm14_reg));??match(VecY); > >?+??predicate((UseSSE?>?0)?&&?(UseAVX?<=?3));??format%{%}??interface(REG_INTER); > >?+%} > >?+operand?rymm15()?%{ > >?+??constraint(ALLOC_IN_RC(ymm15_reg));??match(VecY); > >?+??predicate((UseSSE?>?0)?&&?(UseAVX?<=?3));??format%{%}??interface(REG_INTER); > >?+%} > >?+ > >?+ > >???//----------OPERAND?CLASSES---------------------------------------------------- > >???//?Operand?Classes?are?groups?of?operands?that?are?used?as?to?simplify > >???//?instruction?definitions?by?not?requiring?the?AD?writer?to?specify?separate > > > > > >?The?test?to?reproduce?this?bug.?Please?DO?set?-XX:UseAVX=3 > > > >?import?jdk.incubator.vector.*; > >?import?java.util.Arrays; > >?import?java.util.Random; > >?import?java.lang.reflect.Field; > >?import?java.io.IOException; > >?import?jdk.incubator.vector.Vector.Mask; > >?import?jdk.incubator.vector.Vector.Shape; > >?public?class?VectorTrueTest > >?{ > > > >??????static?Random?random?=?new?Random(); > >??????static?final?IntVector.IntSpecies?Species?=?IntVector.species(Vector.Shape.S_256_BIT); > >??????public?static?int?size?=?1024?*?16; > >??????public?static?int?length?=?Species.length(); > >??????public?static?int?resultSize?=?size?/?length; > >??????static?boolean[]?anyResultV?=?new?boolean[resultSize]; > >??????static?boolean[]?allResultV?=?new?boolean[resultSize]; > >??????static?boolean[]?anyInput?=?new?boolean[size]; > >??????static?boolean[]?allInput?=?new?boolean[size]; > >??????public?static?void?main(String[]?args)?throws??NoSuchFieldException,?SecurityException,?IllegalArgumentException,?IllegalAccessException,?InstantiationException?{ > >??????????long?start0?=?System.currentTimeMillis(); > >??????????long?startv?=?System.currentTimeMillis(); > >??????????long?normalTime?=?0; > >??????????long?vecTime?=?0; > >??????????int?i?=?0; > >??????????for?(i?=?0;?i? >??????????????anyInput[i]?=?false; > >??????????????allInput[i]?=?false; > >??????????} > >??????????for?(i?=?0;?i? >??????????????vecTest(Species); > >??????????} > >??????????for?(i?=?0;?i? >??????????????anyResultV[i]?=?true; > >??????????????allResultV[i]?=?true; > >??????????} > >??????????vecTest(Species); > >??????????for?(i?=?0;?i? >??????????????if?(anyResultV[i]?!=?false)?throw?new?RuntimeException("Wrong?anyTrue?result!?Should?be?all?false,?index?"?+?i); > >??????????????if?(allResultV[i]?!=?false)?throw?new?RuntimeException("Wrong?allTrue?result!?Should?be?all?false,?index?"?+?i); > >??????????} > >??????} > >??????static?void?vecTest(IntVector.IntSpecies?Speciesint?)?{ > >??????????IntVector?v0; > >??????????int?i?=?0; > >??????????int?j?=?0; > >??????????Mask?maskAny?=?Speciesint.maskFromArray(anyInput,?i); > >??????????Mask?maskAll?=?Speciesint.maskFromArray(allInput,?i); > >??????????for?(i?=?0;?i?+?(Speciesint.length())?<=?size;?i?+=?Speciesint.length())?{ > >??????????????allResultV[j]?=?maskAll.allTrue(); > >??????????????anyResultV[j]?=?maskAny.anyTrue(); > >??????????????j++; > >??????????} > >??????????return; > >??????} > >?} > > > >?Regards, > >?Zhuoren > > > > > > > > > From maurizio.cimadamore at oracle.com Fri Jan 4 18:16:17 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Fri, 04 Jan 2019 18:16:17 +0000 Subject: hg: panama/dev: Automatic merge with vectorIntrinsics Message-ID: <201901041816.x04IGHrm025619@aojmv0008.oracle.com> Changeset: 02090c6cb8a3 Author: mcimadamore Date: 2019-01-04 19:21 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/02090c6cb8a3 Automatic merge with vectorIntrinsics From vladimir.x.ivanov at oracle.com Fri Jan 4 18:56:56 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 4 Jan 2019 10:56:56 -0800 Subject: [vector api] Operations on Mask-s In-Reply-To: References: <245a485c-1cfc-0798-300f-e8c88dda65ba@oracle.com> Message-ID: <33e59a58-973b-5721-5039-0e3450de9694@oracle.com> >> On x86 there are 2 ways to represent masks: (1) as high bits in vector registers >> (pre-AVX512) and (2) as opmask registers (k0-k7). (I believe it's similar on ARM >> with NEON and SVE.) >> > > Yes, and since SVE has fully predicate (mask) support, we would like to implement it in the mask register way, but need to keep compatible with non-mask Arches. Speaking of API implementation, I don't see a feasible way to avoid 2 separate representations of masks on backend side. Fortunately, it's still possible to keep unified memory layout and do proper conversions on heap accesses. That's what x86 already do (boolean[] <=> xmm/ymm/zmm, implemented using VectorLoadMask/VectorStoreMask nodes) and it works fine. Moreover, I doubt there's a pressing need to mix 2 representations at runtime. Even if there's a desire to represent different shapes with different representations, shape conversions are explicit (both in API and implementation). >> C2 doesn't have full AVX512 support yet (e.g., no opmask register support in >> register allocator) and it was decided to focus on pre-AVX512 model first. So, >> current implementation represents masks as vectors uniformly across pre- >> AVX512 and AVX512-capable CPUs. And that's probably the main reason why >> Mask hasn't got enough attention yet. >> > > Do you (or anyone else) have any plan to work on opmask register support in register allocator? I'm not aware of anybody working on it right now. I'll let Intel folks comment on their plans. I believe full AVX512 support will be a high priority task for subsequent iterations of x86 backend. Best regards, Vladimir Ivanov From tomasz.kowalczewski at gmail.com Fri Jan 4 19:23:03 2019 From: tomasz.kowalczewski at gmail.com (Tomasz Kowalczewski) Date: Fri, 4 Jan 2019 20:23:03 +0100 Subject: [vector api] Operations on Mask-s In-Reply-To: <245a485c-1cfc-0798-300f-e8c88dda65ba@oracle.com> References: <245a485c-1cfc-0798-300f-e8c88dda65ba@oracle.com> Message-ID: Thank you from thorough answer. I will test the solutions you proposed. What puzzled me initially is that there is actually no "fromLong" method, so API seems asymmetric in this regard (but it can be easily worked around). Thanks, Tomasz On Fri, Jan 4, 2019 at 2:02 AM Vladimir Ivanov wrote: > Tomasz, > > Yes, I had similar experience with complex use cases involving masks. > The API works well for lane-wise cases, but falls apart when cross-lane > operations are needed, requiring user to go through intermediate > representations which aren't optimized well. > > There are different ways to fix that gap on API level, e.g.: > * declare more operations on Mask > * operate on long: Mask <=> long > * operate on Vector: Mask <=> Vector or Mask <: BoolVector <: Vector > > But IMO the main limiting factor to proceed is implementation > considerations. > > On x86 there are 2 ways to represent masks: (1) as high bits in vector > registers (pre-AVX512) and (2) as opmask registers (k0-k7). (I believe > it's similar on ARM with NEON and SVE.) > > C2 doesn't have full AVX512 support yet (e.g., no opmask register > support in register allocator) and it was decided to focus on pre-AVX512 > model first. So, current implementation represents masks as vectors > uniformly across pre-AVX512 and AVX512-capable CPUs. And that's probably > the main reason why Mask hasn't got enough attention yet. > > Regarding workarounds to your immediate problem: with the API in its > current state, either Mask.toLong() or Mask.toVector() can be used, but > considering current state of the implementation (where both haven't been > intrinsified yet), I'd suggest to avoid them altogether in hot code for > now and try the following workaround instead (pseudo-code follows): > > mask0x55 = ... < fromLong(0x55 << 0) > ... // mask constant > mask0xAA = ... < fromLong(0x55 << 1) > ... // mask constant > > compMask0 = vector.lessThan(input) > compMask1 = vector.shiftEL(1).lessThan(input.shiftEL(1)) // lessThan > preserves zero in 1st element > > // equivalent of "(m & 0x55) | ((m << 1) & (0x55 << 1))" > compMask0.and(mask0x55).or(compMask1.and(mask0xAA)) > > The idea is to shift vectors and recompute masks instead. > > (Unfortunately, Vector.shiftEL() hasn't been intrinsified yet as well, > but you can implement an equivalent using Vector.rearrange(Shuffle) > which has enough support on JVM side.) > > Hope that helps. > > Best regards, > Vladimir Ivanov > > On 02/01/2019 01:34, Tomasz Kowalczewski wrote: > > Hi, > > > > I was working on implementing a simple sorting algorithm[1] using > VectorAPI > > and stumbled on something that might be a gap in Mask API. It might be > > intentional for this stage of API maturity - please advise. > > > > I need to modify bits of the mask I get from _mm512_cmp_X_mask (e.g. > > vector.lessThan(input)) by doing and-s and or-s which can be simulated by > > preparing appropriate Mask objects. What I found missing is shift > > operations. Example: > > > > ( compMask & 0x55 ) | ( ( compMask & 0x55 ) << 1) > > > > I cannot do this using Mask object. Most natural alternative would be to > > convert the mask into long, perform these operations and convert it back. > > Unfortunately I was unable to find how to convert long back to Mask. Of > > course this can be done (via boolean[] array) but that does not seem > > efficient :). > > > > Excuse me if I missed some API call. > > > > 1. https://hal.inria.fr/hal-01512970v1/document (page 14, Code 1). > > > -- Tomasz Kowalczewski From sandhya.viswanathan at intel.com Sat Jan 5 01:06:18 2019 From: sandhya.viswanathan at intel.com (Viswanathan, Sandhya) Date: Sat, 5 Jan 2019 01:06:18 +0000 Subject: [vector] gen-src.sh and gen-tests.sh fail on Linux after Jan 2 automatic merge Message-ID: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A49E97@FMSMSX126.amr.corp.intel.com> Vector API gen-src.sh and gen-tests.sh fail on Linux after Jan 3 automatic merge due to changes in Spp.java. I have following webrev which fixes the issue: http://cr.openjdk.java.net/~sviswanathan/vectorIntrinsics/spp/webrev.00/ Please review. Best Regards, Sandhya From vladimir.x.ivanov at oracle.com Sat Jan 5 01:11:39 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 4 Jan 2019 17:11:39 -0800 Subject: [vector] gen-src.sh and gen-tests.sh fail on Linux after Jan 2 automatic merge In-Reply-To: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A49E97@FMSMSX126.amr.corp.intel.com> References: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A49E97@FMSMSX126.amr.corp.intel.com> Message-ID: Looks good. Best regards, Vladimir Ivanov On 04/01/2019 17:06, Viswanathan, Sandhya wrote: > Vector API gen-src.sh and gen-tests.sh fail on Linux after Jan 3 automatic merge due to changes in Spp.java. > > I have following webrev which fixes the issue: > http://cr.openjdk.java.net/~sviswanathan/vectorIntrinsics/spp/webrev.00/ > > Please review. > > Best Regards, > Sandhya > From vladimir.x.ivanov at oracle.com Sat Jan 5 01:31:32 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 4 Jan 2019 17:31:32 -0800 Subject: CFV: New panama Committer: Sandhya Viswanathan Message-ID: <1a3d6307-6dc8-0c31-0f56-b43402c81bea@oracle.com> I hereby nominate Sandhya Viswanathan(openjdk id: sviswanathan) to Committer role in Project Panama. Sandhya is a long-standing contributor to x86 support for Vector API at Intel. Votes are due by January, 11, 2019. Only current panama Committers [1] are eligible to vote on this nomination. Votes must be cast in the open by replying to this mailing list. For Lazy Consensus voting instructions, see [2]. Best regards, Vladimir Ivanov [1] http://openjdk.java.net/census#panama [2] http://openjdk.java.net/projects#committer-vote From vladimir.x.ivanov at oracle.com Sat Jan 5 01:32:04 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 4 Jan 2019 17:32:04 -0800 Subject: CFV: New panama Committer: Sandhya Viswanathan In-Reply-To: <1a3d6307-6dc8-0c31-0f56-b43402c81bea@oracle.com> References: <1a3d6307-6dc8-0c31-0f56-b43402c81bea@oracle.com> Message-ID: <8b2d8cd9-77ae-13aa-4c23-012168e398ce@oracle.com> Vote: yes Best regards, Vladimir Ivanov On 04/01/2019 17:31, Vladimir Ivanov wrote: > I hereby nominate Sandhya Viswanathan(openjdk id: sviswanathan) to > Committer role in Project Panama. From vladimir.x.ivanov at oracle.com Sat Jan 5 01:36:53 2019 From: vladimir.x.ivanov at oracle.com (vladimir.x.ivanov at oracle.com) Date: Sat, 05 Jan 2019 01:36:53 +0000 Subject: hg: panama/dev: Update gen-*.sh scripts according to Spp changes. Message-ID: <201901050136.x051arb0019210@aojmv0008.oracle.com> Changeset: 855cacccf292 Author: sviswanathan Date: 2019-01-04 17:36 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/855cacccf292 Update gen-*.sh scripts according to Spp changes. ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/IntVectorHelper.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/gen-src.sh ! test/jdk/jdk/incubator/vector/gen-template.sh ! test/jdk/jdk/incubator/vector/gen-tests.sh From maurizio.cimadamore at oracle.com Sat Jan 5 01:39:18 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Sat, 05 Jan 2019 01:39:18 +0000 Subject: hg: panama/dev: Automatic merge with vectorIntrinsics Message-ID: <201901050139.x051dJod021787@aojmv0008.oracle.com> Changeset: 4b8c9761dde3 Author: mcimadamore Date: 2019-01-05 02:44 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/4b8c9761dde3 Automatic merge with vectorIntrinsics From vivek.r.deshpande at intel.com Sat Jan 5 03:18:39 2019 From: vivek.r.deshpande at intel.com (Deshpande, Vivek R) Date: Sat, 5 Jan 2019 03:18:39 +0000 Subject: New panama Committer: Sandhya Viswanathan In-Reply-To: <1a3d6307-6dc8-0c31-0f56-b43402c81bea@oracle.com> References: <1a3d6307-6dc8-0c31-0f56-b43402c81bea@oracle.com> Message-ID: <53E8E64DB2403849AFD89B7D4DAC8B2A9A13DB48@ORSMSX106.amr.corp.intel.com> Vote: Yes Regards, Vivek -----Original Message----- From: panama-dev [mailto:panama-dev-bounces at openjdk.java.net] On Behalf Of Vladimir Ivanov Sent: Friday, January 4, 2019 5:32 PM To: 'panama-dev at openjdk.java.net' Subject: CFV: New panama Committer: Sandhya Viswanathan I hereby nominate Sandhya Viswanathan(openjdk id: sviswanathan) to Committer role in Project Panama. Sandhya is a long-standing contributor to x86 support for Vector API at Intel. Votes are due by January, 11, 2019. Only current panama Committers [1] are eligible to vote on this nomination. Votes must be cast in the open by replying to this mailing list. For Lazy Consensus voting instructions, see [2]. Best regards, Vladimir Ivanov [1] http://openjdk.java.net/census#panama [2] http://openjdk.java.net/projects#committer-vote From sundararajan.athijegannathan at oracle.com Mon Jan 7 02:24:38 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Mon, 07 Jan 2019 07:54:38 +0530 Subject: CFV: New panama Committer: Sandhya Viswanathan In-Reply-To: <1a3d6307-6dc8-0c31-0f56-b43402c81bea@oracle.com> References: <1a3d6307-6dc8-0c31-0f56-b43402c81bea@oracle.com> Message-ID: <5C32B866.8000302@oracle.com> Vote: Yes -Sundar On 05/01/19, 7:01 AM, Vladimir Ivanov wrote: > I hereby nominate Sandhya Viswanathan(openjdk id: sviswanathan) to > Committer role in Project Panama. > > Sandhya is a long-standing contributor to x86 support for Vector API > at Intel. > > Votes are due by January, 11, 2019. > > Only current panama Committers [1] are eligible to vote on this > nomination. Votes must be cast in the open by replying to this mailing > list. > > For Lazy Consensus voting instructions, see [2]. > > Best regards, > Vladimir Ivanov > > [1] http://openjdk.java.net/census#panama > [2] http://openjdk.java.net/projects#committer-vote From Ningsheng.Jian at arm.com Mon Jan 7 02:35:33 2019 From: Ningsheng.Jian at arm.com (Ningsheng Jian (Arm Technology China)) Date: Mon, 7 Jan 2019 02:35:33 +0000 Subject: [vector api] RFR: Implement Vector API maxAll/minAll for all data types for AArch64 NEON In-Reply-To: References: Message-ID: Hi Yang, This looks good to me. Verified that current maxAll/minAll cases passed on AArch64 without new failures. Thanks, Ningsheng > -----Original Message----- > From: panama-dev On Behalf Of Yang > Zhang (Arm Technology China) > Sent: Tuesday, December 25, 2018 2:42 PM > To: panama-dev at openjdk.java.net > Cc: nd > Subject: RE: [vector api] RFR: Implement Vector API maxAll/minAll for all data > types for AArch64 NEON > > Hi > > I have a patch which implement Vector API maxAll/minAll for all data types for > AArch64 NEON. > Could you please help to review it? > > http://cr.openjdk.java.net/~yzhang/vectorapi.maxall/webrev.00/ > > Regards, > Yang IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. From Yang.Zhang at arm.com Mon Jan 7 02:40:24 2019 From: Yang.Zhang at arm.com (Yang Zhang (Arm Technology China)) Date: Mon, 7 Jan 2019 02:40:24 +0000 Subject: New panama Committer: Sandhya Viswanathan In-Reply-To: <1a3d6307-6dc8-0c31-0f56-b43402c81bea@oracle.com> References: <1a3d6307-6dc8-0c31-0f56-b43402c81bea@oracle.com> Message-ID: Vote: yes Regards, Yang -----Original Message----- From: panama-dev On Behalf Of Vladimir Ivanov Sent: Saturday, January 5, 2019 9:32 AM To: 'panama-dev at openjdk.java.net' Subject: CFV: New panama Committer: Sandhya Viswanathan I hereby nominate Sandhya Viswanathan(openjdk id: sviswanathan) to Committer role in Project Panama. Sandhya is a long-standing contributor to x86 support for Vector API at Intel. Votes are due by January, 11, 2019. Only current panama Committers [1] are eligible to vote on this nomination. Votes must be cast in the open by replying to this mailing list. For Lazy Consensus voting instructions, see [2]. Best regards, Vladimir Ivanov [1] http://openjdk.java.net/census#panama [2] http://openjdk.java.net/projects#committer-vote From yang.zhang at arm.com Mon Jan 7 02:54:54 2019 From: yang.zhang at arm.com (yang.zhang at arm.com) Date: Mon, 07 Jan 2019 02:54:54 +0000 Subject: hg: panama/dev: Implement Vector API maxAll/minAll for all data types for AArch64 NEON Message-ID: <201901070254.x072st8n007739@aojmv0008.oracle.com> Changeset: 890e996dfd1a Author: yzhang Date: 2019-01-07 10:54 +0800 URL: http://hg.openjdk.java.net/panama/dev/rev/890e996dfd1a Implement Vector API maxAll/minAll for all data types for AArch64 NEON Contributed-by: lauren.walkowski at arm.com, yang.zhang at arm.com ! src/hotspot/cpu/aarch64/aarch64.ad ! src/hotspot/cpu/aarch64/assembler_aarch64.hpp From maurizio.cimadamore at oracle.com Mon Jan 7 02:59:18 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Mon, 07 Jan 2019 02:59:18 +0000 Subject: hg: panama/dev: Automatic merge with vectorIntrinsics Message-ID: <201901070259.x072xI3Q010321@aojmv0008.oracle.com> Changeset: 29c9606b32b8 Author: mcimadamore Date: 2019-01-07 04:04 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/29c9606b32b8 Automatic merge with vectorIntrinsics From sundararajan.athijegannathan at oracle.com Mon Jan 7 13:39:27 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Mon, 07 Jan 2019 19:09:27 +0530 Subject: [foreign] RFR 8216268: duplicate declaration handler has to be a tree visitor Message-ID: <5C33568F.30804@oracle.com> Please review. Bug: https://bugs.openjdk.java.net/browse/JDK-8216268 Webrev: https://cr.openjdk.java.net/~sundar/8216268/webrev.00/index.html Thanks -Sundar From john.r.rose at oracle.com Mon Jan 7 17:10:10 2019 From: john.r.rose at oracle.com (John Rose) Date: Mon, 7 Jan 2019 09:10:10 -0800 Subject: CFV: New panama Committer: Sandhya Viswanathan In-Reply-To: <1a3d6307-6dc8-0c31-0f56-b43402c81bea@oracle.com> References: <1a3d6307-6dc8-0c31-0f56-b43402c81bea@oracle.com> Message-ID: Vote: yes From henry.jen at oracle.com Mon Jan 7 18:00:48 2019 From: henry.jen at oracle.com (Henry Jen) Date: Mon, 7 Jan 2019 10:00:48 -0800 Subject: CFV: New panama Committer: Sandhya Viswanathan In-Reply-To: <1a3d6307-6dc8-0c31-0f56-b43402c81bea@oracle.com> References: <1a3d6307-6dc8-0c31-0f56-b43402c81bea@oracle.com> Message-ID: Vote: yes Cheers, Henry > On Jan 4, 2019, at 5:31 PM, Vladimir Ivanov wrote: > > I hereby nominate Sandhya Viswanathan(openjdk id: sviswanathan) to Committer role in Project Panama. > > Sandhya is a long-standing contributor to x86 support for Vector API at Intel. > > Votes are due by January, 11, 2019. > > Only current panama Committers [1] are eligible to vote on this nomination. Votes must be cast in the open by replying to this mailing list. > > For Lazy Consensus voting instructions, see [2]. > > Best regards, > Vladimir Ivanov > > [1] http://openjdk.java.net/census#panama > [2] http://openjdk.java.net/projects#committer-vote From vladimir.x.ivanov at oracle.com Tue Jan 8 01:16:57 2019 From: vladimir.x.ivanov at oracle.com (vladimir.x.ivanov at oracle.com) Date: Tue, 08 Jan 2019 01:16:57 +0000 Subject: hg: panama/dev: Direct native calls: C2 support Message-ID: <201901080116.x081GwHs010221@aojmv0008.oracle.com> Changeset: 113e76a799c8 Author: vlivanov Date: 2019-01-07 16:07 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/113e76a799c8 Direct native calls: C2 support ! src/hotspot/cpu/x86/directUpcallHandler_x86.cpp ! src/hotspot/cpu/x86/stubGenerator_x86_64.cpp ! src/hotspot/share/opto/callGenerator.cpp ! src/hotspot/share/opto/graphKit.cpp ! src/hotspot/share/opto/graphKit.hpp ! src/hotspot/share/opto/runtime.cpp ! src/hotspot/share/opto/runtime.hpp ! src/hotspot/share/runtime/stubRoutines.cpp ! src/hotspot/share/runtime/stubRoutines.hpp From jbvernee at xs4all.nl Tue Jan 8 15:58:31 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Tue, 08 Jan 2019 16:58:31 +0100 Subject: [foreign] RFR 8216284: Make sure test jars are deleted before/after tests Message-ID: <434cdc241282ded515cea862561cc0b2@xs4all.nl> Hi, I was seeing some errors while running the JextractToolProviderTest test, but the test still passed. The errors were from the test jars not being able to be deleted because some process was using the file. It turns out this was due to JextractToolRunner::loadClass creating a URLClassLoader to load classes from the jar, but the classloader never being closed causing the file to still be in use when trying to delete it. This was throwing IOExceptions which were being printed, but otherwise ignored. With all the errors being thrown and ignored I wasn't sure whether I could trust the outcome of the test. Different tests could also possibly interfere with each other if they use the same jar file name and the jar doesn't get deleted after each test. I've solved the problem by replacing the loadClass method with a classLoader method which returns an AutoCloseable Loader through which the classes can be loaded. The close() method on Loader closes the class loader. I have also turned the errors I was seeing into hard errors to make sure that the tests fail if an IOException occurs unexpectedly. To make that possible I switched JextractToolRunner::deleteFile to using Files.deleteIfExtists instead of delete so that an error will not occur when it is called at the start of a test but there's nothing to delete. JextractToolRunner::classLoader is called with a list of Paths to use as class path entries e.g.: Path helloJar = getOutputFilePath("hello.jar"); deleteFile(helloJar); // make sure jar does not exist at start of test Path helloH = getInputFilePath("hello.h"); run("-o", helloJar.toString(), helloH.toString()).checkSuccess(); // create the jar try(Loader loader = classLoader(helloJar)) { // ... do tests } finally { // class loader is closed after try-block deleteFile(helloJar); // now succeeds to delete jar } I've replaced calls to loadClass with calls to Loader::loadClass, also adding a call to JextractToolRunner::classLoader to the enclosing try-blocks. I've also made sure jars are being deleted before /after a test in some places where that was not being done. Bug: https://bugs.openjdk.java.net/browse/JDK-8216284 Webrev: http://cr.openjdk.java.net/~jvernee/panama/webrevs/8216284/webrev.03/ Thanks, Jorn From sundararajan.athijegannathan at oracle.com Wed Jan 9 02:28:26 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Wed, 09 Jan 2019 07:58:26 +0530 Subject: [foreign] RFR 8216284: Make sure test jars are deleted before/after tests In-Reply-To: <434cdc241282ded515cea862561cc0b2@xs4all.nl> References: <434cdc241282ded515cea862561cc0b2@xs4all.nl> Message-ID: <5C355C4A.8030804@oracle.com> This change looks good. PS. I guess the issue of deleting jar file-in-use is Windows specific. That's why it was never seen on other platforms. -Sundar On 08/01/19, 9:28 PM, Jorn Vernee wrote: > Hi, > > I was seeing some errors while running the JextractToolProviderTest > test, but the test still passed. The errors were from the test jars > not being able to be deleted because some process was using the file. > It turns out this was due to JextractToolRunner::loadClass creating a > URLClassLoader to load classes from the jar, but the classloader never > being closed causing the file to still be in use when trying to delete > it. This was throwing IOExceptions which were being printed, but > otherwise ignored. > > With all the errors being thrown and ignored I wasn't sure whether I > could trust the outcome of the test. Different tests could also > possibly interfere with each other if they use the same jar file name > and the jar doesn't get deleted after each test. > > I've solved the problem by replacing the loadClass method with a > classLoader method which returns an AutoCloseable Loader through which > the classes can be loaded. The close() method on Loader closes the > class loader. I have also turned the errors I was seeing into hard > errors to make sure that the tests fail if an IOException occurs > unexpectedly. To make that possible I switched > JextractToolRunner::deleteFile to using Files.deleteIfExtists instead > of delete so that an error will not occur when it is called at the > start of a test but there's nothing to delete. > > JextractToolRunner::classLoader is called with a list of Paths to use > as class path entries e.g.: > > Path helloJar = getOutputFilePath("hello.jar"); > deleteFile(helloJar); // make sure jar does not exist at start of > test > Path helloH = getInputFilePath("hello.h"); > run("-o", helloJar.toString(), helloH.toString()).checkSuccess(); > // create the jar > try(Loader loader = classLoader(helloJar)) { > // ... do tests > } finally { // class loader is closed after try-block > deleteFile(helloJar); // now succeeds to delete jar > } > > I've replaced calls to loadClass with calls to Loader::loadClass, also > adding a call to JextractToolRunner::classLoader to the enclosing > try-blocks. I've also made sure jars are being deleted before /after a > test in some places where that was not being done. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8216284 > Webrev: > http://cr.openjdk.java.net/~jvernee/panama/webrevs/8216284/webrev.03/ > > Thanks, > Jorn From jbvernee at xs4all.nl Wed Jan 9 02:54:07 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Wed, 09 Jan 2019 03:54:07 +0100 Subject: [foreign] RFR 8216284: Make sure test jars are deleted before/after tests In-Reply-To: <5C355C4A.8030804@oracle.com> References: <434cdc241282ded515cea862561cc0b2@xs4all.nl> <5C355C4A.8030804@oracle.com> Message-ID: > PS. I guess the issue of deleting jar file-in-use is Windows specific. > That's why it was never seen on other platforms. Yeah, I've noticed that the Windows file system can be a little more strict in some cases. Guess Mac/Linux don't care since it's the same process trying to delete the file. Thanks for the review! Jorn Sundararajan Athijegannathan schreef op 2019-01-09 03:28: > This change looks good. > > PS. I guess the issue of deleting jar file-in-use is Windows specific. > That's why it was never seen on other platforms. > > -Sundar > > On 08/01/19, 9:28 PM, Jorn Vernee wrote: >> Hi, >> >> I was seeing some errors while running the JextractToolProviderTest >> test, but the test still passed. The errors were from the test jars >> not being able to be deleted because some process was using the file. >> It turns out this was due to JextractToolRunner::loadClass creating a >> URLClassLoader to load classes from the jar, but the classloader never >> being closed causing the file to still be in use when trying to delete >> it. This was throwing IOExceptions which were being printed, but >> otherwise ignored. >> >> With all the errors being thrown and ignored I wasn't sure whether I >> could trust the outcome of the test. Different tests could also >> possibly interfere with each other if they use the same jar file name >> and the jar doesn't get deleted after each test. >> >> I've solved the problem by replacing the loadClass method with a >> classLoader method which returns an AutoCloseable Loader through which >> the classes can be loaded. The close() method on Loader closes the >> class loader. I have also turned the errors I was seeing into hard >> errors to make sure that the tests fail if an IOException occurs >> unexpectedly. To make that possible I switched >> JextractToolRunner::deleteFile to using Files.deleteIfExtists instead >> of delete so that an error will not occur when it is called at the >> start of a test but there's nothing to delete. >> >> JextractToolRunner::classLoader is called with a list of Paths to use >> as class path entries e.g.: >> >> Path helloJar = getOutputFilePath("hello.jar"); >> deleteFile(helloJar); // make sure jar does not exist at start of >> test >> Path helloH = getInputFilePath("hello.h"); >> run("-o", helloJar.toString(), helloH.toString()).checkSuccess(); >> // create the jar >> try(Loader loader = classLoader(helloJar)) { >> // ... do tests >> } finally { // class loader is closed after try-block >> deleteFile(helloJar); // now succeeds to delete jar >> } >> >> I've replaced calls to loadClass with calls to Loader::loadClass, also >> adding a call to JextractToolRunner::classLoader to the enclosing >> try-blocks. I've also made sure jars are being deleted before /after a >> test in some places where that was not being done. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8216284 >> Webrev: >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8216284/webrev.03/ >> >> Thanks, >> Jorn From jbvernee at xs4all.nl Wed Jan 9 02:55:57 2019 From: jbvernee at xs4all.nl (jbvernee at xs4all.nl) Date: Wed, 09 Jan 2019 02:55:57 +0000 Subject: hg: panama/dev: Make sure test jars are deleted before/after tests Message-ID: <201901090255.x092twAX008031@aojmv0008.oracle.com> Changeset: 01f0daca1db4 Author: jvernee Date: 2019-01-09 03:55 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/01f0daca1db4 Make sure test jars are deleted before/after tests ! test/jdk/com/sun/tools/jextract/ArrayTest.java ! test/jdk/com/sun/tools/jextract/ConstantsTest.java ! test/jdk/com/sun/tools/jextract/EmptyStructTest.java ! test/jdk/com/sun/tools/jextract/JextractToolProviderTest.java ! test/jdk/com/sun/tools/jextract/JextractToolRunner.java ! test/jdk/com/sun/tools/jextract/RedundantDeclsTest.java ! test/jdk/com/sun/tools/jextract/TestDowncall.java ! test/jdk/com/sun/tools/jextract/TestForwardRef.java ! test/jdk/com/sun/tools/jextract/TestUpcall.java ! test/jdk/com/sun/tools/jextract/rpath/RpathTest.java ! test/jdk/com/sun/tools/jextract/testStruct/StructTest.java From maurizio.cimadamore at oracle.com Wed Jan 9 02:58:57 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 09 Jan 2019 02:58:57 +0000 Subject: hg: panama/dev: Automatic merge with foreign Message-ID: <201901090258.x092wvMU009458@aojmv0008.oracle.com> Changeset: edd5f9f61ab8 Author: mcimadamore Date: 2019-01-09 04:04 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/edd5f9f61ab8 Automatic merge with foreign From sundararajan.athijegannathan at oracle.com Wed Jan 9 05:18:55 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Wed, 09 Jan 2019 10:48:55 +0530 Subject: [foreign] RFR 8216268: duplicate declaration handler has to be a tree visitor In-Reply-To: <5C33568F.30804@oracle.com> References: <5C33568F.30804@oracle.com> Message-ID: <5C35843F.5020500@oracle.com> Please review. Updated webrev: https://cr.openjdk.java.net/~sundar/8216268/webrev.01/index.html PS. Updated it for the recent class loading changes in the test framework. -Sundar On 07/01/19, 7:09 PM, Sundararajan Athijegannathan wrote: > Please review. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8216268 > Webrev: https://cr.openjdk.java.net/~sundar/8216268/webrev.00/index.html > > Thanks > -Sundar From maurizio.cimadamore at oracle.com Wed Jan 9 12:14:44 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 9 Jan 2019 12:14:44 +0000 Subject: [foreign] RFR 8216268: duplicate declaration handler has to be a tree visitor In-Reply-To: <5C35843F.5020500@oracle.com> References: <5C33568F.30804@oracle.com> <5C35843F.5020500@oracle.com> Message-ID: Great work. I like it that: 1) It moves duplicate detection logic out of TypedefHandler (which is there for another reason) 2) and also out of ASMCodeFactory One element of risk that I note here, is that the new code isn't exactly a functional replacement for the old one, in that some of the checks (e.g. those in ASMCodeFactory) used to be on a per-header basis, while with your new visitor pass, everything is applied on the header trees produced by the parser. I can imagine situations in which the old code did not detect conflict (e.g. A$foo vs. B$foo) whereas the new approach detects the conflict and generates only one class. I think the new approach is probably for the best and more in sync with where we need to be longer term anyway. But just want to double check on that. Other than that, +1 Maurizio On 09/01/2019 05:18, Sundararajan Athijegannathan wrote: > Please review. > > Updated webrev: > https://cr.openjdk.java.net/~sundar/8216268/webrev.01/index.html > > PS. Updated it for the recent class loading changes in the test > framework. > > -Sundar > > On 07/01/19, 7:09 PM, Sundararajan Athijegannathan wrote: >> Please review. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8216268 >> Webrev: https://cr.openjdk.java.net/~sundar/8216268/webrev.00/index.html >> >> Thanks >> -Sundar From sundararajan.athijegannathan at oracle.com Wed Jan 9 12:41:10 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Wed, 09 Jan 2019 18:11:10 +0530 Subject: [foreign] RFR 8216268: duplicate declaration handler has to be a tree visitor In-Reply-To: References: <5C33568F.30804@oracle.com> <5C35843F.5020500@oracle.com> Message-ID: <5C35EBE6.9080707@oracle.com> Thanks for the review. Yes - we do change the way duplicates are detected and handled - in a tree phase rather than late in codegen. I didn't see any errors in tests as well as with samples. But earlier there was a overwrite warning (for duplicate decl. from the same file) - but that does not occur anymore: File test.h struct Foo; struct Foo; Thanks, -Sundar On 09/01/19, 5:44 PM, Maurizio Cimadamore wrote: > Great work. > > I like it that: > > 1) It moves duplicate detection logic out of TypedefHandler (which is > there for another reason) > 2) and also out of ASMCodeFactory > > One element of risk that I note here, is that the new code isn't > exactly a functional replacement for the old one, in that some of the > checks (e.g. those in ASMCodeFactory) used to be on a per-header > basis, while with your new visitor pass, everything is applied on the > header trees produced by the parser. I can imagine situations in which > the old code did not detect conflict (e.g. A$foo vs. B$foo) whereas > the new approach detects the conflict and generates only one class. > > I think the new approach is probably for the best and more in sync > with where we need to be longer term anyway. But just want to double > check on that. > > Other than that, +1 > > Maurizio > > On 09/01/2019 05:18, Sundararajan Athijegannathan wrote: >> Please review. >> >> Updated webrev: >> https://cr.openjdk.java.net/~sundar/8216268/webrev.01/index.html >> >> PS. Updated it for the recent class loading changes in the test >> framework. >> >> -Sundar >> >> On 07/01/19, 7:09 PM, Sundararajan Athijegannathan wrote: >>> Please review. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8216268 >>> Webrev: >>> https://cr.openjdk.java.net/~sundar/8216268/webrev.00/index.html >>> >>> Thanks >>> -Sundar From sundararajan.athijegannathan at oracle.com Wed Jan 9 12:27:06 2019 From: sundararajan.athijegannathan at oracle.com (sundararajan.athijegannathan at oracle.com) Date: Wed, 09 Jan 2019 12:27:06 +0000 Subject: hg: panama/dev: 8216268: duplicate declaration handler has to be a tree visitor Message-ID: <201901091227.x09CR6g3007945@aojmv0008.oracle.com> Changeset: c2722ee526cf Author: sundar Date: 2019-01-09 18:12 +0530 URL: http://hg.openjdk.java.net/panama/dev/rev/c2722ee526cf 8216268: duplicate declaration handler has to be a tree visitor Reviewed-by: mcimadamore ! src/jdk.jextract/share/classes/com/sun/tools/jextract/AsmCodeFactory.java + src/jdk.jextract/share/classes/com/sun/tools/jextract/DuplicateDeclarationHandler.java ! src/jdk.jextract/share/classes/com/sun/tools/jextract/JextractTool.java ! src/jdk.jextract/share/classes/com/sun/tools/jextract/TypedefHandler.java ! test/jdk/com/sun/tools/jextract/RedundantDeclsTest.java + test/jdk/com/sun/tools/jextract/repeatedDecls.h From maurizio.cimadamore at oracle.com Wed Jan 9 12:29:05 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 09 Jan 2019 12:29:05 +0000 Subject: hg: panama/dev: Automatic merge with foreign Message-ID: <201901091229.x09CT6ek009177@aojmv0008.oracle.com> Changeset: 23c51005f8e6 Author: mcimadamore Date: 2019-01-09 13:34 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/23c51005f8e6 Automatic merge with foreign From maurizio.cimadamore at oracle.com Wed Jan 9 16:30:58 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 09 Jan 2019 16:30:58 +0000 Subject: hg: panama/dev: manual merge with foreign Message-ID: <201901091630.x09GUxwU016377@aojmv0008.oracle.com> Changeset: dfbf76dec2c0 Author: mcimadamore Date: 2019-01-09 16:30 +0000 URL: http://hg.openjdk.java.net/panama/dev/rev/dfbf76dec2c0 manual merge with foreign ! test/jdk/com/sun/tools/jextract/TestUpcall.java From maurizio.cimadamore at oracle.com Wed Jan 9 21:00:13 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 09 Jan 2019 21:00:13 +0000 Subject: hg: panama/dev: 86 new changesets Message-ID: <201901092100.x09L0J0X002290@aojmv0008.oracle.com> Changeset: b561ea19a7b9 Author: igerasim Date: 2019-01-02 15:33 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/b561ea19a7b9 6996807: FieldReflectorKey hash code computation can be improved Reviewed-by: rriggs ! src/java.base/share/classes/java/io/ObjectStreamClass.java Changeset: d3e199e30cfb Author: jwilhelm Date: 2019-01-03 02:26 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/d3e199e30cfb Added tag jdk-13+2 for changeset 50677f43ac3d ! .hgtags Changeset: 3149a923b30e Author: erikj Date: 2019-01-03 11:21 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/3149a923b30e 8215991: Stop hiding exception from ArtifactResolver failures in tests Reviewed-by: tbell, ctornqvi ! test/lib/jdk/test/lib/artifacts/ArtifactResolver.java Changeset: 106fc138542a Author: hseigel Date: 2019-01-03 13:11 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/106fc138542a 8215644: Clean up globalDefinitions_.hpp Summary: Remove non-existent classes from forward declarations, delete unused functions, etc. Reviewed-by: coleenp, kbarrett ! src/hotspot/share/interpreter/templateTable.cpp ! src/hotspot/share/runtime/globals.hpp ! src/hotspot/share/runtime/os.hpp ! src/hotspot/share/utilities/globalDefinitions.hpp ! src/hotspot/share/utilities/globalDefinitions_gcc.hpp ! src/hotspot/share/utilities/globalDefinitions_solstudio.hpp ! src/hotspot/share/utilities/globalDefinitions_visCPP.hpp ! src/hotspot/share/utilities/globalDefinitions_xlc.hpp Changeset: e412d5c096bc Author: coffeys Date: 2019-01-03 17:27 +0000 URL: http://hg.openjdk.java.net/panama/dev/rev/e412d5c096bc 8179943: Typo in javax.net.ssl.SSLSession.removeValue(String) method documentation Reviewed-by: coffeys Contributed-by: roger.calnan at oracle.com ! src/java.base/share/classes/javax/net/ssl/SSLSession.java Changeset: d976ee345d11 Author: coffeys Date: 2019-01-03 17:29 +0000 URL: http://hg.openjdk.java.net/panama/dev/rev/d976ee345d11 8182992: Typo in DatagramPacket constructor API doc Reviewed-by: coffeys Contributed-by: roger.calnan at oracle.com ! src/java.base/share/classes/java/net/DatagramPacket.java Changeset: bf2533105a26 Author: coffeys Date: 2019-01-03 17:46 +0000 URL: http://hg.openjdk.java.net/panama/dev/rev/bf2533105a26 8215912: Various Typos in java.net Method Documentation Reviewed-by: coffeys Contributed-by: roger.calnan at oracle.com ! src/java.base/share/classes/java/net/InetAddress.java ! src/java.base/share/classes/java/net/Socket.java ! src/java.base/share/classes/java/net/URLConnection.java Changeset: 6459eb210418 Author: coffeys Date: 2019-01-03 17:49 +0000 URL: http://hg.openjdk.java.net/panama/dev/rev/6459eb210418 8215911: Various Typos in SQL Method Documentation Reviewed-by: coffeys Contributed-by: roger.calnan at oracle.com ! src/java.sql.rowset/share/classes/com/sun/rowset/JdbcRowSetImpl.java ! src/java.sql.rowset/share/classes/com/sun/rowset/internal/SyncResolverImpl.java Changeset: 247207c768d7 Author: coffeys Date: 2019-01-03 18:19 +0000 URL: http://hg.openjdk.java.net/panama/dev/rev/247207c768d7 Merge Changeset: ddbd6111f564 Author: ecaspole Date: 2019-01-03 13:22 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/ddbd6111f564 8196347: LogCompilation: generate log file on the fly for input to junits Summary: Dynamically generate simple log files Reviewed-by: kvn, thartmann ! src/utils/LogCompilation/pom.xml ! src/utils/LogCompilation/src/test/java/com/sun/hotspot/tools/compiler/TestLogCompilation.java - src/utils/LogCompilation/src/test/resources/hotspot_pid23756.log - src/utils/LogCompilation/src/test/resources/hotspot_pid25109.log - src/utils/LogCompilation/src/test/resources/no_tiered_short.log - src/utils/LogCompilation/src/test/resources/tiered_short.log Changeset: 5f942c387778 Author: ecaspole Date: 2019-01-03 13:22 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/5f942c387778 Merge Changeset: 2345e253e677 Author: gadams Date: 2019-01-03 15:54 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/2345e253e677 8216059: nsk_jvmti_parseoptions still has dependency on tilde separator Reviewed-by: sspitsyn, cjplummer ! test/hotspot/jtreg/vmTestbase/nsk/share/jvmti/jvmti_tools.cpp Changeset: a3e7e08ee427 Author: mbaesken Date: 2019-01-03 16:14 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/a3e7e08ee427 8215961: jdk/jfr/event/os/TestCPUInformation.java fails on AArch64 Reviewed-by: aph, goetz, lucy ! src/hotspot/cpu/aarch64/vm_version_ext_aarch64.cpp ! src/hotspot/cpu/s390/vm_version_ext_s390.cpp ! test/jdk/jdk/jfr/event/os/TestCPUInformation.java Changeset: 697b5f5dec56 Author: jgeorge Date: 2019-01-04 13:41 +0530 URL: http://hg.openjdk.java.net/panama/dev/rev/697b5f5dec56 8213457: serviceability/sa/ClhsdbInspect.java time out Summary: Increase the timeout needed for ClhsdbInspect.java to 480 Reviewed-by: sspitsyn, cjplummer, lmesnik ! test/hotspot/jtreg/ProblemList.txt ! test/hotspot/jtreg/serviceability/sa/ClhsdbInspect.java Changeset: 6f2d65f29de3 Author: erikj Date: 2019-01-04 01:59 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/6f2d65f29de3 8216048: Fix devkit creation in WSL Reviewed-by: erikj Contributed-by: andrewluotechnologies at outlook.com ! make/devkit/createWindowsDevkit2017.sh Changeset: 5c0ec35d0533 Author: egahlin Date: 2019-01-04 14:05 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/5c0ec35d0533 8215771: The jfr tool should pretty print reference chains Reviewed-by: mgronlun ! src/jdk.jfr/share/classes/jdk/jfr/internal/tool/PrettyWriter.java Changeset: 6b7240a24b56 Author: redestad Date: 2019-01-04 16:23 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/6b7240a24b56 8216157: Enable inlining of java_lang_Class::is_primitive Reviewed-by: coleenp, eosterlund, jiangli ! src/hotspot/share/ci/ciInstance.cpp ! src/hotspot/share/ci/ciMethodType.cpp ! src/hotspot/share/classfile/javaClasses.cpp ! src/hotspot/share/classfile/javaClasses.inline.hpp ! src/hotspot/share/prims/jvmtiEnter.xsl ! src/hotspot/share/prims/jvmtiExport.cpp ! src/hotspot/share/prims/jvmtiRedefineClasses.cpp ! src/hotspot/share/prims/jvmtiTrace.cpp ! src/hotspot/share/runtime/reflection.cpp ! src/hotspot/share/services/heapDumper.cpp Changeset: 41779d3b85e1 Author: rriggs Date: 2019-01-04 11:03 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/41779d3b85e1 8216067: Unused local vars in windows/native/libjava/io_util_md.c Reviewed-by: rriggs Contributed-by: andrewluotechnologies at outlook.com ! src/java.base/windows/native/libjava/io_util_md.c Changeset: 41af13b70efd Author: rriggs Date: 2019-01-04 12:30 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/41af13b70efd 8215798: Use {@systemProperty} for definition of org.openjdk.java.util.stream.tripwire property Reviewed-by: lancea ! src/java.base/share/classes/java/util/Spliterator.java Changeset: 62a4355dc9c8 Author: joehw Date: 2019-01-04 10:42 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/62a4355dc9c8 8215330: javax.xml.catalog.CatalogResolverImpl: GroupEntry.matchURI fails to match Reviewed-by: lancea ! src/java.xml/share/classes/javax/xml/catalog/GroupEntry.java + test/jaxp/javax/xml/jaxp/unittest/catalog/GroupTest.java + test/jaxp/javax/xml/jaxp/unittest/catalog/GroupTest.xml Changeset: db1d11c253d8 Author: hseigel Date: 2019-01-04 14:28 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/db1d11c253d8 8216010: Change callers of build_u2_from() to call Bytes::get_Java_u2() instead Summary: Change the callers and delete function build_u2_from() Reviewed-by: kbarrett, jiangli, coleenp ! src/hotspot/share/aot/aotCodeHeap.cpp ! src/hotspot/share/aot/aotCompiledMethod.cpp ! src/hotspot/share/jvmci/compilerRuntime.cpp ! src/hotspot/share/utilities/globalDefinitions.hpp Changeset: 17d568776429 Author: redestad Date: 2019-01-04 20:58 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/17d568776429 8215412: Optimize PrintStream.println methods Reviewed-by: rriggs, dfuchs, forax ! src/java.base/share/classes/java/io/PrintStream.java Changeset: 259c36ef27df Author: coleenp Date: 2019-01-04 15:06 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/259c36ef27df 8215731: Move forward class definitions out of globalDefinitions.hpp Summary: redistribute the forward declarations to the header files that need them. Reviewed-by: dholmes, lfoltan ! src/hotspot/cpu/aarch64/nativeInst_aarch64.hpp ! src/hotspot/cpu/arm/nativeInst_arm_32.hpp ! src/hotspot/cpu/sparc/nativeInst_sparc.hpp ! src/hotspot/cpu/x86/nativeInst_x86.hpp ! src/hotspot/os_cpu/linux_aarch64/globals_linux_aarch64.hpp ! src/hotspot/share/classfile/symbolTable.hpp ! src/hotspot/share/code/compiledMethod.hpp ! src/hotspot/share/code/location.hpp ! src/hotspot/share/code/nmethod.hpp ! src/hotspot/share/compiler/compilerOracle.hpp ! src/hotspot/share/gc/cms/cmsOopClosures.hpp ! src/hotspot/share/gc/g1/dirtyCardQueue.hpp ! src/hotspot/share/gc/g1/g1SATBMarkQueueSet.hpp ! src/hotspot/share/gc/shared/blockOffsetTable.hpp ! src/hotspot/share/gc/shared/ptrQueue.hpp ! src/hotspot/share/gc/shared/satbMarkQueue.hpp ! src/hotspot/share/interpreter/bytecodeTracer.hpp ! src/hotspot/share/interpreter/oopMapCache.hpp ! src/hotspot/share/jfr/leakprofiler/emitEventOperation.hpp ! src/hotspot/share/jfr/leakprofiler/leakProfiler.hpp ! src/hotspot/share/logging/logDecorators.hpp ! src/hotspot/share/memory/allocation.hpp ! src/hotspot/share/memory/virtualspace.hpp ! src/hotspot/share/oops/instanceKlass.hpp ! src/hotspot/share/oops/method.hpp ! src/hotspot/share/oops/oopsHierarchy.hpp ! src/hotspot/share/runtime/compilationPolicy.hpp ! src/hotspot/share/runtime/deoptimization.hpp ! src/hotspot/share/runtime/flags/jvmFlag.hpp ! src/hotspot/share/runtime/frame.hpp ! src/hotspot/share/runtime/jniHandles.hpp ! src/hotspot/share/runtime/os.hpp ! src/hotspot/share/runtime/osThread.hpp ! src/hotspot/share/runtime/stackValue.hpp ! src/hotspot/share/runtime/thread.hpp ! src/hotspot/share/utilities/concurrentHashTable.hpp ! src/hotspot/share/utilities/constantTag.hpp ! src/hotspot/share/utilities/exceptions.hpp ! src/hotspot/share/utilities/globalDefinitions.hpp Changeset: 05429f43e036 Author: ysuenaga Date: 2019-01-05 10:07 +0900 URL: http://hg.openjdk.java.net/panama/dev/rev/05429f43e036 8216155: C4819 warning at libfreetype sources on Windows Reviewed-by: erikj ! make/lib/Awt2dLibraries.gmk Changeset: 22baf8054a40 Author: fyang Date: 2019-01-05 10:48 +0800 URL: http://hg.openjdk.java.net/panama/dev/rev/22baf8054a40 8215951: AArch64: jtreg test vmTestbase/nsk/jvmti/PopFrame/popframe005 segfaults Reviewed-by: aph Contributed-by: nick.gasson at arm.com ! src/hotspot/cpu/aarch64/templateInterpreterGenerator_aarch64.cpp Changeset: 08db5aa02f7b Author: redestad Date: 2019-01-05 20:08 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/08db5aa02f7b 8216189: Remove Klass::compute_is_subtype_of Reviewed-by: hseigel, jiangli ! src/hotspot/share/oops/arrayKlass.cpp ! src/hotspot/share/oops/arrayKlass.hpp ! src/hotspot/share/oops/instanceKlass.cpp ! src/hotspot/share/oops/instanceKlass.hpp ! src/hotspot/share/oops/klass.cpp ! src/hotspot/share/oops/klass.hpp ! src/hotspot/share/oops/objArrayKlass.cpp ! src/hotspot/share/oops/objArrayKlass.hpp ! src/hotspot/share/oops/typeArrayKlass.cpp ! src/hotspot/share/oops/typeArrayKlass.hpp Changeset: af7afdababd3 Author: redestad Date: 2019-01-05 20:11 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/af7afdababd3 8216191: Remove FastSuperclassLimit Reviewed-by: coleenp ! src/hotspot/share/oops/klass.cpp ! src/hotspot/share/oops/klass.hpp ! src/hotspot/share/runtime/globals.hpp Changeset: 3da307766fb1 Author: dholmes Date: 2019-01-06 19:49 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/3da307766fb1 8216188: Remove expired flags in JDK 13 Reviewed-by: kbarrett, ccheung ! src/hotspot/share/runtime/arguments.cpp ! test/hotspot/jtreg/runtime/CommandLine/ObsoleteFlagErrorMessage.java Changeset: c3d6035c11f3 Author: mbaesken Date: 2019-01-04 17:46 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/c3d6035c11f3 8215962: Support ThreadPriorityPolicy mode 1 for non-root users on linux/bsd Reviewed-by: dcubed, dholmes ! src/hotspot/os/bsd/os_bsd.cpp ! src/hotspot/os/linux/os_linux.cpp ! src/hotspot/share/runtime/globals.hpp Changeset: 488773472a63 Author: redestad Date: 2019-01-07 10:21 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/488773472a63 8216197: Remove unused new_hash methods Reviewed-by: kbarrett, dholmes ! src/hotspot/share/oops/metadata.hpp ! src/hotspot/share/oops/oop.cpp ! src/hotspot/share/oops/oop.hpp ! src/hotspot/share/oops/symbol.cpp ! src/hotspot/share/oops/symbol.hpp Changeset: 3d60a1696e19 Author: erikj Date: 2019-01-07 04:56 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/3d60a1696e19 8216267: Fix hotspot-ide-project target on WSL Reviewed-by: erikj Contributed-by: andrewluotechnologies at outlook.com ! make/autoconf/spec.gmk.in ! make/hotspot/ide/CreateVSProject.gmk Changeset: b01fe6b2502c Author: rriggs Date: 2019-01-07 09:29 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/b01fe6b2502c 8216134: (process) ProcessBuilder startPipeline does not hide piped streams Reviewed-by: lancea, bchristi, sgroeger ! src/java.base/windows/classes/java/lang/ProcessImpl.java ! test/jdk/java/lang/ProcessBuilder/PipelineTest.java Changeset: 9339773f2530 Author: redestad Date: 2019-01-07 17:09 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/9339773f2530 8216275: Disable annotation processing lint warnings when building microbenchmarks Reviewed-by: erikj, ecaspole ! make/test/BuildMicrobenchmark.gmk Changeset: 7d1efad039a3 Author: rriggs Date: 2019-01-07 14:15 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/7d1efad039a3 8216205: Java API documentation formatting error in System.getEnv Reviewed-by: lancea, bchristi, alanb ! src/java.base/share/classes/java/lang/System.java Changeset: 03efcf7fc34b Author: kbarrett Date: 2019-01-07 15:20 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/03efcf7fc34b 8215985: ZGC: Simplify reference processing in light of JDK-8175797 Summary: Only self-loop 'next' field for FinalReference deactivation. Reviewed-by: eosterlund, pliden ! src/hotspot/share/gc/z/zReferenceProcessor.cpp ! src/hotspot/share/gc/z/zReferenceProcessor.hpp Changeset: a242fc65ef2e Author: ysuenaga Date: 2019-01-08 10:35 +0900 URL: http://hg.openjdk.java.net/panama/dev/rev/a242fc65ef2e 8216154: C4819 warnings at HotSpot sources on Windows Reviewed-by: kbarrett, tschatzl ! src/hotspot/share/code/codeHeapState.cpp ! src/hotspot/share/compiler/methodMatcher.cpp Changeset: 76f7dbf458fe Author: dzhou Date: 2019-01-07 18:48 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/76f7dbf458fe 8215913: [Test_bug]java/util/Locale/LocaleProvidersRun.java failed on de_DE and ja_JP locale. Reviewed-by: naoto, rgoel, rriggs ! test/jdk/java/util/Locale/LocaleProviders.java ! test/jdk/java/util/Locale/LocaleProvidersRun.java Changeset: f2140eebd91b Author: pmuthuswamy Date: 2019-01-08 11:16 +0530 URL: http://hg.openjdk.java.net/panama/dev/rev/f2140eebd91b 8214738: javadoc should honor styles in doc-files Reviewed-by: jjg ! src/jdk.compiler/share/classes/com/sun/tools/doclint/HtmlTag.java ! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/DocFilesHandlerImpl.java ! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/HtmlDocletWriter.java ! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/markup/Head.java ! test/langtools/jdk/javadoc/doclet/testCopyFiles/TestCopyFiles.java Changeset: c220effa1192 Author: jwilhelm Date: 2019-01-03 02:22 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/c220effa1192 Added tag jdk-12+26 for changeset de9fd809bb47 ! .hgtags Changeset: 36e4c704a88d Author: sdama Date: 2019-01-03 11:21 +0530 URL: http://hg.openjdk.java.net/panama/dev/rev/36e4c704a88d 8208184: IllegalArgumentException while invoking code completion on netbeans IDE Summary: Set Log.useSource and fix the issue in Modules.java when broken module is encountered Reviewed-by: jjg Contributed-by: srinivas.dama at oracle.com, jan.lahoda at oracle.com ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Modules.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/main/JavaCompiler.java ! test/langtools/tools/javac/modules/QueryBeforeEnter.java Changeset: 030429d6baac Author: roland Date: 2018-12-13 17:57 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/030429d6baac 8215044: C2 crash in loopTransform.cpp with assert(cl->trip_count() > 0) failed: peeling a fully unrolled loop Reviewed-by: thartmann, kvn ! src/hotspot/share/opto/loopTransform.cpp + test/hotspot/jtreg/compiler/loopopts/PeelingZeroTripCount.java Changeset: f019e5a7b118 Author: goetz Date: 2018-12-31 14:38 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/f019e5a7b118 8215975: [testbug] Adapt nsk tests to the PPC, S390 and AIX platforms. Summary: Use LIBPATH on AIX, no shared memory connector on any Unix. Reviewed-by: gadams, simonis, sspitsyn ! test/hotspot/jtreg/vmTestbase/nsk/jdi/AttachingConnector/attach/attach002.java ! test/hotspot/jtreg/vmTestbase/nsk/jvmti/Allocate/alloc001/TestDescription.java ! test/hotspot/jtreg/vmTestbase/nsk/jvmti/Allocate/alloc001/alloc001.bash ! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RetransformClasses/retransform003/TestDriver.java ! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetNativeMethodPrefix/SetNativeMethodPrefix002/TestDriver.java ! test/hotspot/jtreg/vmTestbase/nsk/share/jdi/ArgumentHandler.java Changeset: 98580226126d Author: roland Date: 2018-12-14 11:22 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/98580226126d 8215265: C2: range check elimination may allow illegal out of bound access Reviewed-by: thartmann, kvn ! src/hotspot/share/opto/loopTransform.cpp ! src/hotspot/share/opto/loopnode.hpp + test/hotspot/jtreg/compiler/rangechecks/RangeCheckEliminationScaleNotOne.java Changeset: 6a25433b30ed Author: ccheung Date: 2019-01-03 14:33 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/6a25433b30ed 8215947: JVM crash with -XX:+DumpSharedSpaces Summary: disable JIT compilation if -XX:+DumpSharedSpaces is specified by the user Reviewed-by: lfoltan, jiangli ! src/hotspot/share/runtime/arguments.cpp ! test/hotspot/jtreg/runtime/SharedArchiveFile/SharedArchiveFile.java Changeset: 3ab3cb8a8d41 Author: sviswanathan Date: 2019-01-03 14:55 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/3ab3cb8a8d41 8215888: Register to register spill may use AVX 512 move instruction on unsupported platform. Reviewed-by: vlivanov, thartmann ! src/hotspot/cpu/x86/c1_LIRAssembler_x86.cpp ! src/hotspot/cpu/x86/x86.ad Changeset: 6bc1f8d41f4f Author: erikj Date: 2019-01-04 11:00 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/6bc1f8d41f4f 8216021: RunTest.gmk might set concurrency level to 1 on Windows Reviewed-by: ctornqvi, tbell ! make/RunTestsPrebuilt.gmk Changeset: a0eb3da69586 Author: mseledtsov Date: 2019-01-04 15:17 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/a0eb3da69586 8215583: Exclude runtime/handshake/HandshakeWalkSuspendExitTest.java Summary: Added test to problem list Reviewed-by: iignatyev ! test/hotspot/jtreg/ProblemList.txt Changeset: 8970b75f0d37 Author: erikj Date: 2019-01-07 10:00 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/8970b75f0d37 8215400: Warn on usage of trampolines with gcc Reviewed-by: tbell, kbarrett ! make/autoconf/flags-cflags.m4 Changeset: ea921dca7f33 Author: jwilhelm Date: 2019-01-07 13:04 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/ea921dca7f33 8216266: ProblemList PeelingZeroTripCount.java Reviewed-by: thartmann, roland ! test/hotspot/jtreg/ProblemList.txt Changeset: 6af596144294 Author: eosterlund Date: 2019-01-07 12:22 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/6af596144294 8215773: applications/kitchensink/Kitchensink.java crash with "assert(ZAddress::is_marked(addr)) failed: Should be marked" Reviewed-by: coleenp, pliden ! src/hotspot/share/jfr/recorder/checkpoint/types/jfrTypeSetUtils.cpp Changeset: 2692d1bfe83a Author: jwilhelm Date: 2019-01-08 07:29 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/2692d1bfe83a Merge ! .hgtags ! make/autoconf/flags-cflags.m4 ! src/hotspot/share/runtime/arguments.cpp ! test/hotspot/jtreg/ProblemList.txt Changeset: 818b7bf2af49 Author: aivanov Date: 2018-12-11 14:11 +0000 URL: http://hg.openjdk.java.net/panama/dev/rev/818b7bf2af49 8215123: Crash in runtime image built with jlink --compress=2 Reviewed-by: ihse, alanb ! src/java.base/share/native/libjimage/imageDecompressor.cpp Changeset: 760293737af0 Author: psadhukhan Date: 2018-12-12 15:07 +0530 URL: http://hg.openjdk.java.net/panama/dev/rev/760293737af0 Merge - make/scripts/pandoc-manpage-filter.js - make/scripts/pandoc-manpage-filter.sh.template - src/jdk.internal.le/share/classes/jdk/internal/jline/DefaultTerminal2.java - src/jdk.internal.le/share/classes/jdk/internal/jline/NoInterruptUnixTerminal.java - src/jdk.internal.le/share/classes/jdk/internal/jline/OSvTerminal.java - src/jdk.internal.le/share/classes/jdk/internal/jline/Terminal.java - src/jdk.internal.le/share/classes/jdk/internal/jline/Terminal2.java - src/jdk.internal.le/share/classes/jdk/internal/jline/TerminalFactory.java - src/jdk.internal.le/share/classes/jdk/internal/jline/TerminalSupport.java - src/jdk.internal.le/share/classes/jdk/internal/jline/UnixTerminal.java - src/jdk.internal.le/share/classes/jdk/internal/jline/UnsupportedTerminal.java - src/jdk.internal.le/share/classes/jdk/internal/jline/WindowsTerminal.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/ConsoleKeys.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/ConsoleReader.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/CursorBuffer.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/KeyMap.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/KillRing.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/Operation.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/UserInterruptException.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/WCWidth.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/completer/AggregateCompleter.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/completer/AnsiStringsCompleter.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/completer/ArgumentCompleter.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/completer/CandidateListCompletionHandler.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/completer/CandidateListCompletionHandler.properties - src/jdk.internal.le/share/classes/jdk/internal/jline/console/completer/Completer.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/completer/CompletionHandler.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/completer/EnumCompleter.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/completer/FileNameCompleter.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/completer/NullCompleter.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/completer/StringsCompleter.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/completer/package-info.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/history/FileHistory.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/history/History.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/history/MemoryHistory.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/history/PersistentHistory.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/history/package-info.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/internal/ConsoleReaderInputStream.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/internal/ConsoleRunner.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/package-info.java - src/jdk.internal.le/share/classes/jdk/internal/jline/extra/AnsiInterpretingOutputStream.java - src/jdk.internal.le/share/classes/jdk/internal/jline/extra/EditingHistory.java - src/jdk.internal.le/share/classes/jdk/internal/jline/internal/Ansi.java - src/jdk.internal.le/share/classes/jdk/internal/jline/internal/Configuration.java - src/jdk.internal.le/share/classes/jdk/internal/jline/internal/Curses.java - src/jdk.internal.le/share/classes/jdk/internal/jline/internal/InfoCmp.java - src/jdk.internal.le/share/classes/jdk/internal/jline/internal/InputStreamReader.java - src/jdk.internal.le/share/classes/jdk/internal/jline/internal/Log.java - src/jdk.internal.le/share/classes/jdk/internal/jline/internal/NonBlockingInputStream.java - src/jdk.internal.le/share/classes/jdk/internal/jline/internal/Nullable.java - src/jdk.internal.le/share/classes/jdk/internal/jline/internal/Preconditions.java - src/jdk.internal.le/share/classes/jdk/internal/jline/internal/ShutdownHooks.java - src/jdk.internal.le/share/classes/jdk/internal/jline/internal/TerminalLineSettings.java - src/jdk.internal.le/share/classes/jdk/internal/jline/internal/TestAccessible.java - src/jdk.internal.le/share/classes/jdk/internal/jline/internal/Urls.java - src/jdk.internal.le/share/classes/jdk/internal/jline/internal/package-info.java - src/jdk.internal.le/share/classes/jdk/internal/jline/package-info.java - src/jdk.internal.le/windows/native/lible/WindowsTerminal.cpp - src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/replacements/UnsafeAccess.java - src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/resources/jquery/images/ui-bg_flat_0_aaaaaa_40x100.png - src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/resources/jquery/images/ui-bg_flat_75_ffffff_40x100.png - src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/resources/jquery/images/ui-bg_glass_65_ffffff_1x400.png - test/jdk/jdk/internal/jline/console/StripAnsiTest.java - test/jdk/jdk/internal/jline/extra/AnsiInterpretingOutputStreamTest.java - test/jdk/jdk/internal/jline/extra/HistoryTest.java Changeset: 2a39d5fc7e58 Author: kaddepalli Date: 2018-12-14 11:00 +0530 URL: http://hg.openjdk.java.net/panama/dev/rev/2a39d5fc7e58 8196681: Java Access Bridge logging and debug flags dynamically controlled Reviewed-by: serb, sveerabhadra ! src/jdk.accessibility/windows/classes/com/sun/java/accessibility/internal/AccessBridge.java ! src/jdk.accessibility/windows/native/common/AccessBridgeDebug.cpp ! src/jdk.accessibility/windows/native/common/AccessBridgeDebug.h ! src/jdk.accessibility/windows/native/libjavaaccessbridge/AccessBridgeATInstance.cpp ! src/jdk.accessibility/windows/native/libjavaaccessbridge/AccessBridgeJavaEntryPoints.cpp ! src/jdk.accessibility/windows/native/libjavaaccessbridge/JavaAccessBridge.cpp ! src/jdk.accessibility/windows/native/libwindowsaccessbridge/AccessBridgeEventHandler.cpp ! src/jdk.accessibility/windows/native/libwindowsaccessbridge/AccessBridgeJavaVMInstance.cpp ! src/jdk.accessibility/windows/native/libwindowsaccessbridge/AccessBridgeMessageQueue.cpp ! src/jdk.accessibility/windows/native/libwindowsaccessbridge/WinAccessBridge.cpp Changeset: 0434a6393b65 Author: serb Date: 2018-12-15 10:35 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/0434a6393b65 8214461: Some unused classes may be removed Reviewed-by: kaddepalli, prr - src/java.desktop/share/classes/sun/awt/Graphics2Delegate.java ! src/java.desktop/share/classes/sun/awt/SunGraphicsCallback.java - src/java.desktop/share/classes/sun/awt/TracedEventQueue.java - src/java.desktop/share/classes/sun/awt/image/BadDepthException.java ! src/java.desktop/windows/classes/sun/awt/windows/WCanvasPeer.java Changeset: 6cf5fddfb93d Author: kaddepalli Date: 2018-12-17 14:19 +0530 URL: http://hg.openjdk.java.net/panama/dev/rev/6cf5fddfb93d 6714324: Removing a component from a JTabbedPane does not clear its accessibleParent Reviewed-by: serb, sveerabhadra ! src/java.desktop/share/classes/javax/swing/JTabbedPane.java + test/jdk/javax/accessibility/6714324/TabbedPaneMemLeak.java Changeset: eeac4e2558d7 Author: serb Date: 2018-12-19 14:11 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/eeac4e2558d7 8215200: IllegalArgumentException in sun.lwawt.macosx.CPlatformWindow Reviewed-by: dmarkov, kaddepalli ! src/java.desktop/macosx/classes/sun/lwawt/macosx/CPlatformWindow.java + test/jdk/java/awt/Modal/MultipleDialogs/MixOfModalAndNonModalDialogs.java Changeset: 9eee0b148002 Author: aivanov Date: 2018-12-20 12:44 +0000 URL: http://hg.openjdk.java.net/panama/dev/rev/9eee0b148002 8214122: JDWP is broken on 32 bit Windows: transport library missing onLoad entry Reviewed-by: ihse, dcubed ! src/jdk.jdwp.agent/share/native/libjdwp/transport.c Changeset: 450c1abf9c62 Author: kaddepalli Date: 2019-01-04 07:56 +0530 URL: http://hg.openjdk.java.net/panama/dev/rev/450c1abf9c62 8215910: Typo in AWT InvocationEvent Method Documentation. Reviewed-by: serb, aivanov ! src/java.desktop/share/classes/java/awt/event/InvocationEvent.java Changeset: 5ce51ae5c0e5 Author: psadhukhan Date: 2019-01-04 11:40 +0530 URL: http://hg.openjdk.java.net/panama/dev/rev/5ce51ae5c0e5 8215909: Typo in Swing ProcessMouseEvent method documentation Reviewed-by: serb, aivanov ! src/java.desktop/share/classes/javax/swing/MenuElement.java Changeset: 64e7a73195c1 Author: serb Date: 2019-01-05 10:13 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/64e7a73195c1 8215756: Memory leaks in the AWT on macOS Reviewed-by: dmarkov ! src/java.desktop/macosx/native/libawt_lwawt/awt/AWTView.m ! src/java.desktop/macosx/native/libawt_lwawt/awt/AWTWindow.m ! src/java.desktop/macosx/native/libawt_lwawt/awt/CGraphicsEnv.m Changeset: 6c8ce24d0fbf Author: itakiguchi Date: 2019-01-06 19:28 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/6c8ce24d0fbf 8211267: StackOverflowError happened by TextField.setFont(...) Reviewed-by: serb, prr ! src/java.desktop/unix/classes/sun/awt/X11/XTextFieldPeer.java + test/jdk/java/awt/TextField/FontChangeTest/FontChangeTest.java Changeset: 7f1d89aac92a Author: psadhukhan Date: 2019-01-07 11:02 +0530 URL: http://hg.openjdk.java.net/panama/dev/rev/7f1d89aac92a Merge - src/hotspot/share/jfr/recorder/repository/jfrChunkSizeNotifier.cpp - src/hotspot/share/jfr/recorder/repository/jfrChunkSizeNotifier.hpp - src/hotspot/share/runtime/arguments_ext.hpp - src/hotspot/share/services/diagnosticCommand_ext.hpp - src/java.base/share/classes/com/sun/crypto/provider/HmacPKCS12PBESHA1.java - src/java.logging/share/classes/java/util/logging/package.html - src/java.prefs/share/classes/java/util/prefs/package.html - src/java.rmi/share/classes/java/rmi/activation/package.html - src/java.rmi/share/classes/java/rmi/dgc/package.html - src/java.rmi/share/classes/java/rmi/package.html - src/java.rmi/share/classes/java/rmi/registry/package.html - src/java.rmi/share/classes/java/rmi/server/package.html - src/java.rmi/share/classes/javax/rmi/ssl/package.html - src/java.security.jgss/share/classes/org/ietf/jgss/package.html - src/java.smartcardio/share/classes/javax/smartcardio/package.html - src/java.sql.rowset/share/classes/com/sun/rowset/package.html - src/java.sql.rowset/share/classes/com/sun/rowset/providers/package.html - src/java.sql.rowset/share/classes/javax/sql/rowset/serial/package.html - src/java.sql/share/classes/java/sql/package.html - src/java.sql/share/classes/javax/sql/package.html - src/jdk.crypto.mscapi/windows/classes/sun/security/mscapi/Key.java - src/jdk.crypto.mscapi/windows/classes/sun/security/mscapi/KeyStore.java - src/jdk.crypto.mscapi/windows/classes/sun/security/mscapi/RSACipher.java - src/jdk.crypto.mscapi/windows/classes/sun/security/mscapi/RSAKeyPair.java - src/jdk.crypto.mscapi/windows/classes/sun/security/mscapi/RSAKeyPairGenerator.java - src/jdk.crypto.mscapi/windows/classes/sun/security/mscapi/RSAPrivateKey.java - src/jdk.crypto.mscapi/windows/classes/sun/security/mscapi/RSAPublicKey.java - src/jdk.crypto.mscapi/windows/classes/sun/security/mscapi/RSASignature.java - src/utils/LogCompilation/src/test/resources/hotspot_pid23756.log - src/utils/LogCompilation/src/test/resources/hotspot_pid25109.log - src/utils/LogCompilation/src/test/resources/no_tiered_short.log - src/utils/LogCompilation/src/test/resources/tiered_short.log - test/hotspot/jtreg/serviceability/jvmti/HeapMonitor/MyPackage/HeapMonitorEventsForTwoThreadsTest.java - test/jdk/java/net/MulticastSocket/PromiscuousIPv6.java - test/jdk/java/nio/channels/DatagramChannel/PromiscuousIPv6.java - test/langtools/jdk/javadoc/doclet/lib/JavadocTester.java Changeset: 8cc938aa8f74 Author: psadhukhan Date: 2019-01-08 13:40 +0530 URL: http://hg.openjdk.java.net/panama/dev/rev/8cc938aa8f74 Merge Changeset: a257992e1e4d Author: redestad Date: 2019-01-08 10:54 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/a257992e1e4d 8216285: Enable inlining of CollectedHeap::obj-/array-/class_allocate Reviewed-by: ehelin ! src/hotspot/share/gc/shared/collectedHeap.cpp ! src/hotspot/share/gc/shared/collectedHeap.inline.hpp Changeset: 0042eb88035b Author: mdoerr Date: 2019-01-08 11:02 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/0042eb88035b 8216269: [s390] Debug build broken because CodeBlob has not been declared Reviewed-by: shade, coleenp ! src/hotspot/share/code/relocInfo.hpp Changeset: 184c51e48260 Author: redestad Date: 2019-01-08 11:23 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/184c51e48260 8216262: Remove develop flag DelayCompilationDuringStartup Reviewed-by: kvn, thartmann ! src/hotspot/share/interpreter/abstractInterpreter.cpp ! src/hotspot/share/interpreter/invocationCounter.cpp ! src/hotspot/share/interpreter/invocationCounter.hpp ! src/hotspot/share/jvmci/jvmciCompiler.cpp ! src/hotspot/share/prims/jni.cpp ! src/hotspot/share/runtime/compilationPolicy.cpp ! src/hotspot/share/runtime/compilationPolicy.hpp ! src/hotspot/share/runtime/globals.hpp ! src/hotspot/share/runtime/tieredThresholdPolicy.cpp Changeset: c4a64760b1b0 Author: redestad Date: 2019-01-08 16:09 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/c4a64760b1b0 8216359: Remove develop flags TraceCompilationPolicy and TimeCompilationPolicy Reviewed-by: neliasso, thartmann ! src/hotspot/share/runtime/compilationPolicy.cpp ! src/hotspot/share/runtime/compilationPolicy.hpp ! src/hotspot/share/runtime/globals.hpp ! src/hotspot/share/runtime/java.cpp Changeset: 9fff411880fb Author: ecaspole Date: 2019-01-08 10:28 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/9fff411880fb 8076988: reevaluate trivial method policy Summary: Removed some checks to qualify as trivial Reviewed-by: shade, thartmann, dlong ! src/hotspot/share/runtime/tieredThresholdPolicy.cpp Changeset: 48d09a9f4d2b Author: ecaspole Date: 2019-01-08 10:29 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/48d09a9f4d2b Merge Changeset: c92f4465fff1 Author: jcbeyler Date: 2019-01-08 09:55 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/c92f4465fff1 8215495: Set isCopy to JNI_FALSE if len == 0 Summary: Set isCopy in a corner case Reviewed-by: dholmes, phh, minqi ! src/hotspot/share/prims/jni.cpp Changeset: 8663bd437bb8 Author: zgu Date: 2019-01-07 09:17 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/8663bd437bb8 8216199: Local variable arg defined but never used in BCEscapeAnalyzer::compute_escape_for_intrinsic() Summary: Removed unused local variable Reviewed-by: thartmann ! src/hotspot/share/ci/bcEscapeAnalyzer.cpp Changeset: 11464bf82dce Author: zgu Date: 2019-01-07 10:41 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/11464bf82dce 8216200: BCEscapeAnalyzer::ArgumentMap::set_intersect() is incorrect Summary: Removed incorrect/unused method Reviewed-by: thartmann ! src/hotspot/share/ci/bcEscapeAnalyzer.cpp Changeset: 3cdf4d5148a8 Author: sgehwolf Date: 2019-01-08 14:15 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/3cdf4d5148a8 8216366: Add rationale to PER_CPU_SHARES define Reviewed-by: bobv, adinn ! src/hotspot/os/linux/osContainer_linux.cpp Changeset: 28ec06beb091 Author: darcy Date: 2019-01-08 13:04 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/28ec06beb091 8216335: Minor cleanups to javax.annotation.processing and javax.lang.model javadoc Reviewed-by: vromero ! src/java.compiler/share/classes/javax/annotation/processing/AbstractProcessor.java ! src/java.compiler/share/classes/javax/annotation/processing/Filer.java ! src/java.compiler/share/classes/javax/annotation/processing/Processor.java ! src/java.compiler/share/classes/javax/lang/model/element/ModuleElement.java ! src/java.compiler/share/classes/javax/lang/model/util/Elements.java Changeset: 02e648ae46c3 Author: redestad Date: 2019-01-09 01:06 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/02e648ae46c3 8215995: Add specialized toArray methods to immutable collections Reviewed-by: martin, smarks ! src/java.base/share/classes/java/util/ImmutableCollections.java + test/micro/org/openjdk/bench/java/util/ImmutableColls.java Changeset: 7d8676b2487f Author: neliasso Date: 2019-01-09 10:19 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/7d8676b2487f 8216372: ZGC: Put C2 load barrier stub routines in separate codeblobs Reviewed-by: pliden, eosterlund ! src/hotspot/cpu/x86/gc/z/zBarrierSetAssembler_x86.cpp Changeset: 9a3750a63823 Author: hseigel Date: 2019-01-09 08:07 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/9a3750a63823 8215901: [TESTBUG] TestCheckedEnsureLocalCapacity.java fails intermittently Summary: Change pattern match to not require that the matching string start at the beginning of a line. Reviewed-by: dcubed, dholmes ! test/hotspot/jtreg/runtime/jni/checked/TestCheckedEnsureLocalCapacity.java Changeset: 8f79bae1a535 Author: hseigel Date: 2019-01-09 09:01 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/8f79bae1a535 8214442: Improve stack walk API by adding handle marks Summary: Add the missing handle marks. Reviewed-by: zgu, mchung ! src/hotspot/share/classfile/javaClasses.cpp ! src/hotspot/share/prims/stackwalk.cpp Changeset: b58517f0ea0e Author: igerasim Date: 2019-01-09 06:18 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/b58517f0ea0e 8216413: Long.parseLong() is specified to throw unless string contains parsable {@code int}; should be {@code long} Reviewed-by: clanger ! src/java.base/share/classes/java/lang/Long.java Changeset: b1c6d4d7f801 Author: shade Date: 2019-01-09 15:53 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/b1c6d4d7f801 8215724: Epsilon: ArrayStoreExceptionTest.java fails; missing arraycopy check Reviewed-by: eosterlund, lkorinth ! src/hotspot/share/gc/shared/barrierSet.hpp + src/hotspot/share/gc/shared/barrierSet.inline.hpp ! src/hotspot/share/oops/access.inline.hpp + test/hotspot/jtreg/gc/epsilon/TestArraycopyCheckcast.java Changeset: 40187283e6eb Author: ecaspole Date: 2019-01-09 12:02 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/40187283e6eb 8216375: Revert JDK-8145579 after JDK-8076988 is resolved Summary: Remove obsolete code Reviewed-by: thartmann, dlong ! src/hotspot/share/c1/c1_GraphBuilder.cpp Changeset: ae803dd58dbe Author: gadams Date: 2019-01-09 12:09 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/ae803dd58dbe 8213001: vmTestbase/nsk/jvmti/ThreadStart/threadstart002/TestDescription.java debug agent times out Reviewed-by: dcubed, jcbeyler ! test/hotspot/jtreg/vmTestbase/nsk/jvmti/ThreadStart/threadstart002/threadstart002.cpp Changeset: 17539619efe6 Author: redestad Date: 2019-01-09 17:40 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/17539619efe6 8216423: Remove FillDelaySlots Reviewed-by: thartmann ! src/hotspot/share/runtime/globals.hpp Changeset: eda4c6456efb Author: igerasim Date: 2019-01-09 10:59 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/eda4c6456efb 8210788: Javadoc for Thread.join(long, int) should specify that it waits forever when both arguments are zero Reviewed-by: martin, rriggs ! src/java.base/share/classes/java/lang/Thread.java Changeset: bccff579c2ff Author: shade Date: 2019-01-09 20:28 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/bccff579c2ff 8216302: StackTraceElement::fill_in can use cached Class.name Reviewed-by: coleenp, dholmes, mchung ! make/hotspot/symbols/symbols-unix ! src/hotspot/share/classfile/javaClasses.cpp ! src/hotspot/share/classfile/javaClasses.hpp ! src/hotspot/share/include/jvm.h ! src/hotspot/share/prims/jvm.cpp ! src/java.base/share/classes/java/lang/Class.java ! src/java.base/share/native/libjava/Class.c + test/hotspot/jtreg/runtime/StackTrace/StackTraceClassCache.java From maurizio.cimadamore at oracle.com Wed Jan 9 21:03:39 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 09 Jan 2019 21:03:39 +0000 Subject: hg: panama/dev: Automatic merge with default Message-ID: <201901092103.x09L3dLn003255@aojmv0008.oracle.com> Changeset: ad262cd505b4 Author: mcimadamore Date: 2019-01-09 22:08 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/ad262cd505b4 Automatic merge with default ! src/hotspot/share/c1/c1_GraphBuilder.cpp ! src/hotspot/share/classfile/javaClasses.cpp ! src/hotspot/share/classfile/javaClasses.hpp - src/java.desktop/share/classes/sun/awt/Graphics2Delegate.java - src/java.desktop/share/classes/sun/awt/TracedEventQueue.java - src/java.desktop/share/classes/sun/awt/image/BadDepthException.java - src/utils/LogCompilation/src/test/resources/hotspot_pid23756.log - src/utils/LogCompilation/src/test/resources/hotspot_pid25109.log - src/utils/LogCompilation/src/test/resources/no_tiered_short.log - src/utils/LogCompilation/src/test/resources/tiered_short.log From maurizio.cimadamore at oracle.com Wed Jan 9 21:04:01 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 09 Jan 2019 21:04:01 +0000 Subject: hg: panama/dev: Automatic merge with default Message-ID: <201901092104.x09L41x7003569@aojmv0008.oracle.com> Changeset: d3b456d84b4e Author: mcimadamore Date: 2019-01-09 22:09 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/d3b456d84b4e Automatic merge with default ! src/hotspot/cpu/x86/x86.ad ! src/hotspot/share/code/location.hpp ! src/hotspot/share/runtime/globals.hpp ! src/hotspot/share/runtime/stackValue.hpp ! src/hotspot/share/utilities/globalDefinitions.hpp - src/java.desktop/share/classes/sun/awt/Graphics2Delegate.java - src/java.desktop/share/classes/sun/awt/TracedEventQueue.java - src/java.desktop/share/classes/sun/awt/image/BadDepthException.java - src/utils/LogCompilation/src/test/resources/hotspot_pid23756.log - src/utils/LogCompilation/src/test/resources/hotspot_pid25109.log - src/utils/LogCompilation/src/test/resources/no_tiered_short.log - src/utils/LogCompilation/src/test/resources/tiered_short.log From maurizio.cimadamore at oracle.com Wed Jan 9 21:04:22 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 09 Jan 2019 21:04:22 +0000 Subject: hg: panama/dev: Automatic merge with default Message-ID: <201901092104.x09L4Mmp003998@aojmv0008.oracle.com> Changeset: 21f2bda8f11f Author: mcimadamore Date: 2019-01-09 22:09 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/21f2bda8f11f Automatic merge with default ! make/autoconf/spec.gmk.in ! src/java.base/share/classes/java/lang/System.java - src/java.desktop/share/classes/sun/awt/Graphics2Delegate.java - src/java.desktop/share/classes/sun/awt/TracedEventQueue.java - src/java.desktop/share/classes/sun/awt/image/BadDepthException.java - src/utils/LogCompilation/src/test/resources/hotspot_pid23756.log - src/utils/LogCompilation/src/test/resources/hotspot_pid25109.log - src/utils/LogCompilation/src/test/resources/no_tiered_short.log - src/utils/LogCompilation/src/test/resources/tiered_short.log From maurizio.cimadamore at oracle.com Wed Jan 9 21:04:42 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 09 Jan 2019 21:04:42 +0000 Subject: hg: panama/dev: Automatic merge with default Message-ID: <201901092104.x09L4hcr004341@aojmv0008.oracle.com> Changeset: 66a047c6e254 Author: mcimadamore Date: 2019-01-09 22:09 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/66a047c6e254 Automatic merge with default - src/java.desktop/share/classes/sun/awt/Graphics2Delegate.java - src/java.desktop/share/classes/sun/awt/TracedEventQueue.java - src/java.desktop/share/classes/sun/awt/image/BadDepthException.java - src/utils/LogCompilation/src/test/resources/hotspot_pid23756.log - src/utils/LogCompilation/src/test/resources/hotspot_pid25109.log - src/utils/LogCompilation/src/test/resources/no_tiered_short.log - src/utils/LogCompilation/src/test/resources/tiered_short.log From maurizio.cimadamore at oracle.com Wed Jan 9 21:06:22 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 09 Jan 2019 21:06:22 +0000 Subject: hg: panama/dev: Automatic merge with linkToNative Message-ID: <201901092106.x09L6MZA006081@aojmv0008.oracle.com> Changeset: 1f70e3fafb3d Author: mcimadamore Date: 2019-01-09 22:11 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/1f70e3fafb3d Automatic merge with linkToNative From maurizio.cimadamore at oracle.com Wed Jan 9 21:05:03 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 09 Jan 2019 21:05:03 +0000 Subject: hg: panama/dev: Automatic merge with default Message-ID: <201901092105.x09L53hT004666@aojmv0008.oracle.com> Changeset: 389fe3b23e57 Author: mcimadamore Date: 2019-01-09 22:10 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/389fe3b23e57 Automatic merge with default ! src/hotspot/cpu/x86/x86.ad ! src/hotspot/share/c1/c1_GraphBuilder.cpp ! src/hotspot/share/classfile/javaClasses.cpp ! src/hotspot/share/classfile/javaClasses.hpp ! src/hotspot/share/code/location.hpp ! src/hotspot/share/runtime/globals.hpp ! src/hotspot/share/runtime/stackValue.hpp - src/java.desktop/share/classes/sun/awt/Graphics2Delegate.java - src/java.desktop/share/classes/sun/awt/TracedEventQueue.java - src/java.desktop/share/classes/sun/awt/image/BadDepthException.java - src/utils/LogCompilation/src/test/resources/hotspot_pid23756.log - src/utils/LogCompilation/src/test/resources/hotspot_pid25109.log - src/utils/LogCompilation/src/test/resources/no_tiered_short.log - src/utils/LogCompilation/src/test/resources/tiered_short.log From maurizio.cimadamore at oracle.com Wed Jan 9 21:05:24 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 09 Jan 2019 21:05:24 +0000 Subject: hg: panama/dev: Automatic merge with foreign Message-ID: <201901092105.x09L5Ov9004972@aojmv0008.oracle.com> Changeset: 6cf1d3a0f7f1 Author: mcimadamore Date: 2019-01-09 22:10 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/6cf1d3a0f7f1 Automatic merge with foreign ! src/hotspot/cpu/x86/x86.ad ! src/hotspot/share/code/location.hpp ! src/hotspot/share/runtime/globals.hpp ! src/hotspot/share/runtime/stackValue.hpp ! src/hotspot/share/utilities/globalDefinitions.hpp - src/java.desktop/share/classes/sun/awt/Graphics2Delegate.java - src/java.desktop/share/classes/sun/awt/TracedEventQueue.java - src/java.desktop/share/classes/sun/awt/image/BadDepthException.java - src/utils/LogCompilation/src/test/resources/hotspot_pid23756.log - src/utils/LogCompilation/src/test/resources/hotspot_pid25109.log - src/utils/LogCompilation/src/test/resources/no_tiered_short.log - src/utils/LogCompilation/src/test/resources/tiered_short.log From maurizio.cimadamore at oracle.com Wed Jan 9 21:05:42 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 09 Jan 2019 21:05:42 +0000 Subject: hg: panama/dev: Automatic merge with vectorIntrinsics Message-ID: <201901092105.x09L5gWo005301@aojmv0008.oracle.com> Changeset: 298feba06fbe Author: mcimadamore Date: 2019-01-09 22:10 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/298feba06fbe Automatic merge with vectorIntrinsics From maurizio.cimadamore at oracle.com Wed Jan 9 21:06:03 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 09 Jan 2019 21:06:03 +0000 Subject: hg: panama/dev: Automatic merge with foreign Message-ID: <201901092106.x09L63PB005645@aojmv0008.oracle.com> Changeset: e2a301f3d481 Author: mcimadamore Date: 2019-01-09 22:11 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/e2a301f3d481 Automatic merge with foreign ! src/hotspot/share/c1/c1_GraphBuilder.cpp ! src/hotspot/share/classfile/javaClasses.cpp ! src/hotspot/share/classfile/javaClasses.hpp ! src/hotspot/share/runtime/frame.hpp ! src/hotspot/share/runtime/thread.hpp - src/java.desktop/share/classes/sun/awt/Graphics2Delegate.java - src/java.desktop/share/classes/sun/awt/TracedEventQueue.java - src/java.desktop/share/classes/sun/awt/image/BadDepthException.java - src/utils/LogCompilation/src/test/resources/hotspot_pid23756.log - src/utils/LogCompilation/src/test/resources/hotspot_pid25109.log - src/utils/LogCompilation/src/test/resources/no_tiered_short.log - src/utils/LogCompilation/src/test/resources/tiered_short.log From jbvernee at xs4all.nl Wed Jan 9 21:17:06 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Wed, 09 Jan 2019 22:17:06 +0100 Subject: [foreign] RFR: Windows support Message-ID: <8f136c5aa0343d7d41a294ce84457a4e@xs4all.nl> Hi, I have fixed the last few failing tests on Windows so this is an RFR for adding Windows support to Panama FFI. There's a lot of changes so I've split it into 4 patches. We should probably focus on one of these one at a time, but I wanted to include all the changes needed to pass the tests at least. It's probably best to start with [2], since that implements the core functionality. Default lib fixes [1] (needed for the default library to work on Windows) Hotspot, binder tweaks [2] (Changes to the stub generation and shared binder code. Includes the Windows implementation of SystemABI) jextract tweaks [3] (minor jextract portability changes) test tweaks [4] (changes needed to run, and pass the tests on Windows) There are some tests that are disabled: - EmptyStructTest; Empty structs are a GCC extension. - BadBitfieldTest; this is testing for a problem the MSVC ABI doesn't seem to have. - StdioTest; relies on standard system header path, which on Windows is more tricky to find. - ComplexTest; Windows complex types work very differently in C source, so the current test doesn't compile. - LongDoubleTest; long double is only 64 bits on Windows. This might need a replacement test still, but it should work the same as `double` on Windows. - Getpid; No getpid on Windows in the C standard. - UnixSystem; Windows is not unix. (Some tests specifically for Windows could be added, but I'm leaving that for a different RFR) Changes for [1]: - replace default library handle with a findEntryInProcess since Windows doesn't have an RTLD_DEFAULT equivalent. Changes for [2]: - changed upcall/downcall stub generation to follow the MS ABI on windows (using compiler switches) - added SKIPs to shuffle recipe generation when e.g. the first float is passed in the second float register (which can be the case on windows). - moved boxValue and unboxValue into SystemABI, since they turned out to be ABI specific, and made tweaks to the windows version for handeling structs being passed by value. - moved upcall stub natives into shared ABI package. - Let each ABI have it's own VarargsInvoker implementation, since we need to propagate more info about which arguments are variadic on windows. - CallingSequence::storageOffset is no longer relying on Storage::getStorageIndex, since that is not the same as what's actually needed; being the index of the binding. (the 2 can differ on windows, but not on sysv). - Added Windows ABI implementation classes based on the SysV ones. - Let UniversalUpcallHandler write the in-memory-return pointer to RAX since that is required on Windows (and I think this change is portable). Changes for [3]: - instead of hardcoding the size of `long` `unsigned long` and `long double`, delegate to libclang to tell us the size, since these types have different sizes on the 2 ABIs. (This is pretty ad-hoc, so probably needs to be improved. But, I have little experience with the jextract code, so I'm open to suggestion of how to better fix this) Changes for [4]: - added dll exports for native symbols using compiler switches. - added test for small (fits-in-register) struct by-value passing, since windows has special handling for those. - removed depencies on POSIX functions from some tests where it wasn't really needed, so they could run on windows. - propagate clang library name to TestJextractFFI, since on Windows we need to load "libclang". - let Runner test use windows specific files to compare with for bitfields.h and simple.h - use `long long` instead of `long` in some cases since `long long` is 64 bits on SysV and MSx64 Thanks, Jorn [1] : http://cr.openjdk.java.net/~jvernee/panama/webrevs/8211060/webrev.01/ [2] : http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows/webrev.03/ [3] : http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_jextract/webrev.00/ [4] : http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_tests/webrev.02/ From Yang.Zhang at arm.com Thu Jan 10 02:43:44 2019 From: Yang.Zhang at arm.com (Yang Zhang (Arm Technology China)) Date: Thu, 10 Jan 2019 02:43:44 +0000 Subject: [vector api] RFR: Fix the issue about float/double reduce-max/min operations with NaN as an input In-Reply-To: References: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A317E7@FMSMSX126.amr.corp.intel.com> Message-ID: Hi Vladimir, Sandhya Ping again. Could you please help to review this patch? Regards, Yang -----Original Message----- From: Yang Zhang Sent: Thursday, January 10, 2019 10:32 AM To: Yang Zhang (Arm Technology China) Subject: Fwd: [vector api] RFR: Fix the issue about float/double reduce-max/min operations with NaN as an input ---------- Forwarded message --------- From: Yang Zhang (Arm Technology China) Date: Fri, 14 Dec 2018 at 16:31 Subject: RE: [vector api] RFR: Fix the issue about float/double reduce-max/min operations with NaN as an input To: Viswanathan, Sandhya , panama-dev at openjdk.java.net Cc: nd Hi Sandhya Happy holidays. Let us keep on this work after holidays. Regards Yang -----Original Message----- From: Viswanathan, Sandhya Sent: Friday, December 14, 2018 9:43 AM To: Yang Zhang (Arm Technology China) ; panama-dev at openjdk.java.net Cc: nd Subject: RE: [vector api] RFR: Fix the issue about float/double reduce-max/min operations with NaN as an input Hi Yang, The current x86 intrinsics are based on the current definition as in Javadoc for Vector API float(double) min: public abstract FloatVector min(float s) Returns the minimum of this vector and the broadcast of an input scalar. This is a vector binary operation where the operation (a, b) -> a < b ? a : b is applied to lane elements. Parameters: s - the input scalar Returns: the minimum of this vector and the broadcast of an input scalar We are looking into making it closer to Math.min at least in the NaN related behavior. Let us wait till January middle or so to converge on this, it being too close to Christmas now and many are on vacation. Hope that is ok. Happy holidays. It is wonderful to see you and others from Arm technology onboard and contributing to Panama Vector API. Let us keep this collaboration going. Best Regards, Sandhya -----Original Message----- From: panama-dev [mailto:panama-dev-bounces at openjdk.java.net] On Behalf Of Yang Zhang (Arm Technology China) Sent: Wednesday, December 12, 2018 12:37 AM To: panama-dev at openjdk.java.net Cc: nd Subject: [vector api] RFR: Fix the issue about float/double reduce-max/min operations with NaN as an input Hi When I implement reduce-max/min, I found there is an issue about float/double reduce-max/min operations with NaN as an input. In java doc, if either value is NaN, then the result of max/min(a, b) is NaN. But the result of "((a > b) ? a : b)" is affected by sequence of (a, b). If a is NaN, the result is b. If b is NaN, the result is b (NaN). I have a patch which could fix this issue. Could you please help to review it? http://cr.openjdk.java.net/~yzhang/vectorapi.nan/webrev.00/ Ps. With this patch, there are 5 failures on x86 platform. jdk/incubator/vector/Float256VectorTests.java jdk/incubator/vector/Float64VectorTests.java jdk/incubator/vector/FloatMaxVectorTests.java jdk/incubator/vector/Double128VectorTests.java jdk/incubator/vector/DoubleMaxVectorTests.java Webrev: Regards, Yang From maurizio.cimadamore at oracle.com Thu Jan 10 12:44:17 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 10 Jan 2019 12:44:17 +0000 Subject: [foreign] RFR: Windows support In-Reply-To: <8f136c5aa0343d7d41a294ce84457a4e@xs4all.nl> References: <8f136c5aa0343d7d41a294ce84457a4e@xs4all.nl> Message-ID: <04d8d9e4-5251-01c1-bc35-465123b603b8@oracle.com> Hi Jorn, Thanks for the hard work; we are in the process of reviewing the changes. Here's what we'll do: 1) we will create issues for each of the patches (Sundar is doing that as we speak) 2) we will try to get easier changes in sooner rather? than later (that is, patches 1, 3, 4) 3) in parallel, I'll be doing an in-depth review of your binder changes and perhaps start a separate thread on that Thanks Maurizio On 09/01/2019 21:17, Jorn Vernee wrote: > Hi, > > I have fixed the last few failing tests on Windows so this is an RFR > for adding Windows support to Panama FFI. There's a lot of changes so > I've split it into 4 patches. We should probably focus on one of these > one at a time, but I wanted to include all the changes needed to pass > the tests at least. It's probably best to start with [2], since that > implements the core functionality. > > Default lib fixes [1] (needed for the default library to work on Windows) > Hotspot, binder tweaks [2] (Changes to the stub generation and shared > binder code. Includes the Windows implementation of SystemABI) > jextract tweaks [3] (minor jextract portability changes) > test tweaks [4] (changes needed to run, and pass the tests on Windows) > > There are some tests that are disabled: > ? - EmptyStructTest; Empty structs are a GCC extension. > ? - BadBitfieldTest; this is testing for a problem the MSVC ABI > doesn't seem to have. > ? - StdioTest; relies on standard system header path, which on Windows > is more tricky to find. > ? - ComplexTest; Windows complex types work very differently in C > source, so the current test doesn't compile. > ? - LongDoubleTest; long double is only 64 bits on Windows. This might > need a replacement test still, but it should work the same as `double` > on Windows. > ? - Getpid; No getpid on Windows in the C standard. > ? - UnixSystem; Windows is not unix. > > (Some tests specifically for Windows could be added, but I'm leaving > that for a different RFR) > > Changes for [1]: > ? - replace default library handle with a findEntryInProcess since > Windows doesn't have an RTLD_DEFAULT equivalent. > > Changes for [2]: > ? - changed upcall/downcall stub generation to follow the MS ABI on > windows (using compiler switches) > ? - added SKIPs to shuffle recipe generation when e.g. the first float > is passed in the second float register (which can be the case on > windows). > ? - moved boxValue and unboxValue into SystemABI, since they turned > out to be ABI specific, and made tweaks to the windows version for > handeling structs being passed by value. > ? - moved upcall stub natives into shared ABI package. > ? - Let each ABI have it's own VarargsInvoker implementation, since we > need to propagate more info about which arguments are variadic on > windows. > ? - CallingSequence::storageOffset is no longer relying on > Storage::getStorageIndex, since that is not the same as what's > actually needed; being the index of the binding. (the 2 can differ on > windows, but not on sysv). > ? - Added Windows ABI implementation classes based on the SysV ones. > ? - Let UniversalUpcallHandler write the in-memory-return pointer to > RAX since that is required on Windows (and I think this change is > portable). > > Changes for [3]: > ? - instead of hardcoding the size of `long` `unsigned long` and `long > double`, delegate to libclang to tell us the size, since these types > have different sizes on the 2 ABIs. (This is pretty ad-hoc, so > probably needs to be improved. But, I have little experience with the > jextract code, so I'm open to suggestion of how to better fix this) > > Changes for [4]: > ? - added dll exports for native symbols using compiler switches. > ? - added test for small (fits-in-register) struct by-value passing, > since windows has special handling for those. > ? - removed depencies on POSIX functions from some tests where it > wasn't really needed, so they could run on windows. > ? - propagate clang library name to TestJextractFFI, since on Windows > we need to load "libclang". > ? - let Runner test use windows specific files to compare with for > bitfields.h and simple.h > ? - use `long long` instead of `long` in some cases since `long long` > is 64 bits on SysV and MSx64 > > Thanks, > Jorn > > [1] : > http://cr.openjdk.java.net/~jvernee/panama/webrevs/8211060/webrev.01/ > [2] : > http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows/webrev.03/ > [3] : > http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_jextract/webrev.00/ > [4] : > http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_tests/webrev.02/ From sundararajan.athijegannathan at oracle.com Thu Jan 10 13:26:20 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Thu, 10 Jan 2019 18:56:20 +0530 Subject: [foreign] RFR: Windows support In-Reply-To: <04d8d9e4-5251-01c1-bc35-465123b603b8@oracle.com> References: <8f136c5aa0343d7d41a294ce84457a4e@xs4all.nl> <04d8d9e4-5251-01c1-bc35-465123b603b8@oracle.com> Message-ID: <5C3747FC.7070501@oracle.com> The 4 bugs (including the preexisting one) are as follows: 1) Library.getDefault() does not work on Windows https://bugs.openjdk.java.net/browse/JDK-8211060 2) Binder changes are needed to support paname foreign API on Windows https://bugs.openjdk.java.net/browse/JDK-8216485 3) jextract LayoutUtils has to be modified for Windows https://bugs.openjdk.java.net/browse/JDK-8216484 4) jextract tests have to be modified to run on Windows platform https://bugs.openjdk.java.net/browse/JDK-8216483 PS. I'm running the test patch on Mac and Linux to make sure that the testing is fine on other platforms. Thanks, -Sundar On 10/01/19, 6:14 PM, Maurizio Cimadamore wrote: > Hi Jorn, > Thanks for the hard work; we are in the process of reviewing the > changes. Here's what we'll do: > > 1) we will create issues for each of the patches (Sundar is doing that > as we speak) > 2) we will try to get easier changes in sooner rather than later > (that is, patches 1, 3, 4) > 3) in parallel, I'll be doing an in-depth review of your binder > changes and perhaps start a separate thread on that > > Thanks > Maurizio > > > On 09/01/2019 21:17, Jorn Vernee wrote: > >> Hi, >> >> I have fixed the last few failing tests on Windows so this is an RFR >> for adding Windows support to Panama FFI. There's a lot of changes so >> I've split it into 4 patches. We should probably focus on one of >> these one at a time, but I wanted to include all the changes needed >> to pass the tests at least. It's probably best to start with [2], >> since that implements the core functionality. >> >> Default lib fixes [1] (needed for the default library to work on >> Windows) >> Hotspot, binder tweaks [2] (Changes to the stub generation and shared >> binder code. Includes the Windows implementation of SystemABI) >> jextract tweaks [3] (minor jextract portability changes) >> test tweaks [4] (changes needed to run, and pass the tests on Windows) >> >> There are some tests that are disabled: >> - EmptyStructTest; Empty structs are a GCC extension. >> - BadBitfieldTest; this is testing for a problem the MSVC ABI >> doesn't seem to have. >> - StdioTest; relies on standard system header path, which on >> Windows is more tricky to find. >> - ComplexTest; Windows complex types work very differently in C >> source, so the current test doesn't compile. >> - LongDoubleTest; long double is only 64 bits on Windows. This >> might need a replacement test still, but it should work the same as >> `double` on Windows. >> - Getpid; No getpid on Windows in the C standard. >> - UnixSystem; Windows is not unix. >> >> (Some tests specifically for Windows could be added, but I'm leaving >> that for a different RFR) >> >> Changes for [1]: >> - replace default library handle with a findEntryInProcess since >> Windows doesn't have an RTLD_DEFAULT equivalent. >> >> Changes for [2]: >> - changed upcall/downcall stub generation to follow the MS ABI on >> windows (using compiler switches) >> - added SKIPs to shuffle recipe generation when e.g. the first >> float is passed in the second float register (which can be the case >> on windows). >> - moved boxValue and unboxValue into SystemABI, since they turned >> out to be ABI specific, and made tweaks to the windows version for >> handeling structs being passed by value. >> - moved upcall stub natives into shared ABI package. >> - Let each ABI have it's own VarargsInvoker implementation, since >> we need to propagate more info about which arguments are variadic on >> windows. >> - CallingSequence::storageOffset is no longer relying on >> Storage::getStorageIndex, since that is not the same as what's >> actually needed; being the index of the binding. (the 2 can differ on >> windows, but not on sysv). >> - Added Windows ABI implementation classes based on the SysV ones. >> - Let UniversalUpcallHandler write the in-memory-return pointer to >> RAX since that is required on Windows (and I think this change is >> portable). >> >> Changes for [3]: >> - instead of hardcoding the size of `long` `unsigned long` and >> `long double`, delegate to libclang to tell us the size, since these >> types have different sizes on the 2 ABIs. (This is pretty ad-hoc, so >> probably needs to be improved. But, I have little experience with the >> jextract code, so I'm open to suggestion of how to better fix this) >> >> Changes for [4]: >> - added dll exports for native symbols using compiler switches. >> - added test for small (fits-in-register) struct by-value passing, >> since windows has special handling for those. >> - removed depencies on POSIX functions from some tests where it >> wasn't really needed, so they could run on windows. >> - propagate clang library name to TestJextractFFI, since on Windows >> we need to load "libclang". >> - let Runner test use windows specific files to compare with for >> bitfields.h and simple.h >> - use `long long` instead of `long` in some cases since `long long` >> is 64 bits on SysV and MSx64 >> >> Thanks, >> Jorn >> >> [1] : >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8211060/webrev.01/ >> [2] : >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows/webrev.03/ >> [3] : >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_jextract/webrev.00/ >> [4] : >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_tests/webrev.02/ From jbvernee at xs4all.nl Thu Jan 10 13:54:11 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Thu, 10 Jan 2019 14:54:11 +0100 Subject: [foreign] RFR: Windows support In-Reply-To: <5C3747FC.7070501@oracle.com> References: <8f136c5aa0343d7d41a294ce84457a4e@xs4all.nl> <04d8d9e4-5251-01c1-bc35-465123b603b8@oracle.com> <5C3747FC.7070501@oracle.com> Message-ID: <3f4899ea4f4bc1bad612e4d24a6b359c@xs4all.nl> > PS. I'm running the test patch on Mac and Linux to make sure that the > testing is fine on other platforms. Thanks. One thing I forgot to mention; the generated source files for UpcallTest and DowncallTest needed changes, so I've removed them and created the srcgen.sh script in test/jdk/com/sun/tools/jextract to generate the source files instead. I had this hooked up through jtreg, but since the native files are compiled before the sources are generated this didn't generate the files in time, and I never got around to figuring out how to hook the generation into the build system. So, just be aware that after applying the test-tweaks patch you still have to run that srcgen.sh script (relies on java 11 on PATH) to generate the test sources, since the patch deletes them. Jorn Sundararajan Athijegannathan schreef op 2019-01-10 14:26: > The 4 bugs (including the preexisting one) are as follows: > > 1) Library.getDefault() does not work on Windows > > https://bugs.openjdk.java.net/browse/JDK-8211060 > > 2) Binder changes are needed to support paname foreign API on Windows > > https://bugs.openjdk.java.net/browse/JDK-8216485 > > 3) jextract LayoutUtils has to be modified for Windows > > https://bugs.openjdk.java.net/browse/JDK-8216484 > > 4) jextract tests have to be modified to run on Windows platform > > https://bugs.openjdk.java.net/browse/JDK-8216483 > > PS. I'm running the test patch on Mac and Linux to make sure that the > testing is fine on other platforms. > > Thanks, > -Sundar > > On 10/01/19, 6:14 PM, Maurizio Cimadamore wrote: >> Hi Jorn, >> Thanks for the hard work; we are in the process of reviewing the >> changes. Here's what we'll do: >> >> 1) we will create issues for each of the patches (Sundar is doing that >> as we speak) >> 2) we will try to get easier changes in sooner rather than later >> (that is, patches 1, 3, 4) >> 3) in parallel, I'll be doing an in-depth review of your binder >> changes and perhaps start a separate thread on that >> >> Thanks >> Maurizio >> >> >> On 09/01/2019 21:17, Jorn Vernee wrote: >> >>> Hi, >>> >>> I have fixed the last few failing tests on Windows so this is an RFR >>> for adding Windows support to Panama FFI. There's a lot of changes so >>> I've split it into 4 patches. We should probably focus on one of >>> these one at a time, but I wanted to include all the changes needed >>> to pass the tests at least. It's probably best to start with [2], >>> since that implements the core functionality. >>> >>> Default lib fixes [1] (needed for the default library to work on >>> Windows) >>> Hotspot, binder tweaks [2] (Changes to the stub generation and shared >>> binder code. Includes the Windows implementation of SystemABI) >>> jextract tweaks [3] (minor jextract portability changes) >>> test tweaks [4] (changes needed to run, and pass the tests on >>> Windows) >>> >>> There are some tests that are disabled: >>> - EmptyStructTest; Empty structs are a GCC extension. >>> - BadBitfieldTest; this is testing for a problem the MSVC ABI >>> doesn't seem to have. >>> - StdioTest; relies on standard system header path, which on >>> Windows is more tricky to find. >>> - ComplexTest; Windows complex types work very differently in C >>> source, so the current test doesn't compile. >>> - LongDoubleTest; long double is only 64 bits on Windows. This >>> might need a replacement test still, but it should work the same as >>> `double` on Windows. >>> - Getpid; No getpid on Windows in the C standard. >>> - UnixSystem; Windows is not unix. >>> >>> (Some tests specifically for Windows could be added, but I'm leaving >>> that for a different RFR) >>> >>> Changes for [1]: >>> - replace default library handle with a findEntryInProcess since >>> Windows doesn't have an RTLD_DEFAULT equivalent. >>> >>> Changes for [2]: >>> - changed upcall/downcall stub generation to follow the MS ABI on >>> windows (using compiler switches) >>> - added SKIPs to shuffle recipe generation when e.g. the first >>> float is passed in the second float register (which can be the case >>> on windows). >>> - moved boxValue and unboxValue into SystemABI, since they turned >>> out to be ABI specific, and made tweaks to the windows version for >>> handeling structs being passed by value. >>> - moved upcall stub natives into shared ABI package. >>> - Let each ABI have it's own VarargsInvoker implementation, since >>> we need to propagate more info about which arguments are variadic on >>> windows. >>> - CallingSequence::storageOffset is no longer relying on >>> Storage::getStorageIndex, since that is not the same as what's >>> actually needed; being the index of the binding. (the 2 can differ on >>> windows, but not on sysv). >>> - Added Windows ABI implementation classes based on the SysV ones. >>> - Let UniversalUpcallHandler write the in-memory-return pointer to >>> RAX since that is required on Windows (and I think this change is >>> portable). >>> >>> Changes for [3]: >>> - instead of hardcoding the size of `long` `unsigned long` and >>> `long double`, delegate to libclang to tell us the size, since these >>> types have different sizes on the 2 ABIs. (This is pretty ad-hoc, so >>> probably needs to be improved. But, I have little experience with the >>> jextract code, so I'm open to suggestion of how to better fix this) >>> >>> Changes for [4]: >>> - added dll exports for native symbols using compiler switches. >>> - added test for small (fits-in-register) struct by-value passing, >>> since windows has special handling for those. >>> - removed depencies on POSIX functions from some tests where it >>> wasn't really needed, so they could run on windows. >>> - propagate clang library name to TestJextractFFI, since on Windows >>> we need to load "libclang". >>> - let Runner test use windows specific files to compare with for >>> bitfields.h and simple.h >>> - use `long long` instead of `long` in some cases since `long long` >>> is 64 bits on SysV and MSx64 >>> >>> Thanks, >>> Jorn >>> >>> [1] : >>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8211060/webrev.01/ >>> [2] : >>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows/webrev.03/ >>> [3] : >>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_jextract/webrev.00/ >>> [4] : >>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_tests/webrev.02/ From sundararajan.athijegannathan at oracle.com Thu Jan 10 14:18:58 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Thu, 10 Jan 2019 19:48:58 +0530 Subject: [foreign] RFR: Windows support In-Reply-To: <3f4899ea4f4bc1bad612e4d24a6b359c@xs4all.nl> References: <8f136c5aa0343d7d41a294ce84457a4e@xs4all.nl> <04d8d9e4-5251-01c1-bc35-465123b603b8@oracle.com> <5C3747FC.7070501@oracle.com> <3f4899ea4f4bc1bad612e4d24a6b359c@xs4all.nl> Message-ID: <5C375452.8050800@oracle.com> There are two issues: 1) I had to change make/test/JtregNativeJdk.gmk + ifeq ($(OPENJDK_TARGET_OS), windows) + # windows flags + BUILD_JDK_JTREG_LIBRARIES_CFLAGS_libVector := /arch:AVX + BUILD_JDK_JTREG_LIBRARIES_CFLAGS_libUpcall := /arch:AVX + BUILD_JDK_JTREG_LIBRARIES_CFLAGS_libGlobalVariable := /arch:AVX + else + # Enable AVX for vector (Long*) tests + BUILD_JDK_JTREG_LIBRARIES_CFLAGS_libVector := -mavx + BUILD_JDK_JTREG_LIBRARIES_CFLAGS_libUpcall := -mavx + BUILD_JDK_JTREG_LIBRARIES_CFLAGS_libGlobalVariable := -mavx + endif Because I was getting "clang: error: no such file or directory: '/arch:AVX' on Mac. With the above change, I managed to get passed that failure. 2) Three tests failed. com/sun/tools/jextract/TestDowncall.java <../../../test-support/jtreg_test_jdk_jdk_foreign/com/sun/tools/jextract/TestDowncall.jtr>: com/sun/tools/jextract/TestUpcall.java <../../../test-support/jtreg_test_jdk_jdk_foreign/com/sun/tools/jextract/TestUpcall.jtr>: com/sun/tools/jextract/jclang-ffi/TestJextractFFI.java <../../../test-support/jtreg_test_jdk_jdk_foreign/com/sun/tools/jextract/jclang-ffi/TestJextractFFI.jtr>: I think that is srcgen.sh issue you're talking about. If files have to be generated, either we need to automate or pre-generate and be included as part of patch. We cannot expect shell scripts to be manually run before tests (in nightly testing for eg). Please incorporate the above changes and send RFR against the test changes bug. Also please check my change above is fine on Windows. Thanks, -Sundar On 10/01/19, 7:24 PM, Jorn Vernee wrote: >> PS. I'm running the test patch on Mac and Linux to make sure that the >> testing is fine on other platforms. > > Thanks. > > One thing I forgot to mention; the generated source files for > UpcallTest and DowncallTest needed changes, so I've removed them and > created the srcgen.sh script in test/jdk/com/sun/tools/jextract to > generate the source files instead. I had this hooked up through jtreg, > but since the native files are compiled before the sources are > generated this didn't generate the files in time, and I never got > around to figuring out how to hook the generation into the build system. > > So, just be aware that after applying the test-tweaks patch you still > have to run that srcgen.sh script (relies on java 11 on PATH) to > generate the test sources, since the patch deletes them. > > Jorn > > Sundararajan Athijegannathan schreef op 2019-01-10 14:26: >> The 4 bugs (including the preexisting one) are as follows: >> >> 1) Library.getDefault() does not work on Windows >> >> https://bugs.openjdk.java.net/browse/JDK-8211060 >> >> 2) Binder changes are needed to support paname foreign API on Windows >> >> https://bugs.openjdk.java.net/browse/JDK-8216485 >> >> 3) jextract LayoutUtils has to be modified for Windows >> >> https://bugs.openjdk.java.net/browse/JDK-8216484 >> >> 4) jextract tests have to be modified to run on Windows platform >> >> https://bugs.openjdk.java.net/browse/JDK-8216483 >> >> PS. I'm running the test patch on Mac and Linux to make sure that the >> testing is fine on other platforms. >> >> Thanks, >> -Sundar >> >> On 10/01/19, 6:14 PM, Maurizio Cimadamore wrote: >>> Hi Jorn, >>> Thanks for the hard work; we are in the process of reviewing the >>> changes. Here's what we'll do: >>> >>> 1) we will create issues for each of the patches (Sundar is doing >>> that as we speak) >>> 2) we will try to get easier changes in sooner rather than later >>> (that is, patches 1, 3, 4) >>> 3) in parallel, I'll be doing an in-depth review of your binder >>> changes and perhaps start a separate thread on that >>> >>> Thanks >>> Maurizio >>> >>> >>> On 09/01/2019 21:17, Jorn Vernee wrote: >>> >>>> Hi, >>>> >>>> I have fixed the last few failing tests on Windows so this is an >>>> RFR for adding Windows support to Panama FFI. There's a lot of >>>> changes so I've split it into 4 patches. We should probably focus >>>> on one of these one at a time, but I wanted to include all the >>>> changes needed to pass the tests at least. It's probably best to >>>> start with [2], since that implements the core functionality. >>>> >>>> Default lib fixes [1] (needed for the default library to work on >>>> Windows) >>>> Hotspot, binder tweaks [2] (Changes to the stub generation and >>>> shared binder code. Includes the Windows implementation of SystemABI) >>>> jextract tweaks [3] (minor jextract portability changes) >>>> test tweaks [4] (changes needed to run, and pass the tests on Windows) >>>> >>>> There are some tests that are disabled: >>>> - EmptyStructTest; Empty structs are a GCC extension. >>>> - BadBitfieldTest; this is testing for a problem the MSVC ABI >>>> doesn't seem to have. >>>> - StdioTest; relies on standard system header path, which on >>>> Windows is more tricky to find. >>>> - ComplexTest; Windows complex types work very differently in C >>>> source, so the current test doesn't compile. >>>> - LongDoubleTest; long double is only 64 bits on Windows. This >>>> might need a replacement test still, but it should work the same as >>>> `double` on Windows. >>>> - Getpid; No getpid on Windows in the C standard. >>>> - UnixSystem; Windows is not unix. >>>> >>>> (Some tests specifically for Windows could be added, but I'm >>>> leaving that for a different RFR) >>>> >>>> Changes for [1]: >>>> - replace default library handle with a findEntryInProcess since >>>> Windows doesn't have an RTLD_DEFAULT equivalent. >>>> >>>> Changes for [2]: >>>> - changed upcall/downcall stub generation to follow the MS ABI on >>>> windows (using compiler switches) >>>> - added SKIPs to shuffle recipe generation when e.g. the first >>>> float is passed in the second float register (which can be the case >>>> on windows). >>>> - moved boxValue and unboxValue into SystemABI, since they turned >>>> out to be ABI specific, and made tweaks to the windows version for >>>> handeling structs being passed by value. >>>> - moved upcall stub natives into shared ABI package. >>>> - Let each ABI have it's own VarargsInvoker implementation, since >>>> we need to propagate more info about which arguments are variadic >>>> on windows. >>>> - CallingSequence::storageOffset is no longer relying on >>>> Storage::getStorageIndex, since that is not the same as what's >>>> actually needed; being the index of the binding. (the 2 can differ >>>> on windows, but not on sysv). >>>> - Added Windows ABI implementation classes based on the SysV ones. >>>> - Let UniversalUpcallHandler write the in-memory-return pointer >>>> to RAX since that is required on Windows (and I think this change >>>> is portable). >>>> >>>> Changes for [3]: >>>> - instead of hardcoding the size of `long` `unsigned long` and >>>> `long double`, delegate to libclang to tell us the size, since >>>> these types have different sizes on the 2 ABIs. (This is pretty >>>> ad-hoc, so probably needs to be improved. But, I have little >>>> experience with the jextract code, so I'm open to suggestion of how >>>> to better fix this) >>>> >>>> Changes for [4]: >>>> - added dll exports for native symbols using compiler switches. >>>> - added test for small (fits-in-register) struct by-value >>>> passing, since windows has special handling for those. >>>> - removed depencies on POSIX functions from some tests where it >>>> wasn't really needed, so they could run on windows. >>>> - propagate clang library name to TestJextractFFI, since on >>>> Windows we need to load "libclang". >>>> - let Runner test use windows specific files to compare with for >>>> bitfields.h and simple.h >>>> - use `long long` instead of `long` in some cases since `long >>>> long` is 64 bits on SysV and MSx64 >>>> >>>> Thanks, >>>> Jorn >>>> >>>> [1] : >>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8211060/webrev.01/ >>>> [2] : >>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows/webrev.03/ >>>> [3] : >>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_jextract/webrev.00/ >>>> [4] : >>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_tests/webrev.02/ From maurizio.cimadamore at oracle.com Thu Jan 10 14:08:09 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 10 Jan 2019 14:08:09 +0000 Subject: [foreign] RFR: Windows support In-Reply-To: <3f4899ea4f4bc1bad612e4d24a6b359c@xs4all.nl> References: <8f136c5aa0343d7d41a294ce84457a4e@xs4all.nl> <04d8d9e4-5251-01c1-bc35-465123b603b8@oracle.com> <5C3747FC.7070501@oracle.com> <3f4899ea4f4bc1bad612e4d24a6b359c@xs4all.nl> Message-ID: On 10/01/2019 13:54, Jorn Vernee wrote: >> PS. I'm running the test patch on Mac and Linux to make sure that the >> testing is fine on other platforms. > > Thanks. > > One thing I forgot to mention; the generated source files for > UpcallTest and DowncallTest needed changes, so I've removed them and > created the srcgen.sh script in test/jdk/com/sun/tools/jextract to > generate the source files instead. I had this hooked up through jtreg, > but since the native files are compiled before the sources are > generated this didn't generate the files in time, and I never got > around to figuring out how to hook the generation into the build system. Why not just updating the already existing Java files for generating the tests, TestUpcall/DownCallGenerator ? These are in Java and can be executed using the recent java launcher feature (where you can pass a .java file to the 'java' launcher). Relatively easy to maintain. In other words, the idea is that (similarly to what? has been done for some tests in the vector API), whenever we need to tweak the generators we: 1) change the generators 2) rerun them so as to change the headers/impl files used by Upcall/DownCall tests 3) commit all changes So, (3) will see changes in both the generated files and in the generators. Maurizio > > So, just be aware that after applying the test-tweaks patch you still > have to run that srcgen.sh script (relies on java 11 on PATH) to > generate the test sources, since the patch deletes them. > > Jorn > > Sundararajan Athijegannathan schreef op 2019-01-10 14:26: >> The 4 bugs (including the preexisting one) are as follows: >> >> 1) Library.getDefault() does not work on Windows >> >> https://bugs.openjdk.java.net/browse/JDK-8211060 >> >> 2) Binder changes are needed to support paname foreign API on Windows >> >> https://bugs.openjdk.java.net/browse/JDK-8216485 >> >> 3)? jextract LayoutUtils has to be modified for Windows >> >> https://bugs.openjdk.java.net/browse/JDK-8216484 >> >> 4) jextract tests have to be modified to run on Windows platform >> >> https://bugs.openjdk.java.net/browse/JDK-8216483 >> >> PS. I'm running the test patch on Mac and Linux to make sure that the >> testing is fine on other platforms. >> >> Thanks, >> -Sundar >> >> On 10/01/19, 6:14 PM, Maurizio Cimadamore wrote: >>> Hi Jorn, >>> Thanks for the hard work; we are in the process of reviewing the >>> changes. Here's what we'll do: >>> >>> 1) we will create issues for each of the patches (Sundar is doing >>> that as we speak) >>> 2) we will try to get easier changes in sooner rather? than later >>> (that is, patches 1, 3, 4) >>> 3) in parallel, I'll be doing an in-depth review of your binder >>> changes and perhaps start a separate thread on that >>> >>> Thanks >>> Maurizio >>> >>> >>> On 09/01/2019 21:17, Jorn Vernee wrote: >>> >>>> Hi, >>>> >>>> I have fixed the last few failing tests on Windows so this is an >>>> RFR for adding Windows support to Panama FFI. There's a lot of >>>> changes so I've split it into 4 patches. We should probably focus >>>> on one of these one at a time, but I wanted to include all the >>>> changes needed to pass the tests at least. It's probably best to >>>> start with [2], since that implements the core functionality. >>>> >>>> Default lib fixes [1] (needed for the default library to work on >>>> Windows) >>>> Hotspot, binder tweaks [2] (Changes to the stub generation and >>>> shared binder code. Includes the Windows implementation of SystemABI) >>>> jextract tweaks [3] (minor jextract portability changes) >>>> test tweaks [4] (changes needed to run, and pass the tests on Windows) >>>> >>>> There are some tests that are disabled: >>>> ? - EmptyStructTest; Empty structs are a GCC extension. >>>> ? - BadBitfieldTest; this is testing for a problem the MSVC ABI >>>> doesn't seem to have. >>>> ? - StdioTest; relies on standard system header path, which on >>>> Windows is more tricky to find. >>>> ? - ComplexTest; Windows complex types work very differently in C >>>> source, so the current test doesn't compile. >>>> ? - LongDoubleTest; long double is only 64 bits on Windows. This >>>> might need a replacement test still, but it should work the same as >>>> `double` on Windows. >>>> ? - Getpid; No getpid on Windows in the C standard. >>>> ? - UnixSystem; Windows is not unix. >>>> >>>> (Some tests specifically for Windows could be added, but I'm >>>> leaving that for a different RFR) >>>> >>>> Changes for [1]: >>>> ? - replace default library handle with a findEntryInProcess since >>>> Windows doesn't have an RTLD_DEFAULT equivalent. >>>> >>>> Changes for [2]: >>>> ? - changed upcall/downcall stub generation to follow the MS ABI on >>>> windows (using compiler switches) >>>> ? - added SKIPs to shuffle recipe generation when e.g. the first >>>> float is passed in the second float register (which can be the case >>>> on windows). >>>> ? - moved boxValue and unboxValue into SystemABI, since they turned >>>> out to be ABI specific, and made tweaks to the windows version for >>>> handeling structs being passed by value. >>>> ? - moved upcall stub natives into shared ABI package. >>>> ? - Let each ABI have it's own VarargsInvoker implementation, since >>>> we need to propagate more info about which arguments are variadic >>>> on windows. >>>> ? - CallingSequence::storageOffset is no longer relying on >>>> Storage::getStorageIndex, since that is not the same as what's >>>> actually needed; being the index of the binding. (the 2 can differ >>>> on windows, but not on sysv). >>>> ? - Added Windows ABI implementation classes based on the SysV ones. >>>> ? - Let UniversalUpcallHandler write the in-memory-return pointer >>>> to RAX since that is required on Windows (and I think this change >>>> is portable). >>>> >>>> Changes for [3]: >>>> ? - instead of hardcoding the size of `long` `unsigned long` and >>>> `long double`, delegate to libclang to tell us the size, since >>>> these types have different sizes on the 2 ABIs. (This is pretty >>>> ad-hoc, so probably needs to be improved. But, I have little >>>> experience with the jextract code, so I'm open to suggestion of how >>>> to better fix this) >>>> >>>> Changes for [4]: >>>> ? - added dll exports for native symbols using compiler switches. >>>> ? - added test for small (fits-in-register) struct by-value >>>> passing, since windows has special handling for those. >>>> ? - removed depencies on POSIX functions from some tests where it >>>> wasn't really needed, so they could run on windows. >>>> ? - propagate clang library name to TestJextractFFI, since on >>>> Windows we need to load "libclang". >>>> ? - let Runner test use windows specific files to compare with for >>>> bitfields.h and simple.h >>>> ? - use `long long` instead of `long` in some cases since `long >>>> long` is 64 bits on SysV and MSx64 >>>> >>>> Thanks, >>>> Jorn >>>> >>>> [1] : >>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8211060/webrev.01/ >>>> [2] : >>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows/webrev.03/ >>>> [3] : >>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_jextract/webrev.00/ >>>> [4] : >>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_tests/webrev.02/ From jbvernee at xs4all.nl Thu Jan 10 14:13:07 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Thu, 10 Jan 2019 15:13:07 +0100 Subject: [foreign] RFR: Windows support In-Reply-To: References: <8f136c5aa0343d7d41a294ce84457a4e@xs4all.nl> <04d8d9e4-5251-01c1-bc35-465123b603b8@oracle.com> <5C3747FC.7070501@oracle.com> <3f4899ea4f4bc1bad612e4d24a6b359c@xs4all.nl> Message-ID: Maurizio Cimadamore schreef op 2019-01-10 15:08: > On 10/01/2019 13:54, Jorn Vernee wrote: >>> PS. I'm running the test patch on Mac and Linux to make sure that the >>> testing is fine on other platforms. >> >> Thanks. >> >> One thing I forgot to mention; the generated source files for >> UpcallTest and DowncallTest needed changes, so I've removed them and >> created the srcgen.sh script in test/jdk/com/sun/tools/jextract to >> generate the source files instead. I had this hooked up through jtreg, >> but since the native files are compiled before the sources are >> generated this didn't generate the files in time, and I never got >> around to figuring out how to hook the generation into the build >> system. > > Why not just updating the already existing Java files for generating > the tests, TestUpcall/DownCallGenerator ? These are in Java and can be > executed using the recent java launcher feature (where you can pass a > .java file to the 'java' launcher). Relatively easy to maintain. I did that. The script just calls those 2 java generators 4 times to generate all the files :) > In other words, the idea is that (similarly to what? has been done for > some tests in the vector API), whenever we need to tweak the > generators we: > > 1) change the generators > 2) rerun them so as to change the headers/impl files used by > Upcall/DownCall tests > 3) commit all changes I wanted to avoid having huge files/diffs for something that can be generated automatically. I will update the patch to include these files. Jorn > So, (3) will see changes in both the generated files and in the > generators. > > Maurizio > >> >> So, just be aware that after applying the test-tweaks patch you still >> have to run that srcgen.sh script (relies on java 11 on PATH) to >> generate the test sources, since the patch deletes them. >> >> Jorn >> >> Sundararajan Athijegannathan schreef op 2019-01-10 14:26: >>> The 4 bugs (including the preexisting one) are as follows: >>> >>> 1) Library.getDefault() does not work on Windows >>> >>> https://bugs.openjdk.java.net/browse/JDK-8211060 >>> >>> 2) Binder changes are needed to support paname foreign API on Windows >>> >>> https://bugs.openjdk.java.net/browse/JDK-8216485 >>> >>> 3)? jextract LayoutUtils has to be modified for Windows >>> >>> https://bugs.openjdk.java.net/browse/JDK-8216484 >>> >>> 4) jextract tests have to be modified to run on Windows platform >>> >>> https://bugs.openjdk.java.net/browse/JDK-8216483 >>> >>> PS. I'm running the test patch on Mac and Linux to make sure that the >>> testing is fine on other platforms. >>> >>> Thanks, >>> -Sundar >>> >>> On 10/01/19, 6:14 PM, Maurizio Cimadamore wrote: >>>> Hi Jorn, >>>> Thanks for the hard work; we are in the process of reviewing the >>>> changes. Here's what we'll do: >>>> >>>> 1) we will create issues for each of the patches (Sundar is doing >>>> that as we speak) >>>> 2) we will try to get easier changes in sooner rather? than later >>>> (that is, patches 1, 3, 4) >>>> 3) in parallel, I'll be doing an in-depth review of your binder >>>> changes and perhaps start a separate thread on that >>>> >>>> Thanks >>>> Maurizio >>>> >>>> >>>> On 09/01/2019 21:17, Jorn Vernee wrote: >>>> >>>>> Hi, >>>>> >>>>> I have fixed the last few failing tests on Windows so this is an >>>>> RFR for adding Windows support to Panama FFI. There's a lot of >>>>> changes so I've split it into 4 patches. We should probably focus >>>>> on one of these one at a time, but I wanted to include all the >>>>> changes needed to pass the tests at least. It's probably best to >>>>> start with [2], since that implements the core functionality. >>>>> >>>>> Default lib fixes [1] (needed for the default library to work on >>>>> Windows) >>>>> Hotspot, binder tweaks [2] (Changes to the stub generation and >>>>> shared binder code. Includes the Windows implementation of >>>>> SystemABI) >>>>> jextract tweaks [3] (minor jextract portability changes) >>>>> test tweaks [4] (changes needed to run, and pass the tests on >>>>> Windows) >>>>> >>>>> There are some tests that are disabled: >>>>> ? - EmptyStructTest; Empty structs are a GCC extension. >>>>> ? - BadBitfieldTest; this is testing for a problem the MSVC ABI >>>>> doesn't seem to have. >>>>> ? - StdioTest; relies on standard system header path, which on >>>>> Windows is more tricky to find. >>>>> ? - ComplexTest; Windows complex types work very differently in C >>>>> source, so the current test doesn't compile. >>>>> ? - LongDoubleTest; long double is only 64 bits on Windows. This >>>>> might need a replacement test still, but it should work the same as >>>>> `double` on Windows. >>>>> ? - Getpid; No getpid on Windows in the C standard. >>>>> ? - UnixSystem; Windows is not unix. >>>>> >>>>> (Some tests specifically for Windows could be added, but I'm >>>>> leaving that for a different RFR) >>>>> >>>>> Changes for [1]: >>>>> ? - replace default library handle with a findEntryInProcess since >>>>> Windows doesn't have an RTLD_DEFAULT equivalent. >>>>> >>>>> Changes for [2]: >>>>> ? - changed upcall/downcall stub generation to follow the MS ABI on >>>>> windows (using compiler switches) >>>>> ? - added SKIPs to shuffle recipe generation when e.g. the first >>>>> float is passed in the second float register (which can be the case >>>>> on windows). >>>>> ? - moved boxValue and unboxValue into SystemABI, since they turned >>>>> out to be ABI specific, and made tweaks to the windows version for >>>>> handeling structs being passed by value. >>>>> ? - moved upcall stub natives into shared ABI package. >>>>> ? - Let each ABI have it's own VarargsInvoker implementation, since >>>>> we need to propagate more info about which arguments are variadic >>>>> on windows. >>>>> ? - CallingSequence::storageOffset is no longer relying on >>>>> Storage::getStorageIndex, since that is not the same as what's >>>>> actually needed; being the index of the binding. (the 2 can differ >>>>> on windows, but not on sysv). >>>>> ? - Added Windows ABI implementation classes based on the SysV >>>>> ones. >>>>> ? - Let UniversalUpcallHandler write the in-memory-return pointer >>>>> to RAX since that is required on Windows (and I think this change >>>>> is portable). >>>>> >>>>> Changes for [3]: >>>>> ? - instead of hardcoding the size of `long` `unsigned long` and >>>>> `long double`, delegate to libclang to tell us the size, since >>>>> these types have different sizes on the 2 ABIs. (This is pretty >>>>> ad-hoc, so probably needs to be improved. But, I have little >>>>> experience with the jextract code, so I'm open to suggestion of how >>>>> to better fix this) >>>>> >>>>> Changes for [4]: >>>>> ? - added dll exports for native symbols using compiler switches. >>>>> ? - added test for small (fits-in-register) struct by-value >>>>> passing, since windows has special handling for those. >>>>> ? - removed depencies on POSIX functions from some tests where it >>>>> wasn't really needed, so they could run on windows. >>>>> ? - propagate clang library name to TestJextractFFI, since on >>>>> Windows we need to load "libclang". >>>>> ? - let Runner test use windows specific files to compare with for >>>>> bitfields.h and simple.h >>>>> ? - use `long long` instead of `long` in some cases since `long >>>>> long` is 64 bits on SysV and MSx64 >>>>> >>>>> Thanks, >>>>> Jorn >>>>> >>>>> [1] : >>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8211060/webrev.01/ >>>>> [2] : >>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows/webrev.03/ >>>>> [3] : >>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_jextract/webrev.00/ >>>>> [4] : >>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_tests/webrev.02/ From jbvernee at xs4all.nl Thu Jan 10 14:24:27 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Thu, 10 Jan 2019 15:24:27 +0100 Subject: [foreign] RFR: Windows support In-Reply-To: <5C375452.8050800@oracle.com> References: <8f136c5aa0343d7d41a294ce84457a4e@xs4all.nl> <04d8d9e4-5251-01c1-bc35-465123b603b8@oracle.com> <5C3747FC.7070501@oracle.com> <3f4899ea4f4bc1bad612e4d24a6b359c@xs4all.nl> <5C375452.8050800@oracle.com> Message-ID: <7bccacb8e9eb4c6248af48318eb3e20f@xs4all.nl> 2.1) and 2.2) is the srcgen.sh thing. I'm not sure why com/sun/tools/jextract/jclang-ffi/TestJextractFFI.java failed, it looks like your link to the test report got stripped from the email? I changed that test so that it grabs the libclang file name from a system property since it differs on the different platforms ("clang" vs "libclang"). I'll work on an updated webrev for 1) and 2.1) 2.2) in the mean time. Jorn Sundararajan Athijegannathan schreef op 2019-01-10 15:18: > There are two issues: > > 1) I had to change make/test/JtregNativeJdk.gmk > > + ifeq ($(OPENJDK_TARGET_OS), windows) > + # windows flags > + BUILD_JDK_JTREG_LIBRARIES_CFLAGS_libVector := /arch:AVX > + BUILD_JDK_JTREG_LIBRARIES_CFLAGS_libUpcall := /arch:AVX > + BUILD_JDK_JTREG_LIBRARIES_CFLAGS_libGlobalVariable := > /arch:AVX > + else > + # Enable AVX for vector (Long*) tests > + BUILD_JDK_JTREG_LIBRARIES_CFLAGS_libVector := -mavx > + BUILD_JDK_JTREG_LIBRARIES_CFLAGS_libUpcall := -mavx > + BUILD_JDK_JTREG_LIBRARIES_CFLAGS_libGlobalVariable := -mavx > + endif > > Because I was getting "clang: error: no such file or directory: > '/arch:AVX' on Mac. With the above change, I managed to get passed > that failure. > > 2) Three tests failed. > > com/sun/tools/jextract/TestDowncall.java [1]: > com/sun/tools/jextract/TestUpcall.java [2]: > com/sun/tools/jextract/jclang-ffi/TestJextractFFI.java [3]: > > I think that is srcgen.sh issue you're talking about. If files have to > be generated, either we need to automate or pre-generate and be > included as part of patch. We cannot expect shell scripts to be > manually run before tests (in nightly testing for eg). > > Please incorporate the above changes and send RFR against the test > changes bug. Also please check my change above is fine on Windows. > > Thanks, > -Sundar > > On 10/01/19, 7:24 PM, Jorn Vernee wrote: > >>> PS. I'm running the test patch on Mac and Linux to make sure that >>> the >>> testing is fine on other platforms. >> >> Thanks. >> >> One thing I forgot to mention; the generated source files for >> UpcallTest and DowncallTest needed changes, so I've removed them and >> created the srcgen.sh script in test/jdk/com/sun/tools/jextract to >> generate the source files instead. I had this hooked up through >> jtreg, but since the native files are compiled before the sources >> are generated this didn't generate the files in time, and I never >> got around to figuring out how to hook the generation into the build >> system. >> >> So, just be aware that after applying the test-tweaks patch you >> still have to run that srcgen.sh script (relies on java 11 on PATH) >> to generate the test sources, since the patch deletes them. >> >> Jorn >> >> Sundararajan Athijegannathan schreef op 2019-01-10 14:26: >> The 4 bugs (including the preexisting one) are as follows: >> >> 1) Library.getDefault() does not work on Windows >> >> https://bugs.openjdk.java.net/browse/JDK-8211060 >> >> 2) Binder changes are needed to support paname foreign API on >> Windows >> >> https://bugs.openjdk.java.net/browse/JDK-8216485 >> >> 3) jextract LayoutUtils has to be modified for Windows >> >> https://bugs.openjdk.java.net/browse/JDK-8216484 >> >> 4) jextract tests have to be modified to run on Windows platform >> >> https://bugs.openjdk.java.net/browse/JDK-8216483 >> >> PS. I'm running the test patch on Mac and Linux to make sure that >> the >> testing is fine on other platforms. >> >> Thanks, >> -Sundar >> >> On 10/01/19, 6:14 PM, Maurizio Cimadamore wrote: >> Hi Jorn, >> Thanks for the hard work; we are in the process of reviewing the >> changes. Here's what we'll do: >> >> 1) we will create issues for each of the patches (Sundar is doing >> that as we speak) >> 2) we will try to get easier changes in sooner rather than later >> (that is, patches 1, 3, 4) >> 3) in parallel, I'll be doing an in-depth review of your binder >> changes and perhaps start a separate thread on that >> >> Thanks >> Maurizio >> >> On 09/01/2019 21:17, Jorn Vernee wrote: >> >> Hi, >> >> I have fixed the last few failing tests on Windows so this is an RFR >> for adding Windows support to Panama FFI. There's a lot of changes >> so I've split it into 4 patches. We should probably focus on one of >> these one at a time, but I wanted to include all the changes needed >> to pass the tests at least. It's probably best to start with [2], >> since that implements the core functionality. >> >> Default lib fixes [1] (needed for the default library to work on >> Windows) >> Hotspot, binder tweaks [2] (Changes to the stub generation and >> shared binder code. Includes the Windows implementation of >> SystemABI) >> jextract tweaks [3] (minor jextract portability changes) >> test tweaks [4] (changes needed to run, and pass the tests on >> Windows) >> >> There are some tests that are disabled: >> - EmptyStructTest; Empty structs are a GCC extension. >> - BadBitfieldTest; this is testing for a problem the MSVC ABI >> doesn't seem to have. >> - StdioTest; relies on standard system header path, which on >> Windows is more tricky to find. >> - ComplexTest; Windows complex types work very differently in C >> source, so the current test doesn't compile. >> - LongDoubleTest; long double is only 64 bits on Windows. This >> might need a replacement test still, but it should work the same as >> `double` on Windows. >> - Getpid; No getpid on Windows in the C standard. >> - UnixSystem; Windows is not unix. >> >> (Some tests specifically for Windows could be added, but I'm leaving >> that for a different RFR) >> >> Changes for [1]: >> - replace default library handle with a findEntryInProcess since >> Windows doesn't have an RTLD_DEFAULT equivalent. >> >> Changes for [2]: >> - changed upcall/downcall stub generation to follow the MS ABI on >> windows (using compiler switches) >> - added SKIPs to shuffle recipe generation when e.g. the first >> float is passed in the second float register (which can be the case >> on windows). >> - moved boxValue and unboxValue into SystemABI, since they turned >> out to be ABI specific, and made tweaks to the windows version for >> handeling structs being passed by value. >> - moved upcall stub natives into shared ABI package. >> - Let each ABI have it's own VarargsInvoker implementation, since >> we need to propagate more info about which arguments are variadic on >> windows. >> - CallingSequence::storageOffset is no longer relying on >> Storage::getStorageIndex, since that is not the same as what's >> actually needed; being the index of the binding. (the 2 can differ >> on windows, but not on sysv). >> - Added Windows ABI implementation classes based on the SysV ones. >> >> - Let UniversalUpcallHandler write the in-memory-return pointer to >> RAX since that is required on Windows (and I think this change is >> portable). >> >> Changes for [3]: >> - instead of hardcoding the size of `long` `unsigned long` and >> `long double`, delegate to libclang to tell us the size, since these >> types have different sizes on the 2 ABIs. (This is pretty ad-hoc, so >> probably needs to be improved. But, I have little experience with >> the jextract code, so I'm open to suggestion of how to better fix >> this) >> >> Changes for [4]: >> - added dll exports for native symbols using compiler switches. >> - added test for small (fits-in-register) struct by-value passing, >> since windows has special handling for those. >> - removed depencies on POSIX functions from some tests where it >> wasn't really needed, so they could run on windows. >> - propagate clang library name to TestJextractFFI, since on >> Windows we need to load "libclang". >> - let Runner test use windows specific files to compare with for >> bitfields.h and simple.h >> - use `long long` instead of `long` in some cases since `long >> long` is 64 bits on SysV and MSx64 >> >> Thanks, >> Jorn >> >> [1] : >> > http://cr.openjdk.java.net/~jvernee/panama/webrevs/8211060/webrev.01/ >> >> [2] : >> > http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows/webrev.03/ >> >> [3] : >> > http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_jextract/webrev.00/ >> [4] : >> > http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_tests/webrev.02/ > > > Links: > ------ > [1] > http://webmail.xs4all.nl/../../../test-support/jtreg_test_jdk_jdk_foreign/com/sun/tools/jextract/TestDowncall.jtr > [2] > http://webmail.xs4all.nl/../../../test-support/jtreg_test_jdk_jdk_foreign/com/sun/tools/jextract/TestUpcall.jtr > [3] > http://webmail.xs4all.nl/../../../test-support/jtreg_test_jdk_jdk_foreign/com/sun/tools/jextract/jclang-ffi/TestJextractFFI.jtr From sundararajan.athijegannathan at oracle.com Thu Jan 10 15:19:44 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Thu, 10 Jan 2019 20:49:44 +0530 Subject: [foreign] RFR: Windows support In-Reply-To: <7bccacb8e9eb4c6248af48318eb3e20f@xs4all.nl> References: <8f136c5aa0343d7d41a294ce84457a4e@xs4all.nl> <04d8d9e4-5251-01c1-bc35-465123b603b8@oracle.com> <5C3747FC.7070501@oracle.com> <3f4899ea4f4bc1bad612e4d24a6b359c@xs4all.nl> <5C375452.8050800@oracle.com> <7bccacb8e9eb4c6248af48318eb3e20f@xs4all.nl> Message-ID: <5C376290.3070106@oracle.com> clang.dll / libclang.so / libclang.dylib is already taken care by native library loading mechanism. There should not be any need for System property. The test failure stack trace on Mac is as follows: Loading LibClang FFI Exception in thread "main" java.lang.UnsatisfiedLinkError: no libclang in java.library.path: [/Users/SATHIJEG/bin/llvm/lib] at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2700) at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:849) at java.base/java.lang.Runtime.loadLibrary(Runtime.java:829) at java.base/java.lang.System$2.loadLibrary(System.java:2280) at java.base/jdk.internal.foreign.LibrariesHelper.loadLibrary(LibrariesHelper.java:169) at java.base/java.foreign.Libraries.loadLibrary(Libraries.java:100) at jdk.internal.clang/jdk.internal.clang.LibClang.(LibClang.java:43) at jdk.jextract/com.sun.tools.jextract.parser.Parser.parse(Parser.java:71) at jdk.jextract/com.sun.tools.jextract.JextractTool.processHeaders(JextractTool.java:58) at jdk.jextract/com.sun.tools.jextract.Main.run(Main.java:264) at jdk.jextract/com.sun.tools.jextract.Main.main(Main.java:334) -Sundar On 10/01/19, 7:54 PM, Jorn Vernee wrote: > 2.1) and 2.2) is the srcgen.sh thing. I'm not sure why > com/sun/tools/jextract/jclang-ffi/TestJextractFFI.java failed, it > looks like your link to the test report got stripped from the email? I > changed that test so that it grabs the libclang file name from a > system property since it differs on the different platforms ("clang" > vs "libclang"). > > I'll work on an updated webrev for 1) and 2.1) 2.2) in the mean time. > > Jorn > > Sundararajan Athijegannathan schreef op 2019-01-10 15:18: >> There are two issues: >> >> 1) I had to change make/test/JtregNativeJdk.gmk >> >> + ifeq ($(OPENJDK_TARGET_OS), windows) >> + # windows flags >> + BUILD_JDK_JTREG_LIBRARIES_CFLAGS_libVector := /arch:AVX >> + BUILD_JDK_JTREG_LIBRARIES_CFLAGS_libUpcall := /arch:AVX >> + BUILD_JDK_JTREG_LIBRARIES_CFLAGS_libGlobalVariable := >> /arch:AVX >> + else >> + # Enable AVX for vector (Long*) tests >> + BUILD_JDK_JTREG_LIBRARIES_CFLAGS_libVector := -mavx >> + BUILD_JDK_JTREG_LIBRARIES_CFLAGS_libUpcall := -mavx >> + BUILD_JDK_JTREG_LIBRARIES_CFLAGS_libGlobalVariable := -mavx >> + endif >> >> Because I was getting "clang: error: no such file or directory: >> '/arch:AVX' on Mac. With the above change, I managed to get passed >> that failure. >> >> 2) Three tests failed. >> >> com/sun/tools/jextract/TestDowncall.java [1]: >> com/sun/tools/jextract/TestUpcall.java [2]: >> com/sun/tools/jextract/jclang-ffi/TestJextractFFI.java [3]: >> >> I think that is srcgen.sh issue you're talking about. If files have to >> be generated, either we need to automate or pre-generate and be >> included as part of patch. We cannot expect shell scripts to be >> manually run before tests (in nightly testing for eg). >> >> Please incorporate the above changes and send RFR against the test >> changes bug. Also please check my change above is fine on Windows. >> >> Thanks, >> -Sundar >> >> On 10/01/19, 7:24 PM, Jorn Vernee wrote: >> >>>> PS. I'm running the test patch on Mac and Linux to make sure that >>>> the >>>> testing is fine on other platforms. >>> >>> Thanks. >>> >>> One thing I forgot to mention; the generated source files for >>> UpcallTest and DowncallTest needed changes, so I've removed them and >>> created the srcgen.sh script in test/jdk/com/sun/tools/jextract to >>> generate the source files instead. I had this hooked up through >>> jtreg, but since the native files are compiled before the sources >>> are generated this didn't generate the files in time, and I never >>> got around to figuring out how to hook the generation into the build >>> system. >>> >>> So, just be aware that after applying the test-tweaks patch you >>> still have to run that srcgen.sh script (relies on java 11 on PATH) >>> to generate the test sources, since the patch deletes them. >>> >>> Jorn >>> >>> Sundararajan Athijegannathan schreef op 2019-01-10 14:26: >>> The 4 bugs (including the preexisting one) are as follows: >>> >>> 1) Library.getDefault() does not work on Windows >>> >>> https://bugs.openjdk.java.net/browse/JDK-8211060 >>> >>> 2) Binder changes are needed to support paname foreign API on >>> Windows >>> >>> https://bugs.openjdk.java.net/browse/JDK-8216485 >>> >>> 3) jextract LayoutUtils has to be modified for Windows >>> >>> https://bugs.openjdk.java.net/browse/JDK-8216484 >>> >>> 4) jextract tests have to be modified to run on Windows platform >>> >>> https://bugs.openjdk.java.net/browse/JDK-8216483 >>> >>> PS. I'm running the test patch on Mac and Linux to make sure that >>> the >>> testing is fine on other platforms. >>> >>> Thanks, >>> -Sundar >>> >>> On 10/01/19, 6:14 PM, Maurizio Cimadamore wrote: >>> Hi Jorn, >>> Thanks for the hard work; we are in the process of reviewing the >>> changes. Here's what we'll do: >>> >>> 1) we will create issues for each of the patches (Sundar is doing >>> that as we speak) >>> 2) we will try to get easier changes in sooner rather than later >>> (that is, patches 1, 3, 4) >>> 3) in parallel, I'll be doing an in-depth review of your binder >>> changes and perhaps start a separate thread on that >>> >>> Thanks >>> Maurizio >>> >>> On 09/01/2019 21:17, Jorn Vernee wrote: >>> >>> Hi, >>> >>> I have fixed the last few failing tests on Windows so this is an RFR >>> for adding Windows support to Panama FFI. There's a lot of changes >>> so I've split it into 4 patches. We should probably focus on one of >>> these one at a time, but I wanted to include all the changes needed >>> to pass the tests at least. It's probably best to start with [2], >>> since that implements the core functionality. >>> >>> Default lib fixes [1] (needed for the default library to work on >>> Windows) >>> Hotspot, binder tweaks [2] (Changes to the stub generation and >>> shared binder code. Includes the Windows implementation of >>> SystemABI) >>> jextract tweaks [3] (minor jextract portability changes) >>> test tweaks [4] (changes needed to run, and pass the tests on >>> Windows) >>> >>> There are some tests that are disabled: >>> - EmptyStructTest; Empty structs are a GCC extension. >>> - BadBitfieldTest; this is testing for a problem the MSVC ABI >>> doesn't seem to have. >>> - StdioTest; relies on standard system header path, which on >>> Windows is more tricky to find. >>> - ComplexTest; Windows complex types work very differently in C >>> source, so the current test doesn't compile. >>> - LongDoubleTest; long double is only 64 bits on Windows. This >>> might need a replacement test still, but it should work the same as >>> `double` on Windows. >>> - Getpid; No getpid on Windows in the C standard. >>> - UnixSystem; Windows is not unix. >>> >>> (Some tests specifically for Windows could be added, but I'm leaving >>> that for a different RFR) >>> >>> Changes for [1]: >>> - replace default library handle with a findEntryInProcess since >>> Windows doesn't have an RTLD_DEFAULT equivalent. >>> >>> Changes for [2]: >>> - changed upcall/downcall stub generation to follow the MS ABI on >>> windows (using compiler switches) >>> - added SKIPs to shuffle recipe generation when e.g. the first >>> float is passed in the second float register (which can be the case >>> on windows). >>> - moved boxValue and unboxValue into SystemABI, since they turned >>> out to be ABI specific, and made tweaks to the windows version for >>> handeling structs being passed by value. >>> - moved upcall stub natives into shared ABI package. >>> - Let each ABI have it's own VarargsInvoker implementation, since >>> we need to propagate more info about which arguments are variadic on >>> windows. >>> - CallingSequence::storageOffset is no longer relying on >>> Storage::getStorageIndex, since that is not the same as what's >>> actually needed; being the index of the binding. (the 2 can differ >>> on windows, but not on sysv). >>> - Added Windows ABI implementation classes based on the SysV ones. >>> >>> - Let UniversalUpcallHandler write the in-memory-return pointer to >>> RAX since that is required on Windows (and I think this change is >>> portable). >>> >>> Changes for [3]: >>> - instead of hardcoding the size of `long` `unsigned long` and >>> `long double`, delegate to libclang to tell us the size, since these >>> types have different sizes on the 2 ABIs. (This is pretty ad-hoc, so >>> probably needs to be improved. But, I have little experience with >>> the jextract code, so I'm open to suggestion of how to better fix >>> this) >>> >>> Changes for [4]: >>> - added dll exports for native symbols using compiler switches. >>> - added test for small (fits-in-register) struct by-value passing, >>> since windows has special handling for those. >>> - removed depencies on POSIX functions from some tests where it >>> wasn't really needed, so they could run on windows. >>> - propagate clang library name to TestJextractFFI, since on >>> Windows we need to load "libclang". >>> - let Runner test use windows specific files to compare with for >>> bitfields.h and simple.h >>> - use `long long` instead of `long` in some cases since `long >>> long` is 64 bits on SysV and MSx64 >>> >>> Thanks, >>> Jorn >>> >>> [1] : >>> >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8211060/webrev.01/ >>> >>> [2] : >>> >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows/webrev.03/ >>> >>> [3] : >>> >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_jextract/webrev.00/ >> >>> [4] : >>> >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_tests/webrev.02/ >> >> >> >> Links: >> ------ >> [1] >> http://webmail.xs4all.nl/../../../test-support/jtreg_test_jdk_jdk_foreign/com/sun/tools/jextract/TestDowncall.jtr >> >> [2] >> http://webmail.xs4all.nl/../../../test-support/jtreg_test_jdk_jdk_foreign/com/sun/tools/jextract/TestUpcall.jtr >> >> [3] >> http://webmail.xs4all.nl/../../../test-support/jtreg_test_jdk_jdk_foreign/com/sun/tools/jextract/jclang-ffi/TestJextractFFI.jtr >> From jbvernee at xs4all.nl Thu Jan 10 15:09:27 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Thu, 10 Jan 2019 16:09:27 +0100 Subject: [foreign] RFR: Windows support In-Reply-To: <5C376290.3070106@oracle.com> References: <8f136c5aa0343d7d41a294ce84457a4e@xs4all.nl> <04d8d9e4-5251-01c1-bc35-465123b603b8@oracle.com> <5C3747FC.7070501@oracle.com> <3f4899ea4f4bc1bad612e4d24a6b359c@xs4all.nl> <5C375452.8050800@oracle.com> <7bccacb8e9eb4c6248af48318eb3e20f@xs4all.nl> <5C376290.3070106@oracle.com> Message-ID: Well, that's the problem, the .dll doesn't follow the standard library name format. i.e. it's not called clang.dll but libclang.dll Jorn Sundararajan Athijegannathan schreef op 2019-01-10 16:19: > clang.dll / libclang.so / libclang.dylib is already taken care by > native library loading mechanism. There should not be any need for > System property. > > The test failure stack trace on Mac is as follows: > > Loading LibClang FFI > Exception in thread "main" java.lang.UnsatisfiedLinkError: no libclang > in java.library.path: [/Users/SATHIJEG/bin/llvm/lib] > at > java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2700) > at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:849) > at java.base/java.lang.Runtime.loadLibrary(Runtime.java:829) > at java.base/java.lang.System$2.loadLibrary(System.java:2280) > at > java.base/jdk.internal.foreign.LibrariesHelper.loadLibrary(LibrariesHelper.java:169) > at > java.base/java.foreign.Libraries.loadLibrary(Libraries.java:100) > at > jdk.internal.clang/jdk.internal.clang.LibClang.(LibClang.java:43) > at > jdk.jextract/com.sun.tools.jextract.parser.Parser.parse(Parser.java:71) > at > jdk.jextract/com.sun.tools.jextract.JextractTool.processHeaders(JextractTool.java:58) > at jdk.jextract/com.sun.tools.jextract.Main.run(Main.java:264) > at jdk.jextract/com.sun.tools.jextract.Main.main(Main.java:334) > > -Sundar > > On 10/01/19, 7:54 PM, Jorn Vernee wrote: > >> 2.1) and 2.2) is the srcgen.sh thing. I'm not sure why >> com/sun/tools/jextract/jclang-ffi/TestJextractFFI.java failed, it >> looks like your link to the test report got stripped from the email? >> I changed that test so that it grabs the libclang file name from a >> system property since it differs on the different platforms ("clang" >> vs "libclang"). >> >> I'll work on an updated webrev for 1) and 2.1) 2.2) in the mean >> time. >> >> Jorn >> >> Sundararajan Athijegannathan schreef op 2019-01-10 15:18: >> There are two issues: >> >> 1) I had to change make/test/JtregNativeJdk.gmk >> >> + ifeq ($(OPENJDK_TARGET_OS), windows) >> + # windows flags >> + BUILD_JDK_JTREG_LIBRARIES_CFLAGS_libVector := /arch:AVX >> + BUILD_JDK_JTREG_LIBRARIES_CFLAGS_libUpcall := /arch:AVX >> + BUILD_JDK_JTREG_LIBRARIES_CFLAGS_libGlobalVariable := >> /arch:AVX >> + else >> + # Enable AVX for vector (Long*) tests >> + BUILD_JDK_JTREG_LIBRARIES_CFLAGS_libVector := -mavx >> + BUILD_JDK_JTREG_LIBRARIES_CFLAGS_libUpcall := -mavx >> + BUILD_JDK_JTREG_LIBRARIES_CFLAGS_libGlobalVariable := -mavx >> >> + endif >> >> Because I was getting "clang: error: no such file or directory: >> '/arch:AVX' on Mac. With the above change, I managed to get passed >> that failure. >> >> 2) Three tests failed. >> >> com/sun/tools/jextract/TestDowncall.java [1]: >> com/sun/tools/jextract/TestUpcall.java [2]: >> com/sun/tools/jextract/jclang-ffi/TestJextractFFI.java [3]: >> >> I think that is srcgen.sh issue you're talking about. If files have >> to >> be generated, either we need to automate or pre-generate and be >> included as part of patch. We cannot expect shell scripts to be >> manually run before tests (in nightly testing for eg). >> >> Please incorporate the above changes and send RFR against the test >> changes bug. Also please check my change above is fine on Windows. >> >> Thanks, >> -Sundar >> >> On 10/01/19, 7:24 PM, Jorn Vernee wrote: >> >> PS. I'm running the test patch on Mac and Linux to make sure that >> the >> testing is fine on other platforms. >> >> Thanks. >> >> One thing I forgot to mention; the generated source files for >> UpcallTest and DowncallTest needed changes, so I've removed them and >> >> created the srcgen.sh script in test/jdk/com/sun/tools/jextract to >> generate the source files instead. I had this hooked up through >> jtreg, but since the native files are compiled before the sources >> are generated this didn't generate the files in time, and I never >> got around to figuring out how to hook the generation into the build >> >> system. >> >> So, just be aware that after applying the test-tweaks patch you >> still have to run that srcgen.sh script (relies on java 11 on PATH) >> to generate the test sources, since the patch deletes them. >> >> Jorn >> >> Sundararajan Athijegannathan schreef op 2019-01-10 14:26: >> The 4 bugs (including the preexisting one) are as follows: >> >> 1) Library.getDefault() does not work on Windows >> >> https://bugs.openjdk.java.net/browse/JDK-8211060 >> >> 2) Binder changes are needed to support paname foreign API on >> Windows >> >> https://bugs.openjdk.java.net/browse/JDK-8216485 >> >> 3) jextract LayoutUtils has to be modified for Windows >> >> https://bugs.openjdk.java.net/browse/JDK-8216484 >> >> 4) jextract tests have to be modified to run on Windows platform >> >> https://bugs.openjdk.java.net/browse/JDK-8216483 >> >> PS. I'm running the test patch on Mac and Linux to make sure that >> the >> testing is fine on other platforms. >> >> Thanks, >> -Sundar >> >> On 10/01/19, 6:14 PM, Maurizio Cimadamore wrote: >> Hi Jorn, >> Thanks for the hard work; we are in the process of reviewing the >> changes. Here's what we'll do: >> >> 1) we will create issues for each of the patches (Sundar is doing >> that as we speak) >> 2) we will try to get easier changes in sooner rather than later >> (that is, patches 1, 3, 4) >> 3) in parallel, I'll be doing an in-depth review of your binder >> changes and perhaps start a separate thread on that >> >> Thanks >> Maurizio >> >> On 09/01/2019 21:17, Jorn Vernee wrote: >> >> Hi, >> >> I have fixed the last few failing tests on Windows so this is an RFR >> >> for adding Windows support to Panama FFI. There's a lot of changes >> so I've split it into 4 patches. We should probably focus on one of >> these one at a time, but I wanted to include all the changes needed >> to pass the tests at least. It's probably best to start with [2], >> since that implements the core functionality. >> >> Default lib fixes [1] (needed for the default library to work on >> Windows) >> Hotspot, binder tweaks [2] (Changes to the stub generation and >> shared binder code. Includes the Windows implementation of >> SystemABI) >> jextract tweaks [3] (minor jextract portability changes) >> test tweaks [4] (changes needed to run, and pass the tests on >> Windows) >> >> There are some tests that are disabled: >> - EmptyStructTest; Empty structs are a GCC extension. >> - BadBitfieldTest; this is testing for a problem the MSVC ABI >> doesn't seem to have. >> - StdioTest; relies on standard system header path, which on >> Windows is more tricky to find. >> - ComplexTest; Windows complex types work very differently in C >> source, so the current test doesn't compile. >> - LongDoubleTest; long double is only 64 bits on Windows. This >> might need a replacement test still, but it should work the same as >> `double` on Windows. >> - Getpid; No getpid on Windows in the C standard. >> - UnixSystem; Windows is not unix. >> >> (Some tests specifically for Windows could be added, but I'm leaving >> >> that for a different RFR) >> >> Changes for [1]: >> - replace default library handle with a findEntryInProcess since >> Windows doesn't have an RTLD_DEFAULT equivalent. >> >> Changes for [2]: >> - changed upcall/downcall stub generation to follow the MS ABI on >> windows (using compiler switches) >> - added SKIPs to shuffle recipe generation when e.g. the first >> float is passed in the second float register (which can be the case >> on windows). >> - moved boxValue and unboxValue into SystemABI, since they turned >> out to be ABI specific, and made tweaks to the windows version for >> handeling structs being passed by value. >> - moved upcall stub natives into shared ABI package. >> - Let each ABI have it's own VarargsInvoker implementation, since >> we need to propagate more info about which arguments are variadic on >> >> windows. >> - CallingSequence::storageOffset is no longer relying on >> Storage::getStorageIndex, since that is not the same as what's >> actually needed; being the index of the binding. (the 2 can differ >> on windows, but not on sysv). >> - Added Windows ABI implementation classes based on the SysV ones. >> >> - Let UniversalUpcallHandler write the in-memory-return pointer to >> RAX since that is required on Windows (and I think this change is >> portable). >> >> Changes for [3]: >> - instead of hardcoding the size of `long` `unsigned long` and >> `long double`, delegate to libclang to tell us the size, since these >> >> types have different sizes on the 2 ABIs. (This is pretty ad-hoc, so >> >> probably needs to be improved. But, I have little experience with >> the jextract code, so I'm open to suggestion of how to better fix >> this) >> >> Changes for [4]: >> - added dll exports for native symbols using compiler switches. >> - added test for small (fits-in-register) struct by-value passing, >> since windows has special handling for those. >> - removed depencies on POSIX functions from some tests where it >> wasn't really needed, so they could run on windows. >> - propagate clang library name to TestJextractFFI, since on >> Windows we need to load "libclang". >> - let Runner test use windows specific files to compare with for >> bitfields.h and simple.h >> - use `long long` instead of `long` in some cases since `long >> long` is 64 bits on SysV and MSx64 >> >> Thanks, >> Jorn >> >> [1] : > http://cr.openjdk.java.net/~jvernee/panama/webrevs/8211060/webrev.01/ > > >> [2] : > http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows/webrev.03/ > > >> [3] : > > http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_jextract/webrev.00/ > > >> [4] : > > http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_tests/webrev.02/ > > > Links: > ------ > [1] > http://webmail.xs4all.nl/../../../test-support/jtreg_test_jdk_jdk_foreign/com/sun/tools/jextract/TestDowncall.jtr > > [2] > http://webmail.xs4all.nl/../../../test-support/jtreg_test_jdk_jdk_foreign/com/sun/tools/jextract/TestUpcall.jtr > > [3] > http://webmail.xs4all.nl/../../../test-support/jtreg_test_jdk_jdk_foreign/com/sun/tools/jextract/jclang-ffi/TestJextractFFI.jtr From sundararajan.athijegannathan at oracle.com Thu Jan 10 15:28:28 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Thu, 10 Jan 2019 20:58:28 +0530 Subject: [foreign] RFR: Windows support In-Reply-To: References: <8f136c5aa0343d7d41a294ce84457a4e@xs4all.nl> <04d8d9e4-5251-01c1-bc35-465123b603b8@oracle.com> <5C3747FC.7070501@oracle.com> <3f4899ea4f4bc1bad612e4d24a6b359c@xs4all.nl> <5C375452.8050800@oracle.com> <7bccacb8e9eb4c6248af48318eb3e20f@xs4all.nl> <5C376290.3070106@oracle.com> Message-ID: <5C37649C.5030506@oracle.com> Well I'd say we should have Windows specific solution then. i.e., not a system property for all platforms - just for Windows. -Sundar On 10/01/19, 8:39 PM, Jorn Vernee wrote: > Well, that's the problem, the .dll doesn't follow the standard library > name format. i.e. it's not called clang.dll but libclang.dll > > Jorn > > Sundararajan Athijegannathan schreef op 2019-01-10 16:19: >> clang.dll / libclang.so / libclang.dylib is already taken care by >> native library loading mechanism. There should not be any need for >> System property. >> >> The test failure stack trace on Mac is as follows: >> >> Loading LibClang FFI >> Exception in thread "main" java.lang.UnsatisfiedLinkError: no libclang >> in java.library.path: [/Users/SATHIJEG/bin/llvm/lib] >> at >> java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2700) >> at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:849) >> at java.base/java.lang.Runtime.loadLibrary(Runtime.java:829) >> at java.base/java.lang.System$2.loadLibrary(System.java:2280) >> at >> java.base/jdk.internal.foreign.LibrariesHelper.loadLibrary(LibrariesHelper.java:169) >> >> at >> java.base/java.foreign.Libraries.loadLibrary(Libraries.java:100) >> at >> jdk.internal.clang/jdk.internal.clang.LibClang.(LibClang.java:43) >> >> at >> jdk.jextract/com.sun.tools.jextract.parser.Parser.parse(Parser.java:71) >> at >> jdk.jextract/com.sun.tools.jextract.JextractTool.processHeaders(JextractTool.java:58) >> >> at jdk.jextract/com.sun.tools.jextract.Main.run(Main.java:264) >> at jdk.jextract/com.sun.tools.jextract.Main.main(Main.java:334) >> >> -Sundar >> >> On 10/01/19, 7:54 PM, Jorn Vernee wrote: >> >>> 2.1) and 2.2) is the srcgen.sh thing. I'm not sure why >>> com/sun/tools/jextract/jclang-ffi/TestJextractFFI.java failed, it >>> looks like your link to the test report got stripped from the email? >>> I changed that test so that it grabs the libclang file name from a >>> system property since it differs on the different platforms ("clang" >>> vs "libclang"). >>> >>> I'll work on an updated webrev for 1) and 2.1) 2.2) in the mean >>> time. >>> >>> Jorn >>> >>> Sundararajan Athijegannathan schreef op 2019-01-10 15:18: >>> There are two issues: >>> >>> 1) I had to change make/test/JtregNativeJdk.gmk >>> >>> + ifeq ($(OPENJDK_TARGET_OS), windows) >>> + # windows flags >>> + BUILD_JDK_JTREG_LIBRARIES_CFLAGS_libVector := /arch:AVX >>> + BUILD_JDK_JTREG_LIBRARIES_CFLAGS_libUpcall := /arch:AVX >>> + BUILD_JDK_JTREG_LIBRARIES_CFLAGS_libGlobalVariable := >>> /arch:AVX >>> + else >>> + # Enable AVX for vector (Long*) tests >>> + BUILD_JDK_JTREG_LIBRARIES_CFLAGS_libVector := -mavx >>> + BUILD_JDK_JTREG_LIBRARIES_CFLAGS_libUpcall := -mavx >>> + BUILD_JDK_JTREG_LIBRARIES_CFLAGS_libGlobalVariable := -mavx >>> >>> + endif >>> >>> Because I was getting "clang: error: no such file or directory: >>> '/arch:AVX' on Mac. With the above change, I managed to get passed >>> that failure. >>> >>> 2) Three tests failed. >>> >>> com/sun/tools/jextract/TestDowncall.java [1]: >>> com/sun/tools/jextract/TestUpcall.java [2]: >>> com/sun/tools/jextract/jclang-ffi/TestJextractFFI.java [3]: >>> >>> I think that is srcgen.sh issue you're talking about. If files have >>> to >>> be generated, either we need to automate or pre-generate and be >>> included as part of patch. We cannot expect shell scripts to be >>> manually run before tests (in nightly testing for eg). >>> >>> Please incorporate the above changes and send RFR against the test >>> changes bug. Also please check my change above is fine on Windows. >>> >>> Thanks, >>> -Sundar >>> >>> On 10/01/19, 7:24 PM, Jorn Vernee wrote: >>> >>> PS. I'm running the test patch on Mac and Linux to make sure that >>> the >>> testing is fine on other platforms. >>> >>> Thanks. >>> >>> One thing I forgot to mention; the generated source files for >>> UpcallTest and DowncallTest needed changes, so I've removed them and >>> >>> created the srcgen.sh script in test/jdk/com/sun/tools/jextract to >>> generate the source files instead. I had this hooked up through >>> jtreg, but since the native files are compiled before the sources >>> are generated this didn't generate the files in time, and I never >>> got around to figuring out how to hook the generation into the build >>> >>> system. >>> >>> So, just be aware that after applying the test-tweaks patch you >>> still have to run that srcgen.sh script (relies on java 11 on PATH) >>> to generate the test sources, since the patch deletes them. >>> >>> Jorn >>> >>> Sundararajan Athijegannathan schreef op 2019-01-10 14:26: >>> The 4 bugs (including the preexisting one) are as follows: >>> >>> 1) Library.getDefault() does not work on Windows >>> >>> https://bugs.openjdk.java.net/browse/JDK-8211060 >>> >>> 2) Binder changes are needed to support paname foreign API on >>> Windows >>> >>> https://bugs.openjdk.java.net/browse/JDK-8216485 >>> >>> 3) jextract LayoutUtils has to be modified for Windows >>> >>> https://bugs.openjdk.java.net/browse/JDK-8216484 >>> >>> 4) jextract tests have to be modified to run on Windows platform >>> >>> https://bugs.openjdk.java.net/browse/JDK-8216483 >>> >>> PS. I'm running the test patch on Mac and Linux to make sure that >>> the >>> testing is fine on other platforms. >>> >>> Thanks, >>> -Sundar >>> >>> On 10/01/19, 6:14 PM, Maurizio Cimadamore wrote: >>> Hi Jorn, >>> Thanks for the hard work; we are in the process of reviewing the >>> changes. Here's what we'll do: >>> >>> 1) we will create issues for each of the patches (Sundar is doing >>> that as we speak) >>> 2) we will try to get easier changes in sooner rather than later >>> (that is, patches 1, 3, 4) >>> 3) in parallel, I'll be doing an in-depth review of your binder >>> changes and perhaps start a separate thread on that >>> >>> Thanks >>> Maurizio >>> >>> On 09/01/2019 21:17, Jorn Vernee wrote: >>> >>> Hi, >>> >>> I have fixed the last few failing tests on Windows so this is an RFR >>> >>> for adding Windows support to Panama FFI. There's a lot of changes >>> so I've split it into 4 patches. We should probably focus on one of >>> these one at a time, but I wanted to include all the changes needed >>> to pass the tests at least. It's probably best to start with [2], >>> since that implements the core functionality. >>> >>> Default lib fixes [1] (needed for the default library to work on >>> Windows) >>> Hotspot, binder tweaks [2] (Changes to the stub generation and >>> shared binder code. Includes the Windows implementation of >>> SystemABI) >>> jextract tweaks [3] (minor jextract portability changes) >>> test tweaks [4] (changes needed to run, and pass the tests on >>> Windows) >>> >>> There are some tests that are disabled: >>> - EmptyStructTest; Empty structs are a GCC extension. >>> - BadBitfieldTest; this is testing for a problem the MSVC ABI >>> doesn't seem to have. >>> - StdioTest; relies on standard system header path, which on >>> Windows is more tricky to find. >>> - ComplexTest; Windows complex types work very differently in C >>> source, so the current test doesn't compile. >>> - LongDoubleTest; long double is only 64 bits on Windows. This >>> might need a replacement test still, but it should work the same as >>> `double` on Windows. >>> - Getpid; No getpid on Windows in the C standard. >>> - UnixSystem; Windows is not unix. >>> >>> (Some tests specifically for Windows could be added, but I'm leaving >>> >>> that for a different RFR) >>> >>> Changes for [1]: >>> - replace default library handle with a findEntryInProcess since >>> Windows doesn't have an RTLD_DEFAULT equivalent. >>> >>> Changes for [2]: >>> - changed upcall/downcall stub generation to follow the MS ABI on >>> windows (using compiler switches) >>> - added SKIPs to shuffle recipe generation when e.g. the first >>> float is passed in the second float register (which can be the case >>> on windows). >>> - moved boxValue and unboxValue into SystemABI, since they turned >>> out to be ABI specific, and made tweaks to the windows version for >>> handeling structs being passed by value. >>> - moved upcall stub natives into shared ABI package. >>> - Let each ABI have it's own VarargsInvoker implementation, since >>> we need to propagate more info about which arguments are variadic on >>> >>> windows. >>> - CallingSequence::storageOffset is no longer relying on >>> Storage::getStorageIndex, since that is not the same as what's >>> actually needed; being the index of the binding. (the 2 can differ >>> on windows, but not on sysv). >>> - Added Windows ABI implementation classes based on the SysV ones. >>> >>> - Let UniversalUpcallHandler write the in-memory-return pointer to >>> RAX since that is required on Windows (and I think this change is >>> portable). >>> >>> Changes for [3]: >>> - instead of hardcoding the size of `long` `unsigned long` and >>> `long double`, delegate to libclang to tell us the size, since these >>> >>> types have different sizes on the 2 ABIs. (This is pretty ad-hoc, so >>> >>> probably needs to be improved. But, I have little experience with >>> the jextract code, so I'm open to suggestion of how to better fix >>> this) >>> >>> Changes for [4]: >>> - added dll exports for native symbols using compiler switches. >>> - added test for small (fits-in-register) struct by-value passing, >>> since windows has special handling for those. >>> - removed depencies on POSIX functions from some tests where it >>> wasn't really needed, so they could run on windows. >>> - propagate clang library name to TestJextractFFI, since on >>> Windows we need to load "libclang". >>> - let Runner test use windows specific files to compare with for >>> bitfields.h and simple.h >>> - use `long long` instead of `long` in some cases since `long >>> long` is 64 bits on SysV and MSx64 >>> >>> Thanks, >>> Jorn >>> >>> [1] : >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8211060/webrev.01/ >> >> >>> [2] : >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows/webrev.03/ >> >> >>> [3] : >> >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_jextract/webrev.00/ >> >> >> >>> [4] : >> >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_tests/webrev.02/ >> >> >> >> Links: >> ------ >> [1] >> http://webmail.xs4all.nl/../../../test-support/jtreg_test_jdk_jdk_foreign/com/sun/tools/jextract/TestDowncall.jtr >> >> >> [2] >> http://webmail.xs4all.nl/../../../test-support/jtreg_test_jdk_jdk_foreign/com/sun/tools/jextract/TestUpcall.jtr >> >> >> [3] >> http://webmail.xs4all.nl/../../../test-support/jtreg_test_jdk_jdk_foreign/com/sun/tools/jextract/jclang-ffi/TestJextractFFI.jtr >> From jbvernee at xs4all.nl Thu Jan 10 15:11:40 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Thu, 10 Jan 2019 16:11:40 +0100 Subject: [foreign] RFR: Windows support In-Reply-To: References: <8f136c5aa0343d7d41a294ce84457a4e@xs4all.nl> <04d8d9e4-5251-01c1-bc35-465123b603b8@oracle.com> <5C3747FC.7070501@oracle.com> <3f4899ea4f4bc1bad612e4d24a6b359c@xs4all.nl> <5C375452.8050800@oracle.com> <7bccacb8e9eb4c6248af48318eb3e20f@xs4all.nl> <5C376290.3070106@oracle.com> Message-ID: <0e16ecc365ca3eb14601e0280fbfa5d7@xs4all.nl> I also see my mistake now. I think for now I can just check if the test is being ran on Windows and then change the lib name to "libclang" instead of trying to be cute and deriving the name from the filename. Jorn Jorn Vernee schreef op 2019-01-10 16:09: > Well, that's the problem, the .dll doesn't follow the standard library > name format. i.e. it's not called clang.dll but libclang.dll > > Jorn > > Sundararajan Athijegannathan schreef op 2019-01-10 16:19: >> clang.dll / libclang.so / libclang.dylib is already taken care by >> native library loading mechanism. There should not be any need for >> System property. >> >> The test failure stack trace on Mac is as follows: >> >> Loading LibClang FFI >> Exception in thread "main" java.lang.UnsatisfiedLinkError: no libclang >> in java.library.path: [/Users/SATHIJEG/bin/llvm/lib] >> at >> java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2700) >> at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:849) >> at java.base/java.lang.Runtime.loadLibrary(Runtime.java:829) >> at java.base/java.lang.System$2.loadLibrary(System.java:2280) >> at >> java.base/jdk.internal.foreign.LibrariesHelper.loadLibrary(LibrariesHelper.java:169) >> at >> java.base/java.foreign.Libraries.loadLibrary(Libraries.java:100) >> at >> jdk.internal.clang/jdk.internal.clang.LibClang.(LibClang.java:43) >> at >> jdk.jextract/com.sun.tools.jextract.parser.Parser.parse(Parser.java:71) >> at >> jdk.jextract/com.sun.tools.jextract.JextractTool.processHeaders(JextractTool.java:58) >> at jdk.jextract/com.sun.tools.jextract.Main.run(Main.java:264) >> at jdk.jextract/com.sun.tools.jextract.Main.main(Main.java:334) >> >> -Sundar >> >> On 10/01/19, 7:54 PM, Jorn Vernee wrote: >> >>> 2.1) and 2.2) is the srcgen.sh thing. I'm not sure why >>> com/sun/tools/jextract/jclang-ffi/TestJextractFFI.java failed, it >>> looks like your link to the test report got stripped from the email? >>> I changed that test so that it grabs the libclang file name from a >>> system property since it differs on the different platforms ("clang" >>> vs "libclang"). >>> >>> I'll work on an updated webrev for 1) and 2.1) 2.2) in the mean >>> time. >>> >>> Jorn >>> >>> Sundararajan Athijegannathan schreef op 2019-01-10 15:18: >>> There are two issues: >>> >>> 1) I had to change make/test/JtregNativeJdk.gmk >>> >>> + ifeq ($(OPENJDK_TARGET_OS), windows) >>> + # windows flags >>> + BUILD_JDK_JTREG_LIBRARIES_CFLAGS_libVector := /arch:AVX >>> + BUILD_JDK_JTREG_LIBRARIES_CFLAGS_libUpcall := /arch:AVX >>> + BUILD_JDK_JTREG_LIBRARIES_CFLAGS_libGlobalVariable := >>> /arch:AVX >>> + else >>> + # Enable AVX for vector (Long*) tests >>> + BUILD_JDK_JTREG_LIBRARIES_CFLAGS_libVector := -mavx >>> + BUILD_JDK_JTREG_LIBRARIES_CFLAGS_libUpcall := -mavx >>> + BUILD_JDK_JTREG_LIBRARIES_CFLAGS_libGlobalVariable := -mavx >>> >>> + endif >>> >>> Because I was getting "clang: error: no such file or directory: >>> '/arch:AVX' on Mac. With the above change, I managed to get passed >>> that failure. >>> >>> 2) Three tests failed. >>> >>> com/sun/tools/jextract/TestDowncall.java [1]: >>> com/sun/tools/jextract/TestUpcall.java [2]: >>> com/sun/tools/jextract/jclang-ffi/TestJextractFFI.java [3]: >>> >>> I think that is srcgen.sh issue you're talking about. If files have >>> to >>> be generated, either we need to automate or pre-generate and be >>> included as part of patch. We cannot expect shell scripts to be >>> manually run before tests (in nightly testing for eg). >>> >>> Please incorporate the above changes and send RFR against the test >>> changes bug. Also please check my change above is fine on Windows. >>> >>> Thanks, >>> -Sundar >>> >>> On 10/01/19, 7:24 PM, Jorn Vernee wrote: >>> >>> PS. I'm running the test patch on Mac and Linux to make sure that >>> the >>> testing is fine on other platforms. >>> >>> Thanks. >>> >>> One thing I forgot to mention; the generated source files for >>> UpcallTest and DowncallTest needed changes, so I've removed them and >>> >>> created the srcgen.sh script in test/jdk/com/sun/tools/jextract to >>> generate the source files instead. I had this hooked up through >>> jtreg, but since the native files are compiled before the sources >>> are generated this didn't generate the files in time, and I never >>> got around to figuring out how to hook the generation into the build >>> >>> system. >>> >>> So, just be aware that after applying the test-tweaks patch you >>> still have to run that srcgen.sh script (relies on java 11 on PATH) >>> to generate the test sources, since the patch deletes them. >>> >>> Jorn >>> >>> Sundararajan Athijegannathan schreef op 2019-01-10 14:26: >>> The 4 bugs (including the preexisting one) are as follows: >>> >>> 1) Library.getDefault() does not work on Windows >>> >>> https://bugs.openjdk.java.net/browse/JDK-8211060 >>> >>> 2) Binder changes are needed to support paname foreign API on >>> Windows >>> >>> https://bugs.openjdk.java.net/browse/JDK-8216485 >>> >>> 3) jextract LayoutUtils has to be modified for Windows >>> >>> https://bugs.openjdk.java.net/browse/JDK-8216484 >>> >>> 4) jextract tests have to be modified to run on Windows platform >>> >>> https://bugs.openjdk.java.net/browse/JDK-8216483 >>> >>> PS. I'm running the test patch on Mac and Linux to make sure that >>> the >>> testing is fine on other platforms. >>> >>> Thanks, >>> -Sundar >>> >>> On 10/01/19, 6:14 PM, Maurizio Cimadamore wrote: >>> Hi Jorn, >>> Thanks for the hard work; we are in the process of reviewing the >>> changes. Here's what we'll do: >>> >>> 1) we will create issues for each of the patches (Sundar is doing >>> that as we speak) >>> 2) we will try to get easier changes in sooner rather than later >>> (that is, patches 1, 3, 4) >>> 3) in parallel, I'll be doing an in-depth review of your binder >>> changes and perhaps start a separate thread on that >>> >>> Thanks >>> Maurizio >>> >>> On 09/01/2019 21:17, Jorn Vernee wrote: >>> >>> Hi, >>> >>> I have fixed the last few failing tests on Windows so this is an RFR >>> >>> for adding Windows support to Panama FFI. There's a lot of changes >>> so I've split it into 4 patches. We should probably focus on one of >>> these one at a time, but I wanted to include all the changes needed >>> to pass the tests at least. It's probably best to start with [2], >>> since that implements the core functionality. >>> >>> Default lib fixes [1] (needed for the default library to work on >>> Windows) >>> Hotspot, binder tweaks [2] (Changes to the stub generation and >>> shared binder code. Includes the Windows implementation of >>> SystemABI) >>> jextract tweaks [3] (minor jextract portability changes) >>> test tweaks [4] (changes needed to run, and pass the tests on >>> Windows) >>> >>> There are some tests that are disabled: >>> - EmptyStructTest; Empty structs are a GCC extension. >>> - BadBitfieldTest; this is testing for a problem the MSVC ABI >>> doesn't seem to have. >>> - StdioTest; relies on standard system header path, which on >>> Windows is more tricky to find. >>> - ComplexTest; Windows complex types work very differently in C >>> source, so the current test doesn't compile. >>> - LongDoubleTest; long double is only 64 bits on Windows. This >>> might need a replacement test still, but it should work the same as >>> `double` on Windows. >>> - Getpid; No getpid on Windows in the C standard. >>> - UnixSystem; Windows is not unix. >>> >>> (Some tests specifically for Windows could be added, but I'm leaving >>> >>> that for a different RFR) >>> >>> Changes for [1]: >>> - replace default library handle with a findEntryInProcess since >>> Windows doesn't have an RTLD_DEFAULT equivalent. >>> >>> Changes for [2]: >>> - changed upcall/downcall stub generation to follow the MS ABI on >>> windows (using compiler switches) >>> - added SKIPs to shuffle recipe generation when e.g. the first >>> float is passed in the second float register (which can be the case >>> on windows). >>> - moved boxValue and unboxValue into SystemABI, since they turned >>> out to be ABI specific, and made tweaks to the windows version for >>> handeling structs being passed by value. >>> - moved upcall stub natives into shared ABI package. >>> - Let each ABI have it's own VarargsInvoker implementation, since >>> we need to propagate more info about which arguments are variadic on >>> >>> windows. >>> - CallingSequence::storageOffset is no longer relying on >>> Storage::getStorageIndex, since that is not the same as what's >>> actually needed; being the index of the binding. (the 2 can differ >>> on windows, but not on sysv). >>> - Added Windows ABI implementation classes based on the SysV ones. >>> >>> - Let UniversalUpcallHandler write the in-memory-return pointer to >>> RAX since that is required on Windows (and I think this change is >>> portable). >>> >>> Changes for [3]: >>> - instead of hardcoding the size of `long` `unsigned long` and >>> `long double`, delegate to libclang to tell us the size, since these >>> >>> types have different sizes on the 2 ABIs. (This is pretty ad-hoc, so >>> >>> probably needs to be improved. But, I have little experience with >>> the jextract code, so I'm open to suggestion of how to better fix >>> this) >>> >>> Changes for [4]: >>> - added dll exports for native symbols using compiler switches. >>> - added test for small (fits-in-register) struct by-value passing, >>> since windows has special handling for those. >>> - removed depencies on POSIX functions from some tests where it >>> wasn't really needed, so they could run on windows. >>> - propagate clang library name to TestJextractFFI, since on >>> Windows we need to load "libclang". >>> - let Runner test use windows specific files to compare with for >>> bitfields.h and simple.h >>> - use `long long` instead of `long` in some cases since `long >>> long` is 64 bits on SysV and MSx64 >>> >>> Thanks, >>> Jorn >>> >>> [1] : >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8211060/webrev.01/ >> >> >>> [2] : >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows/webrev.03/ >> >> >>> [3] : >> >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_jextract/webrev.00/ >> >> >>> [4] : >> >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_tests/webrev.02/ >> >> >> Links: >> ------ >> [1] >> http://webmail.xs4all.nl/../../../test-support/jtreg_test_jdk_jdk_foreign/com/sun/tools/jextract/TestDowncall.jtr >> >> [2] >> http://webmail.xs4all.nl/../../../test-support/jtreg_test_jdk_jdk_foreign/com/sun/tools/jextract/TestUpcall.jtr >> >> [3] >> http://webmail.xs4all.nl/../../../test-support/jtreg_test_jdk_jdk_foreign/com/sun/tools/jextract/jclang-ffi/TestJextractFFI.jtr From jbvernee at xs4all.nl Thu Jan 10 16:10:37 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Thu, 10 Jan 2019 17:10:37 +0100 Subject: [foreign] RFR 8216483: jextract tests have to be modified to run on Windows platform Message-ID: Hi, Continuation of [1]. updated webrev: http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_tests/webrev.04/ For the TestJextractFFI test I'm now just checking the operating system in the LibClang.java patch file: String libName = System.getProperty("os.name").startsWith("Windows") ? "libclang" : "clang"; Library libclang = Libraries.loadLibrary(MethodHandles.lookup(), libName); The other problem is that the dll is in the bin folder vs lib folder on Windows, and the test was using the linker path which is the lib folder on Windows. There was no build system variable available for the folder that contains the library file, so instead I'm deriving it from the library file path. Cheers, Jorn [1] : https://mail.openjdk.java.net/pipermail/panama-dev/2019-January/003691.html From maurizio.cimadamore at oracle.com Thu Jan 10 17:15:26 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 10 Jan 2019 17:15:26 +0000 Subject: [foreign] RFR 8216485: Binder changes are needed to support paname foreign API on Windows Message-ID: Continuing from [1]. This is a more in depth review of the binder changes. Overall the changes look good, and what I see is more or less what we have discussed over the past few weeks. As a general comment/overarching theme, I think we should try, as much as possible, to reuse code where possible, rather then duplicating it. * I see different idioms at play; for instance, for VarargsInvoker you opted for separate subclasses in the various ABI impl. But for UniversalInvoker you went in with a shared implementation using an helper 'boxer' class. I think should pick a reuse approach (subclassing vs. helpers) and stick with it uniformly. I slightly lean for subclassing. * CallingSequence is still hardwired to the Constants::ARGUMENT_STORAGE_CLASSES in SysV. Not a big deal now (as the two constants classes define enums in a similar way), but I think it's an accident waiting to happen. I think we should eliminate ARGUMENT_STORAGE_CLASSES and RETURN_STORAGE_CLASSES from Constants, and put them in StorageClasses (which is a single source of truth), and then make sure that CallingSequence does not refer to anything inside the specialized 'abi' packages. * I don't think I see a reason for duplicating ArgumentClass - in fact they are the same file * For StorageNames, it is somewhat sad that we have to replicate bigger logic just to define different register arrays. Maybe something that could work is to define registers more explicitly in the abi/x64 package enum Register { ? ? RAX("rax", StorageClass.INTEGER), ??? ... } At which point maybe Constants can also have some EnumSet of integer/vector (and, if needed x87) registers. * StandardCall vs. CallingSequenceBuilderImpl - this is tough; after spending some time staring at the code and the differences, it seems to me that: - ArgumentInfo can be reused across both (the SysV seems more general) - StorageCalculator can be reused across both (again the SysV is more general) - StandardCall::arrangeCall and CallingSequenceBuilderImpl::build are also identical So, the variable parts are/seem to be: examineArgument and classifyType (which is nice, because these are just simple function from Layout to something else), which have ABI dependent logic. Again, we have to decide whether to go with a subclass or helper approach for code reuse. * The alignment routines in CallingSequenceBuilderImpl seem identical to the ones in SysVABI, so perhaps again these could be shared somewhere. That's all I have for now. Cheers Maurizio https://mail.openjdk.java.net/pipermail/panama-dev/2019-January/003680.html From jbvernee at xs4all.nl Thu Jan 10 17:58:52 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Thu, 10 Jan 2019 18:58:52 +0100 Subject: [foreign] RFR 8216485: Binder changes are needed to support paname foreign API on Windows In-Reply-To: References: Message-ID: Hi Maurizio, Thanks for the review, comments inline... Maurizio Cimadamore schreef op 2019-01-10 18:15: > Continuing from [1]. > > This is a more in depth review of the binder changes. > > > Overall the changes look good, and what I see is more or less what we > have discussed over the past few weeks. As a general > comment/overarching theme, I think we should try, as much as possible, > to reuse code where possible, rather then duplicating it. Ok, I thought we would be going the other direction, so I did not spend much time yet on trying to de-duplicate code. I tried to touch the SysV and shared implementation as little as possible to keep maintenance easier, with the intent of de-duplicating code in a later patch if desirable. > * I see different idioms at play; for instance, for VarargsInvoker you > opted for separate subclasses in the various ABI impl. But for > UniversalInvoker you went in with a shared implementation using an > helper 'boxer' class. I think should pick a reuse approach > (subclassing vs. helpers) and stick with it uniformly. I slightly lean > for subclassing. The BOXER is also used by UniversalUpcallHandler, we could go with sub classing for both that and UniversalNativeInvoker, and they would probably delegate to a common implementation. > * CallingSequence is still hardwired to the > Constants::ARGUMENT_STORAGE_CLASSES in SysV. Not a big deal now (as > the two constants classes define enums in a similar way), but I think > it's an accident waiting to happen. I think we should eliminate > ARGUMENT_STORAGE_CLASSES and RETURN_STORAGE_CLASSES from Constants, > and put them in StorageClasses (which is a single source of truth), > and then make sure that CallingSequence does not refer to anything > inside the specialized 'abi' packages. Sounds good. > * I don't think I see a reason for duplicating ArgumentClass - in fact > they are the same file Agreed. > * For StorageNames, it is somewhat sad that we have to replicate > bigger logic just to define different register arrays. Maybe something > that could work is to define registers more explicitly in the abi/x64 > package > > enum Register { > ? ? RAX("rax", StorageClass.INTEGER), > ??? ... > } > > At which point maybe Constants can also have some EnumSet of > integer/vector (and, if needed x87) registers. Agreed. > * StandardCall vs. CallingSequenceBuilderImpl - this is tough; after > spending some time staring at the code and the differences, it seems > to me that: > > - ArgumentInfo can be reused across both (the SysV seems more general) > - StorageCalculator can be reused across both (again the SysV is more > general) > - StandardCall::arrangeCall and CallingSequenceBuilderImpl::build are > also identical > > So, the variable parts are/seem to be: examineArgument and > classifyType (which is nice, because these are just simple function > from Layout to something else), which have ABI dependent logic. Again, > we have to decide whether to go with a subclass or helper approach for > code reuse. - ArgumentInfo/StorageCalculator keep only one counter for the number of registers on Windows while SysV needs 2. - StorageCalculator::addBindings has a special provision for varargs: if(width == 8 && storage.getStorageClass() == StorageClass.VECTOR_ARGUMENT_REGISTER && varargs.contains(arg)) { Storage extraStorage = new Storage(StorageClass.INTEGER_ARGUMENT_REGISTER, nRegs, Constants.INTEGER_REGISTER_SIZE); bindings[StorageClass.INTEGER_ARGUMENT_REGISTER.ordinal()].add(new ArgumentBinding(extraStorage, arg, i * 8)); if (DEBUG) { System.out.println("Argument " + arg.getName() + " will be passed in register " + StorageNames.getStorageName(extraStorage)); } } I don't think those 2 can just be replaced by the SysV implementation, but there's probably some opportunity to de-dupe code in other places. > - StandardCall::arrangeCall and CallingSequenceBuilderImpl::build are > also identical Agree there. This code was moved from AbstractSystemABI to these classes when it got nuked. > * The alignment routines in CallingSequenceBuilderImpl seem identical > to the ones in SysVABI, so perhaps again these could be shared > somewhere. Same here, maybe we should bring back AbstractSystemABI? Or otherwise create an ABI utility class? Jorn > That's all I have for now. > > Cheers > Maurizio > > https://mail.openjdk.java.net/pipermail/panama-dev/2019-January/003680.html From maurizio.cimadamore at oracle.com Thu Jan 10 18:34:39 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 10 Jan 2019 18:34:39 +0000 Subject: [foreign] RFR 8216485: Binder changes are needed to support paname foreign API on Windows In-Reply-To: References: Message-ID: <55f1ade8-c75d-fe78-1b04-1aff15ef8b4a@oracle.com> On 10/01/2019 17:58, Jorn Vernee wrote: > Hi Maurizio, > > Thanks for the review, comments inline... > > Maurizio Cimadamore schreef op 2019-01-10 18:15: >> Continuing from [1]. >> >> This is a more in depth review of the binder changes. >> >> >> Overall the changes look good, and what I see is more or less what we >> have discussed over the past few weeks. As a general >> comment/overarching theme, I think we should try, as much as possible, >> to reuse code where possible, rather then duplicating it. > > Ok, I thought we would be going the other direction, so I did not > spend much time yet on trying to de-duplicate code. I tried to touch > the SysV and shared implementation as little as possible to keep > maintenance easier, with the intent of de-duplicating code in a later > patch if desirable. Yes, I understand. We can also push this as is and work on incremental improvements, but I'd like to unify code paths as possible, as I think maintaining two completely separated code-bases will be trickier in the long run. That said, I like that, at least for prototyping, the new ABI design allows for slotting in new stuff quite easily. > - ArgumentInfo/StorageCalculator keep only one counter for the number > of registers on Windows while SysV needs 2. > - StorageCalculator::addBindings has a special provision for varargs: I might have gone too far here... > > >> * The alignment routines in CallingSequenceBuilderImpl seem identical >> to the ones in SysVABI, so perhaps again these could be shared >> somewhere. > > Same here, maybe we should bring back AbstractSystemABI? Or otherwise > create an ABI utility class? Note that the alignment function is ultimately only used by the CallBuilder. So, if we go for subclassing, I'd say put this in CallingSequenceBuilder - and then have the ABI-dependent details in the Windows/SysV impls. Also, I forgot an important point: please replace "microsoft" with "windows", as that's what's used elsewhere (e.g. in hotspot sources). Maurizio > > Jorn > >> That's all I have for now. >> >> Cheers >> Maurizio >> >> https://mail.openjdk.java.net/pipermail/panama-dev/2019-January/003680.html >> From shravya.rukmannagari at intel.com Thu Jan 10 18:36:14 2019 From: shravya.rukmannagari at intel.com (Rukmannagari, Shravya) Date: Thu, 10 Jan 2019 18:36:14 +0000 Subject: VectorAPI Constant Shift for Byte and Short Intrinsic Support Message-ID: <8D6F463991A1574A8A803B8DA605414F3A82C461@ORSMSX111.amr.corp.intel.com> Hi All, I would like to contribute a patch to add constant shifts support in Java for byte and short datatypes. Could you please review the patch here: http://cr.openjdk.java.net/~srukmannagar/VectorAPI_ByteShortConstantShift/webrev.00/ Thanks, Shravya. From sandhya.viswanathan at intel.com Thu Jan 10 21:29:29 2019 From: sandhya.viswanathan at intel.com (Viswanathan, Sandhya) Date: Thu, 10 Jan 2019 21:29:29 +0000 Subject: VectorAPI Constant Shift for Byte and Short Intrinsic Support In-Reply-To: <8D6F463991A1574A8A803B8DA605414F3A82C461@ORSMSX111.amr.corp.intel.com> References: <8D6F463991A1574A8A803B8DA605414F3A82C461@ORSMSX111.amr.corp.intel.com> Message-ID: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A4D1C9@FMSMSX126.amr.corp.intel.com> Hi Shravya, Could the following line in library_call.cpp be simplified using expression like (type2aelembytes(bt) * BitsPerByte - 1)? 7462 juint mask = (bt == T_LONG) ? (BitsPerLong - 1) : (bt == T_BYTE) ? (BitsPerByte - 1) : (bt == T_SHORT) ? (BitsPerShort - 1) : (BitsPerInt - 1); Rest looks fine. Thanks, Sandhya -----Original Message----- From: panama-dev [mailto:panama-dev-bounces at openjdk.java.net] On Behalf Of Rukmannagari, Shravya Sent: Thursday, January 10, 2019 10:36 AM To: panama-dev Subject: VectorAPI Constant Shift for Byte and Short Intrinsic Support Hi All, I would like to contribute a patch to add constant shifts support in Java for byte and short datatypes. Could you please review the patch here: http://cr.openjdk.java.net/~srukmannagar/VectorAPI_ByteShortConstantShift/webrev.00/ Thanks, Shravya. From vladimir.x.ivanov at oracle.com Thu Jan 10 22:39:22 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Thu, 10 Jan 2019 14:39:22 -0800 Subject: VectorAPI Constant Shift for Byte and Short Intrinsic Support In-Reply-To: <8D6F463991A1574A8A803B8DA605414F3A82C461@ORSMSX111.amr.corp.intel.com> References: <8D6F463991A1574A8A803B8DA605414F3A82C461@ORSMSX111.amr.corp.intel.com> Message-ID: > http://cr.openjdk.java.net/~srukmannagar/VectorAPI_ByteShortConstantShift/webrev.00/ Don't you need to update Javadoc as well? Though Javadoc refers to shift operations, implementation does more than that ("a >> i" vs "a >> (i & 7)" / "a >> (i & 15)"): /** * Arithmetically right shifts (or signed right shifts) this vector by the * broadcast of an input scalar. *

* This is a vector binary operation where the primitive arithmetic right * shift operation ({@code >>}) is applied to lane elements. * * @param s the input scalar; the number of the bits to right shift * @return the result of arithmetically right shifting this vector by the * broadcast of an input scalar */ public abstract ShortVector aShiftR(int s); vs @Override @ForceInline public Short256Vector aShiftR(int s) { return VectorIntrinsics.broadcastInt( VECTOR_OP_RSHIFT, Short256Vector.class, short.class, LENGTH, this, s, (v, i) -> v.uOp((__, a) -> (short) (a >> (i & 15)))); } ================================================= I assume C2 backend support for Short*Vector is already there. src/hotspot/cpu/x86/x86.ad: +instruct vsll32B_avx(vecY dst, vecY src, vecS shift, vecY tmp, vecY tmp2, rRegL scratch) %{ + predicate(UseAVX > 1 && n->as_Vector()->length() == 32); Is it deliberate there's no 32B support with UseAVX=1? On related node, do you need to add *ShiftVB cases into Matcher::match_rule_supported_vector()? Best regards, Vladimir Ivanov From jbvernee at xs4all.nl Thu Jan 10 22:58:18 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Thu, 10 Jan 2019 23:58:18 +0100 Subject: [foreign] RFR 8216485: Binder changes are needed to support paname foreign API on Windows In-Reply-To: <55f1ade8-c75d-fe78-1b04-1aff15ef8b4a@oracle.com> References: <55f1ade8-c75d-fe78-1b04-1aff15ef8b4a@oracle.com> Message-ID: <54351e22f39cf216d2e4782521eeadae@xs4all.nl> Followup on this. CallingSequenceBuilderImpl and StandardCall have more difference than I realized. the arrangeCall method for the latter just lets you pass a NaitveMethodType and the argument layouts are grabbed from that. But on Windows we also have to differentiate between varargs and non-varargs, so the SysV strategy doesn't work for both. I think this is still solvable, of course, but I'd rather save that for a later patch if that's okay with you. I have moved the alignment methods into an abstract base class called CallingSequenceBuilder. The StorageNames thing was only used by the CallingSequenceBuilder implementations, so I've moved that logic into there as well. I've moved ArgumentClass to the abi/x64 package (Same for it's test), and also created a SharedConstants class with some of the x86_64 constant values which are also used by UniversalUpcallHandler (this was also referencing sysv/Constants before, which I changed to package access to avoid that in the future). I've replaced the UniversalBoxer with sub classes of UniversalNativeInvoker and UniversalUpcallHandler. I've changed microsoft -> windows for abi/x64/microsoft and Microsoftx64ABI. updated webrev: http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows/webrev.04/ Jorn Maurizio Cimadamore schreef op 2019-01-10 19:34: > On 10/01/2019 17:58, Jorn Vernee wrote: >> Hi Maurizio, >> >> Thanks for the review, comments inline... >> >> Maurizio Cimadamore schreef op 2019-01-10 18:15: >>> Continuing from [1]. >>> >>> This is a more in depth review of the binder changes. >>> >>> >>> Overall the changes look good, and what I see is more or less what we >>> have discussed over the past few weeks. As a general >>> comment/overarching theme, I think we should try, as much as >>> possible, >>> to reuse code where possible, rather then duplicating it. >> >> Ok, I thought we would be going the other direction, so I did not >> spend much time yet on trying to de-duplicate code. I tried to touch >> the SysV and shared implementation as little as possible to keep >> maintenance easier, with the intent of de-duplicating code in a later >> patch if desirable. > > Yes, I understand. We can also push this as is and work on incremental > improvements, but I'd like to unify code paths as possible, as I think > maintaining two completely separated code-bases will be trickier in > the long run. That said, I like that, at least for prototyping, the > new ABI design allows for slotting in new stuff quite easily. > > > >> - ArgumentInfo/StorageCalculator keep only one counter for the number >> of registers on Windows while SysV needs 2. >> - StorageCalculator::addBindings has a special provision for varargs: > I might have gone too far here... >> >> >>> * The alignment routines in CallingSequenceBuilderImpl seem identical >>> to the ones in SysVABI, so perhaps again these could be shared >>> somewhere. >> >> Same here, maybe we should bring back AbstractSystemABI? Or otherwise >> create an ABI utility class? > > Note that the alignment function is ultimately only used by the > CallBuilder. So, if we go for subclassing, I'd say put this in > CallingSequenceBuilder - and then have the ABI-dependent details in > the Windows/SysV impls. > > > Also, I forgot an important point: please replace "microsoft" with > "windows", as that's what's used elsewhere (e.g. in hotspot sources). > > > Maurizio > > >> >> Jorn >> >>> That's all I have for now. >>> >>> Cheers >>> Maurizio >>> >>> https://mail.openjdk.java.net/pipermail/panama-dev/2019-January/003680.html From sandhya.viswanathan at intel.com Fri Jan 11 00:33:01 2019 From: sandhya.viswanathan at intel.com (Viswanathan, Sandhya) Date: Fri, 11 Jan 2019 00:33:01 +0000 Subject: [vector api] RFR: Fix the issue about float/double reduce-max/min operations with NaN as an input In-Reply-To: References: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A317E7@FMSMSX126.amr.corp.intel.com> Message-ID: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A4D4F5@FMSMSX126.amr.corp.intel.com> Hi Yang, Changing minAll and maxAll to use Math.min() and Math.max() looks good. I have a couple of more inputs below: 1) The Vector API min() and max() methods also need this change. 2) Math.min and Math.max should be used across data types for both integral and floating point. This is to be consistent all across. Best Regards, Sandhya -----Original Message----- From: Yang Zhang (Arm Technology China) [mailto:Yang.Zhang at arm.com] Sent: Wednesday, January 09, 2019 6:44 PM To: 'panama-dev at openjdk.java.net' ; Viswanathan, Sandhya ; Vladimir Ivanov Cc: nd Subject: RE: [vector api] RFR: Fix the issue about float/double reduce-max/min operations with NaN as an input Hi Vladimir, Sandhya Ping again. Could you please help to review this patch? Regards, Yang -----Original Message----- From: Yang Zhang Sent: Thursday, January 10, 2019 10:32 AM To: Yang Zhang (Arm Technology China) Subject: Fwd: [vector api] RFR: Fix the issue about float/double reduce-max/min operations with NaN as an input ---------- Forwarded message --------- From: Yang Zhang (Arm Technology China) Date: Fri, 14 Dec 2018 at 16:31 Subject: RE: [vector api] RFR: Fix the issue about float/double reduce-max/min operations with NaN as an input To: Viswanathan, Sandhya , panama-dev at openjdk.java.net Cc: nd Hi Sandhya Happy holidays. Let us keep on this work after holidays. Regards Yang -----Original Message----- From: Viswanathan, Sandhya Sent: Friday, December 14, 2018 9:43 AM To: Yang Zhang (Arm Technology China) ; panama-dev at openjdk.java.net Cc: nd Subject: RE: [vector api] RFR: Fix the issue about float/double reduce-max/min operations with NaN as an input Hi Yang, The current x86 intrinsics are based on the current definition as in Javadoc for Vector API float(double) min: public abstract FloatVector min(float s) Returns the minimum of this vector and the broadcast of an input scalar. This is a vector binary operation where the operation (a, b) -> a < b ? a : b is applied to lane elements. Parameters: s - the input scalar Returns: the minimum of this vector and the broadcast of an input scalar We are looking into making it closer to Math.min at least in the NaN related behavior. Let us wait till January middle or so to converge on this, it being too close to Christmas now and many are on vacation. Hope that is ok. Happy holidays. It is wonderful to see you and others from Arm technology onboard and contributing to Panama Vector API. Let us keep this collaboration going. Best Regards, Sandhya -----Original Message----- From: panama-dev [mailto:panama-dev-bounces at openjdk.java.net] On Behalf Of Yang Zhang (Arm Technology China) Sent: Wednesday, December 12, 2018 12:37 AM To: panama-dev at openjdk.java.net Cc: nd Subject: [vector api] RFR: Fix the issue about float/double reduce-max/min operations with NaN as an input Hi When I implement reduce-max/min, I found there is an issue about float/double reduce-max/min operations with NaN as an input. In java doc, if either value is NaN, then the result of max/min(a, b) is NaN. But the result of "((a > b) ? a : b)" is affected by sequence of (a, b). If a is NaN, the result is b. If b is NaN, the result is b (NaN). I have a patch which could fix this issue. Could you please help to review it? http://cr.openjdk.java.net/~yzhang/vectorapi.nan/webrev.00/ Ps. With this patch, there are 5 failures on x86 platform. jdk/incubator/vector/Float256VectorTests.java jdk/incubator/vector/Float64VectorTests.java jdk/incubator/vector/FloatMaxVectorTests.java jdk/incubator/vector/Double128VectorTests.java jdk/incubator/vector/DoubleMaxVectorTests.java Webrev: Regards, Yang From sundararajan.athijegannathan at oracle.com Fri Jan 11 02:21:27 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Fri, 11 Jan 2019 07:51:27 +0530 Subject: [foreign] RFR 8216483: jextract tests have to be modified to run on Windows platform In-Reply-To: References: Message-ID: <5C37FDA7.8000602@oracle.com> All test run fine on Mac with this latest patch! I'll test on Linux as well and then send you review comments. -Sundar On 10/01/19, 9:40 PM, Jorn Vernee wrote: > Hi, > > Continuation of [1]. > > updated webrev: > http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_tests/webrev.04/ > > For the TestJextractFFI test I'm now just checking the operating > system in the LibClang.java patch file: > > String libName = System.getProperty("os.name").startsWith("Windows") > ? "libclang" > : "clang"; > Library libclang = Libraries.loadLibrary(MethodHandles.lookup(), > libName); > > The other problem is that the dll is in the bin folder vs lib folder > on Windows, and the test was using the linker path which is the lib > folder on Windows. There was no build system variable available for > the folder that contains the library file, so instead I'm deriving it > from the library file path. > > Cheers, > Jorn > > [1] : > https://mail.openjdk.java.net/pipermail/panama-dev/2019-January/003691.html From Yang.Zhang at arm.com Fri Jan 11 03:23:00 2019 From: Yang.Zhang at arm.com (Yang Zhang (Arm Technology China)) Date: Fri, 11 Jan 2019 03:23:00 +0000 Subject: [vector api] RFR: Fix the issue about float/double reduce-max/min operations with NaN as an input In-Reply-To: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A4D4F5@FMSMSX126.amr.corp.intel.com> References: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A317E7@FMSMSX126.amr.corp.intel.com> <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A4D4F5@FMSMSX126.amr.corp.intel.com> Message-ID: Hi Sandhya 1) Vector Api min()/max() have been updated. Are there any missed? 2) Math.min/max support float/double/int/long only, byte/short need casting. Current ((a Sent: Friday, January 11, 2019 8:33 AM To: Yang Zhang (Arm Technology China) ; 'panama-dev at openjdk.java.net' ; Vladimir Ivanov Cc: nd Subject: RE: [vector api] RFR: Fix the issue about float/double reduce-max/min operations with NaN as an input Hi Yang, Changing minAll and maxAll to use Math.min() and Math.max() looks good. I have a couple of more inputs below: 1) The Vector API min() and max() methods also need this change. 2) Math.min and Math.max should be used across data types for both integral and floating point. This is to be consistent all across. Best Regards, Sandhya -----Original Message----- From: Yang Zhang (Arm Technology China) [mailto:Yang.Zhang at arm.com] Sent: Wednesday, January 09, 2019 6:44 PM To: 'panama-dev at openjdk.java.net' ; Viswanathan, Sandhya ; Vladimir Ivanov Cc: nd Subject: RE: [vector api] RFR: Fix the issue about float/double reduce-max/min operations with NaN as an input Hi Vladimir, Sandhya Ping again. Could you please help to review this patch? Regards, Yang -----Original Message----- From: Yang Zhang Sent: Thursday, January 10, 2019 10:32 AM To: Yang Zhang (Arm Technology China) Subject: Fwd: [vector api] RFR: Fix the issue about float/double reduce-max/min operations with NaN as an input ---------- Forwarded message --------- From: Yang Zhang (Arm Technology China) Date: Fri, 14 Dec 2018 at 16:31 Subject: RE: [vector api] RFR: Fix the issue about float/double reduce-max/min operations with NaN as an input To: Viswanathan, Sandhya , panama-dev at openjdk.java.net Cc: nd Hi Sandhya Happy holidays. Let us keep on this work after holidays. Regards Yang -----Original Message----- From: Viswanathan, Sandhya Sent: Friday, December 14, 2018 9:43 AM To: Yang Zhang (Arm Technology China) ; panama-dev at openjdk.java.net Cc: nd Subject: RE: [vector api] RFR: Fix the issue about float/double reduce-max/min operations with NaN as an input Hi Yang, The current x86 intrinsics are based on the current definition as in Javadoc for Vector API float(double) min: public abstract FloatVector min(float s) Returns the minimum of this vector and the broadcast of an input scalar. This is a vector binary operation where the operation (a, b) -> a < b ? a : b is applied to lane elements. Parameters: s - the input scalar Returns: the minimum of this vector and the broadcast of an input scalar We are looking into making it closer to Math.min at least in the NaN related behavior. Let us wait till January middle or so to converge on this, it being too close to Christmas now and many are on vacation. Hope that is ok. Happy holidays. It is wonderful to see you and others from Arm technology onboard and contributing to Panama Vector API. Let us keep this collaboration going. Best Regards, Sandhya -----Original Message----- From: panama-dev [mailto:panama-dev-bounces at openjdk.java.net] On Behalf Of Yang Zhang (Arm Technology China) Sent: Wednesday, December 12, 2018 12:37 AM To: panama-dev at openjdk.java.net Cc: nd Subject: [vector api] RFR: Fix the issue about float/double reduce-max/min operations with NaN as an input Hi When I implement reduce-max/min, I found there is an issue about float/double reduce-max/min operations with NaN as an input. In java doc, if either value is NaN, then the result of max/min(a, b) is NaN. But the result of "((a > b) ? a : b)" is affected by sequence of (a, b). If a is NaN, the result is b. If b is NaN, the result is b (NaN). I have a patch which could fix this issue. Could you please help to review it? http://cr.openjdk.java.net/~yzhang/vectorapi.nan/webrev.00/ Ps. With this patch, there are 5 failures on x86 platform. jdk/incubator/vector/Float256VectorTests.java jdk/incubator/vector/Float64VectorTests.java jdk/incubator/vector/FloatMaxVectorTests.java jdk/incubator/vector/Double128VectorTests.java jdk/incubator/vector/DoubleMaxVectorTests.java Webrev: Regards, Yang From sundararajan.athijegannathan at oracle.com Fri Jan 11 06:16:35 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Fri, 11 Jan 2019 11:46:35 +0530 Subject: [foreign] RFR 8216483: jextract tests have to be modified to run on Windows platform In-Reply-To: <5C37FDA7.8000602@oracle.com> References: <5C37FDA7.8000602@oracle.com> Message-ID: <5C3834C3.1060801@oracle.com> Looks good. Tests run fine on Linux as well. If all fine on Windows, please push. PS. New test source files should have copyright year 2019 -Sundar On 11/01/19, 7:51 AM, Sundararajan Athijegannathan wrote: > All test run fine on Mac with this latest patch! I'll test on Linux as > well and then send you review comments. > > -Sundar > > On 10/01/19, 9:40 PM, Jorn Vernee wrote: >> Hi, >> >> Continuation of [1]. >> >> updated webrev: >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_tests/webrev.04/ >> >> For the TestJextractFFI test I'm now just checking the operating >> system in the LibClang.java patch file: >> >> String libName = System.getProperty("os.name").startsWith("Windows") >> ? "libclang" >> : "clang"; >> Library libclang = Libraries.loadLibrary(MethodHandles.lookup(), >> libName); >> >> The other problem is that the dll is in the bin folder vs lib folder >> on Windows, and the test was using the linker path which is the lib >> folder on Windows. There was no build system variable available for >> the folder that contains the library file, so instead I'm deriving it >> from the library file path. >> >> Cheers, >> Jorn >> >> [1] : >> https://mail.openjdk.java.net/pipermail/panama-dev/2019-January/003691.html From maurizio.cimadamore at oracle.com Fri Jan 11 10:09:03 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Fri, 11 Jan 2019 10:09:03 +0000 Subject: [foreign] RFR 8216485: Binder changes are needed to support paname foreign API on Windows In-Reply-To: <54351e22f39cf216d2e4782521eeadae@xs4all.nl> References: <55f1ade8-c75d-fe78-1b04-1aff15ef8b4a@oracle.com> <54351e22f39cf216d2e4782521eeadae@xs4all.nl> Message-ID: <8a8542d9-3e9c-f4df-c7f8-ad458980f940@oracle.com> Thanks, I think I'm still somewhat not 100% convinced this is the right stacking, but it's probably better to go ahead with this, and then see if we can simplify things a bit. All tests pass here, so no issues there. Some minor comments below: * One thing with the new SharedConstants is that I noted that (as you also write) there is a dependency from UniversalUpcallHandler (which lives in abi) to SharedConstants (which lives in abi/x64). So we have a dependency from a less specific abi construct to a more specific; and that's probably a bad smell. * The sizeof methods in CallingSequenceBuilder are not used and can be removed * ArgumentClass is in abi/x64, but StorageClass isn't. Not sure, in general, in how do we differentiate between the two cases; in principle, anything that speaks about x87 should probably live under x64, but in reality, the shuffle recipes we generate rely on the fact that we have integer/vector/stack arguments and integer/vector/x87 returns... Maurizio On 10/01/2019 22:58, Jorn Vernee wrote: > Followup on this. > > CallingSequenceBuilderImpl and StandardCall have more difference than > I realized. the arrangeCall method for the latter just lets you pass a > NaitveMethodType and the argument layouts are grabbed from that. But > on Windows we also have to differentiate between varargs and > non-varargs, so the SysV strategy doesn't work for both. I think this > is still solvable, of course, but I'd rather save that for a later > patch if that's okay with you. I have moved the alignment methods into > an abstract base class called CallingSequenceBuilder. The StorageNames > thing was only used by the CallingSequenceBuilder implementations, so > I've moved that logic into there as well. > > I've moved ArgumentClass to the abi/x64 package (Same for it's test), > and also created a SharedConstants class with some of the x86_64 > constant values which are also used by UniversalUpcallHandler (this > was also referencing sysv/Constants before, which I changed to package > access to avoid that in the future). > > I've replaced the UniversalBoxer with sub classes of > UniversalNativeInvoker and UniversalUpcallHandler. > > I've changed microsoft -> windows for abi/x64/microsoft and > Microsoftx64ABI. > > updated webrev: > http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows/webrev.04/ > > Jorn > > Maurizio Cimadamore schreef op 2019-01-10 19:34: >> On 10/01/2019 17:58, Jorn Vernee wrote: >>> Hi Maurizio, >>> >>> Thanks for the review, comments inline... >>> >>> Maurizio Cimadamore schreef op 2019-01-10 18:15: >>>> Continuing from [1]. >>>> >>>> This is a more in depth review of the binder changes. >>>> >>>> >>>> Overall the changes look good, and what I see is more or less what we >>>> have discussed over the past few weeks. As a general >>>> comment/overarching theme, I think we should try, as much as possible, >>>> to reuse code where possible, rather then duplicating it. >>> >>> Ok, I thought we would be going the other direction, so I did not >>> spend much time yet on trying to de-duplicate code. I tried to touch >>> the SysV and shared implementation as little as possible to keep >>> maintenance easier, with the intent of de-duplicating code in a >>> later patch if desirable. >> >> Yes, I understand. We can also push this as is and work on incremental >> improvements, but I'd like to unify code paths as possible, as I think >> maintaining two completely separated code-bases will be trickier in >> the long run. That said, I like that, at least for prototyping, the >> new ABI design allows for slotting in new stuff quite easily. >> >> >> >>> - ArgumentInfo/StorageCalculator keep only one counter for the >>> number of registers on Windows while SysV needs 2. >>> - StorageCalculator::addBindings has a special provision for varargs: >> I might have gone too far here... >>> >>> >>>> * The alignment routines in CallingSequenceBuilderImpl seem identical >>>> to the ones in SysVABI, so perhaps again these could be shared >>>> somewhere. >>> >>> Same here, maybe we should bring back AbstractSystemABI? Or >>> otherwise create an ABI utility class? >> >> Note that the alignment function is ultimately only used by the >> CallBuilder. So, if we go for subclassing, I'd say put this in >> CallingSequenceBuilder - and then have the ABI-dependent details in >> the Windows/SysV impls. >> >> >> Also, I forgot an important point: please replace "microsoft" with >> "windows", as that's what's used elsewhere (e.g. in hotspot sources). >> >> >> Maurizio >> >> >>> >>> Jorn >>> >>>> That's all I have for now. >>>> >>>> Cheers >>>> Maurizio >>>> >>>> https://mail.openjdk.java.net/pipermail/panama-dev/2019-January/003680.html >>>> From henry.jen at oracle.com Fri Jan 11 12:56:55 2019 From: henry.jen at oracle.com (Henry Jen) Date: Fri, 11 Jan 2019 04:56:55 -0800 Subject: [foreign] RFR 8216483: jextract tests have to be modified to run on Windows platform In-Reply-To: References: Message-ID: <07219C0D-E056-48D1-873E-340D9FA5D2A8@oracle.com> > On Jan 10, 2019, at 8:10 AM, Jorn Vernee wrote: > > Hi, > > Continuation of [1]. > > updated webrev: http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_tests/webrev.04/ > > For the TestJextractFFI test I'm now just checking the operating system in the LibClang.java patch file: > > String libName = System.getProperty("os.name").startsWith("Windows") > ? "libclang" > : "clang?; Why do we have this line? The prefix and suffix should be handled by loadLibrary if we are now using absolute path. Cheers, Henry > Library libclang = Libraries.loadLibrary(MethodHandles.lookup(), libName); > > The other problem is that the dll is in the bin folder vs lib folder on Windows, and the test was using the linker path which is the lib folder on Windows. There was no build system variable available for the folder that contains the library file, so instead I'm deriving it from the library file path. > > Cheers, > Jorn > > [1] : https://mail.openjdk.java.net/pipermail/panama-dev/2019-January/003691.html From sundararajan.athijegannathan at oracle.com Fri Jan 11 12:57:32 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Fri, 11 Jan 2019 18:27:32 +0530 Subject: [foreign] RFR: Windows support Message-ID: <5C3892BC.2030506@oracle.com> In response to this email -> https://mail.openjdk.java.net/pipermail/panama-dev/2019-January/003680.html This email is to address webrev for bug https://bugs.openjdk.java.net/browse/JDK-8211060 I tested the default native library changes on Mac and Linux. All tests run tine. We already discussed about Windows limitation (of having to search the entire process for the given symbol because there is no equivalent of RTLD_DEFAULT). We may want to leave an explicit comment in -> src/java.base/windows/native/libjava/jni_util_md.c. Looks good for push. -Sundar From sundararajan.athijegannathan at oracle.com Fri Jan 11 12:59:33 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Fri, 11 Jan 2019 18:29:33 +0530 Subject: [foreign] RFR 8216483: jextract tests have to be modified to run on Windows platform In-Reply-To: <07219C0D-E056-48D1-873E-340D9FA5D2A8@oracle.com> References: <07219C0D-E056-48D1-873E-340D9FA5D2A8@oracle.com> Message-ID: <5C389335.8040901@oracle.com> Hi Henry, The Windows dll name is "libclang.dll" and not "clang.dll" as one would expect! Our auto library name matching would add "lib"/"" as prefix and ".so"/".dylib"/".dll" as suffix - which won't work in this case. -Sundar On 11/01/19, 6:26 PM, Henry Jen wrote: > >> On Jan 10, 2019, at 8:10 AM, Jorn Vernee wrote: >> >> Hi, >> >> Continuation of [1]. >> >> updated webrev: http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_tests/webrev.04/ >> >> For the TestJextractFFI test I'm now just checking the operating system in the LibClang.java patch file: >> >> String libName = System.getProperty("os.name").startsWith("Windows") >> ? "libclang" >> : "clang?; > Why do we have this line? The prefix and suffix should be handled by loadLibrary if we are now using absolute path. > > Cheers, > Henry > >> Library libclang = Libraries.loadLibrary(MethodHandles.lookup(), libName); >> >> The other problem is that the dll is in the bin folder vs lib folder on Windows, and the test was using the linker path which is the lib folder on Windows. There was no build system variable available for the folder that contains the library file, so instead I'm deriving it from the library file path. >> >> Cheers, >> Jorn >> >> [1] : https://mail.openjdk.java.net/pipermail/panama-dev/2019-January/003691.html From henry.jen at oracle.com Fri Jan 11 12:59:38 2019 From: henry.jen at oracle.com (Henry Jen) Date: Fri, 11 Jan 2019 04:59:38 -0800 Subject: [foreign] RFR 8216483: jextract tests have to be modified to run on Windows platform In-Reply-To: <07219C0D-E056-48D1-873E-340D9FA5D2A8@oracle.com> References: <07219C0D-E056-48D1-873E-340D9FA5D2A8@oracle.com> Message-ID: <10626F84-C6B6-41EE-9365-B74C5B28FE34@oracle.com> never mind, I read it wrong. However, on Windows, it?s clang.dll, not libeling.dll iirc. No? Cheers, Henry > On Jan 11, 2019, at 4:56 AM, Henry Jen wrote: > > > >> On Jan 10, 2019, at 8:10 AM, Jorn Vernee wrote: >> >> Hi, >> >> Continuation of [1]. >> >> updated webrev: http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_tests/webrev.04/ >> >> For the TestJextractFFI test I'm now just checking the operating system in the LibClang.java patch file: >> >> String libName = System.getProperty("os.name").startsWith("Windows") >> ? "libclang" >> : "clang?; > > Why do we have this line? The prefix and suffix should be handled by loadLibrary if we are now using absolute path. > > Cheers, > Henry > >> Library libclang = Libraries.loadLibrary(MethodHandles.lookup(), libName); >> >> The other problem is that the dll is in the bin folder vs lib folder on Windows, and the test was using the linker path which is the lib folder on Windows. There was no build system variable available for the folder that contains the library file, so instead I'm deriving it from the library file path. >> >> Cheers, >> Jorn >> >> [1] : https://mail.openjdk.java.net/pipermail/panama-dev/2019-January/003691.html > From jbvernee at xs4all.nl Fri Jan 11 13:00:03 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Fri, 11 Jan 2019 14:00:03 +0100 Subject: [foreign] RFR 8216483: jextract tests have to be modified to run on Windows platform In-Reply-To: <5C3834C3.1060801@oracle.com> References: <5C37FDA7.8000602@oracle.com> <5C3834C3.1060801@oracle.com> Message-ID: I put copyright headers into the files that I created. There are also some other files missing copyright headers I noticed. Should I add them as well, if so, what year should I put in the header? Thanks, Jorn Sundararajan Athijegannathan schreef op 2019-01-11 07:16: > Looks good. > > Tests run fine on Linux as well. If all fine on Windows, please push. > > PS. New test source files should have copyright year 2019 > > -Sundar > > On 11/01/19, 7:51 AM, Sundararajan Athijegannathan wrote: >> All test run fine on Mac with this latest patch! I'll test on Linux as >> well and then send you review comments. >> >> -Sundar >> >> On 10/01/19, 9:40 PM, Jorn Vernee wrote: >>> Hi, >>> >>> Continuation of [1]. >>> >>> updated webrev: >>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_tests/webrev.04/ >>> >>> For the TestJextractFFI test I'm now just checking the operating >>> system in the LibClang.java patch file: >>> >>> String libName = >>> System.getProperty("os.name").startsWith("Windows") >>> ? "libclang" >>> : "clang"; >>> Library libclang = Libraries.loadLibrary(MethodHandles.lookup(), >>> libName); >>> >>> The other problem is that the dll is in the bin folder vs lib folder >>> on Windows, and the test was using the linker path which is the lib >>> folder on Windows. There was no build system variable available for >>> the folder that contains the library file, so instead I'm deriving it >>> from the library file path. >>> >>> Cheers, >>> Jorn >>> >>> [1] : >>> https://mail.openjdk.java.net/pipermail/panama-dev/2019-January/003691.html From jbvernee at xs4all.nl Fri Jan 11 13:03:00 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Fri, 11 Jan 2019 14:03:00 +0100 Subject: [foreign] RFR 8216483: jextract tests have to be modified to run on Windows platform In-Reply-To: <07219C0D-E056-48D1-873E-340D9FA5D2A8@oracle.com> References: <07219C0D-E056-48D1-873E-340D9FA5D2A8@oracle.com> Message-ID: <387f9fdd361f51fcb20b1c7e99f3cfe5@xs4all.nl> We are not using and absolute path, but a library name, which is mapped in a platform dependent manner. But, libclang doesn't follow the standard name format. The expected name would be clang.dll, but the file is actually called libclang.dll, so need to manually pass in the right name. Jorn Henry Jen schreef op 2019-01-11 13:56: >> On Jan 10, 2019, at 8:10 AM, Jorn Vernee wrote: >> >> Hi, >> >> Continuation of [1]. >> >> updated webrev: >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_tests/webrev.04/ >> >> For the TestJextractFFI test I'm now just checking the operating >> system in the LibClang.java patch file: >> >> String libName = >> System.getProperty("os.name").startsWith("Windows") >> ? "libclang" >> : "clang?; > > Why do we have this line? The prefix and suffix should be handled by > loadLibrary if we are now using absolute path. > > Cheers, > Henry > >> Library libclang = Libraries.loadLibrary(MethodHandles.lookup(), >> libName); >> >> The other problem is that the dll is in the bin folder vs lib folder >> on Windows, and the test was using the linker path which is the lib >> folder on Windows. There was no build system variable available for >> the folder that contains the library file, so instead I'm deriving it >> from the library file path. >> >> Cheers, >> Jorn >> >> [1] : >> https://mail.openjdk.java.net/pipermail/panama-dev/2019-January/003691.html From sundararajan.athijegannathan at oracle.com Fri Jan 11 13:03:17 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Fri, 11 Jan 2019 18:33:17 +0530 Subject: [foreign] RFR 8216483: jextract tests have to be modified to run on Windows platform In-Reply-To: References: <5C37FDA7.8000602@oracle.com> <5C3834C3.1060801@oracle.com> Message-ID: <5C389415.5020109@oracle.com> We need copyright in all files. For old files, please use 2018. For new files 2019. PS. But if there are any line number checks in the tests we need to modify the tests after adding copyright. -Sundar On 11/01/19, 6:30 PM, Jorn Vernee wrote: > I put copyright headers into the files that I created. > > There are also some other files missing copyright headers I noticed. > Should I add them as well, if so, what year should I put in the header? > > Thanks, > Jorn > > Sundararajan Athijegannathan schreef op 2019-01-11 07:16: >> Looks good. >> >> Tests run fine on Linux as well. If all fine on Windows, please push. >> >> PS. New test source files should have copyright year 2019 >> >> -Sundar >> >> On 11/01/19, 7:51 AM, Sundararajan Athijegannathan wrote: >>> All test run fine on Mac with this latest patch! I'll test on Linux >>> as well and then send you review comments. >>> >>> -Sundar >>> >>> On 10/01/19, 9:40 PM, Jorn Vernee wrote: >>>> Hi, >>>> >>>> Continuation of [1]. >>>> >>>> updated webrev: >>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_tests/webrev.04/ >>>> >>>> For the TestJextractFFI test I'm now just checking the operating >>>> system in the LibClang.java patch file: >>>> >>>> String libName = >>>> System.getProperty("os.name").startsWith("Windows") >>>> ? "libclang" >>>> : "clang"; >>>> Library libclang = >>>> Libraries.loadLibrary(MethodHandles.lookup(), libName); >>>> >>>> The other problem is that the dll is in the bin folder vs lib >>>> folder on Windows, and the test was using the linker path which is >>>> the lib folder on Windows. There was no build system variable >>>> available for the folder that contains the library file, so instead >>>> I'm deriving it from the library file path. >>>> >>>> Cheers, >>>> Jorn >>>> >>>> [1] : >>>> https://mail.openjdk.java.net/pipermail/panama-dev/2019-January/003691.html From jbvernee at xs4all.nl Fri Jan 11 13:05:35 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Fri, 11 Jan 2019 14:05:35 +0100 Subject: [foreign] RFR 8216483: jextract tests have to be modified to run on Windows platform In-Reply-To: <5C389415.5020109@oracle.com> References: <5C37FDA7.8000602@oracle.com> <5C3834C3.1060801@oracle.com> <5C389415.5020109@oracle.com> Message-ID: <9c768e8a2c341d086a1b2379c05811bd@xs4all.nl> Ok, I will run the tests to check, but I think it uses reflection to do the comparison of those files so should be fine. Jorn Sundararajan Athijegannathan schreef op 2019-01-11 14:03: > We need copyright in all files. For old files, please use 2018. For > new files 2019. > > PS. But if there are any line number checks in the tests we need to > modify the tests after adding copyright. > > -Sundar > > On 11/01/19, 6:30 PM, Jorn Vernee wrote: >> I put copyright headers into the files that I created. >> >> There are also some other files missing copyright headers I noticed. >> Should I add them as well, if so, what year should I put in the >> header? >> >> Thanks, >> Jorn >> >> Sundararajan Athijegannathan schreef op 2019-01-11 07:16: >>> Looks good. >>> >>> Tests run fine on Linux as well. If all fine on Windows, please push. >>> >>> PS. New test source files should have copyright year 2019 >>> >>> -Sundar >>> >>> On 11/01/19, 7:51 AM, Sundararajan Athijegannathan wrote: >>>> All test run fine on Mac with this latest patch! I'll test on Linux >>>> as well and then send you review comments. >>>> >>>> -Sundar >>>> >>>> On 10/01/19, 9:40 PM, Jorn Vernee wrote: >>>>> Hi, >>>>> >>>>> Continuation of [1]. >>>>> >>>>> updated webrev: >>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_tests/webrev.04/ >>>>> >>>>> For the TestJextractFFI test I'm now just checking the operating >>>>> system in the LibClang.java patch file: >>>>> >>>>> String libName = >>>>> System.getProperty("os.name").startsWith("Windows") >>>>> ? "libclang" >>>>> : "clang"; >>>>> Library libclang = >>>>> Libraries.loadLibrary(MethodHandles.lookup(), libName); >>>>> >>>>> The other problem is that the dll is in the bin folder vs lib >>>>> folder on Windows, and the test was using the linker path which is >>>>> the lib folder on Windows. There was no build system variable >>>>> available for the folder that contains the library file, so instead >>>>> I'm deriving it from the library file path. >>>>> >>>>> Cheers, >>>>> Jorn >>>>> >>>>> [1] : >>>>> https://mail.openjdk.java.net/pipermail/panama-dev/2019-January/003691.html From jbvernee at xs4all.nl Fri Jan 11 13:45:57 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Fri, 11 Jan 2019 14:45:57 +0100 Subject: [foreign] RFR 8216485: Binder changes are needed to support paname foreign API on Windows In-Reply-To: <8a8542d9-3e9c-f4df-c7f8-ad458980f940@oracle.com> References: <55f1ade8-c75d-fe78-1b04-1aff15ef8b4a@oracle.com> <54351e22f39cf216d2e4782521eeadae@xs4all.nl> <8a8542d9-3e9c-f4df-c7f8-ad458980f940@oracle.com> Message-ID: <515b6602b94a6aa3e75a64b05907edd9@xs4all.nl> I've removed the sizeof methods. About the other 2 points. These are smells, but they were already present, and I think the problem goes a deeper as well. For instance, lots of places use NativeTypes.UINT64 to represent a "slot" of memory, but that would probably not hold up on 32bit platforms. So maybe we can fully figure out the re-stacking when a non-x64 port comes along and we have a clearer picture about the exact requirements? I can add FIXMEs for now. As a quick fix, we could move all the classes in the abi package, except SystemABI and UpcallStubs, into the abi/x64 package, since there are evidently dependencies on x64 specific stuff. And then when the next port comes along figure things out at that point? Jorn Maurizio Cimadamore schreef op 2019-01-11 11:09: > Thanks, > I think I'm still somewhat not 100% convinced this is the right > stacking, but it's probably better to go ahead with this, and then see > if we can simplify things a bit. All tests pass here, so no issues > there. Some minor comments below: > > * One thing with the new SharedConstants is that I noted that (as you > also write) there is a dependency from UniversalUpcallHandler (which > lives in abi) to SharedConstants (which lives in abi/x64). So we have > a dependency from a less specific abi construct to a more specific; > and that's probably a bad smell. > > * The sizeof methods in CallingSequenceBuilder are not used and can be > removed > > * ArgumentClass is in abi/x64, but StorageClass isn't. Not sure, in > general, in how do we differentiate between the two cases; in > principle, anything that speaks about x87 should probably live under > x64, but in reality, the shuffle recipes we generate rely on the fact > that we have integer/vector/stack arguments and integer/vector/x87 > returns... > > Maurizio > > > On 10/01/2019 22:58, Jorn Vernee wrote: >> Followup on this. >> >> CallingSequenceBuilderImpl and StandardCall have more difference than >> I realized. the arrangeCall method for the latter just lets you pass a >> NaitveMethodType and the argument layouts are grabbed from that. But >> on Windows we also have to differentiate between varargs and >> non-varargs, so the SysV strategy doesn't work for both. I think this >> is still solvable, of course, but I'd rather save that for a later >> patch if that's okay with you. I have moved the alignment methods into >> an abstract base class called CallingSequenceBuilder. The StorageNames >> thing was only used by the CallingSequenceBuilder implementations, so >> I've moved that logic into there as well. >> >> I've moved ArgumentClass to the abi/x64 package (Same for it's test), >> and also created a SharedConstants class with some of the x86_64 >> constant values which are also used by UniversalUpcallHandler (this >> was also referencing sysv/Constants before, which I changed to package >> access to avoid that in the future). >> >> I've replaced the UniversalBoxer with sub classes of >> UniversalNativeInvoker and UniversalUpcallHandler. >> >> I've changed microsoft -> windows for abi/x64/microsoft and >> Microsoftx64ABI. >> >> updated webrev: >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows/webrev.04/ >> >> Jorn >> >> Maurizio Cimadamore schreef op 2019-01-10 19:34: >>> On 10/01/2019 17:58, Jorn Vernee wrote: >>>> Hi Maurizio, >>>> >>>> Thanks for the review, comments inline... >>>> >>>> Maurizio Cimadamore schreef op 2019-01-10 18:15: >>>>> Continuing from [1]. >>>>> >>>>> This is a more in depth review of the binder changes. >>>>> >>>>> >>>>> Overall the changes look good, and what I see is more or less what >>>>> we >>>>> have discussed over the past few weeks. As a general >>>>> comment/overarching theme, I think we should try, as much as >>>>> possible, >>>>> to reuse code where possible, rather then duplicating it. >>>> >>>> Ok, I thought we would be going the other direction, so I did not >>>> spend much time yet on trying to de-duplicate code. I tried to touch >>>> the SysV and shared implementation as little as possible to keep >>>> maintenance easier, with the intent of de-duplicating code in a >>>> later patch if desirable. >>> >>> Yes, I understand. We can also push this as is and work on >>> incremental >>> improvements, but I'd like to unify code paths as possible, as I >>> think >>> maintaining two completely separated code-bases will be trickier in >>> the long run. That said, I like that, at least for prototyping, the >>> new ABI design allows for slotting in new stuff quite easily. >>> >>> >>> >>>> - ArgumentInfo/StorageCalculator keep only one counter for the >>>> number of registers on Windows while SysV needs 2. >>>> - StorageCalculator::addBindings has a special provision for >>>> varargs: >>> I might have gone too far here... >>>> >>>> >>>>> * The alignment routines in CallingSequenceBuilderImpl seem >>>>> identical >>>>> to the ones in SysVABI, so perhaps again these could be shared >>>>> somewhere. >>>> >>>> Same here, maybe we should bring back AbstractSystemABI? Or >>>> otherwise create an ABI utility class? >>> >>> Note that the alignment function is ultimately only used by the >>> CallBuilder. So, if we go for subclassing, I'd say put this in >>> CallingSequenceBuilder - and then have the ABI-dependent details in >>> the Windows/SysV impls. >>> >>> >>> Also, I forgot an important point: please replace "microsoft" with >>> "windows", as that's what's used elsewhere (e.g. in hotspot sources). >>> >>> >>> Maurizio >>> >>> >>>> >>>> Jorn >>>> >>>>> That's all I have for now. >>>>> >>>>> Cheers >>>>> Maurizio >>>>> >>>>> https://mail.openjdk.java.net/pipermail/panama-dev/2019-January/003680.html From maurizio.cimadamore at oracle.com Fri Jan 11 13:51:06 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Fri, 11 Jan 2019 13:51:06 +0000 Subject: [foreign] RFR 8216485: Binder changes are needed to support paname foreign API on Windows In-Reply-To: <515b6602b94a6aa3e75a64b05907edd9@xs4all.nl> References: <55f1ade8-c75d-fe78-1b04-1aff15ef8b4a@oracle.com> <54351e22f39cf216d2e4782521eeadae@xs4all.nl> <8a8542d9-3e9c-f4df-c7f8-ad458980f940@oracle.com> <515b6602b94a6aa3e75a64b05907edd9@xs4all.nl> Message-ID: I'm fine leaving things where they are, for now. Thanks! Maurizio On 11/01/2019 13:45, Jorn Vernee wrote: > I've removed the sizeof methods. > > About the other 2 points. These are smells, but they were already > present, and I think the problem goes a deeper as well. For instance, > lots of places use NativeTypes.UINT64 to represent a "slot" of memory, > but that would probably not hold up on 32bit platforms. So maybe we > can fully figure out the re-stacking when a non-x64 port comes along > and we have a clearer picture about the exact requirements? I can add > FIXMEs for now. > > As a quick fix, we could move all the classes in the abi package, > except SystemABI and UpcallStubs, into the abi/x64 package, since > there are evidently dependencies on x64 specific stuff. And then when > the next port comes along figure things out at that point? > > Jorn > > Maurizio Cimadamore schreef op 2019-01-11 11:09: >> Thanks, >> I think I'm still somewhat not 100% convinced this is the right >> stacking, but it's probably better to go ahead with this, and then see >> if we can simplify things a bit. All tests pass here, so no issues >> there. Some minor comments below: >> >> * One thing with the new SharedConstants is that I noted that (as you >> also write) there is a dependency from UniversalUpcallHandler (which >> lives in abi) to SharedConstants (which lives in abi/x64). So we have >> a dependency from a less specific abi construct to a more specific; >> and that's probably a bad smell. >> >> * The sizeof methods in CallingSequenceBuilder are not used and can >> be removed >> >> * ArgumentClass is in abi/x64, but StorageClass isn't. Not sure, in >> general, in how do we differentiate between the two cases; in >> principle, anything that speaks about x87 should probably live under >> x64, but in reality, the shuffle recipes we generate rely on the fact >> that we have integer/vector/stack arguments and integer/vector/x87 >> returns... >> >> Maurizio >> >> >> On 10/01/2019 22:58, Jorn Vernee wrote: >>> Followup on this. >>> >>> CallingSequenceBuilderImpl and StandardCall have more difference >>> than I realized. the arrangeCall method for the latter just lets you >>> pass a NaitveMethodType and the argument layouts are grabbed from >>> that. But on Windows we also have to differentiate between varargs >>> and non-varargs, so the SysV strategy doesn't work for both. I think >>> this is still solvable, of course, but I'd rather save that for a >>> later patch if that's okay with you. I have moved the alignment >>> methods into an abstract base class called CallingSequenceBuilder. >>> The StorageNames thing was only used by the CallingSequenceBuilder >>> implementations, so I've moved that logic into there as well. >>> >>> I've moved ArgumentClass to the abi/x64 package (Same for it's >>> test), and also created a SharedConstants class with some of the >>> x86_64 constant values which are also used by UniversalUpcallHandler >>> (this was also referencing sysv/Constants before, which I changed to >>> package access to avoid that in the future). >>> >>> I've replaced the UniversalBoxer with sub classes of >>> UniversalNativeInvoker and UniversalUpcallHandler. >>> >>> I've changed microsoft -> windows for abi/x64/microsoft and >>> Microsoftx64ABI. >>> >>> updated webrev: >>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows/webrev.04/ >>> >>> Jorn >>> >>> Maurizio Cimadamore schreef op 2019-01-10 19:34: >>>> On 10/01/2019 17:58, Jorn Vernee wrote: >>>>> Hi Maurizio, >>>>> >>>>> Thanks for the review, comments inline... >>>>> >>>>> Maurizio Cimadamore schreef op 2019-01-10 18:15: >>>>>> Continuing from [1]. >>>>>> >>>>>> This is a more in depth review of the binder changes. >>>>>> >>>>>> >>>>>> Overall the changes look good, and what I see is more or less >>>>>> what we >>>>>> have discussed over the past few weeks. As a general >>>>>> comment/overarching theme, I think we should try, as much as >>>>>> possible, >>>>>> to reuse code where possible, rather then duplicating it. >>>>> >>>>> Ok, I thought we would be going the other direction, so I did not >>>>> spend much time yet on trying to de-duplicate code. I tried to >>>>> touch the SysV and shared implementation as little as possible to >>>>> keep maintenance easier, with the intent of de-duplicating code in >>>>> a later patch if desirable. >>>> >>>> Yes, I understand. We can also push this as is and work on incremental >>>> improvements, but I'd like to unify code paths as possible, as I think >>>> maintaining two completely separated code-bases will be trickier in >>>> the long run. That said, I like that, at least for prototyping, the >>>> new ABI design allows for slotting in new stuff quite easily. >>>> >>>> >>>> >>>>> - ArgumentInfo/StorageCalculator keep only one counter for the >>>>> number of registers on Windows while SysV needs 2. >>>>> - StorageCalculator::addBindings has a special provision for varargs: >>>> I might have gone too far here... >>>>> >>>>> >>>>>> * The alignment routines in CallingSequenceBuilderImpl seem >>>>>> identical >>>>>> to the ones in SysVABI, so perhaps again these could be shared >>>>>> somewhere. >>>>> >>>>> Same here, maybe we should bring back AbstractSystemABI? Or >>>>> otherwise create an ABI utility class? >>>> >>>> Note that the alignment function is ultimately only used by the >>>> CallBuilder. So, if we go for subclassing, I'd say put this in >>>> CallingSequenceBuilder - and then have the ABI-dependent details in >>>> the Windows/SysV impls. >>>> >>>> >>>> Also, I forgot an important point: please replace "microsoft" with >>>> "windows", as that's what's used elsewhere (e.g. in hotspot sources). >>>> >>>> >>>> Maurizio >>>> >>>> >>>>> >>>>> Jorn >>>>> >>>>>> That's all I have for now. >>>>>> >>>>>> Cheers >>>>>> Maurizio >>>>>> >>>>>> https://mail.openjdk.java.net/pipermail/panama-dev/2019-January/003680.html >>>>>> From jbvernee at xs4all.nl Fri Jan 11 14:10:10 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Fri, 11 Jan 2019 15:10:10 +0100 Subject: [foreign] RFR 8216483: jextract tests have to be modified to run on Windows platform In-Reply-To: <9c768e8a2c341d086a1b2379c05811bd@xs4all.nl> References: <5C37FDA7.8000602@oracle.com> <5C3834C3.1060801@oracle.com> <5C389415.5020109@oracle.com> <9c768e8a2c341d086a1b2379c05811bd@xs4all.nl> Message-ID: You're right! There were some tests that checked the correctness of the NativeLocation annotations. I'll update the line numbers in the test files as well. Thanks, Jorn Jorn Vernee schreef op 2019-01-11 14:05: > Ok, I will run the tests to check, but I think it uses reflection to > do the comparison of those files so should be fine. > > Jorn > > Sundararajan Athijegannathan schreef op 2019-01-11 14:03: >> We need copyright in all files. For old files, please use 2018. For >> new files 2019. >> >> PS. But if there are any line number checks in the tests we need to >> modify the tests after adding copyright. >> >> -Sundar >> >> On 11/01/19, 6:30 PM, Jorn Vernee wrote: >>> I put copyright headers into the files that I created. >>> >>> There are also some other files missing copyright headers I noticed. >>> Should I add them as well, if so, what year should I put in the >>> header? >>> >>> Thanks, >>> Jorn >>> >>> Sundararajan Athijegannathan schreef op 2019-01-11 07:16: >>>> Looks good. >>>> >>>> Tests run fine on Linux as well. If all fine on Windows, please >>>> push. >>>> >>>> PS. New test source files should have copyright year 2019 >>>> >>>> -Sundar >>>> >>>> On 11/01/19, 7:51 AM, Sundararajan Athijegannathan wrote: >>>>> All test run fine on Mac with this latest patch! I'll test on Linux >>>>> as well and then send you review comments. >>>>> >>>>> -Sundar >>>>> >>>>> On 10/01/19, 9:40 PM, Jorn Vernee wrote: >>>>>> Hi, >>>>>> >>>>>> Continuation of [1]. >>>>>> >>>>>> updated webrev: >>>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_tests/webrev.04/ >>>>>> >>>>>> For the TestJextractFFI test I'm now just checking the operating >>>>>> system in the LibClang.java patch file: >>>>>> >>>>>> String libName = >>>>>> System.getProperty("os.name").startsWith("Windows") >>>>>> ? "libclang" >>>>>> : "clang"; >>>>>> Library libclang = >>>>>> Libraries.loadLibrary(MethodHandles.lookup(), libName); >>>>>> >>>>>> The other problem is that the dll is in the bin folder vs lib >>>>>> folder on Windows, and the test was using the linker path which is >>>>>> the lib folder on Windows. There was no build system variable >>>>>> available for the folder that contains the library file, so >>>>>> instead I'm deriving it from the library file path. >>>>>> >>>>>> Cheers, >>>>>> Jorn >>>>>> >>>>>> [1] : >>>>>> https://mail.openjdk.java.net/pipermail/panama-dev/2019-January/003691.html From samuel.audet at gmail.com Fri Jan 11 14:30:27 2019 From: samuel.audet at gmail.com (Samuel Audet) Date: Fri, 11 Jan 2019 23:30:27 +0900 Subject: Toward distributions of native libraries Message-ID: <51e5d7d3-6f95-9b33-1858-645bc388e17e@gmail.com> Hello, John, Maurizio, and all Panama developers, I wrote a post about the importance in having distributions of native libraries for Java, and where I see Panama fit in all this: http://bytedeco.org/news/2019/01/11/importance-of-a-distribution/ I hope it provides some context to help understand where you could best focus your efforts. I think I got all the facts straight, but please let me know of any inaccuracies! Thank you Samuel From jbvernee at xs4all.nl Fri Jan 11 15:03:04 2019 From: jbvernee at xs4all.nl (jbvernee at xs4all.nl) Date: Fri, 11 Jan 2019 15:03:04 +0000 Subject: hg: panama/dev: 2 new changesets Message-ID: <201901111503.x0BF35l9014874@aojmv0008.oracle.com> Changeset: 8638cb981169 Author: jvernee Date: 2019-01-11 13:55 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/8638cb981169 test tweaks needed to run on Windows ! make/RunTests.gmk ! make/test/JtregNativeJdk.gmk ! test/jdk/com/sun/tools/jextract/BadBitfieldTest.java ! test/jdk/com/sun/tools/jextract/EmptyStructTest.java ! test/jdk/com/sun/tools/jextract/Runner.java ! test/jdk/com/sun/tools/jextract/TestDowncallGenerator.java ! test/jdk/com/sun/tools/jextract/TestUpcallGenerator.java - test/jdk/com/sun/tools/jextract/TypedefAnonStruct.java ! test/jdk/com/sun/tools/jextract/bitfields.h - test/jdk/com/sun/tools/jextract/bitfields.java + test/jdk/com/sun/tools/jextract/compare/TypedefAnonStruct.java + test/jdk/com/sun/tools/jextract/compare/bitfields.java + test/jdk/com/sun/tools/jextract/compare/globalFuncPointer.java + test/jdk/com/sun/tools/jextract/compare/pad.java + test/jdk/com/sun/tools/jextract/compare/recursive.java + test/jdk/com/sun/tools/jextract/compare/simple.java + test/jdk/com/sun/tools/jextract/compare/windows/bitfields.java + test/jdk/com/sun/tools/jextract/compare/windows/simple.java ! test/jdk/com/sun/tools/jextract/complex/ComplexTest.java ! test/jdk/com/sun/tools/jextract/complex/libcomplex_aux.c ! test/jdk/com/sun/tools/jextract/complex/libcomplex_aux.h ! test/jdk/com/sun/tools/jextract/complex/mycomplex.h - test/jdk/com/sun/tools/jextract/globalFuncPointer.java ! test/jdk/com/sun/tools/jextract/jclang-ffi/TestJextractFFI.java ! test/jdk/com/sun/tools/jextract/jclang-ffi/src/jdk/internal/clang/LibClang.java ! test/jdk/com/sun/tools/jextract/libTestDowncall.c ! test/jdk/com/sun/tools/jextract/libTestDowncall.h ! test/jdk/com/sun/tools/jextract/libTestUpcall.c ! test/jdk/com/sun/tools/jextract/libTestUpcall.h ! test/jdk/com/sun/tools/jextract/missing/libMissing.c ! test/jdk/com/sun/tools/jextract/pad.h - test/jdk/com/sun/tools/jextract/pad.java - test/jdk/com/sun/tools/jextract/recursive.java - test/jdk/com/sun/tools/jextract/simple.java ! test/jdk/com/sun/tools/jextract/staticForwarder/libUtils.c ! test/jdk/com/sun/tools/jextract/staticForwarder/utils.h ! test/jdk/com/sun/tools/jextract/stdio/StdioTest.java ! test/jdk/com/sun/tools/jextract/testArrayFuncParam/funcArrayParam.h ! test/jdk/com/sun/tools/jextract/testArrayFuncParam/libFuncArrayParam.c ! test/jdk/com/sun/tools/jextract/testEnum/enums.h ! test/jdk/com/sun/tools/jextract/testEnum/libEnums.c ! test/jdk/com/sun/tools/jextract/testStruct/libStruct.c ! test/jdk/com/sun/tools/jextract/testStruct/struct.h ! test/jdk/java/foreign/BindLookupTest.java ! test/jdk/java/foreign/LongDoubleTest.java ! test/jdk/java/foreign/StructByValueTest.java ! test/jdk/java/foreign/System/UnixSystem.java ! test/jdk/java/foreign/Upcall/libUpcall.c ! test/jdk/java/foreign/getpid/Getpid.java ! test/jdk/java/foreign/libGlobalVariable.c ! test/jdk/java/foreign/libHello.c ! test/jdk/java/foreign/libPaddedStruct.c ! test/jdk/java/foreign/libRegisterStruct.c ! test/jdk/java/foreign/libstructbyvalue.c ! test/jdk/java/foreign/null/libNull.c ! test/jdk/java/foreign/printf/Util.java ! test/jdk/java/foreign/printf/stdio.java ! test/jdk/java/foreign/security/BindTest.java ! test/jdk/java/foreign/types/libArraysFunc.c ! test/jdk/java/foreign/types/libGlobalFunc.c ! test/jdk/java/foreign/types/libPointerScope.c ! test/jdk/java/foreign/types/libPointers.c Changeset: 387098970f01 Author: jvernee Date: 2019-01-11 16:01 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/387098970f01 Make sure test files have copyright headers ! test/jdk/com/sun/tools/jextract/compare/recursive.java ! test/jdk/com/sun/tools/jextract/compare/simple.java ! test/jdk/com/sun/tools/jextract/compare/windows/simple.java ! test/jdk/com/sun/tools/jextract/recursive.h ! test/jdk/com/sun/tools/jextract/simple.h From maurizio.cimadamore at oracle.com Fri Jan 11 15:10:50 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Fri, 11 Jan 2019 15:10:50 +0000 Subject: hg: panama/dev: Automatic merge with foreign Message-ID: <201901111510.x0BFAo0k020250@aojmv0008.oracle.com> Changeset: 3619463f6bcb Author: mcimadamore Date: 2019-01-11 16:16 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/3619463f6bcb Automatic merge with foreign - test/jdk/com/sun/tools/jextract/TypedefAnonStruct.java - test/jdk/com/sun/tools/jextract/bitfields.java - test/jdk/com/sun/tools/jextract/globalFuncPointer.java - test/jdk/com/sun/tools/jextract/pad.java - test/jdk/com/sun/tools/jextract/recursive.java - test/jdk/com/sun/tools/jextract/simple.java From jbvernee at xs4all.nl Fri Jan 11 15:13:01 2019 From: jbvernee at xs4all.nl (jbvernee at xs4all.nl) Date: Fri, 11 Jan 2019 15:13:01 +0000 Subject: hg: panama/dev: Alternative implementation of default library to make it work on Windows as well Message-ID: <201901111513.x0BFD14Q021162@aojmv0008.oracle.com> Changeset: 2104af107e6e Author: jvernee Date: 2019-01-11 16:11 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/2104af107e6e Alternative implementation of default library to make it work on Windows as well ! src/java.base/share/classes/java/lang/ClassLoader.java ! src/java.base/share/native/libjava/ClassLoader.c ! src/java.base/share/native/libjava/jni_util.h ! src/java.base/unix/native/libjava/jni_util_md.c ! src/java.base/windows/native/libjava/jni_util_md.c From maurizio.cimadamore at oracle.com Fri Jan 11 15:14:35 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Fri, 11 Jan 2019 15:14:35 +0000 Subject: hg: panama/dev: Automatic merge with foreign Message-ID: <201901111514.x0BFEaLO022572@aojmv0008.oracle.com> Changeset: fa129a8f3ee8 Author: mcimadamore Date: 2019-01-11 16:19 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/fa129a8f3ee8 Automatic merge with foreign From maurizio.cimadamore at oracle.com Fri Jan 11 15:10:25 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Fri, 11 Jan 2019 15:10:25 +0000 Subject: hg: panama/dev: Automatic merge with foreign Message-ID: <201901111510.x0BFAPVS019680@aojmv0008.oracle.com> Changeset: 72dba8c6ecc6 Author: mcimadamore Date: 2019-01-11 16:15 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/72dba8c6ecc6 Automatic merge with foreign - test/jdk/com/sun/tools/jextract/TypedefAnonStruct.java - test/jdk/com/sun/tools/jextract/bitfields.java - test/jdk/com/sun/tools/jextract/globalFuncPointer.java - test/jdk/com/sun/tools/jextract/pad.java - test/jdk/com/sun/tools/jextract/recursive.java - test/jdk/com/sun/tools/jextract/simple.java From maurizio.cimadamore at oracle.com Fri Jan 11 15:14:10 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Fri, 11 Jan 2019 15:14:10 +0000 Subject: hg: panama/dev: Automatic merge with foreign Message-ID: <201901111514.x0BFEBfB022105@aojmv0008.oracle.com> Changeset: f626330f517b Author: mcimadamore Date: 2019-01-11 16:19 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/f626330f517b Automatic merge with foreign From maurizio.cimadamore at oracle.com Fri Jan 11 15:48:48 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Fri, 11 Jan 2019 15:48:48 +0000 Subject: Toward distributions of native libraries In-Reply-To: <51e5d7d3-6f95-9b33-1858-645bc388e17e@gmail.com> References: <51e5d7d3-6f95-9b33-1858-645bc388e17e@gmail.com> Message-ID: <479f6425-f3f2-e4ec-abe4-9303996dcd82@oracle.com> One fact to straighten: "faster-than-JNI performance is not a priority" As you can see from [1], a branch was created almost two months ago and is actively worked on. It is not yet 100% stable on large benchmarks, but is good enough to pass all the tests. Vlad will surely post more here when the effort is closer to completion. Maurizio [1] - https://mail.openjdk.java.net/pipermail/panama-dev/2018-November/003250.html On 11/01/2019 14:30, Samuel Audet wrote: > Hello, John, Maurizio, and all Panama developers, > > I wrote a post about the importance in having distributions of native > libraries for Java, and where I see Panama fit in all this: > http://bytedeco.org/news/2019/01/11/importance-of-a-distribution/ > > I hope it provides some context to help understand where you could > best focus your efforts. I think I got all the facts straight, but > please let me know of any inaccuracies! Thank you > > Samuel From jbvernee at xs4all.nl Fri Jan 11 16:01:46 2019 From: jbvernee at xs4all.nl (jbvernee at xs4all.nl) Date: Fri, 11 Jan 2019 16:01:46 +0000 Subject: hg: panama/dev: 8216485: Binder changes are needed to support paname foreign API on Windows Message-ID: <201901111601.x0BG1kdK011096@aojmv0008.oracle.com> Changeset: b22ff94c5458 Author: jvernee Date: 2019-01-11 16:58 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/b22ff94c5458 8216485: Binder changes are needed to support paname foreign API on Windows Reviewed-by: mcimadamore ! src/hotspot/cpu/x86/assembler_x86.hpp ! src/hotspot/cpu/x86/foreign_globals_x86.hpp ! src/hotspot/cpu/x86/universalNativeInvoker_x86.cpp ! src/hotspot/cpu/x86/universalUpcallHandler_x86.cpp ! src/hotspot/share/prims/nativeLookup.cpp - src/hotspot/share/prims/sysVx86ABI.cpp + src/hotspot/share/prims/upcallStubs.cpp ! src/java.base/share/classes/jdk/internal/foreign/abi/CallingSequence.java ! src/java.base/share/classes/jdk/internal/foreign/abi/ShuffleRecipe.java ! src/java.base/share/classes/jdk/internal/foreign/abi/StorageClass.java ! src/java.base/share/classes/jdk/internal/foreign/abi/SystemABI.java ! src/java.base/share/classes/jdk/internal/foreign/abi/UniversalNativeInvoker.java ! src/java.base/share/classes/jdk/internal/foreign/abi/UniversalUpcallHandler.java + src/java.base/share/classes/jdk/internal/foreign/abi/UpcallStubs.java ! src/java.base/share/classes/jdk/internal/foreign/abi/VarargsInvoker.java + src/java.base/share/classes/jdk/internal/foreign/abi/x64/ArgumentClass.java + src/java.base/share/classes/jdk/internal/foreign/abi/x64/CallingSequenceBuilder.java + src/java.base/share/classes/jdk/internal/foreign/abi/x64/SharedConstants.java - src/java.base/share/classes/jdk/internal/foreign/abi/x64/sysv/ArgumentClass.java ! src/java.base/share/classes/jdk/internal/foreign/abi/x64/sysv/Constants.java ! src/java.base/share/classes/jdk/internal/foreign/abi/x64/sysv/StandardCall.java - src/java.base/share/classes/jdk/internal/foreign/abi/x64/sysv/StorageNames.java ! src/java.base/share/classes/jdk/internal/foreign/abi/x64/sysv/SysVx64ABI.java + src/java.base/share/classes/jdk/internal/foreign/abi/x64/sysv/UniversalNativeInvokerImpl.java + src/java.base/share/classes/jdk/internal/foreign/abi/x64/sysv/UniversalUpcallHandlerImpl.java + src/java.base/share/classes/jdk/internal/foreign/abi/x64/sysv/VarargsInvokerImpl.java + src/java.base/share/classes/jdk/internal/foreign/abi/x64/windows/CallingSequenceBuilderImpl.java + src/java.base/share/classes/jdk/internal/foreign/abi/x64/windows/Constants.java + src/java.base/share/classes/jdk/internal/foreign/abi/x64/windows/UniversalNativeInvokerImpl.java + src/java.base/share/classes/jdk/internal/foreign/abi/x64/windows/UniversalUpcallHandlerImpl.java + src/java.base/share/classes/jdk/internal/foreign/abi/x64/windows/VarargsInvokerImpl.java + src/java.base/share/classes/jdk/internal/foreign/abi/x64/windows/Windowsx64ABI.java + test/jdk/java/foreign/abi/x64/ArgumentClassTest.java + test/jdk/java/foreign/abi/x64/BitMatrix.java - test/jdk/java/foreign/abi/x64/sysv/ArgumentClassTest.java - test/jdk/java/foreign/abi/x64/sysv/BitMatrix.java From maurizio.cimadamore at oracle.com Fri Jan 11 16:04:01 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Fri, 11 Jan 2019 16:04:01 +0000 Subject: hg: panama/dev: Automatic merge with foreign Message-ID: <201901111604.x0BG41ah012367@aojmv0008.oracle.com> Changeset: 4dfa05246831 Author: mcimadamore Date: 2019-01-11 17:09 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/4dfa05246831 Automatic merge with foreign ! src/hotspot/cpu/x86/assembler_x86.hpp - src/hotspot/share/prims/sysVx86ABI.cpp - src/java.base/share/classes/jdk/internal/foreign/abi/x64/sysv/ArgumentClass.java - src/java.base/share/classes/jdk/internal/foreign/abi/x64/sysv/StorageNames.java - test/jdk/java/foreign/abi/x64/sysv/ArgumentClassTest.java - test/jdk/java/foreign/abi/x64/sysv/BitMatrix.java From jbvernee at xs4all.nl Fri Jan 11 16:18:51 2019 From: jbvernee at xs4all.nl (jbvernee at xs4all.nl) Date: Fri, 11 Jan 2019 16:18:51 +0000 Subject: hg: panama/dev: Adding missing copyright headers + fixing some incorrect years (retroactive addition to r54162) Message-ID: <201901111618.x0BGIpxZ019752@aojmv0008.oracle.com> Changeset: 59c605f5cfb5 Author: jvernee Date: 2019-01-11 17:18 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/59c605f5cfb5 Adding missing copyright headers + fixing some incorrect years (retroactive addition to r54162) ! src/java.base/share/classes/jdk/internal/foreign/abi/UpcallStubs.java ! src/java.base/share/classes/jdk/internal/foreign/abi/x64/CallingSequenceBuilder.java ! src/java.base/share/classes/jdk/internal/foreign/abi/x64/SharedConstants.java ! src/java.base/share/classes/jdk/internal/foreign/abi/x64/sysv/UniversalNativeInvokerImpl.java ! src/java.base/share/classes/jdk/internal/foreign/abi/x64/sysv/UniversalUpcallHandlerImpl.java ! src/java.base/share/classes/jdk/internal/foreign/abi/x64/sysv/VarargsInvokerImpl.java ! src/java.base/share/classes/jdk/internal/foreign/abi/x64/windows/CallingSequenceBuilderImpl.java ! src/java.base/share/classes/jdk/internal/foreign/abi/x64/windows/Constants.java ! src/java.base/share/classes/jdk/internal/foreign/abi/x64/windows/UniversalNativeInvokerImpl.java ! src/java.base/share/classes/jdk/internal/foreign/abi/x64/windows/UniversalUpcallHandlerImpl.java ! src/java.base/share/classes/jdk/internal/foreign/abi/x64/windows/VarargsInvokerImpl.java ! src/java.base/share/classes/jdk/internal/foreign/abi/x64/windows/Windowsx64ABI.java From maurizio.cimadamore at oracle.com Fri Jan 11 16:24:09 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Fri, 11 Jan 2019 16:24:09 +0000 Subject: hg: panama/dev: Automatic merge with foreign Message-ID: <201901111624.x0BGO9PO023097@aojmv0008.oracle.com> Changeset: 8ab9719b9243 Author: mcimadamore Date: 2019-01-11 17:29 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/8ab9719b9243 Automatic merge with foreign From maurizio.cimadamore at oracle.com Fri Jan 11 17:02:49 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Fri, 11 Jan 2019 17:02:49 +0000 Subject: hg: panama/dev: manual merge with foreign Message-ID: <201901111702.x0BH2nPe009038@aojmv0008.oracle.com> Changeset: 620639776d0c Author: mcimadamore Date: 2019-01-11 17:02 +0000 URL: http://hg.openjdk.java.net/panama/dev/rev/620639776d0c manual merge with foreign ! src/hotspot/share/prims/nativeLookup.cpp - src/hotspot/share/prims/sysVx86ABI.cpp + src/hotspot/share/prims/upcallStubs.cpp ! src/java.base/share/classes/jdk/internal/foreign/abi/UniversalNativeInvoker.java ! src/java.base/share/classes/jdk/internal/foreign/abi/UniversalUpcallHandler.java ! src/java.base/share/classes/jdk/internal/foreign/abi/VarargsInvoker.java + src/java.base/share/classes/jdk/internal/foreign/abi/x64/ArgumentClass.java - src/java.base/share/classes/jdk/internal/foreign/abi/x64/sysv/ArgumentClass.java ! src/java.base/share/classes/jdk/internal/foreign/abi/x64/sysv/Constants.java ! src/java.base/share/classes/jdk/internal/foreign/abi/x64/sysv/StandardCall.java - src/java.base/share/classes/jdk/internal/foreign/abi/x64/sysv/StorageNames.java ! src/java.base/share/classes/jdk/internal/foreign/abi/x64/sysv/SysVx64ABI.java + test/jdk/java/foreign/abi/x64/ArgumentClassTest.java + test/jdk/java/foreign/abi/x64/BitMatrix.java - test/jdk/java/foreign/abi/x64/sysv/ArgumentClassTest.java - test/jdk/java/foreign/abi/x64/sysv/BitMatrix.java From maurizio.cimadamore at oracle.com Fri Jan 11 17:09:27 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Fri, 11 Jan 2019 17:09:27 +0000 Subject: hg: panama/dev: Automatic merge with foreign Message-ID: <201901111709.x0BH9RWW013246@aojmv0008.oracle.com> Changeset: 175e39d93d82 Author: mcimadamore Date: 2019-01-11 18:14 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/175e39d93d82 Automatic merge with foreign From sandhya.viswanathan at intel.com Fri Jan 11 19:16:07 2019 From: sandhya.viswanathan at intel.com (Viswanathan, Sandhya) Date: Fri, 11 Jan 2019 19:16:07 +0000 Subject: [vector api] RFR: Fix the issue about float/double reduce-max/min operations with NaN as an input In-Reply-To: References: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A317E7@FMSMSX126.amr.corp.intel.com> <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A4D4F5@FMSMSX126.amr.corp.intel.com> Message-ID: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A4D9AB@FMSMSX126.amr.corp.intel.com> >>> 1) Vector Api min()/max() have been updated. Are there any missed? e.g. Float128Vector.java and similar occurrences in other files: 586 @Override 587 @ForceInline 588 public Float128Vector min(Vector o) { 589 Objects.requireNonNull(o); 590 Float128Vector v = (Float128Vector)o; 591 return (Float128Vector) VectorIntrinsics.binaryOp( 592 VECTOR_OP_MIN, Float128Vector.class, float.class, LENGTH, 593 this, v, 594 (v1, v2) -> ((Float128Vector)v1).bOp(v2, (i, a, b) -> (float) ((a < b) ? a : b))); 595 } 603 @Override 604 @ForceInline 605 public Float128Vector max(Vector o) { 606 Objects.requireNonNull(o); 607 Float128Vector v = (Float128Vector)o; 608 return VectorIntrinsics.binaryOp( 609 VECTOR_OP_MAX, Float128Vector.class, float.class, LENGTH, 610 this, v, 611 (v1, v2) -> v1.bOp(v2, (i, a, b) -> (float) ((a > b) ? a : b))); 612 } 613 Also FoatVector.java and similar occurrence in DoubleVector.java: 212 /** 213 * Returns the minimum of this vector and the broadcast of an input scalar. 214 *

215 * This is a vector binary operation where the operation 216 * {@code (a, b) -> a < b ? a : b} is applied to lane elements. 217 * 218 * @param s the input scalar 219 * @return the minimum of this vector and the broadcast of an input scalar 220 */ 221 public abstract FloatVector min(float s); 229 /** 230 * Returns the maximum of this vector and the broadcast of an input scalar. 231 *

232 * This is a vector binary operation where the operation 233 * {@code (a, b) -> a > b ? a : b} is applied to lane elements. 234 * 235 * @param s the input scalar 236 * @return the maximum of this vector and the broadcast of an input scalar 237 */ 238 public abstract FloatVector max(float s); Thanks, Sandhya -----Original Message----- From: Yang Zhang (Arm Technology China) [mailto:Yang.Zhang at arm.com] Sent: Thursday, January 10, 2019 7:23 PM To: Viswanathan, Sandhya ; 'panama-dev at openjdk.java.net' ; Vladimir Ivanov Cc: nd Subject: RE: [vector api] RFR: Fix the issue about float/double reduce-max/min operations with NaN as an input Hi Sandhya 1) Vector Api min()/max() have been updated. Are there any missed? 2) Math.min/max support float/double/int/long only, byte/short need casting. Current ((a Sent: Friday, January 11, 2019 8:33 AM To: Yang Zhang (Arm Technology China) ; 'panama-dev at openjdk.java.net' ; Vladimir Ivanov Cc: nd Subject: RE: [vector api] RFR: Fix the issue about float/double reduce-max/min operations with NaN as an input Hi Yang, Changing minAll and maxAll to use Math.min() and Math.max() looks good. I have a couple of more inputs below: 1) The Vector API min() and max() methods also need this change. 2) Math.min and Math.max should be used across data types for both integral and floating point. This is to be consistent all across. Best Regards, Sandhya -----Original Message----- From: Yang Zhang (Arm Technology China) [mailto:Yang.Zhang at arm.com] Sent: Wednesday, January 09, 2019 6:44 PM To: 'panama-dev at openjdk.java.net' ; Viswanathan, Sandhya ; Vladimir Ivanov Cc: nd Subject: RE: [vector api] RFR: Fix the issue about float/double reduce-max/min operations with NaN as an input Hi Vladimir, Sandhya Ping again. Could you please help to review this patch? Regards, Yang -----Original Message----- From: Yang Zhang Sent: Thursday, January 10, 2019 10:32 AM To: Yang Zhang (Arm Technology China) Subject: Fwd: [vector api] RFR: Fix the issue about float/double reduce-max/min operations with NaN as an input ---------- Forwarded message --------- From: Yang Zhang (Arm Technology China) Date: Fri, 14 Dec 2018 at 16:31 Subject: RE: [vector api] RFR: Fix the issue about float/double reduce-max/min operations with NaN as an input To: Viswanathan, Sandhya , panama-dev at openjdk.java.net Cc: nd Hi Sandhya Happy holidays. Let us keep on this work after holidays. Regards Yang -----Original Message----- From: Viswanathan, Sandhya Sent: Friday, December 14, 2018 9:43 AM To: Yang Zhang (Arm Technology China) ; panama-dev at openjdk.java.net Cc: nd Subject: RE: [vector api] RFR: Fix the issue about float/double reduce-max/min operations with NaN as an input Hi Yang, The current x86 intrinsics are based on the current definition as in Javadoc for Vector API float(double) min: public abstract FloatVector min(float s) Returns the minimum of this vector and the broadcast of an input scalar. This is a vector binary operation where the operation (a, b) -> a < b ? a : b is applied to lane elements. Parameters: s - the input scalar Returns: the minimum of this vector and the broadcast of an input scalar We are looking into making it closer to Math.min at least in the NaN related behavior. Let us wait till January middle or so to converge on this, it being too close to Christmas now and many are on vacation. Hope that is ok. Happy holidays. It is wonderful to see you and others from Arm technology onboard and contributing to Panama Vector API. Let us keep this collaboration going. Best Regards, Sandhya -----Original Message----- From: panama-dev [mailto:panama-dev-bounces at openjdk.java.net] On Behalf Of Yang Zhang (Arm Technology China) Sent: Wednesday, December 12, 2018 12:37 AM To: panama-dev at openjdk.java.net Cc: nd Subject: [vector api] RFR: Fix the issue about float/double reduce-max/min operations with NaN as an input Hi When I implement reduce-max/min, I found there is an issue about float/double reduce-max/min operations with NaN as an input. In java doc, if either value is NaN, then the result of max/min(a, b) is NaN. But the result of "((a > b) ? a : b)" is affected by sequence of (a, b). If a is NaN, the result is b. If b is NaN, the result is b (NaN). I have a patch which could fix this issue. Could you please help to review it? http://cr.openjdk.java.net/~yzhang/vectorapi.nan/webrev.00/ Ps. With this patch, there are 5 failures on x86 platform. jdk/incubator/vector/Float256VectorTests.java jdk/incubator/vector/Float64VectorTests.java jdk/incubator/vector/FloatMaxVectorTests.java jdk/incubator/vector/Double128VectorTests.java jdk/incubator/vector/DoubleMaxVectorTests.java Webrev: Regards, Yang From vladimir.x.ivanov at oracle.com Fri Jan 11 19:25:42 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 11 Jan 2019 11:25:42 -0800 Subject: [vector api] RFR: Fix the issue about float/double reduce-max/min operations with NaN as an input In-Reply-To: References: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A317E7@FMSMSX126.amr.corp.intel.com> <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A4D4F5@FMSMSX126.amr.corp.intel.com> Message-ID: On 10/01/2019 19:23, Yang Zhang (Arm Technology China) wrote: > Hi Sandhya > > 1) Vector Api min()/max() have been updated. Are there any missed? > 2) Math.min/max support float/double/int/long only, byte/short need casting. Current ((a -----Original Message----- > From: Viswanathan, Sandhya > Sent: Friday, January 11, 2019 8:33 AM > To: Yang Zhang (Arm Technology China) ; 'panama-dev at openjdk.java.net' ; Vladimir Ivanov > Cc: nd > Subject: RE: [vector api] RFR: Fix the issue about float/double reduce-max/min operations with NaN as an input > > Hi Yang, > > Changing minAll and maxAll to use Math.min() and Math.max() looks good. I have a couple of more inputs below: > 1) The Vector API min() and max() methods also need this change. > 2) Math.min and Math.max should be used across data types for both integral and floating point. > This is to be consistent all across. > > Best Regards, > Sandhya > > > -----Original Message----- > From: Yang Zhang (Arm Technology China) [mailto:Yang.Zhang at arm.com] > Sent: Wednesday, January 09, 2019 6:44 PM > To: 'panama-dev at openjdk.java.net' ; Viswanathan, Sandhya ; Vladimir Ivanov > Cc: nd > Subject: RE: [vector api] RFR: Fix the issue about float/double reduce-max/min operations with NaN as an input > > Hi Vladimir, Sandhya > > Ping again. Could you please help to review this patch? > > Regards, > Yang > > -----Original Message----- > From: Yang Zhang > Sent: Thursday, January 10, 2019 10:32 AM > To: Yang Zhang (Arm Technology China) > Subject: Fwd: [vector api] RFR: Fix the issue about float/double reduce-max/min operations with NaN as an input > > ---------- Forwarded message --------- > From: Yang Zhang (Arm Technology China) > Date: Fri, 14 Dec 2018 at 16:31 > Subject: RE: [vector api] RFR: Fix the issue about float/double reduce-max/min operations with NaN as an input > To: Viswanathan, Sandhya , panama-dev at openjdk.java.net > Cc: nd > > > Hi Sandhya > > Happy holidays. Let us keep on this work after holidays. > > Regards > Yang > > -----Original Message----- > From: Viswanathan, Sandhya > Sent: Friday, December 14, 2018 9:43 AM > To: Yang Zhang (Arm Technology China) ; panama-dev at openjdk.java.net > Cc: nd > Subject: RE: [vector api] RFR: Fix the issue about float/double reduce-max/min operations with NaN as an input > > Hi Yang, > > The current x86 intrinsics are based on the current definition as in Javadoc for Vector API float(double) min: > > public abstract FloatVector min(float s) Returns the minimum of this vector and the broadcast of an input scalar. > This is a vector binary operation where the operation (a, b) -> a < b ? a : b is applied to lane elements. > Parameters: > s - the input scalar > Returns: > the minimum of this vector and the broadcast of an input scalar > > We are looking into making it closer to Math.min at least in the NaN related behavior. Let us wait till January middle or so to converge on this, it being too close to Christmas now and many are on vacation. > Hope that is ok. > > Happy holidays. It is wonderful to see you and others from Arm technology onboard and contributing to Panama Vector API. Let us keep this collaboration going. > > Best Regards, > Sandhya > > > -----Original Message----- > From: panama-dev [mailto:panama-dev-bounces at openjdk.java.net] On Behalf Of Yang Zhang (Arm Technology China) > Sent: Wednesday, December 12, 2018 12:37 AM > To: panama-dev at openjdk.java.net > Cc: nd > Subject: [vector api] RFR: Fix the issue about float/double reduce-max/min operations with NaN as an input > > Hi > > When I implement reduce-max/min, I found there is an issue about float/double reduce-max/min operations with NaN as an input. > In java doc, if either value is NaN, then the result of max/min(a, b) is NaN. But the result of "((a > b) ? a : b)" is affected by sequence of (a, b). If a is NaN, the result is b. If b is NaN, the result is b (NaN). > > I have a patch which could fix this issue. Could you please help to review it? > http://cr.openjdk.java.net/~yzhang/vectorapi.nan/webrev.00/ > > Ps. With this patch, there are 5 failures on x86 platform. > jdk/incubator/vector/Float256VectorTests.java > jdk/incubator/vector/Float64VectorTests.java > jdk/incubator/vector/FloatMaxVectorTests.java > jdk/incubator/vector/Double128VectorTests.java > jdk/incubator/vector/DoubleMaxVectorTests.java > > Webrev: > > Regards, > Yang > From jbvernee at xs4all.nl Sat Jan 12 13:28:18 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Sat, 12 Jan 2019 14:28:18 +0100 Subject: [foreign] RFR 8216484: jextract LayoutUtils has to be modified for Windows Message-ID: <8350815b3f4fc665539424ae23699a2a@xs4all.nl> Hi, Continuing from [1, 2] I have polished the previous patch a little. Instead of doing the minimum needed for Windows support I've replaced all the LayoutUtils.Types constants, and instead am querying the type size from libclang and then caching the result, since libclang seems to know the right size to use for the platform. I have been looking at the libclang API and there doesn't seem to be a way to specify the target platform. I think it always uses the ABI of the platform it was built for. Bug: https://bugs.openjdk.java.net/browse/JDK-8216484 Updated Webrev: http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_jextract/webrev.01/ Cheers, Jorn [1] : https://mail.openjdk.java.net/pipermail/panama-dev/2019-January/003680.html [2] : https://mail.openjdk.java.net/pipermail/panama-dev/2019-January/003683.html From samuel.audet at gmail.com Sun Jan 13 03:42:53 2019 From: samuel.audet at gmail.com (Samuel Audet) Date: Sun, 13 Jan 2019 12:42:53 +0900 Subject: Toward distributions of native libraries In-Reply-To: <479f6425-f3f2-e4ec-abe4-9303996dcd82@oracle.com> References: <51e5d7d3-6f95-9b33-1858-645bc388e17e@gmail.com> <479f6425-f3f2-e4ec-abe4-9303996dcd82@oracle.com> Message-ID: Thanks for the feedback! If it were a priority, and if it were me, I would start by ensuring that the high-level API can also be optimized in the case of AOT, but this isn't happening at the moment. I just shortened it this way because I consider AOT to be just as important as JIT. How do you feel I should phrase this then? That it isn't a priority /only in the case of AOT/? I'll edit it that way if that's accurate! If not, I would welcome additional explanations, for example, see https://github.com/oracle/graal/issues/885 . Thanks Samuel 2019?1?12?(?) 0:48?Maurizio Cimadamore ???maurizio.cimadamore at oracle.com ???????: > One fact to straighten: > > "faster-than-JNI performance is not a priority" > > As you can see from [1], a branch was created almost two months ago and > is actively worked on. It is not yet 100% stable on large benchmarks, > but is good enough to pass all the tests. Vlad will surely post more > here when the effort is closer to completion. > > Maurizio > > [1] - > > https://mail.openjdk.java.net/pipermail/panama-dev/2018-November/003250.html > > > On 11/01/2019 14:30, Samuel Audet wrote: > > Hello, John, Maurizio, and all Panama developers, > > > > I wrote a post about the importance in having distributions of native > > libraries for Java, and where I see Panama fit in all this: > > http://bytedeco.org/news/2019/01/11/importance-of-a-distribution/ > > > > I hope it provides some context to help understand where you could > > best focus your efforts. I think I got all the facts straight, but > > please let me know of any inaccuracies! Thank you > > > > Samuel > From john.r.rose at oracle.com Mon Jan 14 06:04:01 2019 From: john.r.rose at oracle.com (John Rose) Date: Sun, 13 Jan 2019 22:04:01 -0800 Subject: Toward distributions of native libraries In-Reply-To: References: <51e5d7d3-6f95-9b33-1858-645bc388e17e@gmail.com> <479f6425-f3f2-e4ec-abe4-9303996dcd82@oracle.com> Message-ID: On Jan 12, 2019, at 7:42 PM, Samuel Audet wrote: > > Thanks for the feedback! If it were a priority, and if it were me, I would start by ensuring that the high-level API can also be optimized in the case of AOT, but this isn't happening at the moment. I just shortened it this way because I consider AOT to be just as important as JIT. How do you feel I should phrase this then? That it isn't a priority /only in the case of AOT/? I'll edit it that way if that's accurate! If not, I would welcome additional explanations, for example, see https://github.com/oracle/graal/issues/885 . Thanks Performance is important, period. The discussion on #885 peters out with you saying, "but can it work efficiently?". That's a complicated question, to which I believe the answer is almost certainly "yes". We are, fact, planning to make such things "work efficiently", and "such things" include JIT, AOT, and SVM compilation of Panama apps. So I agree with Maurizio's comment, that this bit of your blog post (a very good one!) would better reflect our intentions by omitting the comment about performance. If you wish, you could satisfy our observations by saying something straightforward like "Although the Panama people claim to value high performance, I don't think they can achieve it, especially for AOT." If that's true for you, in which case, wait and see how it all turns out. To put it in a nutshell: You misrepresent our priorities (a set of mental states Maurizio and I can and do authoritatively explain) by reasoning about their feasibility (which reasonable people can disagree about). ? John From sundararajan.athijegannathan at oracle.com Mon Jan 14 06:23:42 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Mon, 14 Jan 2019 11:53:42 +0530 Subject: [foreign] RFR 8216743: Regression: Many panama foreign tests fail on Mac after fix for JDK-8216485 Message-ID: <5C3C2AEE.7040309@oracle.com> Please review. Bug: https://bugs.openjdk.java.net/browse/JDK-8216743 Webrev: https://cr.openjdk.java.net/~sundar/8216743/webrev.00/ Moral: we've to test all the three platforms before pushing panama/foreign changes :) Thanks, -Sundar From sundararajan.athijegannathan at oracle.com Mon Jan 14 06:31:35 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Mon, 14 Jan 2019 12:01:35 +0530 Subject: [foreign] RFR 8216484: jextract LayoutUtils has to be modified for Windows In-Reply-To: <8350815b3f4fc665539424ae23699a2a@xs4all.nl> References: <8350815b3f4fc665539424ae23699a2a@xs4all.nl> Message-ID: <5C3C2CC7.7060106@oracle.com> jextract supports ToolProvider interface. It is possible to create multiple instances of jextract in the same process. LayoutUtils has a static cache which is initialized using sizes obtained from clang library. In theory it is possible that we may get different sizes for types depending on preprocessor settings, 32/64 bit app etc. I think it is better if we associate this cache with the current Context instance. -Sundar On 12/01/19, 6:58 PM, Jorn Vernee wrote: > Hi, > > Continuing from [1, 2] > > I have polished the previous patch a little. > > Instead of doing the minimum needed for Windows support I've replaced > all the LayoutUtils.Types constants, and instead am querying the type > size from libclang and then caching the result, since libclang seems > to know the right size to use for the platform. > > I have been looking at the libclang API and there doesn't seem to be a > way to specify the target platform. I think it always uses the ABI of > the platform it was built for. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8216484 > Updated Webrev: > http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_jextract/webrev.01/ > > Cheers, > Jorn > > [1] : > https://mail.openjdk.java.net/pipermail/panama-dev/2019-January/003680.html > [2] : > https://mail.openjdk.java.net/pipermail/panama-dev/2019-January/003683.html From Yang.Zhang at arm.com Mon Jan 14 09:55:36 2019 From: Yang.Zhang at arm.com (Yang Zhang (Arm Technology China)) Date: Mon, 14 Jan 2019 09:55:36 +0000 Subject: [vector api] RFR: Fix the issue about float/double reduce-max/min operations with NaN as an input In-Reply-To: References: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A317E7@FMSMSX126.amr.corp.intel.com> <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A4D4F5@FMSMSX126.amr.corp.intel.com> Message-ID: Hi Vladimir, Sandhya I have updated the patch according to your comments. Could you please help review it again? http://cr.openjdk.java.net/~yzhang/vectorapi.nan/webrev.01/ Regards Yang -----Original Message----- From: Vladimir Ivanov Sent: Saturday, January 12, 2019 3:26 AM To: Yang Zhang (Arm Technology China) ; Viswanathan, Sandhya ; 'panama-dev at openjdk.java.net' Cc: nd Subject: Re: [vector api] RFR: Fix the issue about float/double reduce-max/min operations with NaN as an input On 10/01/2019 19:23, Yang Zhang (Arm Technology China) wrote: > Hi Sandhya > > 1) Vector Api min()/max() have been updated. Are there any missed? > 2) Math.min/max support float/double/int/long only, byte/short need casting. Current ((a -----Original Message----- > From: Viswanathan, Sandhya > Sent: Friday, January 11, 2019 8:33 AM > To: Yang Zhang (Arm Technology China) ; > 'panama-dev at openjdk.java.net' ; Vladimir > Ivanov > Cc: nd > Subject: RE: [vector api] RFR: Fix the issue about float/double > reduce-max/min operations with NaN as an input > > Hi Yang, > > Changing minAll and maxAll to use Math.min() and Math.max() looks good. I have a couple of more inputs below: > 1) The Vector API min() and max() methods also need this change. > 2) Math.min and Math.max should be used across data types for both integral and floating point. > This is to be consistent all across. > > Best Regards, > Sandhya > > > -----Original Message----- > From: Yang Zhang (Arm Technology China) [mailto:Yang.Zhang at arm.com] > Sent: Wednesday, January 09, 2019 6:44 PM > To: 'panama-dev at openjdk.java.net' ; > Viswanathan, Sandhya ; Vladimir Ivanov > > Cc: nd > Subject: RE: [vector api] RFR: Fix the issue about float/double > reduce-max/min operations with NaN as an input > > Hi Vladimir, Sandhya > > Ping again. Could you please help to review this patch? > > Regards, > Yang > > -----Original Message----- > From: Yang Zhang > Sent: Thursday, January 10, 2019 10:32 AM > To: Yang Zhang (Arm Technology China) > Subject: Fwd: [vector api] RFR: Fix the issue about float/double > reduce-max/min operations with NaN as an input > > ---------- Forwarded message --------- > From: Yang Zhang (Arm Technology China) > Date: Fri, 14 Dec 2018 at 16:31 > Subject: RE: [vector api] RFR: Fix the issue about float/double > reduce-max/min operations with NaN as an input > To: Viswanathan, Sandhya , > panama-dev at openjdk.java.net > Cc: nd > > > Hi Sandhya > > Happy holidays. Let us keep on this work after holidays. > > Regards > Yang > > -----Original Message----- > From: Viswanathan, Sandhya > Sent: Friday, December 14, 2018 9:43 AM > To: Yang Zhang (Arm Technology China) ; > panama-dev at openjdk.java.net > Cc: nd > Subject: RE: [vector api] RFR: Fix the issue about float/double > reduce-max/min operations with NaN as an input > > Hi Yang, > > The current x86 intrinsics are based on the current definition as in Javadoc for Vector API float(double) min: > > public abstract FloatVector min(float s) Returns the minimum of this vector and the broadcast of an input scalar. > This is a vector binary operation where the operation (a, b) -> a < b ? a : b is applied to lane elements. > Parameters: > s - the input scalar > Returns: > the minimum of this vector and the broadcast of an input scalar > > We are looking into making it closer to Math.min at least in the NaN related behavior. Let us wait till January middle or so to converge on this, it being too close to Christmas now and many are on vacation. > Hope that is ok. > > Happy holidays. It is wonderful to see you and others from Arm technology onboard and contributing to Panama Vector API. Let us keep this collaboration going. > > Best Regards, > Sandhya > > > -----Original Message----- > From: panama-dev [mailto:panama-dev-bounces at openjdk.java.net] On > Behalf Of Yang Zhang (Arm Technology China) > Sent: Wednesday, December 12, 2018 12:37 AM > To: panama-dev at openjdk.java.net > Cc: nd > Subject: [vector api] RFR: Fix the issue about float/double > reduce-max/min operations with NaN as an input > > Hi > > When I implement reduce-max/min, I found there is an issue about float/double reduce-max/min operations with NaN as an input. > In java doc, if either value is NaN, then the result of max/min(a, b) is NaN. But the result of "((a > b) ? a : b)" is affected by sequence of (a, b). If a is NaN, the result is b. If b is NaN, the result is b (NaN). > > I have a patch which could fix this issue. Could you please help to review it? > http://cr.openjdk.java.net/~yzhang/vectorapi.nan/webrev.00/ > > Ps. With this patch, there are 5 failures on x86 platform. > jdk/incubator/vector/Float256VectorTests.java > jdk/incubator/vector/Float64VectorTests.java > jdk/incubator/vector/FloatMaxVectorTests.java > jdk/incubator/vector/Double128VectorTests.java > jdk/incubator/vector/DoubleMaxVectorTests.java > > Webrev: > > Regards, > Yang > From maurizio.cimadamore at oracle.com Mon Jan 14 10:54:23 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 14 Jan 2019 10:54:23 +0000 Subject: [foreign] RFR 8216743: Regression: Many panama foreign tests fail on Mac after fix for JDK-8216485 In-Reply-To: <5C3C2AEE.7040309@oracle.com> References: <5C3C2AEE.7040309@oracle.com> Message-ID: <6a2f39c7-efee-ebce-183f-b0067a810660@oracle.com> Looks good Maurizio On 14/01/2019 06:23, Sundararajan Athijegannathan wrote: > Please review. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8216743 > Webrev: https://cr.openjdk.java.net/~sundar/8216743/webrev.00/ > > Moral: we've to test all the three platforms before pushing > panama/foreign changes :) > > Thanks, > -Sundar From maurizio.cimadamore at oracle.com Mon Jan 14 11:44:29 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Mon, 14 Jan 2019 11:44:29 +0000 Subject: hg: panama/dev: Automatic merge with foreign Message-ID: <201901141144.x0EBiT4s008245@aojmv0008.oracle.com> Changeset: cf657ae92581 Author: mcimadamore Date: 2019-01-14 12:49 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/cf657ae92581 Automatic merge with foreign From sundararajan.athijegannathan at oracle.com Mon Jan 14 11:41:57 2019 From: sundararajan.athijegannathan at oracle.com (sundararajan.athijegannathan at oracle.com) Date: Mon, 14 Jan 2019 11:41:57 +0000 Subject: hg: panama/dev: 8216743: Regression: Many panama foreign tests fail on Mac after fix for JDK-8216485 Message-ID: <201901141141.x0EBfwQm006592@aojmv0008.oracle.com> Changeset: bdf098d3d83b Author: sundar Date: 2019-01-14 17:11 +0530 URL: http://hg.openjdk.java.net/panama/dev/rev/bdf098d3d83b 8216743: Regression: Many panama foreign tests fail on Mac after fix for JDK-8216485 Reviewed-by: mcimadamore ! src/java.base/share/classes/jdk/internal/foreign/abi/SystemABI.java From maurizio.cimadamore at oracle.com Mon Jan 14 11:44:03 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Mon, 14 Jan 2019 11:44:03 +0000 Subject: hg: panama/dev: Automatic merge with foreign Message-ID: <201901141144.x0EBi40L007728@aojmv0008.oracle.com> Changeset: 50c6ae0246ca Author: mcimadamore Date: 2019-01-14 12:49 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/50c6ae0246ca Automatic merge with foreign From maurizio.cimadamore at oracle.com Mon Jan 14 14:33:59 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 14 Jan 2019 14:33:59 +0000 Subject: [foreign] RFR 8216484: jextract LayoutUtils has to be modified for Windows In-Reply-To: <5C3C2CC7.7060106@oracle.com> References: <8350815b3f4fc665539424ae23699a2a@xs4all.nl> <5C3C2CC7.7060106@oracle.com> Message-ID: <26c62a72-bbe6-7abb-853f-aa1df4ff8628@oracle.com> Or, remove the cache entirely (we can add it back later if performance really becomes a concern) Maurizio On 14/01/2019 06:31, Sundararajan Athijegannathan wrote: > jextract supports ToolProvider interface. It is possible to create > multiple instances of jextract in the same process. LayoutUtils has? a > static cache which is initialized using sizes obtained from clang > library. In theory it is possible that we may get different sizes for > types depending on preprocessor settings, 32/64 bit app etc. I think > it is better if we associate this cache with the current Context > instance. > > -Sundar > > On 12/01/19, 6:58 PM, Jorn Vernee wrote: >> Hi, >> >> Continuing from [1, 2] >> >> I have polished the previous patch a little. >> >> Instead of doing the minimum needed for Windows support I've replaced >> all the LayoutUtils.Types constants, and instead am querying the type >> size from libclang and then caching the result, since libclang seems >> to know the right size to use for the platform. >> >> I have been looking at the libclang API and there doesn't seem to be >> a way to specify the target platform. I think it always uses the ABI >> of the platform it was built for. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8216484 >> Updated Webrev: >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_jextract/webrev.01/ >> >> Cheers, >> Jorn >> >> [1] : >> https://mail.openjdk.java.net/pipermail/panama-dev/2019-January/003680.html >> [2] : >> https://mail.openjdk.java.net/pipermail/panama-dev/2019-January/003683.html From jbvernee at xs4all.nl Mon Jan 14 15:19:59 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Mon, 14 Jan 2019 16:19:59 +0100 Subject: [foreign] RFR 8216484: jextract LayoutUtils has to be modified for Windows In-Reply-To: <26c62a72-bbe6-7abb-853f-aa1df4ff8628@oracle.com> References: <8350815b3f4fc665539424ae23699a2a@xs4all.nl> <5C3C2CC7.7060106@oracle.com> <26c62a72-bbe6-7abb-853f-aa1df4ff8628@oracle.com> Message-ID: <21749e7bf8f4664d8c50e3ba9fd08d18@xs4all.nl> I think this sounds good. It's probably a premature optimization to cache the Layouts since they are not expensive to create in this case. (plus no-cache is much simpler to implement) updated webrev: http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_jextract/webrev.02/ Jorn Maurizio Cimadamore schreef op 2019-01-14 15:33: > Or, remove the cache entirely (we can add it back later if performance > really becomes a concern) > > Maurizio > > On 14/01/2019 06:31, Sundararajan Athijegannathan wrote: >> jextract supports ToolProvider interface. It is possible to create >> multiple instances of jextract in the same process. LayoutUtils has? a >> static cache which is initialized using sizes obtained from clang >> library. In theory it is possible that we may get different sizes for >> types depending on preprocessor settings, 32/64 bit app etc. I think >> it is better if we associate this cache with the current Context >> instance. >> >> -Sundar >> >> On 12/01/19, 6:58 PM, Jorn Vernee wrote: >>> Hi, >>> >>> Continuing from [1, 2] >>> >>> I have polished the previous patch a little. >>> >>> Instead of doing the minimum needed for Windows support I've replaced >>> all the LayoutUtils.Types constants, and instead am querying the type >>> size from libclang and then caching the result, since libclang seems >>> to know the right size to use for the platform. >>> >>> I have been looking at the libclang API and there doesn't seem to be >>> a way to specify the target platform. I think it always uses the ABI >>> of the platform it was built for. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8216484 >>> Updated Webrev: >>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_jextract/webrev.01/ >>> >>> Cheers, >>> Jorn >>> >>> [1] : >>> https://mail.openjdk.java.net/pipermail/panama-dev/2019-January/003680.html >>> [2] : >>> https://mail.openjdk.java.net/pipermail/panama-dev/2019-January/003683.html From maurizio.cimadamore at oracle.com Mon Jan 14 15:59:36 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 14 Jan 2019 15:59:36 +0000 Subject: [foreign] RFR 8216484: jextract LayoutUtils has to be modified for Windows In-Reply-To: <21749e7bf8f4664d8c50e3ba9fd08d18@xs4all.nl> References: <8350815b3f4fc665539424ae23699a2a@xs4all.nl> <5C3C2CC7.7060106@oracle.com> <26c62a72-bbe6-7abb-853f-aa1df4ff8628@oracle.com> <21749e7bf8f4664d8c50e3ba9fd08d18@xs4all.nl> Message-ID: <4052b963-c6d5-8f21-02f9-57e8fdf95155@oracle.com> Looks good to me Maurizio On 14/01/2019 15:19, Jorn Vernee wrote: > I think this sounds good. It's probably a premature optimization to > cache the Layouts since they are not expensive to create in this case. > (plus no-cache is much simpler to implement) > > updated webrev: > http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_jextract/webrev.02/ > > Jorn > > Maurizio Cimadamore schreef op 2019-01-14 15:33: >> Or, remove the cache entirely (we can add it back later if performance >> really becomes a concern) >> >> Maurizio >> >> On 14/01/2019 06:31, Sundararajan Athijegannathan wrote: >>> jextract supports ToolProvider interface. It is possible to create >>> multiple instances of jextract in the same process. LayoutUtils has? >>> a static cache which is initialized using sizes obtained from clang >>> library. In theory it is possible that we may get different sizes >>> for types depending on preprocessor settings, 32/64 bit app etc. I >>> think it is better if we associate this cache with the current >>> Context instance. >>> >>> -Sundar >>> >>> On 12/01/19, 6:58 PM, Jorn Vernee wrote: >>>> Hi, >>>> >>>> Continuing from [1, 2] >>>> >>>> I have polished the previous patch a little. >>>> >>>> Instead of doing the minimum needed for Windows support I've >>>> replaced all the LayoutUtils.Types constants, and instead am >>>> querying the type size from libclang and then caching the result, >>>> since libclang seems to know the right size to use for the platform. >>>> >>>> I have been looking at the libclang API and there doesn't seem to >>>> be a way to specify the target platform. I think it always uses the >>>> ABI of the platform it was built for. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8216484 >>>> Updated Webrev: >>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_jextract/webrev.01/ >>>> >>>> Cheers, >>>> Jorn >>>> >>>> [1] : >>>> https://mail.openjdk.java.net/pipermail/panama-dev/2019-January/003680.html >>>> [2] : >>>> https://mail.openjdk.java.net/pipermail/panama-dev/2019-January/003683.html From jbvernee at xs4all.nl Mon Jan 14 16:14:43 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Mon, 14 Jan 2019 17:14:43 +0100 Subject: [foreign] RFR 8216484: jextract LayoutUtils has to be modified for Windows In-Reply-To: <4052b963-c6d5-8f21-02f9-57e8fdf95155@oracle.com> References: <8350815b3f4fc665539424ae23699a2a@xs4all.nl> <5C3C2CC7.7060106@oracle.com> <26c62a72-bbe6-7abb-853f-aa1df4ff8628@oracle.com> <21749e7bf8f4664d8c50e3ba9fd08d18@xs4all.nl> <4052b963-c6d5-8f21-02f9-57e8fdf95155@oracle.com> Message-ID: <2c32694e63a1dbb916154cd71ef84e2b@xs4all.nl> Thanks for the reviews! Pushed. Jorn Maurizio Cimadamore schreef op 2019-01-14 16:59: > Looks good to me > > Maurizio > > On 14/01/2019 15:19, Jorn Vernee wrote: >> I think this sounds good. It's probably a premature optimization to >> cache the Layouts since they are not expensive to create in this case. >> (plus no-cache is much simpler to implement) >> >> updated webrev: >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_jextract/webrev.02/ >> >> Jorn >> >> Maurizio Cimadamore schreef op 2019-01-14 15:33: >>> Or, remove the cache entirely (we can add it back later if >>> performance >>> really becomes a concern) >>> >>> Maurizio >>> >>> On 14/01/2019 06:31, Sundararajan Athijegannathan wrote: >>>> jextract supports ToolProvider interface. It is possible to create >>>> multiple instances of jextract in the same process. LayoutUtils has? >>>> a static cache which is initialized using sizes obtained from clang >>>> library. In theory it is possible that we may get different sizes >>>> for types depending on preprocessor settings, 32/64 bit app etc. I >>>> think it is better if we associate this cache with the current >>>> Context instance. >>>> >>>> -Sundar >>>> >>>> On 12/01/19, 6:58 PM, Jorn Vernee wrote: >>>>> Hi, >>>>> >>>>> Continuing from [1, 2] >>>>> >>>>> I have polished the previous patch a little. >>>>> >>>>> Instead of doing the minimum needed for Windows support I've >>>>> replaced all the LayoutUtils.Types constants, and instead am >>>>> querying the type size from libclang and then caching the result, >>>>> since libclang seems to know the right size to use for the >>>>> platform. >>>>> >>>>> I have been looking at the libclang API and there doesn't seem to >>>>> be a way to specify the target platform. I think it always uses the >>>>> ABI of the platform it was built for. >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8216484 >>>>> Updated Webrev: >>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/windows_jextract/webrev.01/ >>>>> >>>>> Cheers, >>>>> Jorn >>>>> >>>>> [1] : >>>>> https://mail.openjdk.java.net/pipermail/panama-dev/2019-January/003680.html >>>>> [2] : >>>>> https://mail.openjdk.java.net/pipermail/panama-dev/2019-January/003683.html From jbvernee at xs4all.nl Mon Jan 14 16:14:01 2019 From: jbvernee at xs4all.nl (jbvernee at xs4all.nl) Date: Mon, 14 Jan 2019 16:14:01 +0000 Subject: hg: panama/dev: 8216484: jextract LayoutUtils has to be modified for Windows Message-ID: <201901141614.x0EGE1UH000841@aojmv0008.oracle.com> Changeset: bbdbe049f87e Author: jvernee Date: 2019-01-14 17:12 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/bbdbe049f87e 8216484: jextract LayoutUtils has to be modified for Windows Summary: remove LayoutUtils.Types and query libclang for size of a type instead Reviewed-by: sundar, mcimadamore ! src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/LayoutUtils.java From maurizio.cimadamore at oracle.com Mon Jan 14 16:19:30 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Mon, 14 Jan 2019 16:19:30 +0000 Subject: hg: panama/dev: Automatic merge with foreign Message-ID: <201901141619.x0EGJUFT003447@aojmv0008.oracle.com> Changeset: d7885ab60131 Author: mcimadamore Date: 2019-01-14 17:24 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/d7885ab60131 Automatic merge with foreign From maurizio.cimadamore at oracle.com Mon Jan 14 16:19:04 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Mon, 14 Jan 2019 16:19:04 +0000 Subject: hg: panama/dev: Automatic merge with foreign Message-ID: <201901141619.x0EGJ5bs002986@aojmv0008.oracle.com> Changeset: 64aa56d24ad0 Author: mcimadamore Date: 2019-01-14 17:24 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/64aa56d24ad0 Automatic merge with foreign From tomasz.kowalczewski at gmail.com Mon Jan 14 17:56:04 2019 From: tomasz.kowalczewski at gmail.com (Tomasz Kowalczewski) Date: Mon, 14 Jan 2019 18:56:04 +0100 Subject: Vector::rearrange(Shuffle) allocations Message-ID: I have a question about vector rearranges. My recent observation is that using IntVector.rearrange causes a lot of allocations (in a JMH microbenchmark). Is this expected? I wrote a few simple (but not trivial) algorithms and they all did not box vectors into objects and all tests were able to run using EpsilonGC. The intrinsic works and vpermd instruction is emitted. There are 64 bytes allocated per benchmark iteration in the following code (the addAll is used so that vector does not escape the method): @Benchmark public int rearrange() { IntVector input = SPECIES_I256.fromArray(inputArray, 0); IntVector rearrangedVector = input.rearrange(SHUFFLE_1); return rearrangedVector.addAll(); } There is no allocation in following code: @Benchmark public int noRearrange() { IntVector input = SPECIES_I256.fromArray(inputArray, 0); return input.addAll(); } Output from jmap -histo for first case: num #instances #bytes class name (module) ------------------------------------------------------- 1: 4122313 198969024 [I (java.base@) 2: 4119794 65916704 jdk.incubator.vector.Int256Vector (jdk.incubator.vector@) 3: 8913 808512 [B (java.base@) 4: 1982 222408 java.lang.Class (java.base@) 5: 8638 207312 java.lang.String (java.base@) Any hints will be much appreciated. -- Thanks, Tomasz Kowalczewski From vladimir.x.ivanov at oracle.com Mon Jan 14 18:39:34 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Mon, 14 Jan 2019 10:39:34 -0800 Subject: [vector api] RFR: Fix the issue about float/double reduce-max/min operations with NaN as an input In-Reply-To: References: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A317E7@FMSMSX126.amr.corp.intel.com> <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A4D4F5@FMSMSX126.amr.corp.intel.com> Message-ID: <1772b48e-c09f-55ee-2647-d569a85a98f4@oracle.com> > http://cr.openjdk.java.net/~yzhang/vectorapi.nan/webrev.01/ src/jdk.incubator.vector/share/classes/jdk/incubator/vector/X-VectorBits.java.template: - (v1, v2) -> (($vectortype$)v1).bOp(v2, (i, a, b) -> ($type$) ((a < b) ? a : b))); +#if[byteOrShort] + (v1, v2) -> v1.bOp(v2, (i, a, b) -> ($type$) Math.min((int) a, (int) b))); +#else[byteOrShort] + (v1, v2) -> v1.bOp(v2, (i, a, b) -> ($type$) Math.min(a, b))); +#end[byteOrShort] I don't see a compelling reason for explicit casts of byte/short arguments to int. It's handled well implicitly. Getting rid of those should significantly simplify the patch. Otherwise, looks good! Best regards, Vladimir Ivanov > -----Original Message----- > From: Vladimir Ivanov > Sent: Saturday, January 12, 2019 3:26 AM > To: Yang Zhang (Arm Technology China) ; Viswanathan, Sandhya ; 'panama-dev at openjdk.java.net' > Cc: nd > Subject: Re: [vector api] RFR: Fix the issue about float/double reduce-max/min operations with NaN as an input > > > > On 10/01/2019 19:23, Yang Zhang (Arm Technology China) wrote: >> Hi Sandhya >> >> 1) Vector Api min()/max() have been updated. Are there any missed? >> 2) Math.min/max support float/double/int/long only, byte/short need casting. Current ((a > For consistency, I would prefer uniform usage of Math.min()/max() across the javadoc/implementation and casts when needed. > > Best regards, > Vladimir Ivanov > >> -----Original Message----- >> From: Viswanathan, Sandhya >> Sent: Friday, January 11, 2019 8:33 AM >> To: Yang Zhang (Arm Technology China) ; >> 'panama-dev at openjdk.java.net' ; Vladimir >> Ivanov >> Cc: nd >> Subject: RE: [vector api] RFR: Fix the issue about float/double >> reduce-max/min operations with NaN as an input >> >> Hi Yang, >> >> Changing minAll and maxAll to use Math.min() and Math.max() looks good. I have a couple of more inputs below: >> 1) The Vector API min() and max() methods also need this change. >> 2) Math.min and Math.max should be used across data types for both integral and floating point. >> This is to be consistent all across. >> >> Best Regards, >> Sandhya >> >> >> -----Original Message----- >> From: Yang Zhang (Arm Technology China) [mailto:Yang.Zhang at arm.com] >> Sent: Wednesday, January 09, 2019 6:44 PM >> To: 'panama-dev at openjdk.java.net' ; >> Viswanathan, Sandhya ; Vladimir Ivanov >> >> Cc: nd >> Subject: RE: [vector api] RFR: Fix the issue about float/double >> reduce-max/min operations with NaN as an input >> >> Hi Vladimir, Sandhya >> >> Ping again. Could you please help to review this patch? >> >> Regards, >> Yang >> >> -----Original Message----- >> From: Yang Zhang >> Sent: Thursday, January 10, 2019 10:32 AM >> To: Yang Zhang (Arm Technology China) >> Subject: Fwd: [vector api] RFR: Fix the issue about float/double >> reduce-max/min operations with NaN as an input >> >> ---------- Forwarded message --------- >> From: Yang Zhang (Arm Technology China) >> Date: Fri, 14 Dec 2018 at 16:31 >> Subject: RE: [vector api] RFR: Fix the issue about float/double >> reduce-max/min operations with NaN as an input >> To: Viswanathan, Sandhya , >> panama-dev at openjdk.java.net >> Cc: nd >> >> >> Hi Sandhya >> >> Happy holidays. Let us keep on this work after holidays. >> >> Regards >> Yang >> >> -----Original Message----- >> From: Viswanathan, Sandhya >> Sent: Friday, December 14, 2018 9:43 AM >> To: Yang Zhang (Arm Technology China) ; >> panama-dev at openjdk.java.net >> Cc: nd >> Subject: RE: [vector api] RFR: Fix the issue about float/double >> reduce-max/min operations with NaN as an input >> >> Hi Yang, >> >> The current x86 intrinsics are based on the current definition as in Javadoc for Vector API float(double) min: >> >> public abstract FloatVector min(float s) Returns the minimum of this vector and the broadcast of an input scalar. >> This is a vector binary operation where the operation (a, b) -> a < b ? a : b is applied to lane elements. >> Parameters: >> s - the input scalar >> Returns: >> the minimum of this vector and the broadcast of an input scalar >> >> We are looking into making it closer to Math.min at least in the NaN related behavior. Let us wait till January middle or so to converge on this, it being too close to Christmas now and many are on vacation. >> Hope that is ok. >> >> Happy holidays. It is wonderful to see you and others from Arm technology onboard and contributing to Panama Vector API. Let us keep this collaboration going. >> >> Best Regards, >> Sandhya >> >> >> -----Original Message----- >> From: panama-dev [mailto:panama-dev-bounces at openjdk.java.net] On >> Behalf Of Yang Zhang (Arm Technology China) >> Sent: Wednesday, December 12, 2018 12:37 AM >> To: panama-dev at openjdk.java.net >> Cc: nd >> Subject: [vector api] RFR: Fix the issue about float/double >> reduce-max/min operations with NaN as an input >> >> Hi >> >> When I implement reduce-max/min, I found there is an issue about float/double reduce-max/min operations with NaN as an input. >> In java doc, if either value is NaN, then the result of max/min(a, b) is NaN. But the result of "((a > b) ? a : b)" is affected by sequence of (a, b). If a is NaN, the result is b. If b is NaN, the result is b (NaN). >> >> I have a patch which could fix this issue. Could you please help to review it? >> http://cr.openjdk.java.net/~yzhang/vectorapi.nan/webrev.00/ >> >> Ps. With this patch, there are 5 failures on x86 platform. >> jdk/incubator/vector/Float256VectorTests.java >> jdk/incubator/vector/Float64VectorTests.java >> jdk/incubator/vector/FloatMaxVectorTests.java >> jdk/incubator/vector/Double128VectorTests.java >> jdk/incubator/vector/DoubleMaxVectorTests.java >> >> Webrev: >> >> Regards, >> Yang >> From Yang.Zhang at arm.com Tue Jan 15 06:04:52 2019 From: Yang.Zhang at arm.com (Yang Zhang (Arm Technology China)) Date: Tue, 15 Jan 2019 06:04:52 +0000 Subject: [vector api] RFR: Fix the issue about float/double reduce-max/min operations with NaN as an input In-Reply-To: <1772b48e-c09f-55ee-2647-d569a85a98f4@oracle.com> References: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A317E7@FMSMSX126.amr.corp.intel.com> <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A4D4F5@FMSMSX126.amr.corp.intel.com> <1772b48e-c09f-55ee-2647-d569a85a98f4@oracle.com> Message-ID: Hi Vladimir I removed explicit casts of byte/short arguments to int. Please review it. http://cr.openjdk.java.net/~yzhang/vectorapi.nan/webrev.02/ Regards Yang -----Original Message----- From: Vladimir Ivanov Sent: Tuesday, January 15, 2019 2:40 AM To: Yang Zhang (Arm Technology China) ; Viswanathan, Sandhya ; 'panama-dev at openjdk.java.net' Cc: nd Subject: Re: [vector api] RFR: Fix the issue about float/double reduce-max/min operations with NaN as an input > http://cr.openjdk.java.net/~yzhang/vectorapi.nan/webrev.01/ src/jdk.incubator.vector/share/classes/jdk/incubator/vector/X-VectorBits.java.template: - (v1, v2) -> (($vectortype$)v1).bOp(v2, (i, a, b) -> ($type$) ((a < b) ? a : b))); +#if[byteOrShort] + (v1, v2) -> v1.bOp(v2, (i, a, b) -> ($type$) Math.min((int) a, (int) b))); +#else[byteOrShort] + (v1, v2) -> v1.bOp(v2, (i, a, b) -> ($type$) Math.min(a, +b))); #end[byteOrShort] I don't see a compelling reason for explicit casts of byte/short arguments to int. It's handled well implicitly. Getting rid of those should significantly simplify the patch. Otherwise, looks good! Best regards, Vladimir Ivanov > -----Original Message----- > From: Vladimir Ivanov > Sent: Saturday, January 12, 2019 3:26 AM > To: Yang Zhang (Arm Technology China) ; > Viswanathan, Sandhya ; > 'panama-dev at openjdk.java.net' > Cc: nd > Subject: Re: [vector api] RFR: Fix the issue about float/double > reduce-max/min operations with NaN as an input > > > > On 10/01/2019 19:23, Yang Zhang (Arm Technology China) wrote: >> Hi Sandhya >> >> 1) Vector Api min()/max() have been updated. Are there any missed? >> 2) Math.min/max support float/double/int/long only, byte/short need casting. Current ((a > For consistency, I would prefer uniform usage of Math.min()/max() across the javadoc/implementation and casts when needed. > > Best regards, > Vladimir Ivanov > >> -----Original Message----- >> From: Viswanathan, Sandhya >> Sent: Friday, January 11, 2019 8:33 AM >> To: Yang Zhang (Arm Technology China) ; >> 'panama-dev at openjdk.java.net' ; Vladimir >> Ivanov >> Cc: nd >> Subject: RE: [vector api] RFR: Fix the issue about float/double >> reduce-max/min operations with NaN as an input >> >> Hi Yang, >> >> Changing minAll and maxAll to use Math.min() and Math.max() looks good. I have a couple of more inputs below: >> 1) The Vector API min() and max() methods also need this change. >> 2) Math.min and Math.max should be used across data types for both integral and floating point. >> This is to be consistent all across. >> >> Best Regards, >> Sandhya >> >> >> -----Original Message----- >> From: Yang Zhang (Arm Technology China) [mailto:Yang.Zhang at arm.com] >> Sent: Wednesday, January 09, 2019 6:44 PM >> To: 'panama-dev at openjdk.java.net' ; >> Viswanathan, Sandhya ; Vladimir Ivanov >> >> Cc: nd >> Subject: RE: [vector api] RFR: Fix the issue about float/double >> reduce-max/min operations with NaN as an input >> >> Hi Vladimir, Sandhya >> >> Ping again. Could you please help to review this patch? >> >> Regards, >> Yang >> >> -----Original Message----- >> From: Yang Zhang >> Sent: Thursday, January 10, 2019 10:32 AM >> To: Yang Zhang (Arm Technology China) >> Subject: Fwd: [vector api] RFR: Fix the issue about float/double >> reduce-max/min operations with NaN as an input >> >> ---------- Forwarded message --------- >> From: Yang Zhang (Arm Technology China) >> Date: Fri, 14 Dec 2018 at 16:31 >> Subject: RE: [vector api] RFR: Fix the issue about float/double >> reduce-max/min operations with NaN as an input >> To: Viswanathan, Sandhya , >> panama-dev at openjdk.java.net >> Cc: nd >> >> >> Hi Sandhya >> >> Happy holidays. Let us keep on this work after holidays. >> >> Regards >> Yang >> >> -----Original Message----- >> From: Viswanathan, Sandhya >> Sent: Friday, December 14, 2018 9:43 AM >> To: Yang Zhang (Arm Technology China) ; >> panama-dev at openjdk.java.net >> Cc: nd >> Subject: RE: [vector api] RFR: Fix the issue about float/double >> reduce-max/min operations with NaN as an input >> >> Hi Yang, >> >> The current x86 intrinsics are based on the current definition as in Javadoc for Vector API float(double) min: >> >> public abstract FloatVector min(float s) Returns the minimum of this vector and the broadcast of an input scalar. >> This is a vector binary operation where the operation (a, b) -> a < b ? a : b is applied to lane elements. >> Parameters: >> s - the input scalar >> Returns: >> the minimum of this vector and the broadcast of an input scalar >> >> We are looking into making it closer to Math.min at least in the NaN related behavior. Let us wait till January middle or so to converge on this, it being too close to Christmas now and many are on vacation. >> Hope that is ok. >> >> Happy holidays. It is wonderful to see you and others from Arm technology onboard and contributing to Panama Vector API. Let us keep this collaboration going. >> >> Best Regards, >> Sandhya >> >> >> -----Original Message----- >> From: panama-dev [mailto:panama-dev-bounces at openjdk.java.net] On >> Behalf Of Yang Zhang (Arm Technology China) >> Sent: Wednesday, December 12, 2018 12:37 AM >> To: panama-dev at openjdk.java.net >> Cc: nd >> Subject: [vector api] RFR: Fix the issue about float/double >> reduce-max/min operations with NaN as an input >> >> Hi >> >> When I implement reduce-max/min, I found there is an issue about float/double reduce-max/min operations with NaN as an input. >> In java doc, if either value is NaN, then the result of max/min(a, b) is NaN. But the result of "((a > b) ? a : b)" is affected by sequence of (a, b). If a is NaN, the result is b. If b is NaN, the result is b (NaN). >> >> I have a patch which could fix this issue. Could you please help to review it? >> http://cr.openjdk.java.net/~yzhang/vectorapi.nan/webrev.00/ >> >> Ps. With this patch, there are 5 failures on x86 platform. >> jdk/incubator/vector/Float256VectorTests.java >> jdk/incubator/vector/Float64VectorTests.java >> jdk/incubator/vector/FloatMaxVectorTests.java >> jdk/incubator/vector/Double128VectorTests.java >> jdk/incubator/vector/DoubleMaxVectorTests.java >> >> Webrev: >> >> Regards, >> Yang >> From Yang.Zhang at arm.com Tue Jan 15 06:38:21 2019 From: Yang.Zhang at arm.com (Yang Zhang (Arm Technology China)) Date: Tue, 15 Jan 2019 06:38:21 +0000 Subject: [vector api] RFR: Fix byte vector test failures about bad ad file and unsupported vector size for AArch64 Message-ID: Hi On AArch64 platform, the following tests failed because of bad ad file and unsupported vector size. jdk/incubator/vector/Byte128VectorTests.java jdk/incubator/vector/Byte256VectorTests.java jdk/incubator/vector/Byte512VectorTests.java jdk/incubator/vector/ByteMaxVectorTests.java I have a patch which could fix these failures. Could you please help to review it? http://cr.openjdk.java.net/~yzhang/vectorapi.vect4/webrev.00/ Regards, Yang From zhuoren.wz at alibaba-inc.com Tue Jan 15 16:37:49 2019 From: zhuoren.wz at alibaba-inc.com (=?UTF-8?B?V2FuZyBaaHVvKFpodW9yZW4p?=) Date: Wed, 16 Jan 2019 00:37:49 +0800 Subject: =?UTF-8?B?W1ZlY3RvckFQSV1tYXNrRnJvbUFycmF5IEludHJpbnNpYyBmYWlsdXJl?= Message-ID: <701fcf82-7ace-4e66-910b-3120753809a8.zhuoren.wz@alibaba-inc.com> Hi, I found intrinsics for maskFromArray always failed. Here is a patch to fix this issue. Benchmarks containing heavy maskFromArray use can benifit from this patch. such as allTrue/anyTrue. This patch consists of two parts. In the JDK part, since boolean is only one byte, it is not correct to perform left shift ARRAY_SHIFT for array index. I listed only one file in the patch to avoid a long letter, but this modification should be applied to all types including the template file. return VectorIntrinsics.load(Int128Mask.class, int.class, LENGTH, - bits, (((long) ix) << ARRAY_SHIFT) + Unsafe.ARRAY_BOOLEAN_BASE_OFFSET, + bits, ((long) ix) + Unsafe.ARRAY_BOOLEAN_BASE_OFFSET, bits, ix, Another part is in hotspot, please check the below patch diff -r 890e996dfd1a src/hotspot/share/opto/library_call.cpp --- a/src/hotspot/share/opto/library_call.cpp Mon Jan 07 10:54:21 2019 +0800 +++ b/src/hotspot/share/opto/library_call.cpp Tue Jan 15 23:45:40 2019 +0800 @@ -7069,13 +7069,13 @@ // Now handle special case where load/store happens from/to byte array but element type is not byte. bool using_byte_array = arr_type != NULL && arr_type->elem()->array_element_basic_type() == T_BYTE && elem_bt != T_BYTE; - - // It must be the case that if it is not special byte array case, there is consistency between - // array and vector element types. - if (!using_byte_array && arr_type != NULL && elem_bt != arr_type->elem()->array_element_basic_type()) { + // Handle loading masks. + ciKlass* vbox_klass = vector_klass->const_oop()->as_instance()->java_lang_Class_klass(); + bool is_mask = vbox_klass->is_vectormask(); + // If there is no consistency between array and vector element types, it must be special byte array case or loading masks + if (!using_byte_array && arr_type != NULL && elem_bt != arr_type->elem()->array_element_basic_type() && !is_mask) { return false; } - // Since we are using byte array, we need to double check that the byte operations are supported by backend. if (using_byte_array) { int byte_num_elem = num_elem * type2aelembytes(elem_bt); @@ -7083,8 +7083,12 @@ return false; // not supported } } - - ciKlass* vbox_klass = vector_klass->const_oop()->as_instance()->java_lang_Class_klass(); + if (is_mask) { + if (!arch_supports_vector(Op_LoadVector, num_elem, T_BOOLEAN, VecMaskUseLoad)) { + return false; // not supported + } + } + const TypeInstPtr* vbox_type = TypeInstPtr::make_exact(TypePtr::NotNull, vbox_klass); if (is_store) { @@ -7114,9 +7118,15 @@ const TypeVect* to_vect_type = TypeVect::make(elem_bt, num_elem); vload = gvn().transform(new VectorReinterpretNode(vload, vload->bottom_type()->is_vect(), to_vect_type)); } else { - vload = gvn().transform(LoadVectorNode::make(0, control(), memory(addr), addr, addr_type, num_elem, elem_bt)); - } - + // Special handle for masks + if (is_mask) { + vload = gvn().transform(LoadVectorNode::make(0, control(), memory(addr), addr, addr_type, num_elem, T_BOOLEAN)); + const TypeVect* to_vect_type = TypeVect::make(elem_bt, num_elem); + vload = gvn().transform(new VectorLoadMaskNode(vload, to_vect_type)); + } else { + vload = gvn().transform(LoadVectorNode::make(0, control(), memory(addr), addr, addr_type, num_elem, elem_bt)); + } + } Node* box = box_vector(vload, vbox_type, elem_bt, num_elem); set_vector_result(box); } --- a/src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Int128Vector.java Mon Jan 07 10:54:21 2019 +0800 +++ b/src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Int128Vector.java Tue Jan 15 23:45:40 2019 +0800 @@ -1338,7 +1338,7 @@ Objects.requireNonNull(bits); ix = VectorIntrinsics.checkIndex(ix, bits.length, LENGTH); return VectorIntrinsics.load(Int128Mask.class, int.class, LENGTH, - bits, (((long) ix) << ARRAY_SHIFT) + Unsafe.ARRAY_BOOLEAN_BASE_OFFSET, + bits, ((long) ix) + Unsafe.ARRAY_BOOLEAN_BASE_OFFSET, bits, ix, (c, idx) -> opm(n -> c[idx + n])); Regards, Zhuoren From jbvernee at xs4all.nl Tue Jan 15 17:12:36 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Tue, 15 Jan 2019 18:12:36 +0100 Subject: [foreign] jextract does not filter typedefs, enums, and structs Message-ID: Hello, Now that I'm done with the initial Windows support I have started looking for interesting Windows APIs to create tests/demos with. Pretty much any part of the Windows API requires users to include Windows.h, which is a huge header that pretty much includes everything and currently requires 14GB of RAM to extract. Trying to jextract a sub-header breaks the pre-processor code in those headers. So, I'm using `--include-symbols` to select just the things I need, but I find TreeFilter [1] lacks filtering of typedefs, structs, and enums. So a lot of unneeded classes are still being generated. Why are these Tree types not currently filtered? Thanks, Jorn [1] : http://hg.openjdk.java.net/panama/dev/file/d7885ab60131/src/jdk.jextract/share/classes/com/sun/tools/jextract/TreeFilter.java From maurizio.cimadamore at oracle.com Tue Jan 15 17:18:15 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 15 Jan 2019 17:18:15 +0000 Subject: [foreign] jextract does not filter typedefs, enums, and structs In-Reply-To: References: Message-ID: <7f7643c7-ccd5-eb6a-2492-4f23434da92f@oracle.com> This is a known problem - we will get there :-) I think it's already a good thing that extraction kind of works (albeit in a suboptimal way) :-) The goal is to use a library-centered approach where if you know which set of symbols you want (which can be inferred e.g. from a DLL, or with a regex), we will have a way to compute the minimum set of 'roots' that are needed in order to define a self-contained extraction unit. Maurizio On 15/01/2019 17:12, Jorn Vernee wrote: > Hello, > > Now that I'm done with the initial Windows support I have started > looking for interesting Windows APIs to create tests/demos with. > Pretty much any part of the Windows API requires users to include > Windows.h, which is a huge header that pretty much includes everything > and currently requires 14GB of RAM to extract. Trying to jextract a > sub-header breaks the pre-processor code in those headers. > > So, I'm using `--include-symbols` to select just the things I need, > but I find TreeFilter [1] lacks filtering of typedefs, structs, and > enums. So a lot of unneeded classes are still being generated. Why are > these Tree types not currently filtered? > > Thanks, > Jorn > > [1] : > http://hg.openjdk.java.net/panama/dev/file/d7885ab60131/src/jdk.jextract/share/classes/com/sun/tools/jextract/TreeFilter.java From vladimir.x.ivanov at oracle.com Tue Jan 15 17:52:29 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 15 Jan 2019 09:52:29 -0800 Subject: [vector api] RFR: Fix the issue about float/double reduce-max/min operations with NaN as an input In-Reply-To: References: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A317E7@FMSMSX126.amr.corp.intel.com> <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A4D4F5@FMSMSX126.amr.corp.intel.com> <1772b48e-c09f-55ee-2647-d569a85a98f4@oracle.com> Message-ID: <8d1cbd2f-1805-a099-7e19-833551d085f6@oracle.com> > http://cr.openjdk.java.net/~yzhang/vectorapi.nan/webrev.02/ Looks good! Thanks. Best regards, Vladimir Ivanov > -----Original Message----- > From: Vladimir Ivanov > Sent: Tuesday, January 15, 2019 2:40 AM > To: Yang Zhang (Arm Technology China) ; Viswanathan, Sandhya ; 'panama-dev at openjdk.java.net' > Cc: nd > Subject: Re: [vector api] RFR: Fix the issue about float/double reduce-max/min operations with NaN as an input > > > >> http://cr.openjdk.java.net/~yzhang/vectorapi.nan/webrev.01/ > > src/jdk.incubator.vector/share/classes/jdk/incubator/vector/X-VectorBits.java.template: > > - (v1, v2) -> (($vectortype$)v1).bOp(v2, (i, a, b) -> > ($type$) ((a < b) ? a : b))); > +#if[byteOrShort] > + (v1, v2) -> v1.bOp(v2, (i, a, b) -> ($type$) Math.min((int) > a, (int) b))); > +#else[byteOrShort] > + (v1, v2) -> v1.bOp(v2, (i, a, b) -> ($type$) Math.min(a, > +b))); #end[byteOrShort] > > I don't see a compelling reason for explicit casts of byte/short arguments to int. It's handled well implicitly. Getting rid of those should significantly simplify the patch. > > Otherwise, looks good! > > Best regards, > Vladimir Ivanov > >> -----Original Message----- >> From: Vladimir Ivanov >> Sent: Saturday, January 12, 2019 3:26 AM >> To: Yang Zhang (Arm Technology China) ; >> Viswanathan, Sandhya ; >> 'panama-dev at openjdk.java.net' >> Cc: nd >> Subject: Re: [vector api] RFR: Fix the issue about float/double >> reduce-max/min operations with NaN as an input >> >> >> >> On 10/01/2019 19:23, Yang Zhang (Arm Technology China) wrote: >>> Hi Sandhya >>> >>> 1) Vector Api min()/max() have been updated. Are there any missed? >>> 2) Math.min/max support float/double/int/long only, byte/short need casting. Current ((a> >> For consistency, I would prefer uniform usage of Math.min()/max() across the javadoc/implementation and casts when needed. >> >> Best regards, >> Vladimir Ivanov >> >>> -----Original Message----- >>> From: Viswanathan, Sandhya >>> Sent: Friday, January 11, 2019 8:33 AM >>> To: Yang Zhang (Arm Technology China) ; >>> 'panama-dev at openjdk.java.net' ; Vladimir >>> Ivanov >>> Cc: nd >>> Subject: RE: [vector api] RFR: Fix the issue about float/double >>> reduce-max/min operations with NaN as an input >>> >>> Hi Yang, >>> >>> Changing minAll and maxAll to use Math.min() and Math.max() looks good. I have a couple of more inputs below: >>> 1) The Vector API min() and max() methods also need this change. >>> 2) Math.min and Math.max should be used across data types for both integral and floating point. >>> This is to be consistent all across. >>> >>> Best Regards, >>> Sandhya >>> >>> >>> -----Original Message----- >>> From: Yang Zhang (Arm Technology China) [mailto:Yang.Zhang at arm.com] >>> Sent: Wednesday, January 09, 2019 6:44 PM >>> To: 'panama-dev at openjdk.java.net' ; >>> Viswanathan, Sandhya ; Vladimir Ivanov >>> >>> Cc: nd >>> Subject: RE: [vector api] RFR: Fix the issue about float/double >>> reduce-max/min operations with NaN as an input >>> >>> Hi Vladimir, Sandhya >>> >>> Ping again. Could you please help to review this patch? >>> >>> Regards, >>> Yang >>> >>> -----Original Message----- >>> From: Yang Zhang >>> Sent: Thursday, January 10, 2019 10:32 AM >>> To: Yang Zhang (Arm Technology China) >>> Subject: Fwd: [vector api] RFR: Fix the issue about float/double >>> reduce-max/min operations with NaN as an input >>> >>> ---------- Forwarded message --------- >>> From: Yang Zhang (Arm Technology China) >>> Date: Fri, 14 Dec 2018 at 16:31 >>> Subject: RE: [vector api] RFR: Fix the issue about float/double >>> reduce-max/min operations with NaN as an input >>> To: Viswanathan, Sandhya , >>> panama-dev at openjdk.java.net >>> Cc: nd >>> >>> >>> Hi Sandhya >>> >>> Happy holidays. Let us keep on this work after holidays. >>> >>> Regards >>> Yang >>> >>> -----Original Message----- >>> From: Viswanathan, Sandhya >>> Sent: Friday, December 14, 2018 9:43 AM >>> To: Yang Zhang (Arm Technology China) ; >>> panama-dev at openjdk.java.net >>> Cc: nd >>> Subject: RE: [vector api] RFR: Fix the issue about float/double >>> reduce-max/min operations with NaN as an input >>> >>> Hi Yang, >>> >>> The current x86 intrinsics are based on the current definition as in Javadoc for Vector API float(double) min: >>> >>> public abstract FloatVector min(float s) Returns the minimum of this vector and the broadcast of an input scalar. >>> This is a vector binary operation where the operation (a, b) -> a < b ? a : b is applied to lane elements. >>> Parameters: >>> s - the input scalar >>> Returns: >>> the minimum of this vector and the broadcast of an input scalar >>> >>> We are looking into making it closer to Math.min at least in the NaN related behavior. Let us wait till January middle or so to converge on this, it being too close to Christmas now and many are on vacation. >>> Hope that is ok. >>> >>> Happy holidays. It is wonderful to see you and others from Arm technology onboard and contributing to Panama Vector API. Let us keep this collaboration going. >>> >>> Best Regards, >>> Sandhya >>> >>> >>> -----Original Message----- >>> From: panama-dev [mailto:panama-dev-bounces at openjdk.java.net] On >>> Behalf Of Yang Zhang (Arm Technology China) >>> Sent: Wednesday, December 12, 2018 12:37 AM >>> To: panama-dev at openjdk.java.net >>> Cc: nd >>> Subject: [vector api] RFR: Fix the issue about float/double >>> reduce-max/min operations with NaN as an input >>> >>> Hi >>> >>> When I implement reduce-max/min, I found there is an issue about float/double reduce-max/min operations with NaN as an input. >>> In java doc, if either value is NaN, then the result of max/min(a, b) is NaN. But the result of "((a > b) ? a : b)" is affected by sequence of (a, b). If a is NaN, the result is b. If b is NaN, the result is b (NaN). >>> >>> I have a patch which could fix this issue. Could you please help to review it? >>> http://cr.openjdk.java.net/~yzhang/vectorapi.nan/webrev.00/ >>> >>> Ps. With this patch, there are 5 failures on x86 platform. >>> jdk/incubator/vector/Float256VectorTests.java >>> jdk/incubator/vector/Float64VectorTests.java >>> jdk/incubator/vector/FloatMaxVectorTests.java >>> jdk/incubator/vector/Double128VectorTests.java >>> jdk/incubator/vector/DoubleMaxVectorTests.java >>> >>> Webrev: >>> >>> Regards, >>> Yang >>> From jbvernee at xs4all.nl Tue Jan 15 18:30:18 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Tue, 15 Jan 2019 19:30:18 +0100 Subject: [foreign] jextract does not filter typedefs, enums, and structs In-Reply-To: <7f7643c7-ccd5-eb6a-2492-4f23434da92f@oracle.com> References: <7f7643c7-ccd5-eb6a-2492-4f23434da92f@oracle.com> Message-ID: Some kind of 'transitive include' is exactly what I was looking for :) (and I found the JBS ticket [1]) Deriving a list of symbols to extract from a library also sounds like a interesting option. Thanks, Jorn [1] : https://bugs.openjdk.java.net/browse/JDK-8214543 Maurizio Cimadamore schreef op 2019-01-15 18:18: > This is a known problem - we will get there :-) > > I think it's already a good thing that extraction kind of works > (albeit in a suboptimal way) :-) > > The goal is to use a library-centered approach where if you know which > set of symbols you want (which can be inferred e.g. from a DLL, or > with a regex), we will have a way to compute the minimum set of > 'roots' that are needed in order to define a self-contained extraction > unit. > > Maurizio > > On 15/01/2019 17:12, Jorn Vernee wrote: >> Hello, >> >> Now that I'm done with the initial Windows support I have started >> looking for interesting Windows APIs to create tests/demos with. >> Pretty much any part of the Windows API requires users to include >> Windows.h, which is a huge header that pretty much includes everything >> and currently requires 14GB of RAM to extract. Trying to jextract a >> sub-header breaks the pre-processor code in those headers. >> >> So, I'm using `--include-symbols` to select just the things I need, >> but I find TreeFilter [1] lacks filtering of typedefs, structs, and >> enums. So a lot of unneeded classes are still being generated. Why are >> these Tree types not currently filtered? >> >> Thanks, >> Jorn >> >> [1] : >> http://hg.openjdk.java.net/panama/dev/file/d7885ab60131/src/jdk.jextract/share/classes/com/sun/tools/jextract/TreeFilter.java From vladimir.x.ivanov at oracle.com Tue Jan 15 18:46:21 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 15 Jan 2019 10:46:21 -0800 Subject: [VectorAPI]maskFromArray Intrinsic failure In-Reply-To: <701fcf82-7ace-4e66-910b-3120753809a8.zhuoren.wz@alibaba-inc.com> References: <701fcf82-7ace-4e66-910b-3120753809a8.zhuoren.wz@alibaba-inc.com> Message-ID: <7b720dbb-91d8-4bef-e680-9474ead264f2@oracle.com> Good catch! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Int128Vector.java: > - bits, (((long) ix) << ARRAY_SHIFT) + Unsafe.ARRAY_BOOLEAN_BASE_OFFSET, > + bits, ((long) ix) + Unsafe.ARRAY_BOOLEAN_BASE_OFFSET, It's better to replace ARRAY_SHIFT with Unsafe.ARRAY_BOOLEAN_INDEX_SCALE. src/hotspot/share/opto/library_call.cpp: > + if (!arch_supports_vector(Op_LoadVector, num_elem, T_BOOLEAN, VecMaskUseLoad)) { Why do you specify VecMaskUseLoad and not VecMaskNotUsed (as in other cases)? Otherwise, looks good for now. Please, send complete webrev for review. As a followup, it would be nice to refactor/cleanup Mask-related code on JDK-JVM boundary (VectorIntrinsics + library_call.cpp). For example, I don't see a reason to pass int.class (and not boolean.class) as the element type for mask-related operations. Best regards, Vladimir Ivanov On 15/01/2019 08:37, Wang Zhuo(Zhuoren) wrote: > Hi, > I found intrinsics for maskFromArray always failed. Here is a patch to fix this issue. > Benchmarks containing heavy maskFromArray use can benifit from this patch. such as allTrue/anyTrue. > > This patch consists of two parts. In the JDK part, since boolean is only one byte, it is not correct to perform left shift ARRAY_SHIFT for array index. I listed only one file in the patch to avoid a long letter, but this modification should be applied to all types including the template file. > return VectorIntrinsics.load(Int128Mask.class, int.class, LENGTH, > - bits, (((long) ix) << ARRAY_SHIFT) + Unsafe.ARRAY_BOOLEAN_BASE_OFFSET, > + bits, ((long) ix) + Unsafe.ARRAY_BOOLEAN_BASE_OFFSET, > bits, ix, > Another part is in hotspot, please check the below patch > > diff -r 890e996dfd1a src/hotspot/share/opto/library_call.cpp > --- a/src/hotspot/share/opto/library_call.cpp Mon Jan 07 10:54:21 2019 +0800 > +++ b/src/hotspot/share/opto/library_call.cpp Tue Jan 15 23:45:40 2019 +0800 > @@ -7069,13 +7069,13 @@ > > // Now handle special case where load/store happens from/to byte array but element type is not byte. > bool using_byte_array = arr_type != NULL && arr_type->elem()->array_element_basic_type() == T_BYTE && elem_bt != T_BYTE; > - > - // It must be the case that if it is not special byte array case, there is consistency between > - // array and vector element types. > - if (!using_byte_array && arr_type != NULL && elem_bt != arr_type->elem()->array_element_basic_type()) { > + // Handle loading masks. > + ciKlass* vbox_klass = vector_klass->const_oop()->as_instance()->java_lang_Class_klass(); > + bool is_mask = vbox_klass->is_vectormask(); > + // If there is no consistency between array and vector element types, it must be special byte array case or loading masks > + if (!using_byte_array && arr_type != NULL && elem_bt != arr_type->elem()->array_element_basic_type() && !is_mask) { > return false; > } > - > // Since we are using byte array, we need to double check that the byte operations are supported by backend. > if (using_byte_array) { > int byte_num_elem = num_elem * type2aelembytes(elem_bt); > @@ -7083,8 +7083,12 @@ > return false; // not supported > } > } > - > - ciKlass* vbox_klass = vector_klass->const_oop()->as_instance()->java_lang_Class_klass(); > + if (is_mask) { > + if (!arch_supports_vector(Op_LoadVector, num_elem, T_BOOLEAN, VecMaskUseLoad)) { > + return false; // not supported > + } > + } > + > const TypeInstPtr* vbox_type = TypeInstPtr::make_exact(TypePtr::NotNull, vbox_klass); > > if (is_store) { > @@ -7114,9 +7118,15 @@ > const TypeVect* to_vect_type = TypeVect::make(elem_bt, num_elem); > vload = gvn().transform(new VectorReinterpretNode(vload, vload->bottom_type()->is_vect(), to_vect_type)); > } else { > - vload = gvn().transform(LoadVectorNode::make(0, control(), memory(addr), addr, addr_type, num_elem, elem_bt)); > - } > - > + // Special handle for masks > + if (is_mask) { > + vload = gvn().transform(LoadVectorNode::make(0, control(), memory(addr), addr, addr_type, num_elem, T_BOOLEAN)); > + const TypeVect* to_vect_type = TypeVect::make(elem_bt, num_elem); > + vload = gvn().transform(new VectorLoadMaskNode(vload, to_vect_type)); > + } else { > + vload = gvn().transform(LoadVectorNode::make(0, control(), memory(addr), addr, addr_type, num_elem, elem_bt)); > + } > + } > Node* box = box_vector(vload, vbox_type, elem_bt, num_elem); > set_vector_result(box); > } > --- a/src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Int128Vector.java Mon Jan 07 10:54:21 2019 +0800 > +++ b/src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Int128Vector.java Tue Jan 15 23:45:40 2019 +0800 > @@ -1338,7 +1338,7 @@ > Objects.requireNonNull(bits); > ix = VectorIntrinsics.checkIndex(ix, bits.length, LENGTH); > return VectorIntrinsics.load(Int128Mask.class, int.class, LENGTH, > - bits, (((long) ix) << ARRAY_SHIFT) + Unsafe.ARRAY_BOOLEAN_BASE_OFFSET, > + bits, ((long) ix) + Unsafe.ARRAY_BOOLEAN_BASE_OFFSET, > bits, ix, > (c, idx) -> opm(n -> c[idx + n])); > > Regards, > Zhuoren > From vladimir.x.ivanov at oracle.com Tue Jan 15 19:20:17 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 15 Jan 2019 11:20:17 -0800 Subject: Results: New panama Committer: Sandhya Viswanathan Message-ID: <050ed1ee-ddd9-3290-1519-53c72a3dc3b3@oracle.com> Voting for Sandhya Viswanathan (openjdk id: sviswanathan) [1] is now closed. Yes: 6 Veto: 0 Abstain: 0 According to the Bylaws definition of Lazy Consensus, this is sufficient to approve the nomination. Best regards, Vladimir Ivanov [1] http://mail.openjdk.java.net/pipermail/panama-dev/2019-January/003645.html From vladimir.x.ivanov at oracle.com Tue Jan 15 19:39:30 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 15 Jan 2019 11:39:30 -0800 Subject: Vector::rearrange(Shuffle) allocations In-Reply-To: References: Message-ID: Good catch, Tomasz! That's a bug in the implementation of rearrange: default implementation (lambda) captures the vector box and it keeps it from being eliminated. The following patch should fix the problem: http://cr.openjdk.java.net/~vlivanov/panama/vector/rearrange/webrev.00/ Best regards, Vladimir Ivanov On 14/01/2019 09:56, Tomasz Kowalczewski wrote: > I have a question about vector rearranges. My recent observation is > that using IntVector.rearrange causes a lot of allocations (in a JMH > microbenchmark). Is this expected? I wrote a few simple (but not > trivial) algorithms and they all did not box vectors into objects and > all tests were able to run using EpsilonGC. The intrinsic works and > vpermd instruction is emitted. > > There are 64 bytes allocated per benchmark iteration in the following > code (the addAll is used so that vector does not escape the method): > > @Benchmark > public int rearrange() { > IntVector input = SPECIES_I256.fromArray(inputArray, 0); > IntVector rearrangedVector = input.rearrange(SHUFFLE_1); > > return rearrangedVector.addAll(); > } > > There is no allocation in following code: > > @Benchmark > public int noRearrange() { > IntVector input = SPECIES_I256.fromArray(inputArray, 0); > > return input.addAll(); > } > > Output from jmap -histo for first case: > > num #instances #bytes class name (module) > ------------------------------------------------------- > 1: 4122313 198969024 [I (java.base@) > 2: 4119794 65916704 jdk.incubator.vector.Int256Vector > (jdk.incubator.vector@) > 3: 8913 808512 [B (java.base@) > 4: 1982 222408 java.lang.Class (java.base@) > 5: 8638 207312 java.lang.String (java.base@) > > Any hints will be much appreciated. > From vladimir.x.ivanov at oracle.com Tue Jan 15 20:19:22 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 15 Jan 2019 12:19:22 -0800 Subject: [vector] RFR: Vector.rearrange() breaks vector box elimination due to lambda capturing in default implementation Message-ID: <167dfaa0-bf44-23a9-97c7-8213ca91d7f3@oracle.com> http://cr.openjdk.java.net/~vlivanov/panama/vector/rearrange/webrev.00/ Vector.rearrange() breaks vector box elimination due to lambda capturing in default implementation. Testing: microbenchmarks + test/jdk/jdk/incubator/vector/ (fastdebug) Best regards, Vladimir Ivanov From shravya.rukmannagari at intel.com Wed Jan 16 01:01:32 2019 From: shravya.rukmannagari at intel.com (Rukmannagari, Shravya) Date: Wed, 16 Jan 2019 01:01:32 +0000 Subject: VectorAPI Constant Shift for Byte and Short Intrinsic Support In-Reply-To: References: <8D6F463991A1574A8A803B8DA605414F3A82C461@ORSMSX111.amr.corp.intel.com> Message-ID: <8D6F463991A1574A8A803B8DA605414F3A82E2E1@ORSMSX111.amr.corp.intel.com> Hi Vladimir, Not adding 32B for AVX=1 was intentional. I updated the patch with your comments. Please find the webrev below. http://cr.openjdk.java.net/~srukmannagar/VectorAPI_ByteShortConstantShift/webrev.01/ Thanks, Shravya. -----Original Message----- From: Vladimir Ivanov [mailto:vladimir.x.ivanov at oracle.com] Sent: Thursday, January 10, 2019 2:39 PM To: Rukmannagari, Shravya ; panama-dev Subject: Re: VectorAPI Constant Shift for Byte and Short Intrinsic Support > http://cr.openjdk.java.net/~srukmannagar/VectorAPI_ByteShortConstantSh > ift/webrev.00/ Don't you need to update Javadoc as well? Though Javadoc refers to shift operations, implementation does more than that ("a >> i" vs "a >> (i & 7)" / "a >> (i & 15)"): /** * Arithmetically right shifts (or signed right shifts) this vector by the * broadcast of an input scalar. *

* This is a vector binary operation where the primitive arithmetic right * shift operation ({@code >>}) is applied to lane elements. * * @param s the input scalar; the number of the bits to right shift * @return the result of arithmetically right shifting this vector by the * broadcast of an input scalar */ public abstract ShortVector aShiftR(int s); vs @Override @ForceInline public Short256Vector aShiftR(int s) { return VectorIntrinsics.broadcastInt( VECTOR_OP_RSHIFT, Short256Vector.class, short.class, LENGTH, this, s, (v, i) -> v.uOp((__, a) -> (short) (a >> (i & 15)))); } ================================================= I assume C2 backend support for Short*Vector is already there. src/hotspot/cpu/x86/x86.ad: +instruct vsll32B_avx(vecY dst, vecY src, vecS shift, vecY tmp, vecY tmp2, rRegL scratch) %{ + predicate(UseAVX > 1 && n->as_Vector()->length() == 32); Is it deliberate there's no 32B support with UseAVX=1? On related node, do you need to add *ShiftVB cases into Matcher::match_rule_supported_vector()? Best regards, Vladimir Ivanov From vladimir.x.ivanov at oracle.com Wed Jan 16 01:20:48 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 15 Jan 2019 17:20:48 -0800 Subject: VectorAPI Constant Shift for Byte and Short Intrinsic Support In-Reply-To: <8D6F463991A1574A8A803B8DA605414F3A82E2E1@ORSMSX111.amr.corp.intel.com> References: <8D6F463991A1574A8A803B8DA605414F3A82C461@ORSMSX111.amr.corp.intel.com> <8D6F463991A1574A8A803B8DA605414F3A82E2E1@ORSMSX111.amr.corp.intel.com> Message-ID: <8c0fb8d9-9b1e-a116-2488-fba22ed1019d@oracle.com> > Not adding 32B for AVX=1 was intentional. I updated the patch with your comments. Please find the webrev below. > http://cr.openjdk.java.net/~srukmannagar/VectorAPI_ByteShortConstantShift/webrev.01/ So, what's expected behavior for 32B variants with AVX=1 then? If you deliberately keep 32B variants not supported for now, I'd expect to see that listed in Matcher::match_rule_supported_vector() as an implementation constraint. The operations shouldn't be intrinsified (because there are no applicable rules in AD file), but Matcher::match_rule_supported_vector() only limits support to UseSSE >= 4. Am I missing something? Best regards, Vladimir Ivanov > -----Original Message----- > From: Vladimir Ivanov [mailto:vladimir.x.ivanov at oracle.com] > Sent: Thursday, January 10, 2019 2:39 PM > To: Rukmannagari, Shravya ; panama-dev > Subject: Re: VectorAPI Constant Shift for Byte and Short Intrinsic Support > > >> http://cr.openjdk.java.net/~srukmannagar/VectorAPI_ByteShortConstantSh >> ift/webrev.00/ > > Don't you need to update Javadoc as well? > > Though Javadoc refers to shift operations, implementation does more than that ("a >> i" vs "a >> (i & 7)" / "a >> (i & 15)"): > > /** > * Arithmetically right shifts (or signed right shifts) this vector by the > * broadcast of an input scalar. > *

> * This is a vector binary operation where the primitive arithmetic right > * shift operation ({@code >>}) is applied to lane elements. > * > * @param s the input scalar; the number of the bits to right shift > * @return the result of arithmetically right shifting this vector by the > * broadcast of an input scalar > */ > public abstract ShortVector aShiftR(int s); > > vs > > @Override > @ForceInline > public Short256Vector aShiftR(int s) { > return VectorIntrinsics.broadcastInt( > VECTOR_OP_RSHIFT, Short256Vector.class, short.class, LENGTH, > this, s, > (v, i) -> v.uOp((__, a) -> (short) (a >> (i & 15)))); > } > > ================================================= > > I assume C2 backend support for Short*Vector is already there. > > src/hotspot/cpu/x86/x86.ad: > > +instruct vsll32B_avx(vecY dst, vecY src, vecS shift, vecY tmp, vecY > tmp2, rRegL scratch) %{ > + predicate(UseAVX > 1 && n->as_Vector()->length() == 32); > > Is it deliberate there's no 32B support with UseAVX=1? > > On related node, do you need to add *ShiftVB cases into > Matcher::match_rule_supported_vector()? > > Best regards, > Vladimir Ivanov > From yang.zhang at arm.com Wed Jan 16 01:58:30 2019 From: yang.zhang at arm.com (yang.zhang at arm.com) Date: Wed, 16 Jan 2019 01:58:30 +0000 Subject: hg: panama/dev: Update max/min/maxAll/minAll Vector Api implementations and test cases Message-ID: <201901160158.x0G1wVW5028861@aojmv0008.oracle.com> Changeset: 9c63e91265fb Author: yzhang Date: 2019-01-15 13:46 +0800 URL: http://hg.openjdk.java.net/panama/dev/rev/9c63e91265fb Update max/min/maxAll/minAll Vector Api implementations and test cases Summary: use Math.min()/Math.max() instead of "((a > b) ? a : b)". Because the result of latter implementation is affected by sequence of (a, b) if NaN is one of inputs. ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Byte128Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Byte256Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Byte512Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Byte64Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/ByteMaxVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/ByteVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Double128Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Double256Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Double512Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Double64Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/DoubleMaxVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/DoubleVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Float128Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Float256Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Float512Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Float64Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/FloatMaxVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/FloatVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Int128Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Int256Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Int512Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Int64Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/IntMaxVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/IntVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Long128Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Long256Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Long512Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Long64Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/LongMaxVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/LongVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Short128Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Short256Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Short512Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Short64Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/ShortMaxVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/ShortVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/X-Vector.java.template ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/X-VectorBits.java.template ! test/jdk/jdk/incubator/vector/Byte128VectorTests.java ! test/jdk/jdk/incubator/vector/Byte256VectorTests.java ! test/jdk/jdk/incubator/vector/Byte512VectorTests.java ! test/jdk/jdk/incubator/vector/Byte64VectorTests.java ! test/jdk/jdk/incubator/vector/ByteMaxVectorTests.java ! test/jdk/jdk/incubator/vector/Double128VectorTests.java ! test/jdk/jdk/incubator/vector/Double256VectorTests.java ! test/jdk/jdk/incubator/vector/Double512VectorTests.java ! test/jdk/jdk/incubator/vector/Double64VectorTests.java ! test/jdk/jdk/incubator/vector/DoubleMaxVectorTests.java ! test/jdk/jdk/incubator/vector/Float128VectorTests.java ! test/jdk/jdk/incubator/vector/Float256VectorTests.java ! test/jdk/jdk/incubator/vector/Float512VectorTests.java ! test/jdk/jdk/incubator/vector/Float64VectorTests.java ! test/jdk/jdk/incubator/vector/FloatMaxVectorTests.java ! test/jdk/jdk/incubator/vector/Int128VectorTests.java ! test/jdk/jdk/incubator/vector/Int256VectorTests.java ! test/jdk/jdk/incubator/vector/Int512VectorTests.java ! test/jdk/jdk/incubator/vector/Int64VectorTests.java ! test/jdk/jdk/incubator/vector/IntMaxVectorTests.java ! test/jdk/jdk/incubator/vector/Long128VectorTests.java ! test/jdk/jdk/incubator/vector/Long256VectorTests.java ! test/jdk/jdk/incubator/vector/Long512VectorTests.java ! test/jdk/jdk/incubator/vector/Long64VectorTests.java ! test/jdk/jdk/incubator/vector/LongMaxVectorTests.java ! test/jdk/jdk/incubator/vector/Short128VectorTests.java ! test/jdk/jdk/incubator/vector/Short256VectorTests.java ! test/jdk/jdk/incubator/vector/Short512VectorTests.java ! test/jdk/jdk/incubator/vector/Short64VectorTests.java ! test/jdk/jdk/incubator/vector/ShortMaxVectorTests.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/ByteScalar.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/DoubleScalar.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/FloatScalar.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/IntScalar.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/LongScalar.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/ShortScalar.java ! test/jdk/jdk/incubator/vector/gen-template.sh ! test/jdk/jdk/incubator/vector/templates/Unit-Reduction-Scalar-Max-op.template ! test/jdk/jdk/incubator/vector/templates/Unit-Reduction-Scalar-Min-op.template From maurizio.cimadamore at oracle.com Wed Jan 16 02:00:35 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 16 Jan 2019 02:00:35 +0000 Subject: hg: panama/dev: Automatic merge with vectorIntrinsics Message-ID: <201901160200.x0G20ZVL000203@aojmv0008.oracle.com> Changeset: bd0b0280c6f9 Author: mcimadamore Date: 2019-01-16 03:00 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/bd0b0280c6f9 Automatic merge with vectorIntrinsics From Ningsheng.Jian at arm.com Wed Jan 16 02:59:05 2019 From: Ningsheng.Jian at arm.com (Ningsheng Jian (Arm Technology China)) Date: Wed, 16 Jan 2019 02:59:05 +0000 Subject: [vector api] RFR: Fix byte vector test failures about bad ad file and unsupported vector size for AArch64 In-Reply-To: References: Message-ID: I've verified that the failures can be fixed by this patch. Thanks! Regards, Ningsheng > -----Original Message----- > From: panama-dev On Behalf Of Yang > Zhang (Arm Technology China) > Sent: Tuesday, January 15, 2019 2:38 PM > To: 'panama-dev at openjdk.java.net' > Subject: [vector api] RFR: Fix byte vector test failures about bad ad file and > unsupported vector size for AArch64 > > Hi > > On AArch64 platform, the following tests failed because of bad ad file and > unsupported vector size. > jdk/incubator/vector/Byte128VectorTests.java > jdk/incubator/vector/Byte256VectorTests.java > jdk/incubator/vector/Byte512VectorTests.java > jdk/incubator/vector/ByteMaxVectorTests.java > > I have a patch which could fix these failures. Could you please help to review it? > http://cr.openjdk.java.net/~yzhang/vectorapi.vect4/webrev.00/ > > Regards, > Yang From yang.zhang at arm.com Wed Jan 16 03:50:00 2019 From: yang.zhang at arm.com (yang.zhang at arm.com) Date: Wed, 16 Jan 2019 03:50:00 +0000 Subject: hg: panama/dev: Fix byte vector test failures about bad ad file and unsupported vector size for AArch64 Message-ID: <201901160350.x0G3o1dZ012686@aojmv0008.oracle.com> Changeset: 25099fb76afb Author: yzhang Date: 2019-01-16 11:49 +0800 URL: http://hg.openjdk.java.net/panama/dev/rev/25099fb76afb Fix byte vector test failures about bad ad file and unsupported vector size for AArch64 Summary: add four-byte vectorization support for vector shift and mul. ! src/hotspot/cpu/aarch64/aarch64.ad From maurizio.cimadamore at oracle.com Wed Jan 16 03:55:23 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 16 Jan 2019 03:55:23 +0000 Subject: hg: panama/dev: Automatic merge with vectorIntrinsics Message-ID: <201901160355.x0G3tNaB016502@aojmv0008.oracle.com> Changeset: 31cf4ed3313f Author: mcimadamore Date: 2019-01-16 04:55 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/31cf4ed3313f Automatic merge with vectorIntrinsics From sundararajan.athijegannathan at oracle.com Wed Jan 16 04:16:00 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Wed, 16 Jan 2019 09:46:00 +0530 Subject: [foreign] jextract does not filter typedefs, enums, and structs In-Reply-To: References: Message-ID: <5C3EB000.2080809@oracle.com> Unlike function symbols, we can't just filter symbols. We've to carefully walk the dependencies of exported functions (all types directly/indirectly used from parameter and return types have to be included). In theory, it should be possible to generate only transitive closure of types needed for the included symbols. As you've found it already, there is a bug filed for that. -Sundar On 15/01/19, 10:42 PM, Jorn Vernee wrote: > Hello, > > Now that I'm done with the initial Windows support I have started > looking for interesting Windows APIs to create tests/demos with. > Pretty much any part of the Windows API requires users to include > Windows.h, which is a huge header that pretty much includes everything > and currently requires 14GB of RAM to extract. Trying to jextract a > sub-header breaks the pre-processor code in those headers. > > So, I'm using `--include-symbols` to select just the things I need, > but I find TreeFilter [1] lacks filtering of typedefs, structs, and > enums. So a lot of unneeded classes are still being generated. Why are > these Tree types not currently filtered? > > Thanks, > Jorn > > [1] : > http://hg.openjdk.java.net/panama/dev/file/d7885ab60131/src/jdk.jextract/share/classes/com/sun/tools/jextract/TreeFilter.java From samuel.audet at gmail.com Wed Jan 16 05:56:55 2019 From: samuel.audet at gmail.com (Samuel Audet) Date: Wed, 16 Jan 2019 14:56:55 +0900 Subject: Toward distributions of native libraries In-Reply-To: References: <51e5d7d3-6f95-9b33-1858-645bc388e17e@gmail.com> <479f6425-f3f2-e4ec-abe4-9303996dcd82@oracle.com> Message-ID: On 1/14/19 3:04 PM, John Rose wrote: > On Jan 12, 2019, at 7:42 PM, Samuel Audet wrote: >> >> Thanks for the feedback! If it were a priority, and if it were me, I would start by ensuring that the high-level API can also be optimized in the case of AOT, but this isn't happening at the moment. I just shortened it this way because I consider AOT to be just as important as JIT. How do you feel I should phrase this then? That it isn't a priority /only in the case of AOT/? I'll edit it that way if that's accurate! If not, I would welcome additional explanations, for example, see https://github.com/oracle/graal/issues/885 . Thanks > > Performance is important, period. The discussion on #885 peters out with you saying, "but can it work efficiently?". That's a complicated question, to which I believe the answer is almost certainly "yes". We are, fact, planning to make such things "work efficiently", and "such things" include JIT, AOT, and SVM compilation of Panama apps. > > So I agree with Maurizio's comment, that this bit of your blog post (a very good one!) would better reflect our intentions by omitting the comment about performance. If you wish, you could satisfy our observations by saying something straightforward like "Although the Panama people claim to value high performance, I don't think they can achieve it, especially for AOT." If that's true for you, in which case, wait and see how it all turns out. > > To put it in a nutshell: You misrepresent our priorities (a set of mental states Maurizio and I can and do authoritatively explain) by reasoning about their feasibility (which reasonable people can disagree about). > I would rather learn to disagree than to ignore each other, so thank you for the feedback! IMO, the question I asked to the SVM team isn't a complicated one. I asked in the simple case of getpid() whether or not we will be able to achieve the same speed as "org.graalvm.nativeimage.c", and although you may believe the answer is "yes", the fact is that it looks like we don't know. So the answer is "probably, but we don't know for sure". What if it turns out to be "no"? Are you prepared to refactor the whole high-level API to something else entirely and maybe even bring back static methods as an option if we're really up against the wall in one or more corner cases where they would help performance? If that's the way you guys work, that's fine too, but let's be clear about it! Anyway, I like to stay objective, so I've rephrased the passage on http://bytedeco.org/news/2019/01/11/importance-of-a-distribution/ without any implications about what you consider important or not: "numbers showing faster-than-JNI performance for both JIT and AOT compilers are still lacking" Sounds good? I'm happy to hear at least that AOT is of importance too! Samuel From kishor.kharbas at intel.com Wed Jan 16 08:20:27 2019 From: kishor.kharbas at intel.com (Kharbas, Kishor) Date: Wed, 16 Jan 2019 08:20:27 +0000 Subject: [Vector] RFR: reshape, resize, rebracket, cast Message-ID: Hi, I have a patch which refactors the above methods as discussed in this thread - https://mail.openjdk.java.net/pipermail/panama-dev/2018-December/003365.html. Please review the changes at - http://cr.openjdk.java.net/~kkharbas/vector-api/webrev-reshape_etc Generation of intrinsics and correct vector boxing elimination have been verified. In the next patch I will add changes to jtreg tests. I wanted to bring forth one small issue(possibly) which programmers might face with this change - We provide specialized types like IntVector, FloatVector, etc for users to define their vectors and they would be able to write code like this with previous methods, FloatVector float256 = SPECIES_FLOAT256.cast(int256); Here FloatSpecies would always return FloatVector. However with this change, since cast() is defined on a vector and takes species of a generic type, it cannot return a specialized Vector like IntVector or FloatVector. User has to explicitly cast the return vector from Vector to specialized Vector or use a generic vector of corresponding element type. For example, FloatVector float256 = (FloatVector)int256.cast(SPECIES_FLOAT256); or Vector float256 = int256.cast(SPECIES_FLOAT256); I am not sure if this is even a problem, but I thought its worth mentioning. Thanks Kishor From maurizio.cimadamore at oracle.com Wed Jan 16 12:07:34 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 16 Jan 2019 12:07:34 +0000 Subject: Lazy memory initialization for Unsafe (calloc vs. malloc) In-Reply-To: References: Message-ID: <310950e3-c22f-32ef-d1ce-546ec0af2f7c@oracle.com> Hi Markus, (moving to panama-dev) I've been pointed at your message by a colleague, I believe it is very relevant to the context of the Panama offheap API, since that API is internally using unsafe to allocate native memory. As far as I'm aware, our Scope class is internally just calling Unsafe::allocateMemory, and leaving memory uninitialized, as per malloc (which means it's up to the application to initialize the memory if really necessary). Adding an alternate way to initialize memory doesn't sound bad - but I have some questions over the proposed approach * while adding a new method to Unsafe seems like the most natural path, please be aware that we have committed to reduce the footprint of that API over time; your proposal goes in the opposite direction. That said, we could still have some internal functionality for performing calloc that is only used internally by the Panama offheap API * I'm not an expert in the area, but I'm wondering whether the lazy zeroing scheme is truly cross-platform as opposed to being just a Linux optimization; of course adding a new functionality that adds benefit on a single platform has a different cost compared to adding something that provides advantages across the board Cheers Maurizio On 05/11/2018 19:49, mail.openjdk.java.net at lemmster.de wrote: > Hi, > > lets assume an application A [1] which is optimized for throughput, not > latency. Additionally, A reverts to sun.misc.Unsafe to store gigabytes > of data on off-heap memory to e.g. improved memory utilization and > performance. > > Due to the fact that Unsafe#allocateMemory returns an uninitialized > (malloc) memory region R, app A has to explicitly initialize R by e.g. > calling Unsafe#setMemory?. > > Since app A is optimized for throughput, there is no real need to > eagerly allocate R. Instead - on OSes with an optimistic memory > allocator such as Linux - app A could benefit from lazy allocation. This > would reduce app startup time that - that today is linear in the size of > R - to constant time. Assuming Unsafe (or some other API) would expose > memory allocation with calloc instead of malloc one would get away > without explicitly calling Unsafe#setMemory or implementing (complex) > lazy initialization into A itself. > > > To check this idea, I added sun.misc.Unsafe#callocateMemory (notice the > "c") to a local jdk12 build. It works as intended. Is this feature a > candidate to be added to the JVM, e.g. as part of project Panama or am I > better off rolling my own solution via JNI? > > Thanks > Markus > > ? Lets assume the application logic cannot deal with uninitialized memory. > > [1] https://github.com/tlaplus/tlaplus > > > > > > From maurizio.cimadamore at oracle.com Wed Jan 16 12:14:54 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 16 Jan 2019 12:14:54 +0000 Subject: [foreign] test target fails on windows Message-ID: <3a81aa9e-6cf3-4e76-76ee-2d436bf43ee3@oracle.com> Hi, I've tried to run tests on Windows the 'usual' way, but I get the following failures - is this something known? c:\cygwin64\home\mcimadam\panama\closed\open\test\jdk\com\sun\tools\jextract\testenum\enums.h(46): error C2220: warning treated as error - no 'object' file generated c:\cygwin64\home\mcimadam\panama\closed\open\test\jdk\com\sun\tools\jextract\testenum\enums.h(46): warning C4309: 'initializing': truncation of constant value c:\cygwin64\home\mcimadam\panama\closed\open\test\jdk\com\sun\tools\jextract\testenum\enums.h(48): warning C4309: 'initializing': truncation of constant value JtregNativeJdk.gmk:106: recipe for target '/cygdrive/c/cygwin64/home/mcimadam/panama/closed/build/windows-x64/support/test/jdk/jtreg/native/support/libEnums/libEnums.obj' failed make[3]: *** [/cygdrive/c/cygwin64/home/mcimadam/panama/closed/build/windows-x64/support/test/jdk/jtreg/native/support/libEnums/libEnums.obj] Error 1 make[3]: *** Waiting for unfinished jobs.... c:/cygwin64/home/mcimadam/panama/closed/open/test/jdk/com/sun/tools/jextract/testStruct/libStruct.c(106): error C2220: warning treated as error - no 'object' file generated c:/cygwin64/home/mcimadam/panama/closed/open/test/jdk/com/sun/tools/jextract/testStruct/libStruct.c(106): warning C4477: 'printf' : format string '%lu' requires an argument of type 'unsigned long', but variadic argument 1 has type '::size_t' c:/cygwin64/home/mcimadam/panama/closed/open/test/jdk/com/sun/tools/jextract/testStruct/libStruct.c(106): note: consider using '%zu' in the format string (**) - the way I run test is this: make run-test-jdk_foreign From maurizio.cimadamore at oracle.com Wed Jan 16 12:21:55 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 16 Jan 2019 12:21:55 +0000 Subject: [foreign] RFR 8217245:Unify functions and layouts Message-ID: Hi, this is a followup for the RFC here [1]. The goal is to put both Layout and Function under a common root, called Descriptor (this name is very broad so we will change it at a later stage). I've put together a new webrev which addresses some of the comments raised in [1]. This is an official request for review. Webrev: http://cr.openjdk.java.net/~mcimadamore/panama/layout%2bfunc_v3/ Cheers Maurizio [1] - https://mail.openjdk.java.net/pipermail/panama-dev/2018-November/003279.html From maurizio.cimadamore at oracle.com Wed Jan 16 12:45:30 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 16 Jan 2019 12:45:30 +0000 Subject: hg: panama/dev: Fix encoding of Panama/foreign doc Message-ID: <201901161245.x0GCjVKu011631@aojmv0008.oracle.com> Changeset: 364dc06dd3ed Author: mcimadamore Date: 2019-01-16 12:45 +0000 URL: http://hg.openjdk.java.net/panama/dev/rev/364dc06dd3ed Fix encoding of Panama/foreign doc ! doc/panama_foreign.html ! doc/panama_foreign.md From zhuoren.wz at alibaba-inc.com Wed Jan 16 12:46:26 2019 From: zhuoren.wz at alibaba-inc.com (=?UTF-8?B?V2FuZyBaaHVvKFpodW9yZW4p?=) Date: Wed, 16 Jan 2019 20:46:26 +0800 Subject: =?UTF-8?B?UmU6IFtWZWN0b3JBUEldbWFza0Zyb21BcnJheSBJbnRyaW5zaWMgZmFpbHVyZQ==?= In-Reply-To: <7b720dbb-91d8-4bef-e680-9474ead264f2@oracle.com> References: <701fcf82-7ace-4e66-910b-3120753809a8.zhuoren.wz@alibaba-inc.com>, <7b720dbb-91d8-4bef-e680-9474ead264f2@oracle.com> Message-ID: <77404c4b-cfa8-4f47-b58f-533ae0d9a1e6.zhuoren.wz@alibaba-inc.com> Webrev files have been uploaded to http://cr.openjdk.java.net/~luchsh/mask_fix_zhuoren/. Please review. Regards, Zhuoren ------------------------------------------------------------------ From:Vladimir Ivanov Sent At:2019 Jan. 16 (Wed.) 02:46 To:Sandler ; panama-dev Cc:li sanhong ; Kingsum Chow Subject:Re: [VectorAPI]maskFromArray Intrinsic failure Good catch! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Int128Vector.java: > - bits, (((long) ix) << ARRAY_SHIFT) + Unsafe.ARRAY_BOOLEAN_BASE_OFFSET, > + bits, ((long) ix) + Unsafe.ARRAY_BOOLEAN_BASE_OFFSET, It's better to replace ARRAY_SHIFT with Unsafe.ARRAY_BOOLEAN_INDEX_SCALE. src/hotspot/share/opto/library_call.cpp: > + if (!arch_supports_vector(Op_LoadVector, num_elem, T_BOOLEAN, VecMaskUseLoad)) { Why do you specify VecMaskUseLoad and not VecMaskNotUsed (as in other cases)? Otherwise, looks good for now. Please, send complete webrev for review. As a followup, it would be nice to refactor/cleanup Mask-related code on JDK-JVM boundary (VectorIntrinsics + library_call.cpp). For example, I don't see a reason to pass int.class (and not boolean.class) as the element type for mask-related operations. Best regards, Vladimir Ivanov On 15/01/2019 08:37, Wang Zhuo(Zhuoren) wrote: > Hi, > I found intrinsics for maskFromArray always failed. Here is a patch to fix this issue. > Benchmarks containing heavy maskFromArray use can benifit from this patch. such as allTrue/anyTrue. > > This patch consists of two parts. In the JDK part, since boolean is only one byte, it is not correct to perform left shift ARRAY_SHIFT for array index. I listed only one file in the patch to avoid a long letter, but this modification should be applied to all types including the template file. > return VectorIntrinsics.load(Int128Mask.class, int.class, LENGTH, > - bits, (((long) ix) << ARRAY_SHIFT) + Unsafe.ARRAY_BOOLEAN_BASE_OFFSET, > + bits, ((long) ix) + Unsafe.ARRAY_BOOLEAN_BASE_OFFSET, > bits, ix, > Another part is in hotspot, please check the below patch > > diff -r 890e996dfd1a src/hotspot/share/opto/library_call.cpp > --- a/src/hotspot/share/opto/library_call.cpp Mon Jan 07 10:54:21 2019 +0800 > +++ b/src/hotspot/share/opto/library_call.cpp Tue Jan 15 23:45:40 2019 +0800 > @@ -7069,13 +7069,13 @@ > > // Now handle special case where load/store happens from/to byte array but element type is not byte. > bool using_byte_array = arr_type != NULL && arr_type->elem()->array_element_basic_type() == T_BYTE && elem_bt != T_BYTE; > - > - // It must be the case that if it is not special byte array case, there is consistency between > - // array and vector element types. > - if (!using_byte_array && arr_type != NULL && elem_bt != arr_type->elem()->array_element_basic_type()) { > + // Handle loading masks. > + ciKlass* vbox_klass = vector_klass->const_oop()->as_instance()->java_lang_Class_klass(); > + bool is_mask = vbox_klass->is_vectormask(); > + // If there is no consistency between array and vector element types, it must be special byte array case or loading masks > + if (!using_byte_array && arr_type != NULL && elem_bt != arr_type->elem()->array_element_basic_type() && !is_mask) { > return false; > } > - > // Since we are using byte array, we need to double check that the byte operations are supported by backend. > if (using_byte_array) { > int byte_num_elem = num_elem * type2aelembytes(elem_bt); > @@ -7083,8 +7083,12 @@ > return false; // not supported > } > } > - > - ciKlass* vbox_klass = vector_klass->const_oop()->as_instance()->java_lang_Class_klass(); > + if (is_mask) { > + if (!arch_supports_vector(Op_LoadVector, num_elem, T_BOOLEAN, VecMaskUseLoad)) { > + return false; // not supported > + } > + } > + > const TypeInstPtr* vbox_type = TypeInstPtr::make_exact(TypePtr::NotNull, vbox_klass); > > if (is_store) { > @@ -7114,9 +7118,15 @@ > const TypeVect* to_vect_type = TypeVect::make(elem_bt, num_elem); > vload = gvn().transform(new VectorReinterpretNode(vload, vload->bottom_type()->is_vect(), to_vect_type)); > } else { > - vload = gvn().transform(LoadVectorNode::make(0, control(), memory(addr), addr, addr_type, num_elem, elem_bt)); > - } > - > + // Special handle for masks > + if (is_mask) { > + vload = gvn().transform(LoadVectorNode::make(0, control(), memory(addr), addr, addr_type, num_elem, T_BOOLEAN)); > + const TypeVect* to_vect_type = TypeVect::make(elem_bt, num_elem); > + vload = gvn().transform(new VectorLoadMaskNode(vload, to_vect_type)); > + } else { > + vload = gvn().transform(LoadVectorNode::make(0, control(), memory(addr), addr, addr_type, num_elem, elem_bt)); > + } > + } > Node* box = box_vector(vload, vbox_type, elem_bt, num_elem); > set_vector_result(box); > } > --- a/src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Int128Vector.java Mon Jan 07 10:54:21 2019 +0800 > +++ b/src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Int128Vector.java Tue Jan 15 23:45:40 2019 +0800 > @@ -1338,7 +1338,7 @@ > Objects.requireNonNull(bits); > ix = VectorIntrinsics.checkIndex(ix, bits.length, LENGTH); > return VectorIntrinsics.load(Int128Mask.class, int.class, LENGTH, > - bits, (((long) ix) << ARRAY_SHIFT) + Unsafe.ARRAY_BOOLEAN_BASE_OFFSET, > + bits, ((long) ix) + Unsafe.ARRAY_BOOLEAN_BASE_OFFSET, > bits, ix, > (c, idx) -> opm(n -> c[idx + n])); > > Regards, > Zhuoren > From maurizio.cimadamore at oracle.com Wed Jan 16 12:50:16 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 16 Jan 2019 12:50:16 +0000 Subject: hg: panama/dev: Automatic merge with foreign Message-ID: <201901161250.x0GCoGPD013786@aojmv0008.oracle.com> Changeset: 4d3d9555a002 Author: mcimadamore Date: 2019-01-16 13:50 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/4d3d9555a002 Automatic merge with foreign From maurizio.cimadamore at oracle.com Wed Jan 16 12:50:45 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 16 Jan 2019 12:50:45 +0000 Subject: hg: panama/dev: Automatic merge with foreign Message-ID: <201901161250.x0GCojbK014243@aojmv0008.oracle.com> Changeset: 1bbfa22b4871 Author: mcimadamore Date: 2019-01-16 13:50 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/1bbfa22b4871 Automatic merge with foreign From sundararajan.athijegannathan at oracle.com Wed Jan 16 12:57:03 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Wed, 16 Jan 2019 18:27:03 +0530 Subject: [foreign] RFR 8217246: jextract should parse single header file with clang Message-ID: <5C3F2A1F.5010906@oracle.com> Please review. Bug: https://bugs.openjdk.java.net/browse/JDK-8217246 Webrev: https://cr.openjdk.java.net/~sundar/8217246/webrev.00/ Thanks, -Sundar From maurizio.cimadamore at oracle.com Wed Jan 16 13:57:44 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 16 Jan 2019 13:57:44 +0000 Subject: [foreign] test target fails on windows In-Reply-To: <3a81aa9e-6cf3-4e76-76ee-2d436bf43ee3@oracle.com> References: <3a81aa9e-6cf3-4e76-76ee-2d436bf43ee3@oracle.com> Message-ID: <373de078-f6a9-3e78-4eba-5c56427f91df@oracle.com> The following patch seems to fix the issue - Jorn, perhaps you have been building/running tests with 'disable-warnings-as-errors' ? diff -r 364dc06dd3ed test/jdk/com/sun/tools/jextract/testEnum/enums.h --- a/test/jdk/com/sun/tools/jextract/testEnum/enums.h? Wed Jan 16 12:45:05 2019 +0000 +++ b/test/jdk/com/sun/tools/jextract/testEnum/enums.h? Wed Jan 16 13:57:29 2019 +0000 @@ -42,10 +42,17 @@ ??? I_VALUE2 = -5345345 ?}; +#ifndef _WIN64 ?enum LongEnum { ??? L_VALUE1 = -4564565645L, ??? L_VALUE2 = 45645645645L ?}; +#else +enum LongEnum { +?? L_VALUE1 = -5345345L, +?? L_VALUE2 = -5345345 +}; +#endif ?EXPORT int i_value1_func(); ?EXPORT int i_value2_func(); Maurizio On 16/01/2019 12:14, Maurizio Cimadamore wrote: > Hi, > I've tried to run tests on Windows the 'usual' way, but I get the > following failures - is this something known? > > c:\cygwin64\home\mcimadam\panama\closed\open\test\jdk\com\sun\tools\jextract\testenum\enums.h(46): > error C2220: warning treated as error - no 'object' file generated > c:\cygwin64\home\mcimadam\panama\closed\open\test\jdk\com\sun\tools\jextract\testenum\enums.h(46): > warning C4309: 'initializing': truncation of constant value > c:\cygwin64\home\mcimadam\panama\closed\open\test\jdk\com\sun\tools\jextract\testenum\enums.h(48): > warning C4309: 'initializing': truncation of constant value > JtregNativeJdk.gmk:106: recipe for target > '/cygdrive/c/cygwin64/home/mcimadam/panama/closed/build/windows-x64/support/test/jdk/jtreg/native/support/libEnums/libEnums.obj' > failed > make[3]: *** > [/cygdrive/c/cygwin64/home/mcimadam/panama/closed/build/windows-x64/support/test/jdk/jtreg/native/support/libEnums/libEnums.obj] > Error 1 > make[3]: *** Waiting for unfinished jobs.... > c:/cygwin64/home/mcimadam/panama/closed/open/test/jdk/com/sun/tools/jextract/testStruct/libStruct.c(106): > error C2220: warning treated as error - no 'object' file generated > c:/cygwin64/home/mcimadam/panama/closed/open/test/jdk/com/sun/tools/jextract/testStruct/libStruct.c(106): > warning C4477: 'printf' : format string '%lu' requires an argument of > type 'unsigned long', but variadic argument 1 has type '::size_t' > c:/cygwin64/home/mcimadam/panama/closed/open/test/jdk/com/sun/tools/jextract/testStruct/libStruct.c(106): > note: consider using '%zu' in the format string > > (**) - the way I run test is this: > > make run-test-jdk_foreign > > From maurizio.cimadamore at oracle.com Wed Jan 16 14:07:00 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 16 Jan 2019 14:07:00 +0000 Subject: [foreign] libclang.dll not copied to JDK image on Windows Message-ID: <0213f0cd-e6dd-f26a-99f6-416e3bf070fe@oracle.com> As the subject implies, there seems to be a build setup issue on windows, as the image I've built doesn't have libclang.dll under the images 'lib' folder. As a result jextract fails with the simplest of example: $ build/windows-x64/images/jdk/bin/jextract.exe foo.h Exception in thread "main" java.lang.UnsatisfiedLinkError: C:\cygwin64\home\MCIMADAM\panama\closed\build\windows-x64\images\jdk\bin\jclang.dll: Can't find dependent libraries ??????? at java.base/java.lang.ClassLoader$NativeLibrary.load0(Native Method) ??????? at java.base/java.lang.ClassLoader$NativeLibrary.load(ClassLoader.java:2451) ??????? at java.base/java.lang.ClassLoader$NativeLibrary.loadLibrary(ClassLoader.java:2523) ??????? at java.base/java.lang.ClassLoader.loadLibrary0(ClassLoader.java:2733) ??????? at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2679) ??????? at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:849) ??????? at java.base/java.lang.System.loadLibrary(System.java:1905) ??????? at jdk.internal.clang/jdk.internal.clang.LibClang.(LibClang.java:35) ??????? at jdk.jextract/com.sun.tools.jextract.parser.Parser.parse(Parser.java:71) ??????? at jdk.jextract/com.sun.tools.jextract.JextractTool.processHeaders(JextractTool.java:58) ??????? at jdk.jextract/com.sun.tools.jextract.Main.run(Main.java:264) ??????? at jdk.jextract/com.sun.tools.jextract.Main.main(Main.java:334) Is this a known issue? This is what I get in the spec.gmk file: CLANG_LIB_PATH:=/cygdrive/c/progra~1/llvm/lib CLANG_INCLUDE_PATH:=/cygdrive/c/progra~1/llvm/include CLANG_INCLUDE_AUX_PATH:=/cygdrive/c/progra~1/llvm/lib/clang/70fe6e~1.0/include CLANG_LIBNAME:=/cygdrive/c/progra~1/llvm/bin/libclang.dll LIBCLANG_CPPFLAGS:=-I/cygdrive/c/progra~1/llvm/include LIBCLANG_LDFLAGS:=/LIBPATH:/cygdrive/c/progra~1/llvm/lib LIBCLANG_LIBS:=/cygdrive/c/progra~1/llvm/lib/libclang.lib All paths involved seem to exist. Maurizio From jbvernee at xs4all.nl Wed Jan 16 15:06:53 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Wed, 16 Jan 2019 16:06:53 +0100 Subject: [foreign] test target fails on windows In-Reply-To: <373de078-f6a9-3e78-4eba-5c56427f91df@oracle.com> References: <3a81aa9e-6cf3-4e76-76ee-2d436bf43ee3@oracle.com> <373de078-f6a9-3e78-4eba-5c56427f91df@oracle.com> Message-ID: <9900ae85ec28a6f3d9e0c7c9b584f1bf@xs4all.nl> Yes, sorry. I have been using `disable-warnings-as-errors` since there was a warning generated by a change in jdk/jdk code a while ago which made `make images` fail. When I noticed the warnings, I had the intent of fixing them later, but I kind of forgot about about it :/ I think your fix looks fine. FWIW, it seems that larger-than-int enum constants are a GCC extension [1]. What about the other warning? I can't reproduce it currently, but I do remember seeing it some time (maybe only from clean build). I always use `make test TEST=jdk_foreign` to run the tests, I'm not sure if there is a difference with `make run-test-jdk_foreign`. Jorn [1] : https://stackoverflow.com/q/366017 Maurizio Cimadamore schreef op 2019-01-16 14:57: > The following patch seems to fix the issue - Jorn, perhaps you have > been building/running tests with 'disable-warnings-as-errors' ? > > diff -r 364dc06dd3ed test/jdk/com/sun/tools/jextract/testEnum/enums.h > --- a/test/jdk/com/sun/tools/jextract/testEnum/enums.h? Wed Jan 16 > 12:45:05 2019 +0000 > +++ b/test/jdk/com/sun/tools/jextract/testEnum/enums.h? Wed Jan 16 > 13:57:29 2019 +0000 > @@ -42,10 +42,17 @@ > ??? I_VALUE2 = -5345345 > ?}; > > +#ifndef _WIN64 > ?enum LongEnum { > ??? L_VALUE1 = -4564565645L, > ??? L_VALUE2 = 45645645645L > ?}; > +#else > +enum LongEnum { > +?? L_VALUE1 = -5345345L, > +?? L_VALUE2 = -5345345 > +}; > +#endif > > ?EXPORT int i_value1_func(); > ?EXPORT int i_value2_func(); > > Maurizio > > On 16/01/2019 12:14, Maurizio Cimadamore wrote: >> Hi, >> I've tried to run tests on Windows the 'usual' way, but I get the >> following failures - is this something known? >> >> c:\cygwin64\home\mcimadam\panama\closed\open\test\jdk\com\sun\tools\jextract\testenum\enums.h(46): >> error C2220: warning treated as error - no 'object' file generated >> c:\cygwin64\home\mcimadam\panama\closed\open\test\jdk\com\sun\tools\jextract\testenum\enums.h(46): >> warning C4309: 'initializing': truncation of constant value >> c:\cygwin64\home\mcimadam\panama\closed\open\test\jdk\com\sun\tools\jextract\testenum\enums.h(48): >> warning C4309: 'initializing': truncation of constant value >> JtregNativeJdk.gmk:106: recipe for target >> '/cygdrive/c/cygwin64/home/mcimadam/panama/closed/build/windows-x64/support/test/jdk/jtreg/native/support/libEnums/libEnums.obj' >> failed >> make[3]: *** >> [/cygdrive/c/cygwin64/home/mcimadam/panama/closed/build/windows-x64/support/test/jdk/jtreg/native/support/libEnums/libEnums.obj] >> Error 1 >> make[3]: *** Waiting for unfinished jobs.... >> c:/cygwin64/home/mcimadam/panama/closed/open/test/jdk/com/sun/tools/jextract/testStruct/libStruct.c(106): >> error C2220: warning treated as error - no 'object' file generated >> c:/cygwin64/home/mcimadam/panama/closed/open/test/jdk/com/sun/tools/jextract/testStruct/libStruct.c(106): >> warning C4477: 'printf' : format string '%lu' requires an argument of >> type 'unsigned long', but variadic argument 1 has type '::size_t' >> c:/cygwin64/home/mcimadam/panama/closed/open/test/jdk/com/sun/tools/jextract/testStruct/libStruct.c(106): >> note: consider using '%zu' in the format string >> >> (**) - the way I run test is this: >> >> make run-test-jdk_foreign >> >> From jbvernee at xs4all.nl Wed Jan 16 15:22:10 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Wed, 16 Jan 2019 16:22:10 +0100 Subject: [foreign] libclang.dll not copied to JDK image on Windows In-Reply-To: <0213f0cd-e6dd-f26a-99f6-416e3bf070fe@oracle.com> References: <0213f0cd-e6dd-f26a-99f6-416e3bf070fe@oracle.com> Message-ID: <10a2080b1a6aab9478135e874f27d22e@xs4all.nl> Afaik the `libclang.dll` file is supposed to be in the `bin` folder after building. It's not copied there for me either currently, but jextract works since I have the llvm/bin folder on the PATH, so the library can be loaded from there. The copying was supposedly working when the Windows build patch came in [1] "This patch allow build image to be completed successfully and libclang.dll copied to bin as expected." And I remember verifying that as well. I guess I just never noticed the regression since I have the llvm/bin folder on my PATH. Jorn [1] : https://mail.openjdk.java.net/pipermail/panama-dev/2018-October/003003.html Maurizio Cimadamore schreef op 2019-01-16 15:07: > As the subject implies, there seems to be a build setup issue on > windows, as the image I've built doesn't have libclang.dll under the > images 'lib' folder. > > As a result jextract fails with the simplest of example: > > $ build/windows-x64/images/jdk/bin/jextract.exe foo.h > Exception in thread "main" java.lang.UnsatisfiedLinkError: > C:\cygwin64\home\MCIMADAM\panama\closed\build\windows-x64\images\jdk\bin\jclang.dll: > Can't find dependent libraries > ??????? at java.base/java.lang.ClassLoader$NativeLibrary.load0(Native > Method) > ??????? at > java.base/java.lang.ClassLoader$NativeLibrary.load(ClassLoader.java:2451) > ??????? at > java.base/java.lang.ClassLoader$NativeLibrary.loadLibrary(ClassLoader.java:2523) > ??????? at > java.base/java.lang.ClassLoader.loadLibrary0(ClassLoader.java:2733) > ??????? at > java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2679) > ??????? at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:849) > ??????? at java.base/java.lang.System.loadLibrary(System.java:1905) > ??????? at > jdk.internal.clang/jdk.internal.clang.LibClang.(LibClang.java:35) > ??????? at > jdk.jextract/com.sun.tools.jextract.parser.Parser.parse(Parser.java:71) > ??????? at > jdk.jextract/com.sun.tools.jextract.JextractTool.processHeaders(JextractTool.java:58) > ??????? at jdk.jextract/com.sun.tools.jextract.Main.run(Main.java:264) > ??????? at jdk.jextract/com.sun.tools.jextract.Main.main(Main.java:334) > > > Is this a known issue? This is what I get in the spec.gmk file: > > CLANG_LIB_PATH:=/cygdrive/c/progra~1/llvm/lib > CLANG_INCLUDE_PATH:=/cygdrive/c/progra~1/llvm/include > CLANG_INCLUDE_AUX_PATH:=/cygdrive/c/progra~1/llvm/lib/clang/70fe6e~1.0/include > CLANG_LIBNAME:=/cygdrive/c/progra~1/llvm/bin/libclang.dll > LIBCLANG_CPPFLAGS:=-I/cygdrive/c/progra~1/llvm/include > LIBCLANG_LDFLAGS:=/LIBPATH:/cygdrive/c/progra~1/llvm/lib > LIBCLANG_LIBS:=/cygdrive/c/progra~1/llvm/lib/libclang.lib > > All paths involved seem to exist. > > Maurizio From maurizio.cimadamore at oracle.com Wed Jan 16 15:30:42 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 16 Jan 2019 15:30:42 +0000 Subject: [foreign] libclang.dll not copied to JDK image on Windows In-Reply-To: <10a2080b1a6aab9478135e874f27d22e@xs4all.nl> References: <0213f0cd-e6dd-f26a-99f6-416e3bf070fe@oracle.com> <10a2080b1a6aab9478135e874f27d22e@xs4all.nl> Message-ID: Yeah - noted the same, working on a fix Maurizio On 16/01/2019 15:22, Jorn Vernee wrote: > Afaik the `libclang.dll` file is supposed to be in the `bin` folder > after building. It's not copied there for me either currently, but > jextract works since I have the llvm/bin folder on the PATH, so the > library can be loaded from there. The copying was supposedly working > when the Windows build patch came in [1] > > ??? "This patch allow build image to be completed successfully and > libclang.dll copied to bin as expected." > > And I remember verifying that as well. I guess I just never noticed > the regression since I have the llvm/bin folder on my PATH. > > Jorn > > [1] : > https://mail.openjdk.java.net/pipermail/panama-dev/2018-October/003003.html > > Maurizio Cimadamore schreef op 2019-01-16 15:07: >> As the subject implies, there seems to be a build setup issue on >> windows, as the image I've built doesn't have libclang.dll under the >> images 'lib' folder. >> >> As a result jextract fails with the simplest of example: >> >> $ build/windows-x64/images/jdk/bin/jextract.exe foo.h >> Exception in thread "main" java.lang.UnsatisfiedLinkError: >> C:\cygwin64\home\MCIMADAM\panama\closed\build\windows-x64\images\jdk\bin\jclang.dll: >> >> Can't find dependent libraries >> ??????? at java.base/java.lang.ClassLoader$NativeLibrary.load0(Native >> Method) >> ??????? at >> java.base/java.lang.ClassLoader$NativeLibrary.load(ClassLoader.java:2451) >> >> ??????? at >> java.base/java.lang.ClassLoader$NativeLibrary.loadLibrary(ClassLoader.java:2523) >> >> ??????? at >> java.base/java.lang.ClassLoader.loadLibrary0(ClassLoader.java:2733) >> ??????? at >> java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2679) >> ??????? at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:849) >> ??????? at java.base/java.lang.System.loadLibrary(System.java:1905) >> ??????? at >> jdk.internal.clang/jdk.internal.clang.LibClang.(LibClang.java:35) >> >> ??????? at >> jdk.jextract/com.sun.tools.jextract.parser.Parser.parse(Parser.java:71) >> ??????? at >> jdk.jextract/com.sun.tools.jextract.JextractTool.processHeaders(JextractTool.java:58) >> >> ??????? at jdk.jextract/com.sun.tools.jextract.Main.run(Main.java:264) >> ??????? at jdk.jextract/com.sun.tools.jextract.Main.main(Main.java:334) >> >> >> Is this a known issue? This is what I get in the spec.gmk file: >> >> CLANG_LIB_PATH:=/cygdrive/c/progra~1/llvm/lib >> CLANG_INCLUDE_PATH:=/cygdrive/c/progra~1/llvm/include >> CLANG_INCLUDE_AUX_PATH:=/cygdrive/c/progra~1/llvm/lib/clang/70fe6e~1.0/include >> >> CLANG_LIBNAME:=/cygdrive/c/progra~1/llvm/bin/libclang.dll >> LIBCLANG_CPPFLAGS:=-I/cygdrive/c/progra~1/llvm/include >> LIBCLANG_LDFLAGS:=/LIBPATH:/cygdrive/c/progra~1/llvm/lib >> LIBCLANG_LIBS:=/cygdrive/c/progra~1/llvm/lib/libclang.lib >> >> All paths involved seem to exist. >> >> Maurizio From maurizio.cimadamore at oracle.com Wed Jan 16 15:32:31 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 16 Jan 2019 15:32:31 +0000 Subject: [foreign] test target fails on windows In-Reply-To: <9900ae85ec28a6f3d9e0c7c9b584f1bf@xs4all.nl> References: <3a81aa9e-6cf3-4e76-76ee-2d436bf43ee3@oracle.com> <373de078-f6a9-3e78-4eba-5c56427f91df@oracle.com> <9900ae85ec28a6f3d9e0c7c9b584f1bf@xs4all.nl> Message-ID: <28e97957-1aa0-d11f-e526-1f10dd7c9979@oracle.com> The way tests should be run is: make run-test-jdk_foreign this is the only way to make sure that all tests are ran (e.g. especially the ClangFFI test) Maurizio On 16/01/2019 15:06, Jorn Vernee wrote: > Yes, sorry. I have been using `disable-warnings-as-errors` since there > was a warning generated by a change in jdk/jdk code a while ago which > made `make images` fail. When I noticed the warnings, I had the intent > of fixing them later, but I kind of forgot about about it :/ > > I think your fix looks fine. FWIW, it seems that larger-than-int enum > constants are a GCC extension [1]. > > What about the other warning? I can't reproduce it currently, but I do > remember seeing it some time (maybe only from clean build). I always > use `make test TEST=jdk_foreign` to run the tests, I'm not sure if > there is a difference with `make run-test-jdk_foreign`. > > Jorn > > [1] : https://stackoverflow.com/q/366017 > > Maurizio Cimadamore schreef op 2019-01-16 14:57: >> The following patch seems to fix the issue - Jorn, perhaps you have >> been building/running tests with 'disable-warnings-as-errors' ? >> >> diff -r 364dc06dd3ed test/jdk/com/sun/tools/jextract/testEnum/enums.h >> --- a/test/jdk/com/sun/tools/jextract/testEnum/enums.h? Wed Jan 16 >> 12:45:05 2019 +0000 >> +++ b/test/jdk/com/sun/tools/jextract/testEnum/enums.h? Wed Jan 16 >> 13:57:29 2019 +0000 >> @@ -42,10 +42,17 @@ >> ??? I_VALUE2 = -5345345 >> ?}; >> >> +#ifndef _WIN64 >> ?enum LongEnum { >> ??? L_VALUE1 = -4564565645L, >> ??? L_VALUE2 = 45645645645L >> ?}; >> +#else >> +enum LongEnum { >> +?? L_VALUE1 = -5345345L, >> +?? L_VALUE2 = -5345345 >> +}; >> +#endif >> >> ?EXPORT int i_value1_func(); >> ?EXPORT int i_value2_func(); >> >> Maurizio >> >> On 16/01/2019 12:14, Maurizio Cimadamore wrote: >>> Hi, >>> I've tried to run tests on Windows the 'usual' way, but I get the >>> following failures - is this something known? >>> >>> c:\cygwin64\home\mcimadam\panama\closed\open\test\jdk\com\sun\tools\jextract\testenum\enums.h(46): >>> error C2220: warning treated as error - no 'object' file generated >>> c:\cygwin64\home\mcimadam\panama\closed\open\test\jdk\com\sun\tools\jextract\testenum\enums.h(46): >>> warning C4309: 'initializing': truncation of constant value >>> c:\cygwin64\home\mcimadam\panama\closed\open\test\jdk\com\sun\tools\jextract\testenum\enums.h(48): >>> warning C4309: 'initializing': truncation of constant value >>> JtregNativeJdk.gmk:106: recipe for target >>> '/cygdrive/c/cygwin64/home/mcimadam/panama/closed/build/windows-x64/support/test/jdk/jtreg/native/support/libEnums/libEnums.obj' >>> failed >>> make[3]: *** >>> [/cygdrive/c/cygwin64/home/mcimadam/panama/closed/build/windows-x64/support/test/jdk/jtreg/native/support/libEnums/libEnums.obj] >>> Error 1 >>> make[3]: *** Waiting for unfinished jobs.... >>> c:/cygwin64/home/mcimadam/panama/closed/open/test/jdk/com/sun/tools/jextract/testStruct/libStruct.c(106): >>> error C2220: warning treated as error - no 'object' file generated >>> c:/cygwin64/home/mcimadam/panama/closed/open/test/jdk/com/sun/tools/jextract/testStruct/libStruct.c(106): >>> warning C4477: 'printf' : format string '%lu' requires an argument >>> of type 'unsigned long', but variadic argument 1 has type '::size_t' >>> c:/cygwin64/home/mcimadam/panama/closed/open/test/jdk/com/sun/tools/jextract/testStruct/libStruct.c(106): >>> note: consider using '%zu' in the format string >>> >>> (**) - the way I run test is this: >>> >>> make run-test-jdk_foreign >>> >>> From jbvernee at xs4all.nl Wed Jan 16 15:51:11 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Wed, 16 Jan 2019 16:51:11 +0100 Subject: [foreign] test target fails on windows In-Reply-To: <28e97957-1aa0-d11f-e526-1f10dd7c9979@oracle.com> References: <3a81aa9e-6cf3-4e76-76ee-2d436bf43ee3@oracle.com> <373de078-f6a9-3e78-4eba-5c56427f91df@oracle.com> <9900ae85ec28a6f3d9e0c7c9b584f1bf@xs4all.nl> <28e97957-1aa0-d11f-e526-1f10dd7c9979@oracle.com> Message-ID: Ok. There seems to have been a build system change along the way that enforced that all test were being run. I remember the ClangFFI test suddenly started showing as a failure instead of vacuously passing. Any ways, running `make run-test-jdk_foreign` shows me the same number of passed tests (58, some are disabled on Windows). But I'll make sure to use that command from now on. Jorn Maurizio Cimadamore schreef op 2019-01-16 16:32: > The way tests should be run is: > > make run-test-jdk_foreign > > this is the only way to make sure that all tests are ran (e.g. > especially the ClangFFI test) > > Maurizio > > On 16/01/2019 15:06, Jorn Vernee wrote: >> Yes, sorry. I have been using `disable-warnings-as-errors` since there >> was a warning generated by a change in jdk/jdk code a while ago which >> made `make images` fail. When I noticed the warnings, I had the intent >> of fixing them later, but I kind of forgot about about it :/ >> >> I think your fix looks fine. FWIW, it seems that larger-than-int enum >> constants are a GCC extension [1]. >> >> What about the other warning? I can't reproduce it currently, but I do >> remember seeing it some time (maybe only from clean build). I always >> use `make test TEST=jdk_foreign` to run the tests, I'm not sure if >> there is a difference with `make run-test-jdk_foreign`. >> >> Jorn >> >> [1] : https://stackoverflow.com/q/366017 >> >> Maurizio Cimadamore schreef op 2019-01-16 14:57: >>> The following patch seems to fix the issue - Jorn, perhaps you have >>> been building/running tests with 'disable-warnings-as-errors' ? >>> >>> diff -r 364dc06dd3ed test/jdk/com/sun/tools/jextract/testEnum/enums.h >>> --- a/test/jdk/com/sun/tools/jextract/testEnum/enums.h? Wed Jan 16 >>> 12:45:05 2019 +0000 >>> +++ b/test/jdk/com/sun/tools/jextract/testEnum/enums.h? Wed Jan 16 >>> 13:57:29 2019 +0000 >>> @@ -42,10 +42,17 @@ >>> ??? I_VALUE2 = -5345345 >>> ?}; >>> >>> +#ifndef _WIN64 >>> ?enum LongEnum { >>> ??? L_VALUE1 = -4564565645L, >>> ??? L_VALUE2 = 45645645645L >>> ?}; >>> +#else >>> +enum LongEnum { >>> +?? L_VALUE1 = -5345345L, >>> +?? L_VALUE2 = -5345345 >>> +}; >>> +#endif >>> >>> ?EXPORT int i_value1_func(); >>> ?EXPORT int i_value2_func(); >>> >>> Maurizio >>> >>> On 16/01/2019 12:14, Maurizio Cimadamore wrote: >>>> Hi, >>>> I've tried to run tests on Windows the 'usual' way, but I get the >>>> following failures - is this something known? >>>> >>>> c:\cygwin64\home\mcimadam\panama\closed\open\test\jdk\com\sun\tools\jextract\testenum\enums.h(46): >>>> error C2220: warning treated as error - no 'object' file generated >>>> c:\cygwin64\home\mcimadam\panama\closed\open\test\jdk\com\sun\tools\jextract\testenum\enums.h(46): >>>> warning C4309: 'initializing': truncation of constant value >>>> c:\cygwin64\home\mcimadam\panama\closed\open\test\jdk\com\sun\tools\jextract\testenum\enums.h(48): >>>> warning C4309: 'initializing': truncation of constant value >>>> JtregNativeJdk.gmk:106: recipe for target >>>> '/cygdrive/c/cygwin64/home/mcimadam/panama/closed/build/windows-x64/support/test/jdk/jtreg/native/support/libEnums/libEnums.obj' >>>> failed >>>> make[3]: *** >>>> [/cygdrive/c/cygwin64/home/mcimadam/panama/closed/build/windows-x64/support/test/jdk/jtreg/native/support/libEnums/libEnums.obj] >>>> Error 1 >>>> make[3]: *** Waiting for unfinished jobs.... >>>> c:/cygwin64/home/mcimadam/panama/closed/open/test/jdk/com/sun/tools/jextract/testStruct/libStruct.c(106): >>>> error C2220: warning treated as error - no 'object' file generated >>>> c:/cygwin64/home/mcimadam/panama/closed/open/test/jdk/com/sun/tools/jextract/testStruct/libStruct.c(106): >>>> warning C4477: 'printf' : format string '%lu' requires an argument >>>> of type 'unsigned long', but variadic argument 1 has type '::size_t' >>>> c:/cygwin64/home/mcimadam/panama/closed/open/test/jdk/com/sun/tools/jextract/testStruct/libStruct.c(106): >>>> note: consider using '%zu' in the format string >>>> >>>> (**) - the way I run test is this: >>>> >>>> make run-test-jdk_foreign >>>> >>>> From henry.jen at oracle.com Wed Jan 16 16:54:15 2019 From: henry.jen at oracle.com (Henry Jen) Date: Wed, 16 Jan 2019 08:54:15 -0800 Subject: [foreign] libclang.dll not copied to JDK image on Windows In-Reply-To: References: <0213f0cd-e6dd-f26a-99f6-416e3bf070fe@oracle.com> <10a2080b1a6aab9478135e874f27d22e@xs4all.nl> Message-ID: I remember this was fixed by copy using CLANG_LIBNAME, https://cr.openjdk.java.net/~henryjen/panama/build-windows/webrev.01/webrev/ but perhaps it got reverted. Another follow up that wasn?t pushed is following to make sure https://cr.openjdk.java.net/~henryjen/panama/libclang-symlinks/webrev/ Cheers, Henry > On Jan 16, 2019, at 7:30 AM, Maurizio Cimadamore wrote: > > Yeah - noted the same, working on a fix > > Maurizio > > On 16/01/2019 15:22, Jorn Vernee wrote: >> Afaik the `libclang.dll` file is supposed to be in the `bin` folder after building. It's not copied there for me either currently, but jextract works since I have the llvm/bin folder on the PATH, so the library can be loaded from there. The copying was supposedly working when the Windows build patch came in [1] >> >> "This patch allow build image to be completed successfully and libclang.dll copied to bin as expected." >> >> And I remember verifying that as well. I guess I just never noticed the regression since I have the llvm/bin folder on my PATH. >> >> Jorn >> >> [1] : https://mail.openjdk.java.net/pipermail/panama-dev/2018-October/003003.html >> >> Maurizio Cimadamore schreef op 2019-01-16 15:07: >>> As the subject implies, there seems to be a build setup issue on >>> windows, as the image I've built doesn't have libclang.dll under the >>> images 'lib' folder. >>> >>> As a result jextract fails with the simplest of example: >>> >>> $ build/windows-x64/images/jdk/bin/jextract.exe foo.h >>> Exception in thread "main" java.lang.UnsatisfiedLinkError: >>> C:\cygwin64\home\MCIMADAM\panama\closed\build\windows-x64\images\jdk\bin\jclang.dll: >>> Can't find dependent libraries >>> at java.base/java.lang.ClassLoader$NativeLibrary.load0(Native Method) >>> at >>> java.base/java.lang.ClassLoader$NativeLibrary.load(ClassLoader.java:2451) >>> at >>> java.base/java.lang.ClassLoader$NativeLibrary.loadLibrary(ClassLoader.java:2523) >>> at java.base/java.lang.ClassLoader.loadLibrary0(ClassLoader.java:2733) >>> at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2679) >>> at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:849) >>> at java.base/java.lang.System.loadLibrary(System.java:1905) >>> at >>> jdk.internal.clang/jdk.internal.clang.LibClang.(LibClang.java:35) >>> at >>> jdk.jextract/com.sun.tools.jextract.parser.Parser.parse(Parser.java:71) >>> at >>> jdk.jextract/com.sun.tools.jextract.JextractTool.processHeaders(JextractTool.java:58) >>> at jdk.jextract/com.sun.tools.jextract.Main.run(Main.java:264) >>> at jdk.jextract/com.sun.tools.jextract.Main.main(Main.java:334) >>> >>> >>> Is this a known issue? This is what I get in the spec.gmk file: >>> >>> CLANG_LIB_PATH:=/cygdrive/c/progra~1/llvm/lib >>> CLANG_INCLUDE_PATH:=/cygdrive/c/progra~1/llvm/include >>> CLANG_INCLUDE_AUX_PATH:=/cygdrive/c/progra~1/llvm/lib/clang/70fe6e~1.0/include >>> CLANG_LIBNAME:=/cygdrive/c/progra~1/llvm/bin/libclang.dll >>> LIBCLANG_CPPFLAGS:=-I/cygdrive/c/progra~1/llvm/include >>> LIBCLANG_LDFLAGS:=/LIBPATH:/cygdrive/c/progra~1/llvm/lib >>> LIBCLANG_LIBS:=/cygdrive/c/progra~1/llvm/lib/libclang.lib >>> >>> All paths involved seem to exist. >>> >>> Maurizio From shravya.rukmannagari at intel.com Wed Jan 16 17:05:23 2019 From: shravya.rukmannagari at intel.com (Rukmannagari, Shravya) Date: Wed, 16 Jan 2019 17:05:23 +0000 Subject: VectorAPI Constant Shift for Byte and Short Intrinsic Support In-Reply-To: <8c0fb8d9-9b1e-a116-2488-fba22ed1019d@oracle.com> References: <8D6F463991A1574A8A803B8DA605414F3A82C461@ORSMSX111.amr.corp.intel.com> <8D6F463991A1574A8A803B8DA605414F3A82E2E1@ORSMSX111.amr.corp.intel.com> <8c0fb8d9-9b1e-a116-2488-fba22ed1019d@oracle.com> Message-ID: <8D6F463991A1574A8A803B8DA605414F3A82E573@ORSMSX111.amr.corp.intel.com> Hi Vladimir, I added a rule in the matcher to support this constraint. Please find the patch below: http://cr.openjdk.java.net/~srukmannagar/VectorAPI_ByteShortConstantShift/webrev.02/ Thanks, Shravya. -----Original Message----- From: Vladimir Ivanov [mailto:vladimir.x.ivanov at oracle.com] Sent: Tuesday, January 15, 2019 5:21 PM To: Rukmannagari, Shravya ; panama-dev Subject: Re: VectorAPI Constant Shift for Byte and Short Intrinsic Support > Not adding 32B for AVX=1 was intentional. I updated the patch with your comments. Please find the webrev below. > http://cr.openjdk.java.net/~srukmannagar/VectorAPI_ByteShortConstantSh > ift/webrev.01/ So, what's expected behavior for 32B variants with AVX=1 then? If you deliberately keep 32B variants not supported for now, I'd expect to see that listed in Matcher::match_rule_supported_vector() as an implementation constraint. The operations shouldn't be intrinsified (because there are no applicable rules in AD file), but Matcher::match_rule_supported_vector() only limits support to UseSSE >= 4. Am I missing something? Best regards, Vladimir Ivanov > -----Original Message----- > From: Vladimir Ivanov [mailto:vladimir.x.ivanov at oracle.com] > Sent: Thursday, January 10, 2019 2:39 PM > To: Rukmannagari, Shravya ; panama-dev > > Subject: Re: VectorAPI Constant Shift for Byte and Short Intrinsic > Support > > >> http://cr.openjdk.java.net/~srukmannagar/VectorAPI_ByteShortConstantS >> h >> ift/webrev.00/ > > Don't you need to update Javadoc as well? > > Though Javadoc refers to shift operations, implementation does more than that ("a >> i" vs "a >> (i & 7)" / "a >> (i & 15)"): > > /** > * Arithmetically right shifts (or signed right shifts) this vector by the > * broadcast of an input scalar. > *

> * This is a vector binary operation where the primitive arithmetic right > * shift operation ({@code >>}) is applied to lane elements. > * > * @param s the input scalar; the number of the bits to right shift > * @return the result of arithmetically right shifting this vector by the > * broadcast of an input scalar > */ > public abstract ShortVector aShiftR(int s); > > vs > > @Override > @ForceInline > public Short256Vector aShiftR(int s) { > return VectorIntrinsics.broadcastInt( > VECTOR_OP_RSHIFT, Short256Vector.class, short.class, LENGTH, > this, s, > (v, i) -> v.uOp((__, a) -> (short) (a >> (i & 15)))); > } > > ================================================= > > I assume C2 backend support for Short*Vector is already there. > > src/hotspot/cpu/x86/x86.ad: > > +instruct vsll32B_avx(vecY dst, vecY src, vecS shift, vecY tmp, vecY > tmp2, rRegL scratch) %{ > + predicate(UseAVX > 1 && n->as_Vector()->length() == 32); > > Is it deliberate there's no 32B support with UseAVX=1? > > On related node, do you need to add *ShiftVB cases into > Matcher::match_rule_supported_vector()? > > Best regards, > Vladimir Ivanov > From maurizio.cimadamore at oracle.com Wed Jan 16 17:17:13 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 16 Jan 2019 17:17:13 +0000 Subject: [foreign] libclang.dll not copied to JDK image on Windows In-Reply-To: References: <0213f0cd-e6dd-f26a-99f6-416e3bf070fe@oracle.com> <10a2080b1a6aab9478135e874f27d22e@xs4all.nl> Message-ID: <97c437d4-1421-0614-7c0e-122b07cf1eab@oracle.com> I put together this patch: http://cr.openjdk.java.net/~mcimadamore/panama/winTest.patch With this, I can build on Windows, and I can run tests. I get test 2 failures, both caused by the fact that jextract cannot find 'common' headers such as and - any idea? Maurizio On 16/01/2019 16:54, Henry Jen wrote: > I remember this was fixed by copy using CLANG_LIBNAME, > > https://cr.openjdk.java.net/~henryjen/panama/build-windows/webrev.01/webrev/ > > but perhaps it got reverted. > > Another follow up that wasn?t pushed is following to make sure > > https://cr.openjdk.java.net/~henryjen/panama/libclang-symlinks/webrev/ > > Cheers, > Henry > >> On Jan 16, 2019, at 7:30 AM, Maurizio Cimadamore wrote: >> >> Yeah - noted the same, working on a fix >> >> Maurizio >> >> On 16/01/2019 15:22, Jorn Vernee wrote: >>> Afaik the `libclang.dll` file is supposed to be in the `bin` folder after building. It's not copied there for me either currently, but jextract works since I have the llvm/bin folder on the PATH, so the library can be loaded from there. The copying was supposedly working when the Windows build patch came in [1] >>> >>> "This patch allow build image to be completed successfully and libclang.dll copied to bin as expected." >>> >>> And I remember verifying that as well. I guess I just never noticed the regression since I have the llvm/bin folder on my PATH. >>> >>> Jorn >>> >>> [1] : https://mail.openjdk.java.net/pipermail/panama-dev/2018-October/003003.html >>> >>> Maurizio Cimadamore schreef op 2019-01-16 15:07: >>>> As the subject implies, there seems to be a build setup issue on >>>> windows, as the image I've built doesn't have libclang.dll under the >>>> images 'lib' folder. >>>> >>>> As a result jextract fails with the simplest of example: >>>> >>>> $ build/windows-x64/images/jdk/bin/jextract.exe foo.h >>>> Exception in thread "main" java.lang.UnsatisfiedLinkError: >>>> C:\cygwin64\home\MCIMADAM\panama\closed\build\windows-x64\images\jdk\bin\jclang.dll: >>>> Can't find dependent libraries >>>> at java.base/java.lang.ClassLoader$NativeLibrary.load0(Native Method) >>>> at >>>> java.base/java.lang.ClassLoader$NativeLibrary.load(ClassLoader.java:2451) >>>> at >>>> java.base/java.lang.ClassLoader$NativeLibrary.loadLibrary(ClassLoader.java:2523) >>>> at java.base/java.lang.ClassLoader.loadLibrary0(ClassLoader.java:2733) >>>> at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2679) >>>> at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:849) >>>> at java.base/java.lang.System.loadLibrary(System.java:1905) >>>> at >>>> jdk.internal.clang/jdk.internal.clang.LibClang.(LibClang.java:35) >>>> at >>>> jdk.jextract/com.sun.tools.jextract.parser.Parser.parse(Parser.java:71) >>>> at >>>> jdk.jextract/com.sun.tools.jextract.JextractTool.processHeaders(JextractTool.java:58) >>>> at jdk.jextract/com.sun.tools.jextract.Main.run(Main.java:264) >>>> at jdk.jextract/com.sun.tools.jextract.Main.main(Main.java:334) >>>> >>>> >>>> Is this a known issue? This is what I get in the spec.gmk file: >>>> >>>> CLANG_LIB_PATH:=/cygdrive/c/progra~1/llvm/lib >>>> CLANG_INCLUDE_PATH:=/cygdrive/c/progra~1/llvm/include >>>> CLANG_INCLUDE_AUX_PATH:=/cygdrive/c/progra~1/llvm/lib/clang/70fe6e~1.0/include >>>> CLANG_LIBNAME:=/cygdrive/c/progra~1/llvm/bin/libclang.dll >>>> LIBCLANG_CPPFLAGS:=-I/cygdrive/c/progra~1/llvm/include >>>> LIBCLANG_LDFLAGS:=/LIBPATH:/cygdrive/c/progra~1/llvm/lib >>>> LIBCLANG_LIBS:=/cygdrive/c/progra~1/llvm/lib/libclang.lib >>>> >>>> All paths involved seem to exist. >>>> >>>> Maurizio From maurizio.cimadamore at oracle.com Wed Jan 16 17:27:10 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 16 Jan 2019 17:27:10 +0000 Subject: [foreign] libclang.dll not copied to JDK image on Windows In-Reply-To: <97c437d4-1421-0614-7c0e-122b07cf1eab@oracle.com> References: <0213f0cd-e6dd-f26a-99f6-416e3bf070fe@oracle.com> <10a2080b1a6aab9478135e874f27d22e@xs4all.nl> <97c437d4-1421-0614-7c0e-122b07cf1eab@oracle.com> Message-ID: <0472b145-c7ef-7b0c-1294-c9d88c2b063a@oracle.com> On 16/01/2019 17:17, Maurizio Cimadamore wrote: > I get test 2 failures, both caused by the fact that jextract cannot > find 'common' headers such as and - any idea? Ok, I think the problem here is that clang expects Visual Studio to be installed: clang -cc1 version 7.0.0 based upon LLVM 7.0.0 default target x86_64-pc-win32 ignoring nonexistent directory "C:/Program Files/Microsoft Visual Studio 10.0/VC/include" ignoring nonexistent directory "C:/Program Files/Microsoft Visual Studio 9.0/VC/include" ignoring nonexistent directory "C:/Program Files/Microsoft Visual Studio 9.0/VC/PlatformSDK/Include" ignoring nonexistent directory "C:/Program Files/Microsoft Visual Studio 8/VC/include" ignoring nonexistent directory "C:/Program Files/Microsoft Visual Studio 8/VC/PlatformSDK/Include" #include "..." search starts here: #include <...> search starts here: The build I have does not use a standard Visual Studio install, hence the issue. Maurizio From maurizio.cimadamore at oracle.com Wed Jan 16 17:55:38 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 16 Jan 2019 17:55:38 +0000 Subject: [foreign] RFR 8217246: jextract should parse single header file with clang In-Reply-To: <5C3F2A1F.5010906@oracle.com> References: <5C3F2A1F.5010906@oracle.com> Message-ID: Looks good - I have one question: do we still need to call 'distinct' in JextractTool::processHeaders ? That is, shouldn't it be the case that now the parser will only give us a given cursor only once? I added this step because, since we were doing multiple clang passes, it was sometimes necessary to filter out duplicates (e.g. if two input headers both included time.h) - but now that we unify the headers under a single clang compilation unit, shouldn't the clang API already give us unique entities? [I'm saying this because I suspect that this distinct() call is causing much of the overhead introduced by the recent jextract refactoring] Maurizio On 16/01/2019 12:57, Sundararajan Athijegannathan wrote: > Please review. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8217246 > Webrev: https://cr.openjdk.java.net/~sundar/8217246/webrev.00/ > > Thanks, > -Sundar From vladimir.x.ivanov at oracle.com Wed Jan 16 17:59:48 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Wed, 16 Jan 2019 09:59:48 -0800 Subject: VectorAPI Constant Shift for Byte and Short Intrinsic Support In-Reply-To: <8D6F463991A1574A8A803B8DA605414F3A82E573@ORSMSX111.amr.corp.intel.com> References: <8D6F463991A1574A8A803B8DA605414F3A82C461@ORSMSX111.amr.corp.intel.com> <8D6F463991A1574A8A803B8DA605414F3A82E2E1@ORSMSX111.amr.corp.intel.com> <8c0fb8d9-9b1e-a116-2488-fba22ed1019d@oracle.com> <8D6F463991A1574A8A803B8DA605414F3A82E573@ORSMSX111.amr.corp.intel.com> Message-ID: <6f17b743-6dc6-a426-e8ab-18cfcb456799@oracle.com> > I added a rule in the matcher to support this constraint. Please find the patch below: > http://cr.openjdk.java.net/~srukmannagar/VectorAPI_ByteShortConstantShift/webrev.02/ Looks good! Best regards, Vladimir Ivanov > > Thanks, > Shravya. > > -----Original Message----- > From: Vladimir Ivanov [mailto:vladimir.x.ivanov at oracle.com] > Sent: Tuesday, January 15, 2019 5:21 PM > To: Rukmannagari, Shravya ; panama-dev > Subject: Re: VectorAPI Constant Shift for Byte and Short Intrinsic Support > > >> Not adding 32B for AVX=1 was intentional. I updated the patch with your comments. Please find the webrev below. >> http://cr.openjdk.java.net/~srukmannagar/VectorAPI_ByteShortConstantSh >> ift/webrev.01/ > > So, what's expected behavior for 32B variants with AVX=1 then? > If you deliberately keep 32B variants not supported for now, I'd expect to see that listed in Matcher::match_rule_supported_vector() as an implementation constraint. > > The operations shouldn't be intrinsified (because there are no applicable rules in AD file), but Matcher::match_rule_supported_vector() > only limits support to UseSSE >= 4. > > Am I missing something? > > Best regards, > Vladimir Ivanov > >> -----Original Message----- >> From: Vladimir Ivanov [mailto:vladimir.x.ivanov at oracle.com] >> Sent: Thursday, January 10, 2019 2:39 PM >> To: Rukmannagari, Shravya ; panama-dev >> >> Subject: Re: VectorAPI Constant Shift for Byte and Short Intrinsic >> Support >> >> >>> http://cr.openjdk.java.net/~srukmannagar/VectorAPI_ByteShortConstantS >>> h >>> ift/webrev.00/ >> >> Don't you need to update Javadoc as well? >> >> Though Javadoc refers to shift operations, implementation does more than that ("a >> i" vs "a >> (i & 7)" / "a >> (i & 15)"): >> >> /** >> * Arithmetically right shifts (or signed right shifts) this vector by the >> * broadcast of an input scalar. >> *

>> * This is a vector binary operation where the primitive arithmetic right >> * shift operation ({@code >>}) is applied to lane elements. >> * >> * @param s the input scalar; the number of the bits to right shift >> * @return the result of arithmetically right shifting this vector by the >> * broadcast of an input scalar >> */ >> public abstract ShortVector aShiftR(int s); >> >> vs >> >> @Override >> @ForceInline >> public Short256Vector aShiftR(int s) { >> return VectorIntrinsics.broadcastInt( >> VECTOR_OP_RSHIFT, Short256Vector.class, short.class, LENGTH, >> this, s, >> (v, i) -> v.uOp((__, a) -> (short) (a >> (i & 15)))); >> } >> >> ================================================= >> >> I assume C2 backend support for Short*Vector is already there. >> >> src/hotspot/cpu/x86/x86.ad: >> >> +instruct vsll32B_avx(vecY dst, vecY src, vecS shift, vecY tmp, vecY >> tmp2, rRegL scratch) %{ >> + predicate(UseAVX > 1 && n->as_Vector()->length() == 32); >> >> Is it deliberate there's no 32B support with UseAVX=1? >> >> On related node, do you need to add *ShiftVB cases into >> Matcher::match_rule_supported_vector()? >> >> Best regards, >> Vladimir Ivanov >> From maurizio.cimadamore at oracle.com Wed Jan 16 18:17:25 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 16 Jan 2019 18:17:25 +0000 Subject: RFR 8217268: Windows tests fail to build Message-ID: Hi, This is an official RFR for this patch: http://cr.openjdk.java.net/~mcimadamore/panama/8217268/ This takes care of the warnings causing build failure when running tests with make; it also addresses the problem of copying libclang.dll in the right folder (I did that in a very minimalistic way, to prevent issues in other platforms, we can revise this code at a later point if needed). With this, Windows builds and all tests pass except for two: SystemHeadersTest TestJextractFFI These tests depend on system headers - and clang by default, on Windows resolves system headers using the standard MSVC install path. If MSVC is not installed, clang will fail to resolve system headers (unless an explicit -I option is passed to jextract). I don't think there's a way to fix these tests in a way that they will work in our internal build/test setup - given that the machines in such setup do not have a standard MSVC install. That said, I think this fix makes Windows support much more reliable when it comes to testing. Cheers Maurizio From maurizio.cimadamore at oracle.com Wed Jan 16 18:22:06 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 16 Jan 2019 18:22:06 +0000 Subject: hg: panama/dev: Add support for windows jib profile/devkit Message-ID: <201901161822.x0GIM61i016067@aojmv0008.oracle.com> Changeset: cbf5c43555b7 Author: mcimadamore Date: 2019-01-16 18:21 +0000 URL: http://hg.openjdk.java.net/panama/dev/rev/cbf5c43555b7 Add support for windows jib profile/devkit Contributed-by: erik.joelsson at oracle.com ! make/conf/jib-profiles.js ! make/devkit/createLibclangBundle.sh From maurizio.cimadamore at oracle.com Wed Jan 16 18:25:01 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 16 Jan 2019 18:25:01 +0000 Subject: hg: panama/dev: Automatic merge with foreign Message-ID: <201901161825.x0GIP1Qd019069@aojmv0008.oracle.com> Changeset: 341dded452d7 Author: mcimadamore Date: 2019-01-16 19:24 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/341dded452d7 Automatic merge with foreign From maurizio.cimadamore at oracle.com Wed Jan 16 18:25:26 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 16 Jan 2019 18:25:26 +0000 Subject: hg: panama/dev: Automatic merge with foreign Message-ID: <201901161825.x0GIPRoC019982@aojmv0008.oracle.com> Changeset: 914a7f5aa56b Author: mcimadamore Date: 2019-01-16 19:25 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/914a7f5aa56b Automatic merge with foreign From henry.jen at oracle.com Wed Jan 16 18:34:28 2019 From: henry.jen at oracle.com (Henry Jen) Date: Wed, 16 Jan 2019 10:34:28 -0800 Subject: RFR 8217268: Windows tests fail to build In-Reply-To: References: Message-ID: Looks good. Cheers, Henry > On Jan 16, 2019, at 10:17 AM, Maurizio Cimadamore wrote: > > Hi, > This is an official RFR for this patch: > > http://cr.openjdk.java.net/~mcimadamore/panama/8217268/ > > This takes care of the warnings causing build failure when running tests with make; it also addresses the problem of copying libclang.dll in the right folder (I did that in a very minimalistic way, to prevent issues in other platforms, we can revise this code at a later point if needed). > > With this, Windows builds and all tests pass except for two: > > SystemHeadersTest > TestJextractFFI > > These tests depend on system headers - and clang by default, on Windows resolves system headers using the standard MSVC install path. If MSVC is not installed, clang will fail to resolve system headers (unless an explicit -I option is passed to jextract). I don't think there's a way to fix these tests in a way that they will work in our internal build/test setup - given that the machines in such setup do not have a standard MSVC install. > > That said, I think this fix makes Windows support much more reliable when it comes to testing. > > Cheers > Maurizio > From jbvernee at xs4all.nl Wed Jan 16 18:36:37 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Wed, 16 Jan 2019 19:36:37 +0100 Subject: RFR 8217268: Windows tests fail to build In-Reply-To: References: Message-ID: <241e123a0906d23aa79fe6b7e0473951@xs4all.nl> All tests pass on my end with this patch as well. Jorn Henry Jen schreef op 2019-01-16 19:34: > Looks good. > > Cheers, > Henry > >> On Jan 16, 2019, at 10:17 AM, Maurizio Cimadamore >> wrote: >> >> Hi, >> This is an official RFR for this patch: >> >> http://cr.openjdk.java.net/~mcimadamore/panama/8217268/ >> >> This takes care of the warnings causing build failure when running >> tests with make; it also addresses the problem of copying libclang.dll >> in the right folder (I did that in a very minimalistic way, to prevent >> issues in other platforms, we can revise this code at a later point if >> needed). >> >> With this, Windows builds and all tests pass except for two: >> >> SystemHeadersTest >> TestJextractFFI >> >> These tests depend on system headers - and clang by default, on >> Windows resolves system headers using the standard MSVC install path. >> If MSVC is not installed, clang will fail to resolve system headers >> (unless an explicit -I option is passed to jextract). I don't think >> there's a way to fix these tests in a way that they will work in our >> internal build/test setup - given that the machines in such setup do >> not have a standard MSVC install. >> >> That said, I think this fix makes Windows support much more reliable >> when it comes to testing. >> >> Cheers >> Maurizio >> From maurizio.cimadamore at oracle.com Wed Jan 16 19:13:18 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 16 Jan 2019 19:13:18 +0000 Subject: hg: panama/dev: 8217268: Windows tests fail to build Message-ID: <201901161913.x0GJDIDn013320@aojmv0008.oracle.com> Changeset: 0b6500132478 Author: mcimadamore Date: 2019-01-16 19:12 +0000 URL: http://hg.openjdk.java.net/panama/dev/rev/0b6500132478 8217268: Windows tests fail to build ! make/lib/Lib-jdk.internal.clang.gmk ! test/jdk/com/sun/tools/jextract/testEnum/enums.h ! test/jdk/com/sun/tools/jextract/testStruct/libStruct.c ! test/jdk/java/foreign/types/libPointers.c From maurizio.cimadamore at oracle.com Wed Jan 16 19:15:01 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 16 Jan 2019 19:15:01 +0000 Subject: hg: panama/dev: Automatic merge with foreign Message-ID: <201901161915.x0GJF10B014687@aojmv0008.oracle.com> Changeset: 0cbab1918503 Author: mcimadamore Date: 2019-01-16 20:14 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/0cbab1918503 Automatic merge with foreign From maurizio.cimadamore at oracle.com Wed Jan 16 19:15:28 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 16 Jan 2019 19:15:28 +0000 Subject: hg: panama/dev: Automatic merge with foreign Message-ID: <201901161915.x0GJFSE0015161@aojmv0008.oracle.com> Changeset: 8c51e32e2575 Author: mcimadamore Date: 2019-01-16 20:15 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/8c51e32e2575 Automatic merge with foreign From sandhya.viswanathan at intel.com Wed Jan 16 20:32:59 2019 From: sandhya.viswanathan at intel.com (Viswanathan, Sandhya) Date: Wed, 16 Jan 2019 20:32:59 +0000 Subject: [Vector] RFR: reshape, resize, rebracket, cast In-Reply-To: References: Message-ID: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A5012D@FMSMSX126.amr.corp.intel.com> Adding Brian, John, Joe and Vladimir to the thread. Please do give your feedback on the patch from Kishor (http://cr.openjdk.java.net/~kkharbas/vector-api/webrev-reshape_etc) implementing the reinterpret API as suggested by Brian and Vladimir in thread https://mail.openjdk.java.net/pipermail/panama-dev/2018-December/003365.html. Best Regards, Sandhya -----Original Message----- From: panama-dev [mailto:panama-dev-bounces at openjdk.java.net] On Behalf Of Kharbas, Kishor Sent: Wednesday, January 16, 2019 12:20 AM To: panama-dev at openjdk.java.net Subject: [Vector] RFR: reshape, resize, rebracket, cast Hi, I have a patch which refactors the above methods as discussed in this thread - https://mail.openjdk.java.net/pipermail/panama-dev/2018-December/003365.html. Please review the changes at - http://cr.openjdk.java.net/~kkharbas/vector-api/webrev-reshape_etc Generation of intrinsics and correct vector boxing elimination have been verified. In the next patch I will add changes to jtreg tests. I wanted to bring forth one small issue(possibly) which programmers might face with this change - We provide specialized types like IntVector, FloatVector, etc for users to define their vectors and they would be able to write code like this with previous methods, FloatVector float256 = SPECIES_FLOAT256.cast(int256); Here FloatSpecies would always return FloatVector. However with this change, since cast() is defined on a vector and takes species of a generic type, it cannot return a specialized Vector like IntVector or FloatVector. User has to explicitly cast the return vector from Vector to specialized Vector or use a generic vector of corresponding element type. For example, FloatVector float256 = (FloatVector)int256.cast(SPECIES_FLOAT256); or Vector float256 = int256.cast(SPECIES_FLOAT256); I am not sure if this is even a problem, but I thought its worth mentioning. Thanks Kishor From shravya.rukmannagari at intel.com Wed Jan 16 20:38:28 2019 From: shravya.rukmannagari at intel.com (shravya.rukmannagari at intel.com) Date: Wed, 16 Jan 2019 20:38:28 +0000 Subject: hg: panama/dev: Contant Shift support for Byte and Short datatypes in VectorAPI Message-ID: <201901162038.x0GKcTYb024011@aojmv0008.oracle.com> Changeset: 90ddf1dc631d Author: srukmannagar Date: 2019-01-16 12:37 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/90ddf1dc631d Contant Shift support for Byte and Short datatypes in VectorAPI ! src/hotspot/cpu/x86/assembler_x86.cpp ! src/hotspot/cpu/x86/assembler_x86.hpp ! src/hotspot/cpu/x86/stubGenerator_x86_64.cpp ! src/hotspot/cpu/x86/stubRoutines_x86.hpp ! src/hotspot/cpu/x86/stubRoutines_x86_64.cpp ! src/hotspot/cpu/x86/x86.ad ! src/hotspot/share/opto/library_call.cpp ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Byte128Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Byte256Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Byte512Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Byte64Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/ByteMaxVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/ByteVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Int128Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Int256Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Int512Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Int64Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/IntMaxVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/IntVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Long128Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Long256Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Long512Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Long64Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/LongMaxVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/LongVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Short128Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Short256Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Short512Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Short64Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/ShortMaxVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/ShortVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/X-Vector.java.template ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/X-VectorBits.java.template ! test/jdk/jdk/incubator/vector/Byte128VectorTests.java ! test/jdk/jdk/incubator/vector/Byte256VectorTests.java ! test/jdk/jdk/incubator/vector/Byte512VectorTests.java ! test/jdk/jdk/incubator/vector/Byte64VectorTests.java ! test/jdk/jdk/incubator/vector/ByteMaxVectorTests.java ! test/jdk/jdk/incubator/vector/Double128VectorTests.java ! test/jdk/jdk/incubator/vector/Double256VectorTests.java ! test/jdk/jdk/incubator/vector/Double512VectorTests.java ! test/jdk/jdk/incubator/vector/Double64VectorTests.java ! test/jdk/jdk/incubator/vector/DoubleMaxVectorTests.java ! test/jdk/jdk/incubator/vector/Float128VectorTests.java ! test/jdk/jdk/incubator/vector/Float256VectorTests.java ! test/jdk/jdk/incubator/vector/Float512VectorTests.java ! test/jdk/jdk/incubator/vector/Float64VectorTests.java ! test/jdk/jdk/incubator/vector/FloatMaxVectorTests.java ! test/jdk/jdk/incubator/vector/Int128VectorTests.java ! test/jdk/jdk/incubator/vector/Int256VectorTests.java ! test/jdk/jdk/incubator/vector/Int512VectorTests.java ! test/jdk/jdk/incubator/vector/Int64VectorTests.java ! test/jdk/jdk/incubator/vector/IntMaxVectorTests.java ! test/jdk/jdk/incubator/vector/Long128VectorTests.java ! test/jdk/jdk/incubator/vector/Long256VectorTests.java ! test/jdk/jdk/incubator/vector/Long512VectorTests.java ! test/jdk/jdk/incubator/vector/Long64VectorTests.java ! test/jdk/jdk/incubator/vector/LongMaxVectorTests.java ! test/jdk/jdk/incubator/vector/Short128VectorTests.java ! test/jdk/jdk/incubator/vector/Short256VectorTests.java ! test/jdk/jdk/incubator/vector/Short512VectorTests.java ! test/jdk/jdk/incubator/vector/Short64VectorTests.java ! test/jdk/jdk/incubator/vector/ShortMaxVectorTests.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Byte128Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Byte256Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Byte512Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Byte64Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/ByteMaxVector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/ByteScalar.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Double128Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Double256Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Double512Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Double64Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/DoubleMaxVector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/DoubleScalar.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Float128Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Float256Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Float512Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Float64Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/FloatMaxVector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/FloatScalar.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Int128Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Int256Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Int512Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Int64Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/IntMaxVector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/IntScalar.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Long128Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Long256Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Long512Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Long64Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/LongMaxVector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/LongScalar.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Short128Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Short256Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Short512Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Short64Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/ShortMaxVector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/ShortScalar.java ! test/jdk/jdk/incubator/vector/gen-template.sh From vladimir.x.ivanov at oracle.com Wed Jan 16 20:40:40 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Wed, 16 Jan 2019 12:40:40 -0800 Subject: [VectorAPI]maskFromArray Intrinsic failure In-Reply-To: <77404c4b-cfa8-4f47-b58f-533ae0d9a1e6.zhuoren.wz@alibaba-inc.com> References: <701fcf82-7ace-4e66-910b-3120753809a8.zhuoren.wz@alibaba-inc.com> <7b720dbb-91d8-4bef-e680-9474ead264f2@oracle.com> <77404c4b-cfa8-4f47-b58f-533ae0d9a1e6.zhuoren.wz@alibaba-inc.com> Message-ID: <4c086c97-ac8c-a07b-4535-40194338243e@oracle.com> Looks good. Best regards, Vladimir Ivanov On 16/01/2019 04:46, Wang Zhuo(Zhuoren) wrote: > Webrev files have been uploaded to > http://cr.openjdk.java.net/~luchsh/mask_fix_zhuoren/.? Please review. > > Regards, > Zhuoren > > ------------------------------------------------------------------ > From:Vladimir Ivanov > Sent At:2019 Jan. 16 (Wed.) 02:46 > To:Sandler ; panama-dev > > Cc:li sanhong ; Kingsum Chow > > Subject:Re: [VectorAPI]maskFromArray Intrinsic failure > > Good?catch! > > src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Int128Vector.java: > ?>?-?????????????????????????????????????????bits,?(((long)?ix)?<< > ARRAY_SHIFT)?+?Unsafe.ARRAY_BOOLEAN_BASE_OFFSET, > ?>?+?????????????????????????????????????????bits,?((long)?ix)?+ > Unsafe.ARRAY_BOOLEAN_BASE_OFFSET, > > It's?better?to?replace?ARRAY_SHIFT?with?Unsafe.ARRAY_BOOLEAN_INDEX_SCALE. > > > src/hotspot/share/opto/library_call.cpp: > > ?>?+????if?(!arch_supports_vector(Op_LoadVector,?num_elem,?T_BOOLEAN, > VecMaskUseLoad))?{ > > Why?do?you?specify?VecMaskUseLoad?and?not?VecMaskNotUsed?(as?in?other > cases)? > > Otherwise,?looks?good?for?now.?Please,?send?complete?webrev?for?review. > > As?a?followup,?it?would?be?nice?to?refactor/cleanup?Mask-related?code?on > > JDK-JVM?boundary?(VectorIntrinsics?+?library_call.cpp).?For?example,?I > don't?see?a?reason?to?pass?int.class?(and?not?boolean.class)?as?the > element?type?for?mask-related?operations. > > Best?regards, > Vladimir?Ivanov > > On?15/01/2019?08:37,?Wang?Zhuo(Zhuoren)?wrote: > >?Hi, > >?I?found?intrinsics?for?maskFromArray?always?failed.?Here?is?a?patch?to?fix?this?issue. > >?Benchmarks?containing?heavy?maskFromArray?use?can?benifit?from?this?patch.?such?as?allTrue/anyTrue. > > > >?This?patch?consists?of?two?parts.?In?the?JDK?part,?since?boolean?is?only?one?byte,??it?is?not?correct?to?perform?left?shift?ARRAY_SHIFT?for?array?index.?I?listed?only?one?file?in?the?patch?to?avoid?a?long?letter,?but?this?modification?should?be?applied?to?all?types?including?the?template?file. > >???????????????return?VectorIntrinsics.load(Int128Mask.class,?int.class,?LENGTH, > >?-?????????????????????????????????????????bits,?(((long)?ix)?< >?+?????????????????????????????????????????bits,?((long)?ix)?+?Unsafe.ARRAY_BOOLEAN_BASE_OFFSET, > >????????????????????????????????????????????bits,?ix, > >?Another?part?is?in?hotspot,?please?check?the?below?patch > > > >?diff?-r?890e996dfd1a?src/hotspot/share/opto/library_call.cpp > >?---?a/src/hotspot/share/opto/library_call.cpp???Mon?Jan?07?10:54:21?2019?+0800 > >?+++?b/src/hotspot/share/opto/library_call.cpp???Tue?Jan?15?23:45:40?2019?+0800 > >?@@?-7069,13?+7069,13?@@ > > > >?????//?Now?handle?special?case?where?load/store?happens?from/to?byte?array?but?element?type?is?not?byte. > >?????bool?using_byte_array?=?arr_type?!=?NULL?&&?arr_type->elem()->array_element_basic_type()?==?T_BYTE?&&?elem_bt?!=?T_BYTE; > >?- > >?-??//?It?must?be?the?case?that?if?it?is?not?special?byte?array?case,?there?is?consistency?between > >?-??//?array?and?vector?element?types. > >?-??if?(!using_byte_array?&&?arr_type?!=?NULL?&&?elem_bt?!=?arr_type->elem()->array_element_basic_type())?{ > >?+??//?Handle?loading?masks. > >?+??ciKlass*?vbox_klass?=?vector_klass->const_oop()->as_instance()->java_lang_Class_klass(); > >?+??bool?is_mask?=?vbox_klass->is_vectormask(); > >?+??//?If?there?is?no?consistency?between?array?and?vector?element?types,?it?must?be?special?byte?array?case?or?loading?masks > >?+??if?(!using_byte_array?&&?arr_type?!=?NULL?&&?elem_bt?!=?arr_type->elem()->array_element_basic_type()?&&?!is_mask)?{ > >???????return?false; > >?????} > >?- > >?????//?Since?we?are?using?byte?array,?we?need?to?double?check?that?the?byte?operations?are?supported?by?backend. > >?????if?(using_byte_array)?{ > >???????int?byte_num_elem?=?num_elem?*?type2aelembytes(elem_bt); > >?@@?-7083,8?+7083,12?@@ > >?????????return?false;?//?not?supported > >???????} > >?????} > >?- > >?-??ciKlass*?vbox_klass?=?vector_klass->const_oop()->as_instance()->java_lang_Class_klass(); > >?+??if?(is_mask)?{ > >?+????if?(!arch_supports_vector(Op_LoadVector,?num_elem,?T_BOOLEAN,?VecMaskUseLoad))?{ > >?+??????return?false;?//?not?supported > >?+????} > >?+??} > >?+ > >?????const?TypeInstPtr*?vbox_type?=?TypeInstPtr::make_exact(TypePtr::NotNull,?vbox_klass); > > > >?????if?(is_store)?{ > >?@@?-7114,9?+7118,15?@@ > >?????????const?TypeVect*?to_vect_type?=?TypeVect::make(elem_bt,?num_elem); > >?????????vload?=?gvn().transform(new?VectorReinterpretNode(vload,?vload->bottom_type()->is_vect(),?to_vect_type)); > >???????}?else?{ > >?-??????vload?=?gvn().transform(LoadVectorNode::make(0,?control(),?memory(addr),?addr,?addr_type,?num_elem,?elem_bt)); > >?-????} > >?- > >?+??????//?Special?handle?for?masks > >?+??????if?(is_mask)?{ > >?+??????????vload?=?gvn().transform(LoadVectorNode::make(0,?control(),?memory(addr),?addr,?addr_type,?num_elem,?T_BOOLEAN)); > >?+??????????const?TypeVect*?to_vect_type?=?TypeVect::make(elem_bt,?num_elem); > >?+??????????vload?=?gvn().transform(new?VectorLoadMaskNode(vload,?to_vect_type)); > >?+??????}?else?{ > >?+??????????vload?=?gvn().transform(LoadVectorNode::make(0,?control(),?memory(addr),?addr,?addr_type,?num_elem,?elem_bt)); > >?+??????} > >?+????} > >???????Node*?box?=?box_vector(vload,?vbox_type,?elem_bt,?num_elem); > >???????set_vector_result(box); > >?????} > >?---?a/src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Int128Vector.java?????Mon?Jan?07?10:54:21?2019?+0800 > >?+++?b/src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Int128Vector.java?????Tue?Jan?15?23:45:40?2019?+0800 > >?@@?-1338,7?+1338,7?@@ > >???????????????Objects.requireNonNull(bits); > >???????????????ix?=?VectorIntrinsics.checkIndex(ix,?bits.length,?LENGTH); > >???????????????return?VectorIntrinsics.load(Int128Mask.class,?int.class,?LENGTH, > >?-?????????????????????????????????????????bits,?(((long)?ix)?< >?+?????????????????????????????????????????bits,?((long)?ix)?+?Unsafe.ARRAY_BOOLEAN_BASE_OFFSET, > >????????????????????????????????????????????bits,?ix, > >????????????????????????????????????????????(c,?idx)?->?opm(n?->?c[idx?+?n])); > > > >?Regards, > >?Zhuoren > > > From maurizio.cimadamore at oracle.com Wed Jan 16 20:40:24 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 16 Jan 2019 20:40:24 +0000 Subject: hg: panama/dev: Automatic merge with vectorIntrinsics Message-ID: <201901162040.x0GKePjQ026663@aojmv0008.oracle.com> Changeset: 28d2e6ae9c3d Author: mcimadamore Date: 2019-01-16 21:40 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/28d2e6ae9c3d Automatic merge with vectorIntrinsics ! src/hotspot/cpu/x86/assembler_x86.hpp From maurizio.cimadamore at oracle.com Wed Jan 16 21:06:34 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 16 Jan 2019 21:06:34 +0000 Subject: hg: panama/dev: 149 new changesets Message-ID: <201901162106.x0GL6mnP008293@aojmv0008.oracle.com> Changeset: 716c746165b2 Author: cushon Date: 2019-01-08 17:37 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/716c746165b2 8216403: Allocate fewer EnumSets in JavacFileManager#list Reviewed-by: vromero, redestad ! src/jdk.compiler/share/classes/com/sun/tools/javac/file/JavacFileManager.java Changeset: 386df79e2011 Author: goetz Date: 2019-01-08 09:29 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/386df79e2011 8216271: Make AllocateOldGenAt an unsupported option on AIX. Reviewed-by: shade, tschatzl ! src/hotspot/share/runtime/arguments.cpp ! test/hotspot/jtreg/gc/nvdimm/TestAllocateOldGenAt.java ! test/hotspot/jtreg/gc/nvdimm/TestAllocateOldGenAtError.java ! test/hotspot/jtreg/gc/nvdimm/TestAllocateOldGenAtMultiple.java ! test/hotspot/jtreg/gc/nvdimm/TestHumongousObjectsOnNvdimm.java ! test/hotspot/jtreg/gc/nvdimm/TestOldObjectsOnNvdimm.java ! test/hotspot/jtreg/gc/nvdimm/TestYoungObjectsOnDram.java Changeset: 299fe76c25c7 Author: jlahoda Date: 2019-01-08 16:31 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/299fe76c25c7 8215438: jshell tool: Ctrl-D causes EOF Summary: Properly handling EndOfFileException so that jshell can be closed with Ctrl-D. Reviewed-by: rfield ! src/jdk.jshell/share/classes/jdk/internal/jshell/tool/ConsoleIOContext.java ! test/langtools/jdk/jshell/ReplToolTesting.java ! test/langtools/jdk/jshell/ToolBasicTest.java Changeset: df97e2c0f9ae Author: naoto Date: 2019-01-08 10:05 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/df97e2c0f9ae 8216176: Clarify the singleton description in j.t.c.JapaneseEra class Reviewed-by: rriggs ! src/java.base/share/classes/java/time/chrono/JapaneseEra.java Changeset: 0f9a83a93e52 Author: fyang Date: 2019-01-05 10:48 +0800 URL: http://hg.openjdk.java.net/panama/dev/rev/0f9a83a93e52 8215951: AArch64: jtreg test vmTestbase/nsk/jvmti/PopFrame/popframe005 segfaults Reviewed-by: aph Contributed-by: nick.gasson at arm.com ! src/hotspot/cpu/aarch64/templateInterpreterGenerator_aarch64.cpp Changeset: 9db2dda367c6 Author: darcy Date: 2019-01-08 16:26 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/9db2dda367c6 8216322: Missing since information in deprecation of constructor visitors Reviewed-by: vromero ! src/java.compiler/share/classes/javax/lang/model/util/AbstractAnnotationValueVisitor7.java ! src/java.compiler/share/classes/javax/lang/model/util/AbstractElementVisitor7.java ! src/java.compiler/share/classes/javax/lang/model/util/AbstractTypeVisitor7.java ! src/java.compiler/share/classes/javax/lang/model/util/ElementKindVisitor7.java ! src/java.compiler/share/classes/javax/lang/model/util/ElementScanner7.java ! src/java.compiler/share/classes/javax/lang/model/util/SimpleAnnotationValueVisitor7.java ! src/java.compiler/share/classes/javax/lang/model/util/SimpleElementVisitor7.java ! src/java.compiler/share/classes/javax/lang/model/util/SimpleTypeVisitor7.java ! src/java.compiler/share/classes/javax/lang/model/util/TypeKindVisitor7.java Changeset: ef41d615b3f0 Author: ljiang Date: 2019-01-09 00:25 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/ef41d615b3f0 8215994: JDK 12 l10n resource file update - msg drop 10 Reviewed-by: billyh, ssadetsky, naoto ! src/java.base/share/classes/sun/security/tools/keytool/Resources_ja.java ! src/java.base/share/classes/sun/security/tools/keytool/Resources_zh_CN.java ! src/java.base/share/classes/sun/security/util/AuthResources_ja.java ! src/jdk.compiler/share/classes/com/sun/tools/doclint/resources/doclint_ja.properties ! src/jdk.compiler/share/classes/com/sun/tools/doclint/resources/doclint_zh_CN.properties ! src/jdk.compiler/share/classes/com/sun/tools/javac/resources/compiler_ja.properties ! src/jdk.compiler/share/classes/com/sun/tools/javac/resources/compiler_zh_CN.properties ! src/jdk.compiler/share/classes/com/sun/tools/javac/resources/javac_ja.properties ! src/jdk.compiler/share/classes/com/sun/tools/javac/resources/javac_zh_CN.properties ! src/jdk.compiler/share/classes/com/sun/tools/javac/resources/launcher_ja.properties ! src/jdk.compiler/share/classes/com/sun/tools/javac/resources/launcher_zh_CN.properties ! src/jdk.jartool/share/classes/sun/security/tools/jarsigner/Resources_ja.java ! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/resources/standard_ja.properties ! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/resources/standard_zh_CN.properties ! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/toolkit/resources/doclets_ja.properties ! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/toolkit/resources/doclets_zh_CN.properties ! src/jdk.jdeps/share/classes/com/sun/tools/jdeps/resources/jdeps_ja.properties ! src/jdk.jdeps/share/classes/com/sun/tools/jdeps/resources/jdeps_zh_CN.properties ! src/jdk.jshell/share/classes/jdk/internal/jshell/tool/resources/l10n_ja.properties ! src/jdk.jshell/share/classes/jdk/internal/jshell/tool/resources/l10n_zh_CN.properties Changeset: c14b7b6a9b2f Author: roland Date: 2018-12-07 17:56 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/c14b7b6a9b2f 8214862: assert(proj != __null) at compile.cpp:3251 Reviewed-by: kvn, thartmann ! src/hotspot/share/opto/callnode.cpp ! src/hotspot/share/opto/callnode.hpp ! src/hotspot/share/opto/compile.cpp ! src/hotspot/share/opto/compile.hpp ! src/hotspot/share/opto/node.cpp ! src/hotspot/share/opto/phaseX.cpp + test/hotspot/jtreg/compiler/inlining/StringConcatInfiniteLoop.java Changeset: 3f4f81fbc989 Author: pliden Date: 2019-01-09 10:18 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/3f4f81fbc989 8216385: ZGC: Fix building without C2 Reviewed-by: shade, eosterlund ! src/hotspot/os_cpu/linux_x86/gc/z/zArguments_linux_x86.cpp Changeset: 8be214962266 Author: adinn Date: 2018-12-19 11:45 +0000 URL: http://hg.openjdk.java.net/panama/dev/rev/8be214962266 8209414: AArch64: method handle invocation does not respect JVMTI interp_only mode Reviewed-by: adinn Contributed-by: nick.gasson at arm.com ! src/hotspot/cpu/aarch64/methodHandles_aarch64.cpp Changeset: df6cbf676c70 Author: pliden Date: 2019-01-09 13:31 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/df6cbf676c70 8215708: ZGC: Add missing LoadBarrierNode::size_of() Reviewed-by: eosterlund, neliasso ! src/hotspot/share/gc/z/c2/zBarrierSetC2.cpp ! src/hotspot/share/gc/z/c2/zBarrierSetC2.hpp Changeset: bae765528fcc Author: vromero Date: 2019-01-09 08:07 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/bae765528fcc 8215510: j.l.c.ClassDesc is accepting descriptors not allowed by the spec Reviewed-by: goetz ! src/java.base/share/classes/java/lang/constant/ClassDesc.java ! src/java.base/share/classes/java/lang/constant/ConstantUtils.java ! src/java.base/share/classes/java/lang/constant/DirectMethodHandleDescImpl.java ! src/java.base/share/classes/java/lang/constant/DynamicCallSiteDesc.java ! src/java.base/share/classes/java/lang/constant/DynamicConstantDesc.java ! src/java.base/share/classes/java/lang/constant/ReferenceClassDescImpl.java ! test/jdk/java/lang/constant/ClassDescTest.java ! test/jdk/java/lang/constant/NameValidationTest.java ! test/jdk/java/lang/constant/boottest/java.base/java/lang/constant/ConstantUtilsTest.java Changeset: b11483a74e5d Author: coleenp Date: 2019-01-09 07:52 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/b11483a74e5d 8215575: C2 crash: assert(get_instanceKlass()->is_loaded()) failed: must be at least loaded Summary: Set InstanceKlass::loaded before adding classes to the subklass list, which can be read concurrently by the compiler. Reviewed-by: dholmes, eosterlund ! src/hotspot/share/classfile/systemDictionary.cpp ! src/hotspot/share/oops/instanceKlass.cpp ! src/hotspot/share/oops/instanceKlass.hpp Changeset: 4ff3f9d83fe5 Author: neliasso Date: 2019-01-09 15:36 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/4ff3f9d83fe5 8215755: ZGC: split_barrier_thru_phi: check number of inputs of phi Reviewed-by: pliden, thartmann ! src/hotspot/share/gc/z/c2/zBarrierSetC2.cpp Changeset: f15d443f9731 Author: jlaskey Date: 2019-01-09 11:13 -0400 URL: http://hg.openjdk.java.net/panama/dev/rev/f15d443f9731 8215681: Remove compiler support for Raw String Literals from JDK 12 Reviewed-by: mcimadamore, jlahoda, sundar ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Preview.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Source.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/parser/JavaTokenizer.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/parser/UnicodeReader.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/resources/compiler.properties ! test/langtools/jdk/jshell/ToolSimpleTest.java - test/langtools/tools/javac/RawStringLiteralLang.java - test/langtools/tools/javac/RawStringLiteralLangAPI.java - test/langtools/tools/javac/diags/examples/RawStringLiteral.java Changeset: e7738fd1c974 Author: jjg Date: 2019-01-09 08:49 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/e7738fd1c974 8215308: pandoc-html-manpage-filter.js does not work for [un]pack200 Reviewed-by: erikj ! make/scripts/pandoc-html-manpage-filter.js Changeset: 76a4b08fdf59 Author: mr Date: 2019-01-09 08:52 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/76a4b08fdf59 8210669: Some launcher tests assume a pre-JDK 9 run-time image layout Reviewed-by: mchung ! test/jdk/tools/launcher/ExecutionEnvironment.java ! test/jdk/tools/launcher/Test7029048.java ! test/jdk/tools/launcher/TestHelper.java Changeset: b5f085197234 Author: jlaskey Date: 2019-01-09 15:23 -0400 URL: http://hg.openjdk.java.net/panama/dev/rev/b5f085197234 8215112: String::transform spec clarification Reviewed-by: smarks ! src/java.base/share/classes/java/lang/String.java Changeset: ee1f64096d7c Author: jlaskey Date: 2019-01-09 16:41 -0400 URL: http://hg.openjdk.java.net/panama/dev/rev/ee1f64096d7c 8215489: Remove String::align Reviewed-by: vromero, sundar ! src/java.base/share/classes/java/lang/String.java - test/jdk/java/lang/String/AlignIndent.java + test/jdk/java/lang/String/Indent.java Changeset: 32c6cc430526 Author: jwilhelm Date: 2019-01-09 22:59 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/32c6cc430526 Merge ! src/hotspot/share/classfile/systemDictionary.cpp ! src/hotspot/share/oops/instanceKlass.cpp ! src/hotspot/share/oops/instanceKlass.hpp ! src/hotspot/share/runtime/arguments.cpp ! src/jdk.compiler/share/classes/com/sun/tools/javac/code/Source.java - test/jdk/java/lang/String/AlignIndent.java + test/jdk/java/lang/String/Indent.java - test/langtools/tools/javac/RawStringLiteralLang.java - test/langtools/tools/javac/RawStringLiteralLangAPI.java - test/langtools/tools/javac/diags/examples/RawStringLiteral.java Changeset: 642346a11059 Author: iignatyev Date: 2019-01-09 17:12 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/642346a11059 8216441: problem list org.graalvm.compiler.hotspot.test.ExplicitExceptionTest Reviewed-by: dholmes ! test/hotspot/jtreg/ProblemList-graal.txt Changeset: e3641318f540 Author: jwilhelm Date: 2019-01-10 04:52 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/e3641318f540 Added tag jdk-13+3 for changeset 642346a11059 ! .hgtags Changeset: e1cc790f0c06 Author: roland Date: 2019-01-10 13:54 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/e1cc790f0c06 8216482: Shenandoah: typo in ShenandoahBarrierSetC2::clone_barrier_at_expansion() causes failed compilations Reviewed-by: thartmann, shade, rkennke ! src/hotspot/share/gc/shenandoah/c2/shenandoahBarrierSetC2.cpp Changeset: dece421843f3 Author: hseigel Date: 2019-01-10 10:15 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/dece421843f3 8207964: [TESTBUG] Change stressTime to default to 30 for nsk tests Summary: Change the default from 60 seconds to 30 seconds. Reviewed-by: coleenp, dholmes ! test/hotspot/jtreg/vmTestbase/nsk/share/test/StressOptions.java ! test/hotspot/jtreg/vmTestbase/nsk/share/test/Stresser.java ! test/hotspot/jtreg/vmTestbase/vm/share/options/test/SimpleExampleWithOptionsAnnotation.java Changeset: 443abf0dc2ed Author: robm Date: 2019-01-10 07:54 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/443abf0dc2ed 8214440: ldap over a TLS connection negotiate failed with "javax.net.ssl.SSLPeerUnverifiedException: hostname of the server '' does not match the hostname in the server's certificate" Reviewed-by: vtewari, xuelei ! src/java.naming/share/classes/com/sun/jndi/ldap/ext/StartTlsResponseImpl.java Changeset: 38716f9d2239 Author: redestad Date: 2019-01-10 16:50 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/38716f9d2239 8216428: Remove IgnoreLibthreadGPFault Reviewed-by: dholmes, eosterlund ! src/hotspot/os_cpu/solaris_x86/os_solaris_x86.cpp ! src/hotspot/share/runtime/globals.hpp Changeset: c024fcb88ede Author: gziemski Date: 2019-01-10 11:16 -0600 URL: http://hg.openjdk.java.net/panama/dev/rev/c024fcb88ede 8215155: Remove get_insert() from concurrent hashtable and gtests Summary: Replaced get_insert() with get()/insert() in gtest, removed get_insert() API from cht implementation. Reviewed-by: coleenp, rehn ! src/hotspot/share/utilities/concurrentHashTable.hpp ! src/hotspot/share/utilities/concurrentHashTable.inline.hpp ! test/hotspot/gtest/utilities/test_concurrentHashtable.cpp Changeset: f6ab4cc4c70e Author: erikj Date: 2019-01-10 09:11 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/f6ab4cc4c70e 8216278: Fix devkit and basic Jib support on WSL Reviewed-by: tbell ! make/autoconf/basics.m4 ! make/autoconf/toolchain_windows.m4 ! make/conf/jib-profiles.js Changeset: 7327a62f3c04 Author: erikj Date: 2019-01-10 10:28 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/7327a62f3c04 8216489: Issues with ModulePackages attribute generation on incremental build Reviewed-by: redestad, alanb, tbell ! make/ExplodedImageOptimize.gmk Changeset: e81edc1f6f7e Author: darcy Date: 2019-01-10 10:34 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/e81edc1f6f7e 8216404: Elements.getPackageOf should handle modules Reviewed-by: jlahoda ! src/java.compiler/share/classes/javax/lang/model/util/Elements.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/model/JavacElements.java ! test/langtools/tools/javac/processing/model/util/elements/TestGetPackageOf.java Changeset: 8bea4144b21c Author: lancea Date: 2019-01-10 14:32 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/8bea4144b21c 8216362: Better error message handling when there is an invalid Manifest Reviewed-by: lancea, rriggs, mullan Contributed-by: Philipp Kunz ! src/java.base/share/classes/java/util/jar/Manifest.java + test/jdk/java/util/jar/Manifest/IncludeInExceptionsTest.java Changeset: 9807daeb47c4 Author: coleenp Date: 2019-01-10 15:13 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/9807daeb47c4 8216167: Update include guards to reflect correct directories Summary: Use script and some manual fixup to fix directores names in include guards. Reviewed-by: lfoltan, eosterlund, kbarrett ! src/hotspot/cpu/aarch64/assembler_aarch64.hpp ! src/hotspot/cpu/aarch64/assembler_aarch64.inline.hpp ! src/hotspot/cpu/aarch64/bytecodes_aarch64.hpp ! src/hotspot/cpu/aarch64/bytes_aarch64.hpp ! src/hotspot/cpu/aarch64/c1_Defs_aarch64.hpp ! src/hotspot/cpu/aarch64/c1_FpuStackSim_aarch64.hpp ! src/hotspot/cpu/aarch64/c1_FrameMap_aarch64.hpp ! src/hotspot/cpu/aarch64/c1_LIRAssembler_aarch64.hpp ! src/hotspot/cpu/aarch64/c1_LinearScan_aarch64.hpp ! src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.hpp ! src/hotspot/cpu/aarch64/c1_globals_aarch64.hpp ! src/hotspot/cpu/aarch64/c2_globals_aarch64.hpp ! src/hotspot/cpu/aarch64/codeBuffer_aarch64.hpp ! src/hotspot/cpu/aarch64/copy_aarch64.hpp ! src/hotspot/cpu/aarch64/depChecker_aarch64.hpp ! src/hotspot/cpu/aarch64/disassembler_aarch64.hpp ! src/hotspot/cpu/aarch64/frame_aarch64.hpp ! src/hotspot/cpu/aarch64/frame_aarch64.inline.hpp ! src/hotspot/cpu/aarch64/gc/shared/cardTableBarrierSetAssembler_aarch64.hpp ! src/hotspot/cpu/aarch64/globalDefinitions_aarch64.hpp ! src/hotspot/cpu/aarch64/globals_aarch64.hpp ! src/hotspot/cpu/aarch64/icache_aarch64.hpp ! src/hotspot/cpu/aarch64/interp_masm_aarch64.hpp ! src/hotspot/cpu/aarch64/interpreterRT_aarch64.hpp ! src/hotspot/cpu/aarch64/javaFrameAnchor_aarch64.hpp ! src/hotspot/cpu/aarch64/jniTypes_aarch64.hpp ! src/hotspot/cpu/aarch64/macroAssembler_aarch64.hpp ! src/hotspot/cpu/aarch64/macroAssembler_aarch64.inline.hpp ! src/hotspot/cpu/aarch64/nativeInst_aarch64.hpp ! src/hotspot/cpu/aarch64/registerMap_aarch64.hpp ! src/hotspot/cpu/aarch64/register_aarch64.hpp ! src/hotspot/cpu/aarch64/relocInfo_aarch64.hpp ! src/hotspot/cpu/aarch64/stubRoutines_aarch64.hpp ! src/hotspot/cpu/aarch64/templateTable_aarch64.hpp ! src/hotspot/cpu/aarch64/vmStructs_aarch64.hpp ! src/hotspot/cpu/aarch64/vm_version_aarch64.hpp ! src/hotspot/cpu/aarch64/vm_version_ext_aarch64.hpp ! src/hotspot/cpu/aarch64/vmreg_aarch64.hpp ! src/hotspot/cpu/aarch64/vmreg_aarch64.inline.hpp ! src/hotspot/cpu/arm/assembler_arm.hpp ! src/hotspot/cpu/arm/assembler_arm.inline.hpp ! src/hotspot/cpu/arm/assembler_arm_32.hpp ! src/hotspot/cpu/arm/bytes_arm.hpp ! src/hotspot/cpu/arm/c1_Defs_arm.hpp ! src/hotspot/cpu/arm/c1_FpuStackSim_arm.hpp ! src/hotspot/cpu/arm/c1_FrameMap_arm.hpp ! src/hotspot/cpu/arm/c1_LIRAssembler_arm.hpp ! src/hotspot/cpu/arm/c1_LinearScan_arm.hpp ! src/hotspot/cpu/arm/c1_MacroAssembler_arm.hpp ! src/hotspot/cpu/arm/c1_globals_arm.hpp ! src/hotspot/cpu/arm/c2_globals_arm.hpp ! src/hotspot/cpu/arm/codeBuffer_arm.hpp ! src/hotspot/cpu/arm/copy_arm.hpp ! src/hotspot/cpu/arm/depChecker_arm.hpp ! src/hotspot/cpu/arm/disassembler_arm.hpp ! src/hotspot/cpu/arm/frame_arm.hpp ! src/hotspot/cpu/arm/frame_arm.inline.hpp ! src/hotspot/cpu/arm/gc/shared/cardTableBarrierSetAssembler_arm.hpp ! src/hotspot/cpu/arm/globalDefinitions_arm.hpp ! src/hotspot/cpu/arm/globals_arm.hpp ! src/hotspot/cpu/arm/icache_arm.hpp ! src/hotspot/cpu/arm/interp_masm_arm.hpp ! src/hotspot/cpu/arm/interpreterRT_arm.hpp ! src/hotspot/cpu/arm/javaFrameAnchor_arm.hpp ! src/hotspot/cpu/arm/jniTypes_arm.hpp ! src/hotspot/cpu/arm/macroAssembler_arm.hpp ! src/hotspot/cpu/arm/macroAssembler_arm.inline.hpp ! src/hotspot/cpu/arm/nativeInst_arm.hpp ! src/hotspot/cpu/arm/nativeInst_arm_32.hpp ! src/hotspot/cpu/arm/registerMap_arm.hpp ! src/hotspot/cpu/arm/register_arm.hpp ! src/hotspot/cpu/arm/relocInfo_arm.hpp ! src/hotspot/cpu/arm/stubRoutines_arm.hpp ! src/hotspot/cpu/arm/templateTable_arm.hpp ! src/hotspot/cpu/arm/vmStructs_arm.hpp ! src/hotspot/cpu/arm/vm_version_arm.hpp ! src/hotspot/cpu/arm/vm_version_ext_arm.hpp ! src/hotspot/cpu/arm/vmreg_arm.hpp ! src/hotspot/cpu/arm/vmreg_arm.inline.hpp ! src/hotspot/cpu/ppc/assembler_ppc.hpp ! src/hotspot/cpu/ppc/assembler_ppc.inline.hpp ! src/hotspot/cpu/ppc/bytes_ppc.hpp ! src/hotspot/cpu/ppc/c1_Defs_ppc.hpp ! src/hotspot/cpu/ppc/c1_FpuStackSim_ppc.hpp ! src/hotspot/cpu/ppc/c1_FrameMap_ppc.hpp ! src/hotspot/cpu/ppc/c1_LIRAssembler_ppc.hpp ! src/hotspot/cpu/ppc/c1_LinearScan_ppc.hpp ! src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.hpp ! src/hotspot/cpu/ppc/c1_globals_ppc.hpp ! src/hotspot/cpu/ppc/c2_globals_ppc.hpp ! src/hotspot/cpu/ppc/codeBuffer_ppc.hpp ! src/hotspot/cpu/ppc/copy_ppc.hpp ! src/hotspot/cpu/ppc/depChecker_ppc.hpp ! src/hotspot/cpu/ppc/disassembler_ppc.hpp ! src/hotspot/cpu/ppc/frame_ppc.hpp ! src/hotspot/cpu/ppc/frame_ppc.inline.hpp ! src/hotspot/cpu/ppc/globalDefinitions_ppc.hpp ! src/hotspot/cpu/ppc/globals_ppc.hpp ! src/hotspot/cpu/ppc/icache_ppc.hpp ! src/hotspot/cpu/ppc/interp_masm_ppc.hpp ! src/hotspot/cpu/ppc/interpreterRT_ppc.hpp ! src/hotspot/cpu/ppc/javaFrameAnchor_ppc.hpp ! src/hotspot/cpu/ppc/jniTypes_ppc.hpp ! src/hotspot/cpu/ppc/macroAssembler_ppc.hpp ! src/hotspot/cpu/ppc/macroAssembler_ppc.inline.hpp ! src/hotspot/cpu/ppc/nativeInst_ppc.hpp ! src/hotspot/cpu/ppc/registerMap_ppc.hpp ! src/hotspot/cpu/ppc/register_ppc.hpp ! src/hotspot/cpu/ppc/relocInfo_ppc.hpp ! src/hotspot/cpu/ppc/stubRoutines_ppc.hpp ! src/hotspot/cpu/ppc/templateTable_ppc.hpp ! src/hotspot/cpu/ppc/vmStructs_ppc.hpp ! src/hotspot/cpu/ppc/vm_version_ext_ppc.hpp ! src/hotspot/cpu/ppc/vm_version_ppc.hpp ! src/hotspot/cpu/ppc/vmreg_ppc.hpp ! src/hotspot/cpu/ppc/vmreg_ppc.inline.hpp ! src/hotspot/cpu/s390/assembler_s390.hpp ! src/hotspot/cpu/s390/assembler_s390.inline.hpp ! src/hotspot/cpu/s390/bytes_s390.hpp ! src/hotspot/cpu/s390/c1_Defs_s390.hpp ! src/hotspot/cpu/s390/c1_FpuStackSim_s390.hpp ! src/hotspot/cpu/s390/c1_FrameMap_s390.hpp ! src/hotspot/cpu/s390/c1_LIRAssembler_s390.hpp ! src/hotspot/cpu/s390/c1_LinearScan_s390.hpp ! src/hotspot/cpu/s390/c1_MacroAssembler_s390.hpp ! src/hotspot/cpu/s390/c1_globals_s390.hpp ! src/hotspot/cpu/s390/c2_globals_s390.hpp ! src/hotspot/cpu/s390/codeBuffer_s390.hpp ! src/hotspot/cpu/s390/copy_s390.hpp ! src/hotspot/cpu/s390/depChecker_s390.hpp ! src/hotspot/cpu/s390/disassembler_s390.hpp ! src/hotspot/cpu/s390/frame_s390.hpp ! src/hotspot/cpu/s390/frame_s390.inline.hpp ! src/hotspot/cpu/s390/globalDefinitions_s390.hpp ! src/hotspot/cpu/s390/globals_s390.hpp ! src/hotspot/cpu/s390/icache_s390.hpp ! src/hotspot/cpu/s390/interp_masm_s390.hpp ! src/hotspot/cpu/s390/interpreterRT_s390.hpp ! src/hotspot/cpu/s390/javaFrameAnchor_s390.hpp ! src/hotspot/cpu/s390/jniTypes_s390.hpp ! src/hotspot/cpu/s390/macroAssembler_s390.hpp ! src/hotspot/cpu/s390/macroAssembler_s390.inline.hpp ! src/hotspot/cpu/s390/nativeInst_s390.hpp ! src/hotspot/cpu/s390/registerMap_s390.hpp ! src/hotspot/cpu/s390/registerSaver_s390.hpp ! src/hotspot/cpu/s390/register_s390.hpp ! src/hotspot/cpu/s390/relocInfo_s390.hpp ! src/hotspot/cpu/s390/stubRoutines_s390.hpp ! src/hotspot/cpu/s390/templateTable_s390.hpp ! src/hotspot/cpu/s390/vmStructs_s390.hpp ! src/hotspot/cpu/s390/vm_version_ext_s390.hpp ! src/hotspot/cpu/s390/vm_version_s390.hpp ! src/hotspot/cpu/s390/vmreg_s390.hpp ! src/hotspot/cpu/s390/vmreg_s390.inline.hpp ! src/hotspot/cpu/sparc/assembler_sparc.hpp ! src/hotspot/cpu/sparc/assembler_sparc.inline.hpp ! src/hotspot/cpu/sparc/bytes_sparc.hpp ! src/hotspot/cpu/sparc/c1_Defs_sparc.hpp ! src/hotspot/cpu/sparc/c1_FpuStackSim_sparc.hpp ! src/hotspot/cpu/sparc/c1_FrameMap_sparc.hpp ! src/hotspot/cpu/sparc/c1_LIRAssembler_sparc.hpp ! src/hotspot/cpu/sparc/c1_LinearScan_sparc.hpp ! src/hotspot/cpu/sparc/c1_MacroAssembler_sparc.hpp ! src/hotspot/cpu/sparc/c1_globals_sparc.hpp ! src/hotspot/cpu/sparc/c2_globals_sparc.hpp ! src/hotspot/cpu/sparc/codeBuffer_sparc.hpp ! src/hotspot/cpu/sparc/copy_sparc.hpp ! src/hotspot/cpu/sparc/depChecker_sparc.hpp ! src/hotspot/cpu/sparc/disassembler_sparc.hpp ! src/hotspot/cpu/sparc/frame_sparc.hpp ! src/hotspot/cpu/sparc/frame_sparc.inline.hpp ! src/hotspot/cpu/sparc/globalDefinitions_sparc.hpp ! src/hotspot/cpu/sparc/globals_sparc.hpp ! src/hotspot/cpu/sparc/icache_sparc.hpp ! src/hotspot/cpu/sparc/interp_masm_sparc.hpp ! src/hotspot/cpu/sparc/interpreterRT_sparc.hpp ! src/hotspot/cpu/sparc/javaFrameAnchor_sparc.hpp ! src/hotspot/cpu/sparc/jniTypes_sparc.hpp ! src/hotspot/cpu/sparc/macroAssembler_sparc.hpp ! src/hotspot/cpu/sparc/macroAssembler_sparc.inline.hpp ! src/hotspot/cpu/sparc/nativeInst_sparc.hpp ! src/hotspot/cpu/sparc/registerMap_sparc.hpp ! src/hotspot/cpu/sparc/register_sparc.hpp ! src/hotspot/cpu/sparc/relocInfo_sparc.hpp ! src/hotspot/cpu/sparc/stubRoutines_sparc.hpp ! src/hotspot/cpu/sparc/templateTable_sparc.hpp ! src/hotspot/cpu/sparc/vmStructs_sparc.hpp ! src/hotspot/cpu/sparc/vm_version_ext_sparc.hpp ! src/hotspot/cpu/sparc/vm_version_sparc.hpp ! src/hotspot/cpu/sparc/vmreg_sparc.hpp ! src/hotspot/cpu/sparc/vmreg_sparc.inline.hpp ! src/hotspot/cpu/x86/assembler_x86.hpp ! src/hotspot/cpu/x86/assembler_x86.inline.hpp ! src/hotspot/cpu/x86/bytes_x86.hpp ! src/hotspot/cpu/x86/c1_Defs_x86.hpp ! src/hotspot/cpu/x86/c1_FpuStackSim_x86.hpp ! src/hotspot/cpu/x86/c1_FrameMap_x86.hpp ! src/hotspot/cpu/x86/c1_LIRAssembler_x86.hpp ! src/hotspot/cpu/x86/c1_LinearScan_x86.hpp ! src/hotspot/cpu/x86/c1_MacroAssembler_x86.hpp ! src/hotspot/cpu/x86/c1_globals_x86.hpp ! src/hotspot/cpu/x86/c2_globals_x86.hpp ! src/hotspot/cpu/x86/codeBuffer_x86.hpp ! src/hotspot/cpu/x86/copy_x86.hpp ! src/hotspot/cpu/x86/depChecker_x86.hpp ! src/hotspot/cpu/x86/disassembler_x86.hpp ! src/hotspot/cpu/x86/frame_x86.hpp ! src/hotspot/cpu/x86/frame_x86.inline.hpp ! src/hotspot/cpu/x86/globalDefinitions_x86.hpp ! src/hotspot/cpu/x86/globals_x86.hpp ! src/hotspot/cpu/x86/icache_x86.hpp ! src/hotspot/cpu/x86/interp_masm_x86.hpp ! src/hotspot/cpu/x86/interpreterRT_x86.hpp ! src/hotspot/cpu/x86/javaFrameAnchor_x86.hpp ! src/hotspot/cpu/x86/jniTypes_x86.hpp ! src/hotspot/cpu/x86/macroAssembler_x86.hpp ! src/hotspot/cpu/x86/macroAssembler_x86.inline.hpp ! src/hotspot/cpu/x86/nativeInst_x86.hpp ! src/hotspot/cpu/x86/rdtsc_x86.hpp ! src/hotspot/cpu/x86/registerMap_x86.hpp ! src/hotspot/cpu/x86/register_x86.hpp ! src/hotspot/cpu/x86/relocInfo_x86.hpp ! src/hotspot/cpu/x86/stubRoutines_x86.hpp ! src/hotspot/cpu/x86/templateTable_x86.hpp ! src/hotspot/cpu/x86/vmStructs_x86.hpp ! src/hotspot/cpu/x86/vm_version_ext_x86.hpp ! src/hotspot/cpu/x86/vm_version_x86.hpp ! src/hotspot/cpu/x86/vmreg_x86.hpp ! src/hotspot/cpu/x86/vmreg_x86.inline.hpp ! src/hotspot/cpu/zero/assembler_zero.hpp ! src/hotspot/cpu/zero/assembler_zero.inline.hpp ! src/hotspot/cpu/zero/bytecodeInterpreter_zero.hpp ! src/hotspot/cpu/zero/bytecodeInterpreter_zero.inline.hpp ! src/hotspot/cpu/zero/bytes_zero.hpp ! src/hotspot/cpu/zero/codeBuffer_zero.hpp ! src/hotspot/cpu/zero/copy_zero.hpp ! src/hotspot/cpu/zero/cppInterpreter_zero.hpp ! src/hotspot/cpu/zero/depChecker_zero.hpp ! src/hotspot/cpu/zero/disassembler_zero.hpp ! src/hotspot/cpu/zero/entryFrame_zero.hpp ! src/hotspot/cpu/zero/entry_zero.hpp ! src/hotspot/cpu/zero/fakeStubFrame_zero.hpp ! src/hotspot/cpu/zero/frame_zero.hpp ! src/hotspot/cpu/zero/frame_zero.inline.hpp ! src/hotspot/cpu/zero/globalDefinitions_zero.hpp ! src/hotspot/cpu/zero/globals_zero.hpp ! src/hotspot/cpu/zero/icache_zero.hpp ! src/hotspot/cpu/zero/interp_masm_zero.hpp ! src/hotspot/cpu/zero/interpreterFrame_zero.hpp ! src/hotspot/cpu/zero/interpreterRT_zero.hpp ! src/hotspot/cpu/zero/javaFrameAnchor_zero.hpp ! src/hotspot/cpu/zero/jniTypes_zero.hpp ! src/hotspot/cpu/zero/macroAssembler_zero.hpp ! src/hotspot/cpu/zero/macroAssembler_zero.inline.hpp ! src/hotspot/cpu/zero/nativeInst_zero.hpp ! src/hotspot/cpu/zero/registerMap_zero.hpp ! src/hotspot/cpu/zero/register_zero.hpp ! src/hotspot/cpu/zero/relocInfo_zero.hpp ! src/hotspot/cpu/zero/stack_zero.hpp ! src/hotspot/cpu/zero/stack_zero.inline.hpp ! src/hotspot/cpu/zero/stubRoutines_zero.hpp ! src/hotspot/cpu/zero/vmStructs_zero.hpp ! src/hotspot/cpu/zero/vm_version_ext_zero.hpp ! src/hotspot/cpu/zero/vm_version_zero.hpp ! src/hotspot/cpu/zero/vmreg_zero.hpp ! src/hotspot/cpu/zero/vmreg_zero.inline.hpp ! src/hotspot/os/aix/c1_globals_aix.hpp ! src/hotspot/os/aix/c2_globals_aix.hpp ! src/hotspot/os/aix/globals_aix.hpp ! src/hotspot/os/aix/libo4.hpp ! src/hotspot/os/aix/libodm_aix.hpp ! src/hotspot/os/aix/libperfstat_aix.hpp ! src/hotspot/os/aix/loadlib_aix.hpp ! src/hotspot/os/aix/misc_aix.hpp ! src/hotspot/os/aix/osThread_aix.hpp ! src/hotspot/os/aix/os_aix.hpp ! src/hotspot/os/aix/os_aix.inline.hpp ! src/hotspot/os/aix/os_share_aix.hpp ! src/hotspot/os/aix/porting_aix.hpp ! src/hotspot/os/aix/vmStructs_aix.hpp ! src/hotspot/os/bsd/c1_globals_bsd.hpp ! src/hotspot/os/bsd/c2_globals_bsd.hpp ! src/hotspot/os/bsd/decoder_machO.hpp ! src/hotspot/os/bsd/globals_bsd.hpp ! src/hotspot/os/bsd/osThread_bsd.hpp ! src/hotspot/os/bsd/os_bsd.hpp ! src/hotspot/os/bsd/os_bsd.inline.hpp ! src/hotspot/os/bsd/os_share_bsd.hpp ! src/hotspot/os/bsd/semaphore_bsd.hpp ! src/hotspot/os/bsd/vmStructs_bsd.hpp ! src/hotspot/os/linux/c1_globals_linux.hpp ! src/hotspot/os/linux/c2_globals_linux.hpp ! src/hotspot/os/linux/globals_linux.hpp ! src/hotspot/os/linux/osContainer_linux.hpp ! src/hotspot/os/linux/osThread_linux.hpp ! src/hotspot/os/linux/os_linux.hpp ! src/hotspot/os/linux/os_linux.inline.hpp ! src/hotspot/os/linux/os_share_linux.hpp ! src/hotspot/os/linux/vmStructs_linux.hpp ! src/hotspot/os/posix/os_posix.hpp ! src/hotspot/os/posix/semaphore_posix.hpp ! src/hotspot/os/solaris/c1_globals_solaris.hpp ! src/hotspot/os/solaris/c2_globals_solaris.hpp ! src/hotspot/os/solaris/globals_solaris.hpp ! src/hotspot/os/solaris/osThread_solaris.hpp ! src/hotspot/os/solaris/os_share_solaris.hpp ! src/hotspot/os/solaris/os_solaris.hpp ! src/hotspot/os/solaris/os_solaris.inline.hpp ! src/hotspot/os/solaris/vmStructs_solaris.hpp ! src/hotspot/os/windows/c1_globals_windows.hpp ! src/hotspot/os/windows/c2_globals_windows.hpp ! src/hotspot/os/windows/globals_windows.hpp ! src/hotspot/os/windows/iphlp_interface.hpp ! src/hotspot/os/windows/osThread_windows.hpp ! src/hotspot/os/windows/os_share_windows.hpp ! src/hotspot/os/windows/os_windows.hpp ! src/hotspot/os/windows/os_windows.inline.hpp ! src/hotspot/os/windows/pdh_interface.hpp ! src/hotspot/os/windows/semaphore_windows.hpp ! src/hotspot/os/windows/symbolengine.hpp ! src/hotspot/os/windows/vmStructs_windows.hpp ! src/hotspot/os/windows/windbghelp.hpp ! src/hotspot/os_cpu/aix_ppc/atomic_aix_ppc.hpp ! src/hotspot/os_cpu/aix_ppc/bytes_aix_ppc.inline.hpp ! src/hotspot/os_cpu/aix_ppc/globals_aix_ppc.hpp ! src/hotspot/os_cpu/aix_ppc/orderAccess_aix_ppc.hpp ! src/hotspot/os_cpu/aix_ppc/os_aix_ppc.hpp ! src/hotspot/os_cpu/aix_ppc/prefetch_aix_ppc.inline.hpp ! src/hotspot/os_cpu/aix_ppc/thread_aix_ppc.hpp ! src/hotspot/os_cpu/aix_ppc/vmStructs_aix_ppc.hpp ! src/hotspot/os_cpu/bsd_x86/atomic_bsd_x86.hpp ! src/hotspot/os_cpu/bsd_x86/bytes_bsd_x86.inline.hpp ! src/hotspot/os_cpu/bsd_x86/copy_bsd_x86.inline.hpp ! src/hotspot/os_cpu/bsd_x86/globals_bsd_x86.hpp ! src/hotspot/os_cpu/bsd_x86/orderAccess_bsd_x86.hpp ! src/hotspot/os_cpu/bsd_x86/os_bsd_x86.hpp ! src/hotspot/os_cpu/bsd_x86/os_bsd_x86.inline.hpp ! src/hotspot/os_cpu/bsd_x86/prefetch_bsd_x86.inline.hpp ! src/hotspot/os_cpu/bsd_x86/thread_bsd_x86.hpp ! src/hotspot/os_cpu/bsd_x86/vmStructs_bsd_x86.hpp ! src/hotspot/os_cpu/bsd_zero/atomic_bsd_zero.hpp ! src/hotspot/os_cpu/bsd_zero/bytes_bsd_zero.inline.hpp ! src/hotspot/os_cpu/bsd_zero/globals_bsd_zero.hpp ! src/hotspot/os_cpu/bsd_zero/orderAccess_bsd_zero.hpp ! src/hotspot/os_cpu/bsd_zero/os_bsd_zero.hpp ! src/hotspot/os_cpu/bsd_zero/prefetch_bsd_zero.inline.hpp ! src/hotspot/os_cpu/bsd_zero/thread_bsd_zero.hpp ! src/hotspot/os_cpu/bsd_zero/vmStructs_bsd_zero.hpp ! src/hotspot/os_cpu/linux_aarch64/atomic_linux_aarch64.hpp ! src/hotspot/os_cpu/linux_aarch64/bytes_linux_aarch64.inline.hpp ! src/hotspot/os_cpu/linux_aarch64/copy_linux_aarch64.inline.hpp ! src/hotspot/os_cpu/linux_aarch64/globals_linux_aarch64.hpp ! src/hotspot/os_cpu/linux_aarch64/orderAccess_linux_aarch64.hpp ! src/hotspot/os_cpu/linux_aarch64/os_linux_aarch64.hpp ! src/hotspot/os_cpu/linux_aarch64/os_linux_aarch64.inline.hpp ! src/hotspot/os_cpu/linux_aarch64/prefetch_linux_aarch64.inline.hpp ! src/hotspot/os_cpu/linux_aarch64/thread_linux_aarch64.hpp ! src/hotspot/os_cpu/linux_aarch64/vmStructs_linux_aarch64.hpp ! src/hotspot/os_cpu/linux_arm/atomic_linux_arm.hpp ! src/hotspot/os_cpu/linux_arm/bytes_linux_arm.inline.hpp ! src/hotspot/os_cpu/linux_arm/copy_linux_arm.inline.hpp ! src/hotspot/os_cpu/linux_arm/globals_linux_arm.hpp ! src/hotspot/os_cpu/linux_arm/orderAccess_linux_arm.hpp ! src/hotspot/os_cpu/linux_arm/os_linux_arm.hpp ! src/hotspot/os_cpu/linux_arm/prefetch_linux_arm.inline.hpp ! src/hotspot/os_cpu/linux_arm/thread_linux_arm.hpp ! src/hotspot/os_cpu/linux_arm/vmStructs_linux_arm.hpp ! src/hotspot/os_cpu/linux_ppc/atomic_linux_ppc.hpp ! src/hotspot/os_cpu/linux_ppc/bytes_linux_ppc.inline.hpp ! src/hotspot/os_cpu/linux_ppc/globals_linux_ppc.hpp ! src/hotspot/os_cpu/linux_ppc/orderAccess_linux_ppc.hpp ! src/hotspot/os_cpu/linux_ppc/os_linux_ppc.hpp ! src/hotspot/os_cpu/linux_ppc/prefetch_linux_ppc.inline.hpp ! src/hotspot/os_cpu/linux_ppc/thread_linux_ppc.hpp ! src/hotspot/os_cpu/linux_ppc/vmStructs_linux_ppc.hpp ! src/hotspot/os_cpu/linux_s390/atomic_linux_s390.hpp ! src/hotspot/os_cpu/linux_s390/bytes_linux_s390.inline.hpp ! src/hotspot/os_cpu/linux_s390/globals_linux_s390.hpp ! src/hotspot/os_cpu/linux_s390/orderAccess_linux_s390.hpp ! src/hotspot/os_cpu/linux_s390/os_linux_s390.hpp ! src/hotspot/os_cpu/linux_s390/prefetch_linux_s390.inline.hpp ! src/hotspot/os_cpu/linux_s390/thread_linux_s390.hpp ! src/hotspot/os_cpu/linux_s390/vmStructs_linux_s390.hpp ! src/hotspot/os_cpu/linux_sparc/atomic_linux_sparc.hpp ! src/hotspot/os_cpu/linux_sparc/globals_linux_sparc.hpp ! src/hotspot/os_cpu/linux_sparc/orderAccess_linux_sparc.hpp ! src/hotspot/os_cpu/linux_sparc/os_linux_sparc.hpp ! src/hotspot/os_cpu/linux_sparc/prefetch_linux_sparc.inline.hpp ! src/hotspot/os_cpu/linux_sparc/thread_linux_sparc.hpp ! src/hotspot/os_cpu/linux_sparc/vmStructs_linux_sparc.hpp ! src/hotspot/os_cpu/linux_x86/atomic_linux_x86.hpp ! src/hotspot/os_cpu/linux_x86/bytes_linux_x86.inline.hpp ! src/hotspot/os_cpu/linux_x86/copy_linux_x86.inline.hpp ! src/hotspot/os_cpu/linux_x86/gc/z/zAddress_linux_x86.inline.hpp ! src/hotspot/os_cpu/linux_x86/gc/z/zBackingFile_linux_x86.hpp ! src/hotspot/os_cpu/linux_x86/gc/z/zBackingPath_linux_x86.hpp ! src/hotspot/os_cpu/linux_x86/gc/z/zGlobals_linux_x86.hpp ! src/hotspot/os_cpu/linux_x86/gc/z/zPhysicalMemoryBacking_linux_x86.hpp ! src/hotspot/os_cpu/linux_x86/globals_linux_x86.hpp ! src/hotspot/os_cpu/linux_x86/orderAccess_linux_x86.hpp ! src/hotspot/os_cpu/linux_x86/os_linux_x86.hpp ! src/hotspot/os_cpu/linux_x86/os_linux_x86.inline.hpp ! src/hotspot/os_cpu/linux_x86/prefetch_linux_x86.inline.hpp ! src/hotspot/os_cpu/linux_x86/thread_linux_x86.hpp ! src/hotspot/os_cpu/linux_x86/vmStructs_linux_x86.hpp ! src/hotspot/os_cpu/linux_zero/atomic_linux_zero.hpp ! src/hotspot/os_cpu/linux_zero/bytes_linux_zero.inline.hpp ! src/hotspot/os_cpu/linux_zero/globals_linux_zero.hpp ! src/hotspot/os_cpu/linux_zero/orderAccess_linux_zero.hpp ! src/hotspot/os_cpu/linux_zero/os_linux_zero.hpp ! src/hotspot/os_cpu/linux_zero/prefetch_linux_zero.inline.hpp ! src/hotspot/os_cpu/linux_zero/thread_linux_zero.hpp ! src/hotspot/os_cpu/linux_zero/vmStructs_linux_zero.hpp ! src/hotspot/os_cpu/solaris_sparc/atomic_solaris_sparc.hpp ! src/hotspot/os_cpu/solaris_sparc/count_trailing_zeros_solaris_sparc.hpp ! src/hotspot/os_cpu/solaris_sparc/globals_solaris_sparc.hpp ! src/hotspot/os_cpu/solaris_sparc/orderAccess_solaris_sparc.hpp ! src/hotspot/os_cpu/solaris_sparc/os_solaris_sparc.hpp ! src/hotspot/os_cpu/solaris_sparc/prefetch_solaris_sparc.inline.hpp ! src/hotspot/os_cpu/solaris_sparc/thread_solaris_sparc.hpp ! src/hotspot/os_cpu/solaris_sparc/vmStructs_solaris_sparc.hpp ! src/hotspot/os_cpu/solaris_x86/atomic_solaris_x86.hpp ! src/hotspot/os_cpu/solaris_x86/bytes_solaris_x86.inline.hpp ! src/hotspot/os_cpu/solaris_x86/copy_solaris_x86.inline.hpp ! src/hotspot/os_cpu/solaris_x86/count_trailing_zeros_solaris_x86.hpp ! src/hotspot/os_cpu/solaris_x86/globals_solaris_x86.hpp ! src/hotspot/os_cpu/solaris_x86/orderAccess_solaris_x86.hpp ! src/hotspot/os_cpu/solaris_x86/os_solaris_x86.hpp ! src/hotspot/os_cpu/solaris_x86/os_solaris_x86.inline.hpp ! src/hotspot/os_cpu/solaris_x86/prefetch_solaris_x86.inline.hpp ! src/hotspot/os_cpu/solaris_x86/thread_solaris_x86.hpp ! src/hotspot/os_cpu/solaris_x86/vmStructs_solaris_x86.hpp ! src/hotspot/os_cpu/windows_x86/atomic_windows_x86.hpp ! src/hotspot/os_cpu/windows_x86/bytes_windows_x86.inline.hpp ! src/hotspot/os_cpu/windows_x86/copy_windows_x86.inline.hpp ! src/hotspot/os_cpu/windows_x86/globals_windows_x86.hpp ! src/hotspot/os_cpu/windows_x86/orderAccess_windows_x86.hpp ! src/hotspot/os_cpu/windows_x86/os_windows_x86.hpp ! src/hotspot/os_cpu/windows_x86/os_windows_x86.inline.hpp ! src/hotspot/os_cpu/windows_x86/prefetch_windows_x86.inline.hpp ! src/hotspot/os_cpu/windows_x86/thread_windows_x86.hpp ! src/hotspot/os_cpu/windows_x86/unwind_windows_x86.hpp ! src/hotspot/os_cpu/windows_x86/vmStructs_windows_x86.hpp ! src/hotspot/share/adlc/adlc.hpp ! src/hotspot/share/adlc/adlparse.hpp ! src/hotspot/share/adlc/archDesc.hpp ! src/hotspot/share/adlc/arena.hpp ! src/hotspot/share/adlc/dict2.hpp ! src/hotspot/share/adlc/filebuff.hpp ! src/hotspot/share/adlc/forms.hpp ! src/hotspot/share/adlc/formsopt.hpp ! src/hotspot/share/adlc/formssel.hpp ! src/hotspot/share/aot/aotCodeHeap.hpp ! src/hotspot/share/aot/aotCompiledMethod.hpp ! src/hotspot/share/aot/aotLoader.hpp ! src/hotspot/share/aot/aotLoader.inline.hpp ! src/hotspot/share/aot/compiledIC_aot.hpp ! src/hotspot/share/asm/assembler.hpp ! src/hotspot/share/asm/assembler.inline.hpp ! src/hotspot/share/asm/codeBuffer.hpp ! src/hotspot/share/asm/macroAssembler.hpp ! src/hotspot/share/asm/macroAssembler.inline.hpp ! src/hotspot/share/asm/register.hpp ! src/hotspot/share/c1/c1_CFGPrinter.hpp ! src/hotspot/share/c1/c1_Canonicalizer.hpp ! src/hotspot/share/c1/c1_CodeStubs.hpp ! src/hotspot/share/c1/c1_Compilation.hpp ! src/hotspot/share/c1/c1_Compiler.hpp ! src/hotspot/share/c1/c1_Decorators.hpp ! src/hotspot/share/c1/c1_Defs.hpp ! src/hotspot/share/c1/c1_FpuStackSim.hpp ! src/hotspot/share/c1/c1_FrameMap.hpp ! src/hotspot/share/c1/c1_GraphBuilder.hpp ! src/hotspot/share/c1/c1_IR.hpp ! src/hotspot/share/c1/c1_Instruction.hpp ! src/hotspot/share/c1/c1_InstructionPrinter.hpp ! src/hotspot/share/c1/c1_LIR.hpp ! src/hotspot/share/c1/c1_LIRAssembler.hpp ! src/hotspot/share/c1/c1_LIRGenerator.hpp ! src/hotspot/share/c1/c1_LinearScan.hpp ! src/hotspot/share/c1/c1_MacroAssembler.hpp ! src/hotspot/share/c1/c1_Optimizer.hpp ! src/hotspot/share/c1/c1_RangeCheckElimination.hpp ! src/hotspot/share/c1/c1_Runtime1.hpp ! src/hotspot/share/c1/c1_ValueMap.hpp ! src/hotspot/share/c1/c1_ValueSet.hpp ! src/hotspot/share/c1/c1_ValueSet.inline.hpp ! src/hotspot/share/c1/c1_ValueStack.hpp ! src/hotspot/share/c1/c1_ValueType.hpp ! src/hotspot/share/c1/c1_globals.hpp ! src/hotspot/share/ci/bcEscapeAnalyzer.hpp ! src/hotspot/share/ci/ciArray.hpp ! src/hotspot/share/ci/ciArrayKlass.hpp ! src/hotspot/share/ci/ciBaseObject.hpp ! src/hotspot/share/ci/ciCallProfile.hpp ! src/hotspot/share/ci/ciCallSite.hpp ! src/hotspot/share/ci/ciClassList.hpp ! src/hotspot/share/ci/ciConstant.hpp ! src/hotspot/share/ci/ciConstantPoolCache.hpp ! src/hotspot/share/ci/ciEnv.hpp ! src/hotspot/share/ci/ciExceptionHandler.hpp ! src/hotspot/share/ci/ciField.hpp ! src/hotspot/share/ci/ciFlags.hpp ! src/hotspot/share/ci/ciInstance.hpp ! src/hotspot/share/ci/ciInstanceKlass.hpp ! src/hotspot/share/ci/ciKlass.hpp ! src/hotspot/share/ci/ciMemberName.hpp ! src/hotspot/share/ci/ciMetadata.hpp ! src/hotspot/share/ci/ciMethod.hpp ! src/hotspot/share/ci/ciMethodBlocks.hpp ! src/hotspot/share/ci/ciMethodData.hpp ! src/hotspot/share/ci/ciMethodHandle.hpp ! src/hotspot/share/ci/ciMethodType.hpp ! src/hotspot/share/ci/ciNullObject.hpp ! src/hotspot/share/ci/ciObjArray.hpp ! src/hotspot/share/ci/ciObjArrayKlass.hpp ! src/hotspot/share/ci/ciObject.hpp ! src/hotspot/share/ci/ciObjectFactory.hpp ! src/hotspot/share/ci/ciReplay.hpp ! src/hotspot/share/ci/ciSignature.hpp ! src/hotspot/share/ci/ciStreams.hpp ! src/hotspot/share/ci/ciSymbol.hpp ! src/hotspot/share/ci/ciType.hpp ! src/hotspot/share/ci/ciTypeArray.hpp ! src/hotspot/share/ci/ciTypeArrayKlass.hpp ! src/hotspot/share/ci/ciTypeFlow.hpp ! src/hotspot/share/ci/ciUtilities.hpp ! src/hotspot/share/ci/ciUtilities.inline.hpp ! src/hotspot/share/ci/compilerInterface.hpp ! src/hotspot/share/classfile/altHashing.hpp ! src/hotspot/share/classfile/bytecodeAssembler.hpp ! src/hotspot/share/classfile/classFileParser.hpp ! src/hotspot/share/classfile/classFileStream.hpp ! src/hotspot/share/classfile/classListParser.hpp ! src/hotspot/share/classfile/classLoader.hpp ! src/hotspot/share/classfile/classLoader.inline.hpp ! src/hotspot/share/classfile/classLoaderData.hpp ! src/hotspot/share/classfile/classLoaderData.inline.hpp ! src/hotspot/share/classfile/classLoaderDataGraph.hpp ! src/hotspot/share/classfile/classLoaderDataGraph.inline.hpp ! src/hotspot/share/classfile/classLoaderExt.hpp ! src/hotspot/share/classfile/classLoaderHierarchyDCmd.hpp ! src/hotspot/share/classfile/classLoaderStats.hpp ! src/hotspot/share/classfile/compactHashtable.hpp ! src/hotspot/share/classfile/defaultMethods.hpp ! src/hotspot/share/classfile/dictionary.hpp ! src/hotspot/share/classfile/dictionary.inline.hpp ! src/hotspot/share/classfile/javaAssertions.hpp ! src/hotspot/share/classfile/javaClasses.hpp ! src/hotspot/share/classfile/javaClasses.inline.hpp ! src/hotspot/share/classfile/klassFactory.hpp ! src/hotspot/share/classfile/loaderConstraints.hpp ! src/hotspot/share/classfile/metadataOnStackMark.hpp ! src/hotspot/share/classfile/moduleEntry.hpp ! src/hotspot/share/classfile/modules.hpp ! src/hotspot/share/classfile/packageEntry.hpp ! src/hotspot/share/classfile/placeholders.hpp ! src/hotspot/share/classfile/protectionDomainCache.hpp ! src/hotspot/share/classfile/resolutionErrors.hpp ! src/hotspot/share/classfile/sharedPathsMiscInfo.hpp ! src/hotspot/share/classfile/stackMapFrame.hpp ! src/hotspot/share/classfile/stackMapTable.hpp ! src/hotspot/share/classfile/stackMapTableFormat.hpp ! src/hotspot/share/classfile/stringTable.hpp ! src/hotspot/share/classfile/symbolTable.hpp ! src/hotspot/share/classfile/systemDictionary.hpp ! src/hotspot/share/classfile/systemDictionaryShared.hpp ! src/hotspot/share/classfile/verificationType.hpp ! src/hotspot/share/classfile/verifier.hpp ! src/hotspot/share/classfile/vmSymbols.hpp ! src/hotspot/share/code/codeBlob.hpp ! src/hotspot/share/code/codeCache.hpp ! src/hotspot/share/code/compiledIC.hpp ! src/hotspot/share/code/compiledMethod.hpp ! src/hotspot/share/code/compiledMethod.inline.hpp ! src/hotspot/share/code/compressedStream.hpp ! src/hotspot/share/code/debugInfo.hpp ! src/hotspot/share/code/debugInfoRec.hpp ! src/hotspot/share/code/dependencies.hpp ! src/hotspot/share/code/dependencyContext.hpp ! src/hotspot/share/code/exceptionHandlerTable.hpp ! src/hotspot/share/code/icBuffer.hpp ! src/hotspot/share/code/location.hpp ! src/hotspot/share/code/nativeInst.hpp ! src/hotspot/share/code/nmethod.hpp ! src/hotspot/share/code/oopRecorder.hpp ! src/hotspot/share/code/pcDesc.hpp ! src/hotspot/share/code/relocInfo.hpp ! src/hotspot/share/code/relocInfo_ext.hpp ! src/hotspot/share/code/scopeDesc.hpp ! src/hotspot/share/code/stubs.hpp ! src/hotspot/share/code/vmreg.hpp ! src/hotspot/share/code/vmreg.inline.hpp ! src/hotspot/share/code/vtableStubs.hpp ! src/hotspot/share/compiler/abstractCompiler.hpp ! src/hotspot/share/compiler/compileBroker.hpp ! src/hotspot/share/compiler/compileLog.hpp ! src/hotspot/share/compiler/compileTask.hpp ! src/hotspot/share/compiler/compilerDefinitions.hpp ! src/hotspot/share/compiler/compilerDirectives.hpp ! src/hotspot/share/compiler/compilerOracle.hpp ! src/hotspot/share/compiler/directivesParser.hpp ! src/hotspot/share/compiler/disassembler.hpp ! src/hotspot/share/compiler/methodLiveness.hpp ! src/hotspot/share/compiler/methodMatcher.hpp ! src/hotspot/share/compiler/oopMap.hpp ! src/hotspot/share/gc/cms/adaptiveFreeList.hpp ! src/hotspot/share/gc/cms/allocationStats.hpp ! src/hotspot/share/gc/cms/cmsCollectorPolicy.hpp ! src/hotspot/share/gc/cms/cmsHeap.hpp ! src/hotspot/share/gc/cms/cmsLockVerifier.hpp ! src/hotspot/share/gc/cms/cmsOopClosures.hpp ! src/hotspot/share/gc/cms/cmsOopClosures.inline.hpp ! src/hotspot/share/gc/cms/cmsVMOperations.hpp ! src/hotspot/share/gc/cms/compactibleFreeListSpace.hpp ! src/hotspot/share/gc/cms/concurrentMarkSweepGeneration.hpp ! src/hotspot/share/gc/cms/concurrentMarkSweepGeneration.inline.hpp ! src/hotspot/share/gc/cms/concurrentMarkSweepThread.hpp ! src/hotspot/share/gc/cms/freeChunk.hpp ! src/hotspot/share/gc/cms/gSpaceCounters.hpp ! src/hotspot/share/gc/cms/jvmFlagConstraintsCMS.hpp ! src/hotspot/share/gc/cms/parNewGeneration.hpp ! src/hotspot/share/gc/cms/parNewGeneration.inline.hpp ! src/hotspot/share/gc/cms/parOopClosures.hpp ! src/hotspot/share/gc/cms/parOopClosures.inline.hpp ! src/hotspot/share/gc/cms/promotionInfo.hpp ! src/hotspot/share/gc/cms/promotionInfo.inline.hpp ! src/hotspot/share/gc/cms/vmStructs_cms.hpp ! src/hotspot/share/gc/cms/yieldingWorkgroup.hpp ! src/hotspot/share/gc/epsilon/epsilonBarrierSet.hpp ! src/hotspot/share/gc/epsilon/epsilonCollectorPolicy.hpp ! src/hotspot/share/gc/epsilon/epsilonHeap.hpp ! src/hotspot/share/gc/epsilon/epsilonMemoryPool.hpp ! src/hotspot/share/gc/epsilon/epsilonMonitoringSupport.hpp ! src/hotspot/share/gc/epsilon/epsilonThreadLocalData.hpp ! src/hotspot/share/gc/epsilon/epsilon_globals.hpp ! src/hotspot/share/gc/epsilon/vmStructs_epsilon.hpp ! src/hotspot/share/gc/g1/c2/g1BarrierSetC2.hpp ! src/hotspot/share/gc/g1/collectionSetChooser.hpp ! src/hotspot/share/gc/g1/dirtyCardQueue.hpp ! src/hotspot/share/gc/g1/evacuationInfo.hpp ! src/hotspot/share/gc/g1/g1AllocRegion.hpp ! src/hotspot/share/gc/g1/g1AllocRegion.inline.hpp ! src/hotspot/share/gc/g1/g1Allocator.hpp ! src/hotspot/share/gc/g1/g1Allocator.inline.hpp ! src/hotspot/share/gc/g1/g1Analytics.hpp ! src/hotspot/share/gc/g1/g1BarrierSet.hpp ! src/hotspot/share/gc/g1/g1BarrierSet.inline.hpp ! src/hotspot/share/gc/g1/g1BiasedArray.hpp ! src/hotspot/share/gc/g1/g1BlockOffsetTable.hpp ! src/hotspot/share/gc/g1/g1BlockOffsetTable.inline.hpp ! src/hotspot/share/gc/g1/g1CardCounts.hpp ! src/hotspot/share/gc/g1/g1CardTable.hpp ! src/hotspot/share/gc/g1/g1CardTable.inline.hpp ! src/hotspot/share/gc/g1/g1CodeBlobClosure.hpp ! src/hotspot/share/gc/g1/g1CodeCacheRemSet.hpp ! src/hotspot/share/gc/g1/g1CodeRootSetTable.hpp ! src/hotspot/share/gc/g1/g1CollectedHeap.hpp ! src/hotspot/share/gc/g1/g1CollectedHeap.inline.hpp ! src/hotspot/share/gc/g1/g1CollectionSet.hpp ! src/hotspot/share/gc/g1/g1CollectorPolicy.hpp ! src/hotspot/share/gc/g1/g1CollectorState.hpp ! src/hotspot/share/gc/g1/g1ConcurrentMark.hpp ! src/hotspot/share/gc/g1/g1ConcurrentMark.inline.hpp ! src/hotspot/share/gc/g1/g1ConcurrentMarkBitMap.hpp ! src/hotspot/share/gc/g1/g1ConcurrentMarkBitMap.inline.hpp ! src/hotspot/share/gc/g1/g1ConcurrentMarkObjArrayProcessor.hpp ! src/hotspot/share/gc/g1/g1ConcurrentMarkObjArrayProcessor.inline.hpp ! src/hotspot/share/gc/g1/g1ConcurrentMarkThread.hpp ! src/hotspot/share/gc/g1/g1ConcurrentMarkThread.inline.hpp ! src/hotspot/share/gc/g1/g1ConcurrentRefine.hpp ! src/hotspot/share/gc/g1/g1ConcurrentRefineThread.hpp ! src/hotspot/share/gc/g1/g1EdenRegions.hpp ! src/hotspot/share/gc/g1/g1EvacFailure.hpp ! src/hotspot/share/gc/g1/g1EvacStats.hpp ! src/hotspot/share/gc/g1/g1EvacStats.inline.hpp ! src/hotspot/share/gc/g1/g1FromCardCache.hpp ! src/hotspot/share/gc/g1/g1FullGCMarker.inline.hpp ! src/hotspot/share/gc/g1/g1FullGCOopClosures.inline.hpp ! src/hotspot/share/gc/g1/g1FullGCScope.hpp ! src/hotspot/share/gc/g1/g1GCPhaseTimes.hpp ! src/hotspot/share/gc/g1/g1HRPrinter.hpp ! src/hotspot/share/gc/g1/g1HeapRegionEventSender.hpp ! src/hotspot/share/gc/g1/g1HeapRegionTraceType.hpp ! src/hotspot/share/gc/g1/g1HeapSizingPolicy.hpp ! src/hotspot/share/gc/g1/g1HeapTransition.hpp ! src/hotspot/share/gc/g1/g1HeapVerifier.hpp ! src/hotspot/share/gc/g1/g1HeterogeneousCollectorPolicy.hpp ! src/hotspot/share/gc/g1/g1HeterogeneousHeapPolicy.hpp ! src/hotspot/share/gc/g1/g1HeterogeneousHeapYoungGenSizer.hpp ! src/hotspot/share/gc/g1/g1HotCardCache.hpp ! src/hotspot/share/gc/g1/g1IHOPControl.hpp ! src/hotspot/share/gc/g1/g1InCSetState.hpp ! src/hotspot/share/gc/g1/g1InitialMarkToMixedTimeTracker.hpp ! src/hotspot/share/gc/g1/g1MMUTracker.hpp ! src/hotspot/share/gc/g1/g1MemoryPool.hpp ! src/hotspot/share/gc/g1/g1MonitoringSupport.hpp ! src/hotspot/share/gc/g1/g1OopClosures.hpp ! src/hotspot/share/gc/g1/g1OopClosures.inline.hpp ! src/hotspot/share/gc/g1/g1PageBasedVirtualSpace.hpp ! src/hotspot/share/gc/g1/g1ParScanThreadState.hpp ! src/hotspot/share/gc/g1/g1ParScanThreadState.inline.hpp ! src/hotspot/share/gc/g1/g1Policy.hpp ! src/hotspot/share/gc/g1/g1Predictions.hpp ! src/hotspot/share/gc/g1/g1RegionMarkStatsCache.hpp ! src/hotspot/share/gc/g1/g1RegionMarkStatsCache.inline.hpp ! src/hotspot/share/gc/g1/g1RegionToSpaceMapper.hpp ! src/hotspot/share/gc/g1/g1RemSet.hpp ! src/hotspot/share/gc/g1/g1RemSetSummary.hpp ! src/hotspot/share/gc/g1/g1RemSetTrackingPolicy.hpp ! src/hotspot/share/gc/g1/g1RootClosures.hpp ! src/hotspot/share/gc/g1/g1RootProcessor.hpp ! src/hotspot/share/gc/g1/g1SATBMarkQueueSet.hpp ! src/hotspot/share/gc/g1/g1StringDedup.hpp ! src/hotspot/share/gc/g1/g1StringDedupQueue.hpp ! src/hotspot/share/gc/g1/g1StringDedupStat.hpp ! src/hotspot/share/gc/g1/g1SurvivorRegions.hpp ! src/hotspot/share/gc/g1/g1VMOperations.hpp ! src/hotspot/share/gc/g1/g1YCTypes.hpp ! src/hotspot/share/gc/g1/g1YoungGenSizer.hpp ! src/hotspot/share/gc/g1/g1YoungRemSetSamplingThread.hpp ! src/hotspot/share/gc/g1/g1_globals.hpp ! src/hotspot/share/gc/g1/heapRegion.hpp ! src/hotspot/share/gc/g1/heapRegion.inline.hpp ! src/hotspot/share/gc/g1/heapRegionBounds.hpp ! src/hotspot/share/gc/g1/heapRegionBounds.inline.hpp ! src/hotspot/share/gc/g1/heapRegionManager.hpp ! src/hotspot/share/gc/g1/heapRegionManager.inline.hpp ! src/hotspot/share/gc/g1/heapRegionRemSet.hpp ! src/hotspot/share/gc/g1/heapRegionSet.hpp ! src/hotspot/share/gc/g1/heapRegionSet.inline.hpp ! src/hotspot/share/gc/g1/heapRegionTracer.hpp ! src/hotspot/share/gc/g1/heapRegionType.hpp ! src/hotspot/share/gc/g1/heterogeneousHeapRegionManager.hpp ! src/hotspot/share/gc/g1/jvmFlagConstraintsG1.hpp ! src/hotspot/share/gc/g1/sparsePRT.hpp ! src/hotspot/share/gc/g1/survRateGroup.hpp ! src/hotspot/share/gc/g1/vmStructs_g1.hpp ! src/hotspot/share/gc/parallel/adjoiningGenerations.hpp ! src/hotspot/share/gc/parallel/adjoiningGenerationsForHeteroHeap.hpp ! src/hotspot/share/gc/parallel/adjoiningVirtualSpaces.hpp ! src/hotspot/share/gc/parallel/asPSOldGen.hpp ! src/hotspot/share/gc/parallel/asPSYoungGen.hpp ! src/hotspot/share/gc/parallel/gcAdaptivePolicyCounters.hpp ! src/hotspot/share/gc/parallel/gcTaskManager.hpp ! src/hotspot/share/gc/parallel/gcTaskThread.hpp ! src/hotspot/share/gc/parallel/generationSizer.hpp ! src/hotspot/share/gc/parallel/heterogeneousGenerationSizer.hpp ! src/hotspot/share/gc/parallel/immutableSpace.hpp ! src/hotspot/share/gc/parallel/jvmFlagConstraintsParallel.hpp ! src/hotspot/share/gc/parallel/mutableNUMASpace.hpp ! src/hotspot/share/gc/parallel/mutableSpace.hpp ! src/hotspot/share/gc/parallel/objectStartArray.hpp ! src/hotspot/share/gc/parallel/objectStartArray.inline.hpp ! src/hotspot/share/gc/parallel/parMarkBitMap.hpp ! src/hotspot/share/gc/parallel/parMarkBitMap.inline.hpp ! src/hotspot/share/gc/parallel/parallelArguments.hpp ! src/hotspot/share/gc/parallel/parallelScavengeHeap.hpp ! src/hotspot/share/gc/parallel/parallelScavengeHeap.inline.hpp ! src/hotspot/share/gc/parallel/pcTasks.hpp ! src/hotspot/share/gc/parallel/psAdaptiveSizePolicy.hpp ! src/hotspot/share/gc/parallel/psCardTable.hpp ! src/hotspot/share/gc/parallel/psClosure.inline.hpp ! src/hotspot/share/gc/parallel/psCompactionManager.hpp ! src/hotspot/share/gc/parallel/psCompactionManager.inline.hpp ! src/hotspot/share/gc/parallel/psFileBackedVirtualspace.hpp ! src/hotspot/share/gc/parallel/psGCAdaptivePolicyCounters.hpp ! src/hotspot/share/gc/parallel/psGenerationCounters.hpp ! src/hotspot/share/gc/parallel/psMarkSweep.hpp ! src/hotspot/share/gc/parallel/psMarkSweepDecorator.hpp ! src/hotspot/share/gc/parallel/psMemoryPool.hpp ! src/hotspot/share/gc/parallel/psOldGen.hpp ! src/hotspot/share/gc/parallel/psParallelCompact.hpp ! src/hotspot/share/gc/parallel/psParallelCompact.inline.hpp ! src/hotspot/share/gc/parallel/psPromotionLAB.hpp ! src/hotspot/share/gc/parallel/psPromotionLAB.inline.hpp ! src/hotspot/share/gc/parallel/psPromotionManager.hpp ! src/hotspot/share/gc/parallel/psPromotionManager.inline.hpp ! src/hotspot/share/gc/parallel/psScavenge.hpp ! src/hotspot/share/gc/parallel/psScavenge.inline.hpp ! src/hotspot/share/gc/parallel/psTasks.hpp ! src/hotspot/share/gc/parallel/psVMOperations.hpp ! src/hotspot/share/gc/parallel/psVirtualspace.hpp ! src/hotspot/share/gc/parallel/psYoungGen.hpp ! src/hotspot/share/gc/parallel/spaceCounters.hpp ! src/hotspot/share/gc/parallel/vmStructs_parallelgc.hpp ! src/hotspot/share/gc/serial/cSpaceCounters.hpp ! src/hotspot/share/gc/serial/defNewGeneration.hpp ! src/hotspot/share/gc/serial/defNewGeneration.inline.hpp ! src/hotspot/share/gc/serial/genMarkSweep.hpp ! src/hotspot/share/gc/serial/markSweep.hpp ! src/hotspot/share/gc/serial/markSweep.inline.hpp ! src/hotspot/share/gc/serial/serialHeap.hpp ! src/hotspot/share/gc/serial/tenuredGeneration.hpp ! src/hotspot/share/gc/serial/tenuredGeneration.inline.hpp ! src/hotspot/share/gc/serial/vmStructs_serial.hpp ! src/hotspot/share/gc/shared/accessBarrierSupport.hpp ! src/hotspot/share/gc/shared/accessBarrierSupport.inline.hpp ! src/hotspot/share/gc/shared/adaptiveSizePolicy.hpp ! src/hotspot/share/gc/shared/ageTable.hpp ! src/hotspot/share/gc/shared/ageTable.inline.hpp ! src/hotspot/share/gc/shared/ageTableTracer.hpp ! src/hotspot/share/gc/shared/allocTracer.hpp ! src/hotspot/share/gc/shared/barrierSet.hpp ! src/hotspot/share/gc/shared/barrierSet.inline.hpp ! src/hotspot/share/gc/shared/barrierSetConfig.hpp ! src/hotspot/share/gc/shared/barrierSetConfig.inline.hpp ! src/hotspot/share/gc/shared/barrierSetNMethod.hpp ! src/hotspot/share/gc/shared/blockOffsetTable.hpp ! src/hotspot/share/gc/shared/blockOffsetTable.inline.hpp ! src/hotspot/share/gc/shared/cardGeneration.hpp ! src/hotspot/share/gc/shared/cardGeneration.inline.hpp ! src/hotspot/share/gc/shared/cardTable.hpp ! src/hotspot/share/gc/shared/cardTableBarrierSet.hpp ! src/hotspot/share/gc/shared/cardTableBarrierSet.inline.hpp ! src/hotspot/share/gc/shared/cardTableRS.hpp ! src/hotspot/share/gc/shared/collectedHeap.hpp ! src/hotspot/share/gc/shared/collectedHeap.inline.hpp ! src/hotspot/share/gc/shared/collectorCounters.hpp ! src/hotspot/share/gc/shared/collectorPolicy.hpp ! src/hotspot/share/gc/shared/concurrentGCPhaseManager.hpp ! src/hotspot/share/gc/shared/concurrentGCThread.hpp ! src/hotspot/share/gc/shared/copyFailedInfo.hpp ! src/hotspot/share/gc/shared/gcBehaviours.hpp ! src/hotspot/share/gc/shared/gcCause.hpp ! src/hotspot/share/gc/shared/gcConfiguration.hpp ! src/hotspot/share/gc/shared/gcHeapSummary.hpp ! src/hotspot/share/gc/shared/gcId.hpp ! src/hotspot/share/gc/shared/gcLocker.hpp ! src/hotspot/share/gc/shared/gcLocker.inline.hpp ! src/hotspot/share/gc/shared/gcName.hpp ! src/hotspot/share/gc/shared/gcPolicyCounters.hpp ! src/hotspot/share/gc/shared/gcStats.hpp ! src/hotspot/share/gc/shared/gcTimer.hpp ! src/hotspot/share/gc/shared/gcTrace.hpp ! src/hotspot/share/gc/shared/gcTraceTime.hpp ! src/hotspot/share/gc/shared/gcTraceTime.inline.hpp ! src/hotspot/share/gc/shared/gcUtil.hpp ! src/hotspot/share/gc/shared/gcUtil.inline.hpp ! src/hotspot/share/gc/shared/gcVMOperations.hpp ! src/hotspot/share/gc/shared/gcWhen.hpp ! src/hotspot/share/gc/shared/genCollectedHeap.hpp ! src/hotspot/share/gc/shared/genMemoryPools.hpp ! src/hotspot/share/gc/shared/genOopClosures.hpp ! src/hotspot/share/gc/shared/genOopClosures.inline.hpp ! src/hotspot/share/gc/shared/generation.hpp ! src/hotspot/share/gc/shared/generationCounters.hpp ! src/hotspot/share/gc/shared/generationSpec.hpp ! src/hotspot/share/gc/shared/hSpaceCounters.hpp ! src/hotspot/share/gc/shared/isGCActiveMark.hpp ! src/hotspot/share/gc/shared/jvmFlagConstraintsGC.hpp ! src/hotspot/share/gc/shared/markBitMap.hpp ! src/hotspot/share/gc/shared/markBitMap.inline.hpp ! src/hotspot/share/gc/shared/memAllocator.hpp ! src/hotspot/share/gc/shared/memset_with_concurrent_readers.hpp ! src/hotspot/share/gc/shared/modRefBarrierSet.hpp ! src/hotspot/share/gc/shared/modRefBarrierSet.inline.hpp ! src/hotspot/share/gc/shared/objectCountEventSender.hpp ! src/hotspot/share/gc/shared/oopStorage.hpp ! src/hotspot/share/gc/shared/oopStorage.inline.hpp ! src/hotspot/share/gc/shared/owstTaskTerminator.hpp ! src/hotspot/share/gc/shared/parallelCleaning.hpp ! src/hotspot/share/gc/shared/plab.hpp ! src/hotspot/share/gc/shared/plab.inline.hpp ! src/hotspot/share/gc/shared/preservedMarks.hpp ! src/hotspot/share/gc/shared/preservedMarks.inline.hpp ! src/hotspot/share/gc/shared/referencePolicy.hpp ! src/hotspot/share/gc/shared/referenceProcessor.hpp ! src/hotspot/share/gc/shared/referenceProcessor.inline.hpp ! src/hotspot/share/gc/shared/referenceProcessorPhaseTimes.hpp ! src/hotspot/share/gc/shared/referenceProcessorStats.hpp ! src/hotspot/share/gc/shared/softRefGenPolicy.hpp ! src/hotspot/share/gc/shared/softRefPolicy.hpp ! src/hotspot/share/gc/shared/space.hpp ! src/hotspot/share/gc/shared/space.inline.hpp ! src/hotspot/share/gc/shared/spaceDecorator.hpp ! src/hotspot/share/gc/shared/stringdedup/stringDedup.hpp ! src/hotspot/share/gc/shared/stringdedup/stringDedup.inline.hpp ! src/hotspot/share/gc/shared/stringdedup/stringDedupQueue.hpp ! src/hotspot/share/gc/shared/stringdedup/stringDedupQueue.inline.hpp ! src/hotspot/share/gc/shared/stringdedup/stringDedupStat.hpp ! src/hotspot/share/gc/shared/stringdedup/stringDedupTable.hpp ! src/hotspot/share/gc/shared/stringdedup/stringDedupThread.hpp ! src/hotspot/share/gc/shared/stringdedup/stringDedupThread.inline.hpp ! src/hotspot/share/gc/shared/strongRootsScope.hpp ! src/hotspot/share/gc/shared/taskqueue.hpp ! src/hotspot/share/gc/shared/taskqueue.inline.hpp ! src/hotspot/share/gc/shared/threadLocalAllocBuffer.hpp ! src/hotspot/share/gc/shared/threadLocalAllocBuffer.inline.hpp ! src/hotspot/share/gc/shared/weakProcessor.hpp ! src/hotspot/share/gc/shared/weakProcessor.inline.hpp ! src/hotspot/share/gc/shared/workerDataArray.hpp ! src/hotspot/share/gc/shared/workerDataArray.inline.hpp ! src/hotspot/share/gc/shared/workerManager.hpp ! src/hotspot/share/gc/shared/workerPolicy.hpp ! src/hotspot/share/gc/shared/workgroup.hpp ! src/hotspot/share/gc/shenandoah/c1/shenandoahBarrierSetC1.hpp ! src/hotspot/share/gc/shenandoah/c2/shenandoahBarrierSetC2.hpp ! src/hotspot/share/gc/shenandoah/c2/shenandoahSupport.hpp ! src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.hpp ! src/hotspot/share/gc/shenandoah/heuristics/shenandoahAggressiveHeuristics.hpp ! src/hotspot/share/gc/shenandoah/heuristics/shenandoahCompactHeuristics.hpp ! src/hotspot/share/gc/shenandoah/heuristics/shenandoahPassiveHeuristics.hpp ! src/hotspot/share/gc/shenandoah/heuristics/shenandoahStaticHeuristics.hpp ! src/hotspot/share/gc/shenandoah/heuristics/shenandoahTraversalHeuristics.hpp ! src/hotspot/share/gc/shenandoah/shenandoahAllocRequest.hpp ! src/hotspot/share/gc/shenandoah/shenandoahAllocTracker.hpp ! src/hotspot/share/gc/shenandoah/shenandoahAsserts.hpp ! src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.hpp ! src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.inline.hpp ! src/hotspot/share/gc/shenandoah/shenandoahBrooksPointer.hpp ! src/hotspot/share/gc/shenandoah/shenandoahBrooksPointer.inline.hpp ! src/hotspot/share/gc/shenandoah/shenandoahCodeRoots.hpp ! src/hotspot/share/gc/shenandoah/shenandoahCollectionSet.hpp ! src/hotspot/share/gc/shenandoah/shenandoahCollectionSet.inline.hpp ! src/hotspot/share/gc/shenandoah/shenandoahCollectorPolicy.hpp ! src/hotspot/share/gc/shenandoah/shenandoahConcurrentMark.hpp ! src/hotspot/share/gc/shenandoah/shenandoahConcurrentMark.inline.hpp ! src/hotspot/share/gc/shenandoah/shenandoahControlThread.hpp ! src/hotspot/share/gc/shenandoah/shenandoahEvacOOMHandler.hpp ! src/hotspot/share/gc/shenandoah/shenandoahFreeSet.hpp ! src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp ! src/hotspot/share/gc/shenandoah/shenandoahHeap.inline.hpp ! src/hotspot/share/gc/shenandoah/shenandoahHeapLock.hpp ! src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.hpp ! src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.inline.hpp ! src/hotspot/share/gc/shenandoah/shenandoahHeapRegionCounters.hpp ! src/hotspot/share/gc/shenandoah/shenandoahHeapRegionSet.hpp ! src/hotspot/share/gc/shenandoah/shenandoahHeapRegionSet.inline.hpp ! src/hotspot/share/gc/shenandoah/shenandoahHeuristics.hpp ! src/hotspot/share/gc/shenandoah/shenandoahMarkCompact.hpp ! src/hotspot/share/gc/shenandoah/shenandoahMarkingContext.hpp ! src/hotspot/share/gc/shenandoah/shenandoahMarkingContext.inline.hpp ! src/hotspot/share/gc/shenandoah/shenandoahMemoryPool.hpp ! src/hotspot/share/gc/shenandoah/shenandoahMetrics.hpp ! src/hotspot/share/gc/shenandoah/shenandoahMonitoringSupport.hpp ! src/hotspot/share/gc/shenandoah/shenandoahNumberSeq.hpp ! src/hotspot/share/gc/shenandoah/shenandoahOopClosures.hpp ! src/hotspot/share/gc/shenandoah/shenandoahOopClosures.inline.hpp ! src/hotspot/share/gc/shenandoah/shenandoahPacer.hpp ! src/hotspot/share/gc/shenandoah/shenandoahPacer.inline.hpp ! src/hotspot/share/gc/shenandoah/shenandoahPhaseTimings.hpp ! src/hotspot/share/gc/shenandoah/shenandoahRootProcessor.hpp ! src/hotspot/share/gc/shenandoah/shenandoahRuntime.hpp ! src/hotspot/share/gc/shenandoah/shenandoahSATBMarkQueueSet.hpp ! src/hotspot/share/gc/shenandoah/shenandoahSharedVariables.hpp ! src/hotspot/share/gc/shenandoah/shenandoahStrDedupQueue.hpp ! src/hotspot/share/gc/shenandoah/shenandoahStrDedupQueue.inline.hpp ! src/hotspot/share/gc/shenandoah/shenandoahStringDedup.hpp ! src/hotspot/share/gc/shenandoah/shenandoahTaskqueue.hpp ! src/hotspot/share/gc/shenandoah/shenandoahTaskqueue.inline.hpp ! src/hotspot/share/gc/shenandoah/shenandoahTimingTracker.hpp ! src/hotspot/share/gc/shenandoah/shenandoahTracer.hpp ! src/hotspot/share/gc/shenandoah/shenandoahTraversalGC.hpp ! src/hotspot/share/gc/shenandoah/shenandoahTraversalGC.inline.hpp ! src/hotspot/share/gc/shenandoah/shenandoahUtils.hpp ! src/hotspot/share/gc/shenandoah/shenandoahVMOperations.hpp ! src/hotspot/share/gc/shenandoah/shenandoahVerifier.hpp ! src/hotspot/share/gc/shenandoah/shenandoahWorkGroup.hpp ! src/hotspot/share/gc/shenandoah/shenandoahWorkerPolicy.hpp ! src/hotspot/share/gc/shenandoah/shenandoah_globals.hpp ! src/hotspot/share/gc/shenandoah/vmStructs_shenandoah.hpp ! src/hotspot/share/gc/z/vmStructs_z.hpp ! src/hotspot/share/gc/z/zForwardingTable.hpp ! src/hotspot/share/gc/z/zForwardingTable.inline.hpp ! src/hotspot/share/interpreter/abstractInterpreter.hpp ! src/hotspot/share/interpreter/bytecode.hpp ! src/hotspot/share/interpreter/bytecode.inline.hpp ! src/hotspot/share/interpreter/bytecodeHistogram.hpp ! src/hotspot/share/interpreter/bytecodeInterpreter.hpp ! src/hotspot/share/interpreter/bytecodeInterpreter.inline.hpp ! src/hotspot/share/interpreter/bytecodeInterpreterProfiling.hpp ! src/hotspot/share/interpreter/bytecodeStream.hpp ! src/hotspot/share/interpreter/bytecodeTracer.hpp ! src/hotspot/share/interpreter/bytecodes.hpp ! src/hotspot/share/interpreter/cppInterpreter.hpp ! src/hotspot/share/interpreter/cppInterpreterGenerator.hpp ! src/hotspot/share/interpreter/interp_masm.hpp ! src/hotspot/share/interpreter/interpreter.hpp ! src/hotspot/share/interpreter/interpreterRuntime.hpp ! src/hotspot/share/interpreter/invocationCounter.hpp ! src/hotspot/share/interpreter/linkResolver.hpp ! src/hotspot/share/interpreter/oopMapCache.hpp ! src/hotspot/share/interpreter/rewriter.hpp ! src/hotspot/share/interpreter/templateInterpreter.hpp ! src/hotspot/share/interpreter/templateInterpreterGenerator.hpp ! src/hotspot/share/interpreter/templateTable.hpp ! src/hotspot/share/jfr/dcmd/jfrDcmds.hpp ! src/hotspot/share/jfr/instrumentation/jfrEventClassTransformer.hpp ! src/hotspot/share/jfr/instrumentation/jfrJvmtiAgent.hpp ! src/hotspot/share/jfr/jfr.hpp ! src/hotspot/share/jfr/jfrEvents.hpp ! src/hotspot/share/jfr/jni/jfrGetAllEventClasses.hpp ! src/hotspot/share/jfr/jni/jfrJavaCall.hpp ! src/hotspot/share/jfr/jni/jfrJavaSupport.hpp ! src/hotspot/share/jfr/jni/jfrJniMethod.hpp ! src/hotspot/share/jfr/jni/jfrJniMethodRegistration.hpp ! src/hotspot/share/jfr/jni/jfrUpcalls.hpp ! src/hotspot/share/jfr/leakprofiler/chains/bfsClosure.hpp ! src/hotspot/share/jfr/leakprofiler/chains/bitset.hpp ! src/hotspot/share/jfr/leakprofiler/chains/dfsClosure.hpp ! src/hotspot/share/jfr/leakprofiler/chains/edge.hpp ! src/hotspot/share/jfr/leakprofiler/chains/edgeQueue.hpp ! src/hotspot/share/jfr/leakprofiler/chains/edgeStore.hpp ! src/hotspot/share/jfr/leakprofiler/chains/edgeUtils.hpp ! src/hotspot/share/jfr/leakprofiler/chains/objectSampleMarker.hpp ! src/hotspot/share/jfr/leakprofiler/chains/rootSetClosure.hpp ! src/hotspot/share/jfr/leakprofiler/checkpoint/objectSampleCheckpoint.hpp ! src/hotspot/share/jfr/leakprofiler/checkpoint/objectSampleDescription.hpp ! src/hotspot/share/jfr/leakprofiler/checkpoint/objectSampleWriter.hpp ! src/hotspot/share/jfr/leakprofiler/checkpoint/rootResolver.hpp ! src/hotspot/share/jfr/leakprofiler/emitEventOperation.hpp ! src/hotspot/share/jfr/leakprofiler/leakProfiler.hpp ! src/hotspot/share/jfr/leakprofiler/sampling/objectSample.hpp ! src/hotspot/share/jfr/leakprofiler/sampling/objectSampler.hpp ! src/hotspot/share/jfr/leakprofiler/sampling/sampleList.hpp ! src/hotspot/share/jfr/leakprofiler/sampling/samplePriorityQueue.hpp ! src/hotspot/share/jfr/leakprofiler/startOperation.hpp ! src/hotspot/share/jfr/leakprofiler/stopOperation.hpp ! src/hotspot/share/jfr/leakprofiler/utilities/granularTimer.hpp ! src/hotspot/share/jfr/leakprofiler/utilities/rootType.hpp ! src/hotspot/share/jfr/leakprofiler/utilities/saveRestore.hpp ! src/hotspot/share/jfr/leakprofiler/utilities/unifiedOop.hpp ! src/hotspot/share/jfr/metadata/jfrSerializer.hpp ! src/hotspot/share/jfr/periodic/jfrModuleEvent.hpp ! src/hotspot/share/jfr/periodic/jfrNetworkUtilization.hpp ! src/hotspot/share/jfr/periodic/jfrOSInterface.hpp ! src/hotspot/share/jfr/periodic/jfrThreadCPULoadEvent.hpp ! src/hotspot/share/jfr/periodic/jfrThreadDumpEvent.hpp ! src/hotspot/share/jfr/periodic/sampling/jfrCallTrace.hpp ! src/hotspot/share/jfr/periodic/sampling/jfrThreadSampler.hpp ! src/hotspot/share/jfr/recorder/checkpoint/jfrCheckpointBlob.hpp ! src/hotspot/share/jfr/recorder/checkpoint/jfrCheckpointManager.hpp ! src/hotspot/share/jfr/recorder/checkpoint/jfrCheckpointWriter.hpp ! src/hotspot/share/jfr/recorder/checkpoint/jfrMetadataEvent.hpp ! src/hotspot/share/jfr/recorder/checkpoint/types/jfrThreadGroup.hpp ! src/hotspot/share/jfr/recorder/checkpoint/types/jfrThreadState.hpp ! src/hotspot/share/jfr/recorder/checkpoint/types/jfrType.hpp ! src/hotspot/share/jfr/recorder/checkpoint/types/jfrTypeManager.hpp ! src/hotspot/share/jfr/recorder/checkpoint/types/jfrTypeSet.hpp ! src/hotspot/share/jfr/recorder/checkpoint/types/jfrTypeSetUtils.hpp ! src/hotspot/share/jfr/recorder/checkpoint/types/jfrTypeSetWriter.hpp ! src/hotspot/share/jfr/recorder/checkpoint/types/traceid/jfrTraceId.hpp ! src/hotspot/share/jfr/recorder/checkpoint/types/traceid/jfrTraceId.inline.hpp ! src/hotspot/share/jfr/recorder/checkpoint/types/traceid/jfrTraceIdBits.inline.hpp ! src/hotspot/share/jfr/recorder/checkpoint/types/traceid/jfrTraceIdEpoch.hpp ! src/hotspot/share/jfr/recorder/checkpoint/types/traceid/jfrTraceIdMacros.hpp ! src/hotspot/share/jfr/recorder/jfrEventSetting.hpp ! src/hotspot/share/jfr/recorder/jfrEventSetting.inline.hpp ! src/hotspot/share/jfr/recorder/jfrRecorder.hpp ! src/hotspot/share/jfr/recorder/repository/jfrChunkRotation.hpp ! src/hotspot/share/jfr/recorder/repository/jfrChunkState.hpp ! src/hotspot/share/jfr/recorder/repository/jfrChunkWriter.hpp ! src/hotspot/share/jfr/recorder/repository/jfrEmergencyDump.hpp ! src/hotspot/share/jfr/recorder/repository/jfrRepository.hpp ! src/hotspot/share/jfr/recorder/service/jfrEvent.hpp ! src/hotspot/share/jfr/recorder/service/jfrMemorySizer.hpp ! src/hotspot/share/jfr/recorder/service/jfrOptionSet.hpp ! src/hotspot/share/jfr/recorder/service/jfrPostBox.hpp ! src/hotspot/share/jfr/recorder/service/jfrRecorderService.hpp ! src/hotspot/share/jfr/recorder/service/jfrRecorderThread.hpp ! src/hotspot/share/jfr/recorder/stacktrace/jfrStackTraceRepository.hpp ! src/hotspot/share/jfr/recorder/storage/jfrBuffer.hpp ! src/hotspot/share/jfr/recorder/storage/jfrMemorySpace.hpp ! src/hotspot/share/jfr/recorder/storage/jfrMemorySpace.inline.hpp ! src/hotspot/share/jfr/recorder/storage/jfrMemorySpaceRetrieval.hpp ! src/hotspot/share/jfr/recorder/storage/jfrStorage.hpp ! src/hotspot/share/jfr/recorder/storage/jfrStorageControl.hpp ! src/hotspot/share/jfr/recorder/storage/jfrStorageUtils.hpp ! src/hotspot/share/jfr/recorder/storage/jfrStorageUtils.inline.hpp ! src/hotspot/share/jfr/recorder/storage/jfrVirtualMemory.hpp ! src/hotspot/share/jfr/recorder/stringpool/jfrStringPool.hpp ! src/hotspot/share/jfr/recorder/stringpool/jfrStringPoolBuffer.hpp ! src/hotspot/share/jfr/recorder/stringpool/jfrStringPoolWriter.hpp ! src/hotspot/share/jfr/support/jfrAllocationTracer.hpp ! src/hotspot/share/jfr/support/jfrEventClass.hpp ! src/hotspot/share/jfr/support/jfrFlush.hpp ! src/hotspot/share/jfr/support/jfrIntrinsics.hpp ! src/hotspot/share/jfr/support/jfrKlassExtension.hpp ! src/hotspot/share/jfr/support/jfrStackTraceMark.hpp ! src/hotspot/share/jfr/support/jfrThreadExtension.hpp ! src/hotspot/share/jfr/support/jfrThreadId.hpp ! src/hotspot/share/jfr/support/jfrThreadLocal.hpp ! src/hotspot/share/jfr/support/jfrTraceIdExtension.hpp ! src/hotspot/share/jfr/utilities/jfrAllocation.hpp ! src/hotspot/share/jfr/utilities/jfrBigEndian.hpp ! src/hotspot/share/jfr/utilities/jfrDoublyLinkedList.hpp ! src/hotspot/share/jfr/utilities/jfrHashtable.hpp ! src/hotspot/share/jfr/utilities/jfrIterator.hpp ! src/hotspot/share/jfr/utilities/jfrJavaLog.hpp ! src/hotspot/share/jfr/utilities/jfrLogTagSets.hpp ! src/hotspot/share/jfr/utilities/jfrRefCountPointer.hpp ! src/hotspot/share/jfr/utilities/jfrResourceManager.hpp ! src/hotspot/share/jfr/utilities/jfrSpinlockHelper.hpp ! src/hotspot/share/jfr/utilities/jfrTime.hpp ! src/hotspot/share/jfr/utilities/jfrTimeConverter.hpp ! src/hotspot/share/jfr/utilities/jfrTryLock.hpp ! src/hotspot/share/jfr/utilities/jfrTypes.hpp ! src/hotspot/share/jfr/writers/jfrBigEndianWriter.hpp ! src/hotspot/share/jfr/writers/jfrEncoders.hpp ! src/hotspot/share/jfr/writers/jfrEncoding.hpp ! src/hotspot/share/jfr/writers/jfrEventWriterHost.hpp ! src/hotspot/share/jfr/writers/jfrEventWriterHost.inline.hpp ! src/hotspot/share/jfr/writers/jfrJavaEventWriter.hpp ! src/hotspot/share/jfr/writers/jfrMemoryWriterHost.hpp ! src/hotspot/share/jfr/writers/jfrMemoryWriterHost.inline.hpp ! src/hotspot/share/jfr/writers/jfrNativeEventWriter.hpp ! src/hotspot/share/jfr/writers/jfrPosition.hpp ! src/hotspot/share/jfr/writers/jfrPosition.inline.hpp ! src/hotspot/share/jfr/writers/jfrStorageAdapter.hpp ! src/hotspot/share/jfr/writers/jfrStorageHost.hpp ! src/hotspot/share/jfr/writers/jfrStorageHost.inline.hpp ! src/hotspot/share/jfr/writers/jfrStreamWriterHost.hpp ! src/hotspot/share/jfr/writers/jfrStreamWriterHost.inline.hpp ! src/hotspot/share/jfr/writers/jfrWriterHost.hpp ! src/hotspot/share/jfr/writers/jfrWriterHost.inline.hpp ! src/hotspot/share/jvmci/compilerRuntime.hpp ! src/hotspot/share/jvmci/jvmciCodeInstaller.hpp ! src/hotspot/share/jvmci/jvmciCompiler.hpp ! src/hotspot/share/jvmci/jvmciCompilerToVM.hpp ! src/hotspot/share/jvmci/jvmciEnv.hpp ! src/hotspot/share/jvmci/jvmciJavaClasses.hpp ! src/hotspot/share/jvmci/jvmciRuntime.hpp ! src/hotspot/share/jvmci/jvmci_globals.hpp ! src/hotspot/share/jvmci/systemDictionary_jvmci.hpp ! src/hotspot/share/jvmci/vmStructs_compiler_runtime.hpp ! src/hotspot/share/jvmci/vmStructs_jvmci.hpp ! src/hotspot/share/jvmci/vmSymbols_jvmci.hpp ! src/hotspot/share/libadt/dict.hpp ! src/hotspot/share/libadt/set.hpp ! src/hotspot/share/libadt/vectset.hpp ! src/hotspot/share/logging/log.hpp ! src/hotspot/share/logging/logConfiguration.hpp ! src/hotspot/share/logging/logDecorations.hpp ! src/hotspot/share/logging/logDecorators.hpp ! src/hotspot/share/logging/logDiagnosticCommand.hpp ! src/hotspot/share/logging/logFileOutput.hpp ! src/hotspot/share/logging/logFileStreamOutput.hpp ! src/hotspot/share/logging/logHandle.hpp ! src/hotspot/share/logging/logLevel.hpp ! src/hotspot/share/logging/logMessage.hpp ! src/hotspot/share/logging/logMessageBuffer.hpp ! src/hotspot/share/logging/logOutput.hpp ! src/hotspot/share/logging/logOutputList.hpp ! src/hotspot/share/logging/logPrefix.hpp ! src/hotspot/share/logging/logSelection.hpp ! src/hotspot/share/logging/logSelectionList.hpp ! src/hotspot/share/logging/logStream.hpp ! src/hotspot/share/logging/logTag.hpp ! src/hotspot/share/logging/logTagSet.hpp ! src/hotspot/share/logging/logTagSetDescriptions.hpp ! src/hotspot/share/logging/logTag_ext.hpp ! src/hotspot/share/memory/allocation.hpp ! src/hotspot/share/memory/allocation.inline.hpp ! src/hotspot/share/memory/arena.hpp ! src/hotspot/share/memory/binaryTreeDictionary.hpp ! src/hotspot/share/memory/binaryTreeDictionary.inline.hpp ! src/hotspot/share/memory/filemap.hpp ! src/hotspot/share/memory/freeList.hpp ! src/hotspot/share/memory/guardedMemory.hpp ! src/hotspot/share/memory/heap.hpp ! src/hotspot/share/memory/heapInspection.hpp ! src/hotspot/share/memory/heapShared.hpp ! src/hotspot/share/memory/heapShared.inline.hpp ! src/hotspot/share/memory/iterator.hpp ! src/hotspot/share/memory/iterator.inline.hpp ! src/hotspot/share/memory/memRegion.hpp ! src/hotspot/share/memory/metadataFactory.hpp ! src/hotspot/share/memory/metaspace.hpp ! src/hotspot/share/memory/metaspace/blockFreelist.hpp ! src/hotspot/share/memory/metaspace/chunkManager.hpp ! src/hotspot/share/memory/metaspace/metaDebug.hpp ! src/hotspot/share/memory/metaspace/metabase.hpp ! src/hotspot/share/memory/metaspace/metablock.hpp ! src/hotspot/share/memory/metaspace/metachunk.hpp ! src/hotspot/share/memory/metaspace/metaspaceCommon.hpp ! src/hotspot/share/memory/metaspace/metaspaceDCmd.hpp ! src/hotspot/share/memory/metaspace/metaspaceStatistics.hpp ! src/hotspot/share/memory/metaspace/occupancyMap.hpp ! src/hotspot/share/memory/metaspace/printCLDMetaspaceInfoClosure.hpp ! src/hotspot/share/memory/metaspace/printMetaspaceInfoKlassClosure.hpp ! src/hotspot/share/memory/metaspace/smallBlocks.hpp ! src/hotspot/share/memory/metaspace/spaceManager.hpp ! src/hotspot/share/memory/metaspace/virtualSpaceList.hpp ! src/hotspot/share/memory/metaspace/virtualSpaceNode.hpp ! src/hotspot/share/memory/metaspaceChunkFreeListSummary.hpp ! src/hotspot/share/memory/metaspaceClosure.hpp ! src/hotspot/share/memory/metaspaceCounters.hpp ! src/hotspot/share/memory/metaspaceGCThresholdUpdater.hpp ! src/hotspot/share/memory/metaspaceShared.hpp ! src/hotspot/share/memory/metaspaceShared.inline.hpp ! src/hotspot/share/memory/metaspaceTracer.hpp ! src/hotspot/share/memory/oopFactory.hpp ! src/hotspot/share/memory/padded.hpp ! src/hotspot/share/memory/padded.inline.hpp ! src/hotspot/share/memory/referenceType.hpp ! src/hotspot/share/memory/resourceArea.hpp ! src/hotspot/share/memory/resourceArea.inline.hpp ! src/hotspot/share/memory/universe.hpp ! src/hotspot/share/memory/virtualspace.hpp ! src/hotspot/share/metaprogramming/conditional.hpp ! src/hotspot/share/metaprogramming/decay.hpp ! src/hotspot/share/metaprogramming/enableIf.hpp ! src/hotspot/share/metaprogramming/integralConstant.hpp ! src/hotspot/share/metaprogramming/isConst.hpp ! src/hotspot/share/metaprogramming/isFloatingPoint.hpp ! src/hotspot/share/metaprogramming/isIntegral.hpp ! src/hotspot/share/metaprogramming/isPointer.hpp ! src/hotspot/share/metaprogramming/isRegisteredEnum.hpp ! src/hotspot/share/metaprogramming/isSame.hpp ! src/hotspot/share/metaprogramming/isSigned.hpp ! src/hotspot/share/metaprogramming/isVolatile.hpp ! src/hotspot/share/metaprogramming/primitiveConversions.hpp ! src/hotspot/share/metaprogramming/removeCV.hpp ! src/hotspot/share/metaprogramming/removePointer.hpp ! src/hotspot/share/metaprogramming/removeReference.hpp ! src/hotspot/share/oops/accessBackend.inline.hpp ! src/hotspot/share/oops/annotations.hpp ! src/hotspot/share/oops/array.hpp ! src/hotspot/share/oops/array.inline.hpp ! src/hotspot/share/oops/arrayKlass.hpp ! src/hotspot/share/oops/arrayKlass.inline.hpp ! src/hotspot/share/oops/arrayOop.hpp ! src/hotspot/share/oops/compiledICHolder.hpp ! src/hotspot/share/oops/constMethod.hpp ! src/hotspot/share/oops/constantPool.hpp ! src/hotspot/share/oops/constantPool.inline.hpp ! src/hotspot/share/oops/cpCache.hpp ! src/hotspot/share/oops/cpCache.inline.hpp ! src/hotspot/share/oops/fieldInfo.hpp ! src/hotspot/share/oops/fieldStreams.hpp ! src/hotspot/share/oops/generateOopMap.hpp ! src/hotspot/share/oops/instanceClassLoaderKlass.hpp ! src/hotspot/share/oops/instanceClassLoaderKlass.inline.hpp ! src/hotspot/share/oops/instanceKlass.hpp ! src/hotspot/share/oops/instanceKlass.inline.hpp ! src/hotspot/share/oops/instanceMirrorKlass.hpp ! src/hotspot/share/oops/instanceMirrorKlass.inline.hpp ! src/hotspot/share/oops/instanceOop.hpp ! src/hotspot/share/oops/instanceRefKlass.hpp ! src/hotspot/share/oops/instanceRefKlass.inline.hpp ! src/hotspot/share/oops/klass.hpp ! src/hotspot/share/oops/klass.inline.hpp ! src/hotspot/share/oops/klassVtable.hpp ! src/hotspot/share/oops/markOop.hpp ! src/hotspot/share/oops/markOop.inline.hpp ! src/hotspot/share/oops/metadata.hpp ! src/hotspot/share/oops/method.hpp ! src/hotspot/share/oops/method.inline.hpp ! src/hotspot/share/oops/methodCounters.hpp ! src/hotspot/share/oops/methodData.hpp ! src/hotspot/share/oops/methodData.inline.hpp ! src/hotspot/share/oops/objArrayKlass.hpp ! src/hotspot/share/oops/objArrayKlass.inline.hpp ! src/hotspot/share/oops/objArrayOop.hpp ! src/hotspot/share/oops/objArrayOop.inline.hpp ! src/hotspot/share/oops/oop.hpp ! src/hotspot/share/oops/oop.inline.hpp ! src/hotspot/share/oops/oopHandle.hpp ! src/hotspot/share/oops/oopHandle.inline.hpp ! src/hotspot/share/oops/oopsHierarchy.hpp ! src/hotspot/share/oops/reflectionAccessorImplKlassHelper.hpp ! src/hotspot/share/oops/symbol.hpp ! src/hotspot/share/oops/typeArrayKlass.hpp ! src/hotspot/share/oops/typeArrayKlass.inline.hpp ! src/hotspot/share/oops/typeArrayOop.hpp ! src/hotspot/share/oops/typeArrayOop.inline.hpp ! src/hotspot/share/oops/verifyOopClosure.hpp ! src/hotspot/share/oops/weakHandle.hpp ! src/hotspot/share/oops/weakHandle.inline.hpp ! src/hotspot/share/opto/ad.hpp ! src/hotspot/share/opto/addnode.hpp ! src/hotspot/share/opto/adlcVMDeps.hpp ! src/hotspot/share/opto/arraycopynode.hpp ! src/hotspot/share/opto/block.hpp ! src/hotspot/share/opto/c2_globals.hpp ! src/hotspot/share/opto/c2compiler.hpp ! src/hotspot/share/opto/callGenerator.hpp ! src/hotspot/share/opto/callnode.hpp ! src/hotspot/share/opto/castnode.hpp ! src/hotspot/share/opto/cfgnode.hpp ! src/hotspot/share/opto/chaitin.hpp ! src/hotspot/share/opto/coalesce.hpp ! src/hotspot/share/opto/compile.hpp ! src/hotspot/share/opto/connode.hpp ! src/hotspot/share/opto/convertnode.hpp ! src/hotspot/share/opto/countbitsnode.hpp ! src/hotspot/share/opto/divnode.hpp ! src/hotspot/share/opto/escape.hpp ! src/hotspot/share/opto/graphKit.hpp ! src/hotspot/share/opto/idealGraphPrinter.hpp ! src/hotspot/share/opto/idealKit.hpp ! src/hotspot/share/opto/indexSet.hpp ! src/hotspot/share/opto/intrinsicnode.hpp ! src/hotspot/share/opto/live.hpp ! src/hotspot/share/opto/locknode.hpp ! src/hotspot/share/opto/loopnode.hpp ! src/hotspot/share/opto/machnode.hpp ! src/hotspot/share/opto/macro.hpp ! src/hotspot/share/opto/matcher.hpp ! src/hotspot/share/opto/mathexactnode.hpp ! src/hotspot/share/opto/memnode.hpp ! src/hotspot/share/opto/movenode.hpp ! src/hotspot/share/opto/mulnode.hpp ! src/hotspot/share/opto/multnode.hpp ! src/hotspot/share/opto/narrowptrnode.hpp ! src/hotspot/share/opto/node.hpp ! src/hotspot/share/opto/opaquenode.hpp ! src/hotspot/share/opto/opcodes.hpp ! src/hotspot/share/opto/optoreg.hpp ! src/hotspot/share/opto/output.hpp ! src/hotspot/share/opto/parse.hpp ! src/hotspot/share/opto/phase.hpp ! src/hotspot/share/opto/phaseX.hpp ! src/hotspot/share/opto/phasetype.hpp ! src/hotspot/share/opto/regalloc.hpp ! src/hotspot/share/opto/regmask.hpp ! src/hotspot/share/opto/replacednodes.hpp ! src/hotspot/share/opto/rootnode.hpp ! src/hotspot/share/opto/runtime.hpp ! src/hotspot/share/opto/stringopts.hpp ! src/hotspot/share/opto/subnode.hpp ! src/hotspot/share/opto/superword.hpp ! src/hotspot/share/opto/type.hpp ! src/hotspot/share/opto/vectornode.hpp ! src/hotspot/share/prims/forte.hpp ! src/hotspot/share/prims/jniCheck.hpp ! src/hotspot/share/prims/jniExport.hpp ! src/hotspot/share/prims/jniFastGetField.hpp ! src/hotspot/share/prims/jvm_misc.hpp ! src/hotspot/share/prims/jvmtiAgentThread.hpp ! src/hotspot/share/prims/jvmtiClassFileReconstituter.hpp ! src/hotspot/share/prims/jvmtiCodeBlobEvents.hpp ! src/hotspot/share/prims/jvmtiEnter.inline.hpp ! src/hotspot/share/prims/jvmtiEnvBase.hpp ! src/hotspot/share/prims/jvmtiEnvThreadState.hpp ! src/hotspot/share/prims/jvmtiEventController.hpp ! src/hotspot/share/prims/jvmtiEventController.inline.hpp ! src/hotspot/share/prims/jvmtiExport.hpp ! src/hotspot/share/prims/jvmtiExtensions.hpp ! src/hotspot/share/prims/jvmtiGetLoadedClasses.hpp ! src/hotspot/share/prims/jvmtiImpl.hpp ! src/hotspot/share/prims/jvmtiManageCapabilities.hpp ! src/hotspot/share/prims/jvmtiRawMonitor.hpp ! src/hotspot/share/prims/jvmtiRedefineClasses.hpp ! src/hotspot/share/prims/jvmtiTagMap.hpp ! src/hotspot/share/prims/jvmtiThreadState.hpp ! src/hotspot/share/prims/jvmtiThreadState.inline.hpp ! src/hotspot/share/prims/jvmtiTrace.hpp ! src/hotspot/share/prims/jvmtiUtil.hpp ! src/hotspot/share/prims/methodComparator.hpp ! src/hotspot/share/prims/methodHandles.hpp ! src/hotspot/share/prims/nativeLookup.hpp ! src/hotspot/share/prims/resolvedMethodTable.hpp ! src/hotspot/share/prims/stackwalk.hpp ! src/hotspot/share/prims/unsafe.hpp ! src/hotspot/share/prims/wbtestmethods/parserTests.hpp ! src/hotspot/share/prims/whitebox.hpp ! src/hotspot/share/prims/whitebox.inline.hpp ! src/hotspot/share/runtime/arguments.hpp ! src/hotspot/share/runtime/atomic.hpp ! src/hotspot/share/runtime/basicLock.hpp ! src/hotspot/share/runtime/biasedLocking.hpp ! src/hotspot/share/runtime/compilationPolicy.hpp ! src/hotspot/share/runtime/deoptimization.hpp ! src/hotspot/share/runtime/extendedPC.hpp ! src/hotspot/share/runtime/fieldDescriptor.hpp ! src/hotspot/share/runtime/fieldDescriptor.inline.hpp ! src/hotspot/share/runtime/fieldType.hpp ! src/hotspot/share/runtime/flags/flagSetting.hpp ! src/hotspot/share/runtime/flags/jvmFlag.hpp ! src/hotspot/share/runtime/flags/jvmFlagConstraintList.hpp ! src/hotspot/share/runtime/flags/jvmFlagConstraintsCompiler.hpp ! src/hotspot/share/runtime/flags/jvmFlagConstraintsRuntime.hpp ! src/hotspot/share/runtime/flags/jvmFlagRangeList.hpp ! src/hotspot/share/runtime/flags/jvmFlagWriteableList.hpp ! src/hotspot/share/runtime/frame.hpp ! src/hotspot/share/runtime/frame.inline.hpp ! src/hotspot/share/runtime/globals.hpp ! src/hotspot/share/runtime/globals_ext.hpp ! src/hotspot/share/runtime/globals_extension.hpp ! src/hotspot/share/runtime/handles.hpp ! src/hotspot/share/runtime/handles.inline.hpp ! src/hotspot/share/runtime/handshake.hpp ! src/hotspot/share/runtime/icache.hpp ! src/hotspot/share/runtime/init.hpp ! src/hotspot/share/runtime/interfaceSupport.inline.hpp ! src/hotspot/share/runtime/java.hpp ! src/hotspot/share/runtime/javaCalls.hpp ! src/hotspot/share/runtime/javaFrameAnchor.hpp ! src/hotspot/share/runtime/jfieldIDWorkaround.hpp ! src/hotspot/share/runtime/jniHandles.hpp ! src/hotspot/share/runtime/jniHandles.inline.hpp ! src/hotspot/share/runtime/jniPeriodicChecker.hpp ! src/hotspot/share/runtime/memprofiler.hpp ! src/hotspot/share/runtime/monitorChunk.hpp ! src/hotspot/share/runtime/mutex.hpp ! src/hotspot/share/runtime/mutexLocker.hpp ! src/hotspot/share/runtime/objectMonitor.hpp ! src/hotspot/share/runtime/objectMonitor.inline.hpp ! src/hotspot/share/runtime/orderAccess.hpp ! src/hotspot/share/runtime/os.hpp ! src/hotspot/share/runtime/os.inline.hpp ! src/hotspot/share/runtime/osThread.hpp ! src/hotspot/share/runtime/os_ext.hpp ! src/hotspot/share/runtime/os_perf.hpp ! src/hotspot/share/runtime/park.hpp ! src/hotspot/share/runtime/perfData.hpp ! src/hotspot/share/runtime/perfData.inline.hpp ! src/hotspot/share/runtime/perfMemory.hpp ! src/hotspot/share/runtime/prefetch.hpp ! src/hotspot/share/runtime/prefetch.inline.hpp ! src/hotspot/share/runtime/reflection.hpp ! src/hotspot/share/runtime/reflectionUtils.hpp ! src/hotspot/share/runtime/registerMap.hpp ! src/hotspot/share/runtime/relocator.hpp ! src/hotspot/share/runtime/rframe.hpp ! src/hotspot/share/runtime/rtmLocking.hpp ! src/hotspot/share/runtime/safepoint.hpp ! src/hotspot/share/runtime/safepointMechanism.hpp ! src/hotspot/share/runtime/safepointMechanism.inline.hpp ! src/hotspot/share/runtime/safepointVerifiers.hpp ! src/hotspot/share/runtime/semaphore.hpp ! src/hotspot/share/runtime/semaphore.inline.hpp ! src/hotspot/share/runtime/serviceThread.hpp ! src/hotspot/share/runtime/sharedRuntime.hpp ! src/hotspot/share/runtime/sharedRuntimeMath.hpp ! src/hotspot/share/runtime/signature.hpp ! src/hotspot/share/runtime/stackValue.hpp ! src/hotspot/share/runtime/stackValueCollection.hpp ! src/hotspot/share/runtime/statSampler.hpp ! src/hotspot/share/runtime/stubCodeGenerator.hpp ! src/hotspot/share/runtime/stubRoutines.hpp ! src/hotspot/share/runtime/sweeper.hpp ! src/hotspot/share/runtime/synchronizer.hpp ! src/hotspot/share/runtime/task.hpp ! src/hotspot/share/runtime/thread.hpp ! src/hotspot/share/runtime/thread.inline.hpp ! src/hotspot/share/runtime/threadCritical.hpp ! src/hotspot/share/runtime/threadHeapSampler.hpp ! src/hotspot/share/runtime/threadLocalStorage.hpp ! src/hotspot/share/runtime/threadSMR.hpp ! src/hotspot/share/runtime/threadSMR.inline.hpp ! src/hotspot/share/runtime/threadStatisticalInfo.hpp ! src/hotspot/share/runtime/tieredThresholdPolicy.hpp ! src/hotspot/share/runtime/timer.hpp ! src/hotspot/share/runtime/timerTrace.hpp ! src/hotspot/share/runtime/unhandledOops.hpp ! src/hotspot/share/runtime/vframe.hpp ! src/hotspot/share/runtime/vframe.inline.hpp ! src/hotspot/share/runtime/vframeArray.hpp ! src/hotspot/share/runtime/vframe_hp.hpp ! src/hotspot/share/runtime/vmOperations.hpp ! src/hotspot/share/runtime/vmStructs.hpp ! src/hotspot/share/runtime/vmThread.hpp ! src/hotspot/share/runtime/vm_version.hpp ! src/hotspot/share/services/allocationSite.hpp ! src/hotspot/share/services/attachListener.hpp ! src/hotspot/share/services/classLoadingService.hpp ! src/hotspot/share/services/diagnosticArgument.hpp ! src/hotspot/share/services/diagnosticCommand.hpp ! src/hotspot/share/services/diagnosticFramework.hpp ! src/hotspot/share/services/dtraceAttacher.hpp ! src/hotspot/share/services/gcNotifier.hpp ! src/hotspot/share/services/heapDumper.hpp ! src/hotspot/share/services/lowMemoryDetector.hpp ! src/hotspot/share/services/mallocSiteTable.hpp ! src/hotspot/share/services/mallocTracker.hpp ! src/hotspot/share/services/mallocTracker.inline.hpp ! src/hotspot/share/services/management.hpp ! src/hotspot/share/services/memBaseline.hpp ! src/hotspot/share/services/memReporter.hpp ! src/hotspot/share/services/memTracker.hpp ! src/hotspot/share/services/memoryManager.hpp ! src/hotspot/share/services/memoryPool.hpp ! src/hotspot/share/services/memoryService.hpp ! src/hotspot/share/services/memoryUsage.hpp ! src/hotspot/share/services/nmtCommon.hpp ! src/hotspot/share/services/nmtDCmd.hpp ! src/hotspot/share/services/runtimeService.hpp ! src/hotspot/share/services/threadService.hpp ! src/hotspot/share/services/virtualMemoryTracker.hpp ! src/hotspot/share/services/writeableFlags.hpp ! src/hotspot/share/utilities/accessFlags.hpp ! src/hotspot/share/utilities/align.hpp ! src/hotspot/share/utilities/bitMap.hpp ! src/hotspot/share/utilities/bitMap.inline.hpp ! src/hotspot/share/utilities/breakpoint.hpp ! src/hotspot/share/utilities/bytes.hpp ! src/hotspot/share/utilities/chunkedList.hpp ! src/hotspot/share/utilities/compilerWarnings.hpp ! src/hotspot/share/utilities/concurrentHashTable.hpp ! src/hotspot/share/utilities/concurrentHashTable.inline.hpp ! src/hotspot/share/utilities/concurrentHashTableTasks.inline.hpp ! src/hotspot/share/utilities/constantTag.hpp ! src/hotspot/share/utilities/copy.hpp ! src/hotspot/share/utilities/count_trailing_zeros.hpp ! src/hotspot/share/utilities/debug.hpp ! src/hotspot/share/utilities/decoder.hpp ! src/hotspot/share/utilities/decoder_elf.hpp ! src/hotspot/share/utilities/defaultStream.hpp ! src/hotspot/share/utilities/dtrace.hpp ! src/hotspot/share/utilities/dtrace_disabled.hpp ! src/hotspot/share/utilities/elfFile.hpp ! src/hotspot/share/utilities/elfFuncDescTable.hpp ! src/hotspot/share/utilities/elfStringTable.hpp ! src/hotspot/share/utilities/elfSymbolTable.hpp ! src/hotspot/share/utilities/events.hpp ! src/hotspot/share/utilities/exceptions.hpp ! src/hotspot/share/utilities/fakeRttiSupport.hpp ! src/hotspot/share/utilities/formatBuffer.hpp ! src/hotspot/share/utilities/globalCounter.hpp ! src/hotspot/share/utilities/globalCounter.inline.hpp ! src/hotspot/share/utilities/globalDefinitions.hpp ! src/hotspot/share/utilities/globalDefinitions_gcc.hpp ! src/hotspot/share/utilities/globalDefinitions_solstudio.hpp ! src/hotspot/share/utilities/globalDefinitions_visCPP.hpp ! src/hotspot/share/utilities/globalDefinitions_xlc.hpp ! src/hotspot/share/utilities/growableArray.hpp ! src/hotspot/share/utilities/hashtable.hpp ! src/hotspot/share/utilities/hashtable.inline.hpp ! src/hotspot/share/utilities/histogram.hpp ! src/hotspot/share/utilities/intHisto.hpp ! src/hotspot/share/utilities/json.hpp ! src/hotspot/share/utilities/linkedlist.hpp ! src/hotspot/share/utilities/macros.hpp ! src/hotspot/share/utilities/nativeCallStack.hpp ! src/hotspot/share/utilities/numberSeq.hpp ! src/hotspot/share/utilities/ostream.hpp ! src/hotspot/share/utilities/pair.hpp ! src/hotspot/share/utilities/preserveException.hpp ! src/hotspot/share/utilities/quickSort.hpp ! src/hotspot/share/utilities/resourceHash.hpp ! src/hotspot/share/utilities/sizes.hpp ! src/hotspot/share/utilities/stack.hpp ! src/hotspot/share/utilities/stack.inline.hpp ! src/hotspot/share/utilities/stringUtils.hpp ! src/hotspot/share/utilities/ticks.hpp ! src/hotspot/share/utilities/utf8.hpp ! src/hotspot/share/utilities/vmError.hpp ! src/hotspot/share/utilities/xmlstream.hpp Changeset: 05c5c27b5a54 Author: jlaskey Date: 2019-01-09 18:17 -0400 URL: http://hg.openjdk.java.net/panama/dev/rev/05c5c27b5a54 8215493: String::indent inconsistency with blank lines Reviewed-by: rriggs, smarks ! src/java.base/share/classes/java/lang/String.java ! test/jdk/java/lang/String/Indent.java Changeset: 0740588d1f8a Author: jwilhelm Date: 2019-01-10 02:47 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/0740588d1f8a Added tag jdk-12+27 for changeset f15d443f9731 ! .hgtags Changeset: 3fc330702246 Author: tschatzl Date: 2019-01-10 12:14 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/3fc330702246 8216316: Tests fail due to too low specified TLAB size Reviewed-by: goetz, sangheki Contributed-by: goetz.lindenmaier at sap.com, thomas.schatzl at oracle.com ! test/hotspot/jtreg/compiler/interpreter/TestVerifyStackAfterDeopt.java ! test/jdk/jdk/jfr/event/gc/detailed/TestPromotionEventWithG1.java Changeset: ed36ff53642c Author: robm Date: 2019-01-10 07:54 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/ed36ff53642c 8214440: ldap over a TLS connection negotiate failed with "javax.net.ssl.SSLPeerUnverifiedException: hostname of the server '' does not match the hostname in the server's certificate" Reviewed-by: vtewari, xuelei ! src/java.naming/share/classes/com/sun/jndi/ldap/ext/StartTlsResponseImpl.java Changeset: fbab5d82f3d7 Author: jiangli Date: 2019-01-10 13:03 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/fbab5d82f3d7 8214827: Incorrect call ClassLoaders.toFileURL("jrt:/java.compiler") Summary: Use URL constructor for jrt URL in SystemDictionaryShared::get_shared_protection_domain(). Reviewed-by: ccheung, iklam, dholmes, coleenp ! src/hotspot/share/classfile/systemDictionaryShared.cpp ! test/hotspot/jtreg/runtime/appcds/ProtectionDomain.java ! test/hotspot/jtreg/runtime/appcds/test-classes/JimageClassProtDomain.java ! test/hotspot/jtreg/runtime/appcds/test-classes/ProtDomain.java ! test/hotspot/jtreg/runtime/appcds/test-classes/ProtDomainB.java Changeset: 10621b0e8e38 Author: shade Date: 2019-01-09 15:53 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/10621b0e8e38 8215724: Epsilon: ArrayStoreExceptionTest.java fails; missing arraycopy check Reviewed-by: eosterlund, lkorinth ! src/hotspot/share/gc/shared/barrierSet.hpp + src/hotspot/share/gc/shared/barrierSet.inline.hpp ! src/hotspot/share/oops/access.inline.hpp + test/hotspot/jtreg/gc/epsilon/TestArraycopyCheckcast.java Changeset: 2e1fd6414c4b Author: jwilhelm Date: 2019-01-10 21:52 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/2e1fd6414c4b Merge ! .hgtags ! test/jdk/java/lang/String/Indent.java Changeset: e832101ff63c Author: mbaesken Date: 2019-01-09 14:46 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/e832101ff63c 8216355: missing NULL checks in libnet in interface iteration and potential resource leak in getMacAddress Reviewed-by: clanger, rwestberg ! src/java.base/unix/native/libnet/Inet6AddressImpl.c ! src/java.base/unix/native/libnet/NetworkInterface.c Changeset: d87633b62f1f Author: thartmann Date: 2019-01-11 09:59 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/d87633b62f1f 8216480: Typo in test/hotspot/jtreg/compiler/graalunit/README.md Summary: Removed -vmoptions: Reviewed-by: epavlova ! test/hotspot/jtreg/compiler/graalunit/README.md Changeset: 47bc06170313 Author: rehn Date: 2019-01-11 10:58 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/47bc06170313 8214271: Fast primitive to wake many threads Reviewed-by: dholmes, dcubed + src/hotspot/os/linux/waitBarrier_linux.cpp + src/hotspot/os/linux/waitBarrier_linux.hpp + src/hotspot/share/utilities/waitBarrier.hpp + src/hotspot/share/utilities/waitBarrier_generic.cpp + src/hotspot/share/utilities/waitBarrier_generic.hpp + test/hotspot/gtest/utilities/test_waitBarrier.cpp Changeset: 61a385765c9b Author: ysuenaga Date: 2019-01-11 23:32 +0900 URL: http://hg.openjdk.java.net/panama/dev/rev/61a385765c9b 8216486: Possibility of integer overflow in JfrThreadSampler::run() Reviewed-by: rehn, sgehwolf ! src/hotspot/share/jfr/periodic/sampling/jfrThreadSampler.cpp Changeset: bd8df96decba Author: dfuchs Date: 2019-01-11 14:48 +0000 URL: http://hg.openjdk.java.net/panama/dev/rev/bd8df96decba 8216498: Confusing and unneeded wrapping of SSLHandshakeException Summary: [httpclient] Avoid wrapping SSLHandshakeException in plain IOException Reviewed-by: chegar ! src/java.net.http/share/classes/jdk/internal/net/http/HttpClientImpl.java ! src/java.net.http/share/classes/jdk/internal/net/http/common/Utils.java ! test/jdk/java/net/httpclient/ShortResponseBody.java Changeset: 5170dc2bcf64 Author: mbalao Date: 2018-11-29 13:36 -0300 URL: http://hg.openjdk.java.net/panama/dev/rev/5170dc2bcf64 6913047: Long term memory leak when using PKCS11 and JCE exceeds 32 bit process address space Summary: Extract cryptographic keys within NSS PKCS11 software tokens for memory management purposes. Reviewed-by: valeriep ! src/java.base/share/lib/security/default.policy ! src/jdk.crypto.cryptoki/share/classes/sun/security/pkcs11/P11Cipher.java ! src/jdk.crypto.cryptoki/share/classes/sun/security/pkcs11/P11DHKeyFactory.java ! src/jdk.crypto.cryptoki/share/classes/sun/security/pkcs11/P11DSAKeyFactory.java ! src/jdk.crypto.cryptoki/share/classes/sun/security/pkcs11/P11Digest.java ! src/jdk.crypto.cryptoki/share/classes/sun/security/pkcs11/P11ECDHKeyAgreement.java ! src/jdk.crypto.cryptoki/share/classes/sun/security/pkcs11/P11ECKeyFactory.java ! src/jdk.crypto.cryptoki/share/classes/sun/security/pkcs11/P11Key.java ! src/jdk.crypto.cryptoki/share/classes/sun/security/pkcs11/P11KeyAgreement.java ! src/jdk.crypto.cryptoki/share/classes/sun/security/pkcs11/P11KeyStore.java ! src/jdk.crypto.cryptoki/share/classes/sun/security/pkcs11/P11Mac.java ! src/jdk.crypto.cryptoki/share/classes/sun/security/pkcs11/P11RSACipher.java ! src/jdk.crypto.cryptoki/share/classes/sun/security/pkcs11/P11RSAKeyFactory.java ! src/jdk.crypto.cryptoki/share/classes/sun/security/pkcs11/P11SecretKeyFactory.java ! src/jdk.crypto.cryptoki/share/classes/sun/security/pkcs11/P11Signature.java ! src/jdk.crypto.cryptoki/share/classes/sun/security/pkcs11/P11TlsKeyMaterialGenerator.java ! src/jdk.crypto.cryptoki/share/classes/sun/security/pkcs11/P11TlsMasterSecretGenerator.java ! src/jdk.crypto.cryptoki/share/classes/sun/security/pkcs11/P11TlsPrfGenerator.java ! src/jdk.crypto.cryptoki/share/classes/sun/security/pkcs11/wrapper/PKCS11.java ! src/jdk.crypto.cryptoki/share/native/libj2pkcs11/p11_keymgmt.c ! src/jdk.crypto.cryptoki/share/native/libj2pkcs11/pkcs11t.h ! src/jdk.crypto.cryptoki/share/native/libj2pkcs11/pkcs11wrapper.h Changeset: e348b0160d61 Author: bpb Date: 2019-01-11 08:20 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/e348b0160d61 8216172: File.renameTo(File dest) should check for NPE at the very beginning Reviewed-by: lancea ! src/java.base/share/classes/java/io/File.java Changeset: 754312b616de Author: darcy Date: 2019-01-11 09:57 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/754312b616de 8208371: Provided supported mechanims to create a ModuleElement for an unnamed module Reviewed-by: jjg ! src/java.compiler/share/classes/javax/lang/model/util/Elements.java ! test/langtools/tools/javac/processing/model/element/TestModuleElementNames.java Changeset: fbc921683f02 Author: sgehwolf Date: 2019-01-11 13:34 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/fbc921683f02 8216559: [JFR] Native libraries not correctly parsed from /proc/self/maps Summary: Use %7s for the dev scan format as major:minor may be up to that length Reviewed-by: mgronlun, jwilhelm ! src/hotspot/os/linux/os_linux.cpp ! test/jdk/jdk/jfr/event/runtime/TestNativeLibrariesEvent.java Changeset: b7dca420fa0c Author: ccheung Date: 2019-01-11 14:05 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/b7dca420fa0c 8216184: CDS/appCDS tests failed on Windows due to long path to a classlist file Summary: use os::open() instead of fopen() Reviewed-by: iklam, dholmes ! src/hotspot/share/classfile/classListParser.cpp ! src/hotspot/share/memory/metaspaceShared.cpp + test/hotspot/jtreg/runtime/appcds/LongClassListPath.java ! test/hotspot/jtreg/runtime/appcds/jigsaw/classpathtests/DummyClassesInBootClassPath.java Changeset: 2969ff55c29b Author: naoto Date: 2019-01-11 14:24 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/2969ff55c29b 8216140: Correct UnicodeDecoder U+FFFE handling Reviewed-by: rriggs ! src/java.base/share/classes/sun/nio/cs/UnicodeDecoder.java + test/jdk/sun/nio/cs/TestUnicodeReversedBOM.java Changeset: 5d7e4d832868 Author: shade Date: 2019-01-12 13:33 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/5d7e4d832868 8216589: s390x build failures after JDK-8216167 (Update include guards to reflect correct directories) Reviewed-by: dholmes ! src/hotspot/cpu/s390/codeBuffer_s390.hpp ! src/hotspot/os_cpu/linux_s390/globals_linux_s390.hpp Changeset: 424e4908b4b8 Author: redestad Date: 2019-01-13 12:50 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/424e4908b4b8 8216424: Remove TimeLivenessAnalysis Reviewed-by: kvn, thartmann ! src/hotspot/share/compiler/methodLiveness.cpp ! src/hotspot/share/compiler/methodLiveness.hpp ! src/hotspot/share/runtime/globals.hpp ! src/hotspot/share/runtime/java.cpp Changeset: febc37adfe80 Author: pliden Date: 2019-01-13 17:33 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/febc37adfe80 8216595: Fix broken builds after JDK-8216424 Reviewed-by: redestad, alanb ! src/hotspot/share/compiler/methodLiveness.cpp Changeset: 57d8566a2732 Author: dholmes Date: 2019-01-13 16:54 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/57d8566a2732 8214816: os::read() should not transition to _thread_blocked with safepoint check on Solaris Reviewed-by: jiangli, mgronlun ! src/hotspot/os/aix/os_aix.cpp ! src/hotspot/os/aix/os_aix.inline.hpp ! src/hotspot/os/bsd/os_bsd.cpp ! src/hotspot/os/bsd/os_bsd.inline.hpp ! src/hotspot/os/linux/os_linux.cpp ! src/hotspot/os/linux/os_linux.inline.hpp ! src/hotspot/os/posix/os_posix.cpp ! src/hotspot/os/solaris/os_solaris.cpp ! src/hotspot/os/windows/os_windows.cpp ! src/hotspot/os/windows/os_windows.inline.hpp ! src/hotspot/share/compiler/directivesParser.cpp ! src/hotspot/share/runtime/arguments.cpp ! src/hotspot/share/runtime/os.hpp ! src/hotspot/share/runtime/os.inline.hpp Changeset: 0b6d6db878b6 Author: jgeorge Date: 2019-01-14 09:30 +0530 URL: http://hg.openjdk.java.net/panama/dev/rev/0b6d6db878b6 8215544: SA: Modify ClhsdbLauncher to add sudo privileges to enable MacOS tests on Mach5 Summary: Check if 'sudo' privileges can be added for executing macOS tests, and if so, add these privileges before executing the tests Reviewed-by: jcbeyler, phh, sballal ! test/hotspot/jtreg/serviceability/sa/ClhsdbFindPC.java ! test/hotspot/jtreg/serviceability/sa/ClhsdbJstack.java ! test/hotspot/jtreg/serviceability/sa/ClhsdbLauncher.java + test/lib/jdk/test/lib/SA/SATestUtils.java Changeset: c02949731190 Author: mdoerr Date: 2019-01-14 09:26 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/c02949731190 8216560: gtest build broken on PPC64 and aarch64 Reviewed-by: shade ! test/hotspot/gtest/threadHelper.inline.hpp Changeset: 26a53519c82f Author: thartmann Date: 2019-01-14 09:48 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/26a53519c82f 8213249: compiler/graalunit/HotspotTest.java failed in ExplicitExceptionTest Summary: Added -XX:-OmitStackTraceInFastThrow to test flags to avoid empty exception message. Reviewed-by: epavlova, iignatyev, dlong, kvn ! test/hotspot/jtreg/ProblemList-graal.txt ! test/hotspot/jtreg/compiler/graalunit/common/GraalUnitTestLauncher.java Changeset: f3a42c440fcb Author: pmuthuswamy Date: 2019-01-14 15:09 +0530 URL: http://hg.openjdk.java.net/panama/dev/rev/f3a42c440fcb 8199892: Missing landmarks when generating docs using html sources Reviewed-by: jjg ! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/DocFilesHandlerImpl.java + test/langtools/jdk/javadoc/doclet/testHtmlLandmarkRegions/TestHtmlLandmarkRegions.java - test/langtools/jdk/javadoc/doclet/testHtmlLandmarkRegions/TestHtmlLankmarkRegions.java Changeset: 14d078fd74cb Author: dfuchs Date: 2019-01-14 10:46 +0000 URL: http://hg.openjdk.java.net/panama/dev/rev/14d078fd74cb 8216478: Cleanup HttpResponseImpl back reference to HttpConnection Summary: Retain a reference to Exchange and HttpConnection only when necessary, i.e. for WebSocket initial connection. Reviewed-by: chegar ! src/java.net.http/share/classes/jdk/internal/net/http/HttpResponseImpl.java Changeset: 5328dd5d2be8 Author: rkennke Date: 2019-01-14 12:49 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/5328dd5d2be8 8216973: Kick up cleanup phases in the right places Reviewed-by: shade ! src/hotspot/share/gc/shenandoah/shenandoahControlThread.cpp Changeset: bbc79e0ec9ee Author: goetz Date: 2019-01-14 00:00 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/bbc79e0ec9ee 8216265: [testbug] Introduce Platform.sharedLibraryPathVariableName() and adapt all tests. Summary: Also cleanup some switches over OSes and use File.pathSeparator. Reviewed-by: dholmes, mdoerr ! test/hotspot/jtreg/gtest/GTestWrapper.java ! test/hotspot/jtreg/runtime/signal/SigTestDriver.java ! test/hotspot/jtreg/vmTestbase/ExecDriver.java ! test/hotspot/jtreg/vmTestbase/nsk/jvmti/RetransformClasses/retransform003/TestDriver.java ! test/hotspot/jtreg/vmTestbase/nsk/jvmti/SetNativeMethodPrefix/SetNativeMethodPrefix002/TestDriver.java ! test/jdk/com/sun/jdi/PrivateTransportTest.java ! test/jdk/java/nio/channels/spi/SelectorProvider/inheritedChannel/InheritedChannelTest.java ! test/jdk/javax/net/ssl/DTLS/DTLSBufferOverflowUnderflowTest.java ! test/jdk/javax/net/ssl/DTLS/DTLSDataExchangeTest.java ! test/jdk/javax/net/ssl/DTLS/DTLSEnginesClosureTest.java ! test/jdk/javax/net/ssl/DTLS/DTLSHandshakeTest.java ! test/jdk/javax/net/ssl/DTLS/DTLSHandshakeWithReplicatedPacketsTest.java ! test/jdk/javax/net/ssl/DTLS/DTLSMFLNTest.java ! test/jdk/javax/net/ssl/DTLS/DTLSNotEnabledRC4Test.java ! test/jdk/javax/net/ssl/DTLS/DTLSRehandshakeTest.java ! test/jdk/javax/net/ssl/DTLS/DTLSRehandshakeWithDataExTest.java ! test/jdk/javax/net/ssl/DTLS/DTLSUnsupportedCiphersTest.java ! test/jdk/javax/net/ssl/DTLSv10/DTLSv10BufferOverflowUnderflowTest.java ! test/jdk/javax/net/ssl/DTLSv10/DTLSv10DataExchangeTest.java ! test/jdk/javax/net/ssl/DTLSv10/DTLSv10EnginesClosureTest.java ! test/jdk/javax/net/ssl/DTLSv10/DTLSv10HandshakeTest.java ! test/jdk/javax/net/ssl/DTLSv10/DTLSv10HandshakeWithReplicatedPacketsTest.java ! test/jdk/javax/net/ssl/DTLSv10/DTLSv10MFLNTest.java ! test/jdk/javax/net/ssl/DTLSv10/DTLSv10NotEnabledRC4Test.java ! test/jdk/javax/net/ssl/DTLSv10/DTLSv10RehandshakeTest.java ! test/jdk/javax/net/ssl/DTLSv10/DTLSv10RehandshakeWithDataExTest.java ! test/jdk/javax/net/ssl/DTLSv10/DTLSv10SequenceNumberTest.java ! test/jdk/javax/net/ssl/DTLSv10/DTLSv10UnsupportedCiphersTest.java ! test/jdk/javax/net/ssl/TLS/TLSDataExchangeTest.java ! test/jdk/javax/net/ssl/TLS/TLSEnginesClosureTest.java ! test/jdk/javax/net/ssl/TLS/TLSHandshakeTest.java ! test/jdk/javax/net/ssl/TLS/TLSMFLNTest.java ! test/jdk/javax/net/ssl/TLS/TLSNotEnabledRC4Test.java ! test/jdk/javax/net/ssl/TLS/TLSRehandshakeTest.java ! test/jdk/javax/net/ssl/TLS/TLSRehandshakeWithDataExTest.java ! test/jdk/javax/net/ssl/TLS/TLSUnsupportedCiphersTest.java ! test/jdk/javax/net/ssl/TLSv1/TLSDataExchangeTest.java ! test/jdk/javax/net/ssl/TLSv1/TLSEnginesClosureTest.java ! test/jdk/javax/net/ssl/TLSv1/TLSHandshakeTest.java ! test/jdk/javax/net/ssl/TLSv1/TLSMFLNTest.java ! test/jdk/javax/net/ssl/TLSv1/TLSNotEnabledRC4Test.java ! test/jdk/javax/net/ssl/TLSv1/TLSRehandshakeTest.java ! test/jdk/javax/net/ssl/TLSv1/TLSRehandshakeWithDataExTest.java ! test/jdk/javax/net/ssl/TLSv1/TLSUnsupportedCiphersTest.java ! test/jdk/javax/net/ssl/TLSv11/TLSDataExchangeTest.java ! test/jdk/javax/net/ssl/TLSv11/TLSEnginesClosureTest.java ! test/jdk/javax/net/ssl/TLSv11/TLSHandshakeTest.java ! test/jdk/javax/net/ssl/TLSv11/TLSMFLNTest.java ! test/jdk/javax/net/ssl/TLSv11/TLSNotEnabledRC4Test.java ! test/jdk/javax/net/ssl/TLSv11/TLSRehandshakeTest.java ! test/jdk/javax/net/ssl/TLSv11/TLSRehandshakeWithDataExTest.java ! test/jdk/javax/net/ssl/TLSv11/TLSUnsupportedCiphersTest.java ! test/jdk/javax/net/ssl/TLSv12/TLSEnginesClosureTest.java ! test/jdk/sun/security/krb5/auto/BasicProc.java ! test/jdk/sun/security/krb5/auto/KDC.java ! test/jdk/sun/security/krb5/auto/NoAddresses.java ! test/jdk/sun/security/krb5/auto/ReplayCacheTestProc.java ! test/jdk/sun/security/krb5/auto/principalProperty/PrincipalSystemPropTest.java ! test/jdk/tools/launcher/ExecutionEnvironment.java ! test/jdk/tools/launcher/JliLaunchTest.java ! test/jdk/tools/launcher/Test7029048.java ! test/jdk/vm/JniInvocationTest.java ! test/lib/jdk/test/lib/Platform.java Changeset: cf80666df79a Author: shade Date: 2019-01-14 17:20 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/cf80666df79a 8216308: StackTraceElement::fill_in can use injected Class source-file Reviewed-by: coleenp, dholmes ! src/hotspot/share/classfile/javaClasses.cpp ! src/hotspot/share/classfile/javaClasses.hpp ! src/hotspot/share/classfile/vmSymbols.hpp Changeset: f5e601ad26a8 Author: vromero Date: 2019-01-14 12:24 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/f5e601ad26a8 8215482: check for cycles in type variables can provoke NPE Reviewed-by: mcimadamore ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Attr.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/MemberEnter.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/comp/TypeEnter.java + test/langtools/tools/javac/T8215482/NPETypeVarWithOuterBoundTest.java Changeset: 72fdf46a274e Author: zgu Date: 2019-01-14 12:51 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/72fdf46a274e 8215549: Shenandoah deduplication cleans up table/queue twice Reviewed-by: rkennke ! src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp ! src/hotspot/share/gc/shenandoah/shenandoahPhaseTimings.hpp ! src/hotspot/share/gc/shenandoah/shenandoahStringDedup.cpp ! src/hotspot/share/gc/shenandoah/shenandoahStringDedup.hpp Changeset: b002e4ee60b0 Author: bchristi Date: 2019-01-14 11:22 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/b002e4ee60b0 8216401: Allow "file:" URLs in Class-Path of local JARs Reviewed-by: alanb, mchung ! src/java.base/share/classes/jdk/internal/loader/URLClassPath.java + test/jdk/jdk/internal/loader/URLClassPath/JarClassPathFileEntry.java Changeset: 4b469f5f4bf2 Author: eosterlund Date: 2019-01-10 18:10 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/4b469f5f4bf2 8215889: assert(!_unloading) failed: This oop is not available to unloading class loader data with ZGC Reviewed-by: coleenp, neliasso ! src/hotspot/share/ci/ciEnv.hpp ! src/hotspot/share/ci/ciMethodData.cpp ! src/hotspot/share/ci/ciMethodData.hpp ! src/hotspot/share/ci/ciObjectFactory.cpp ! src/hotspot/share/ci/ciObjectFactory.hpp ! src/hotspot/share/oops/instanceKlass.cpp ! src/hotspot/share/oops/methodData.cpp ! src/hotspot/share/oops/methodData.hpp Changeset: 6b37a7ba9b66 Author: itakiguchi Date: 2019-01-11 09:37 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/6b37a7ba9b66 8211841: [testbug] sun/nio/cs/OLD/TestIBMDB.java does not compile (aix) Reviewed-by: alanb, goetz ! test/jdk/ProblemList.txt ! test/jdk/sun/nio/cs/OLD/IBM1383_OLD.java ! test/jdk/sun/nio/cs/OLD/IBM942_OLD.java ! test/jdk/sun/nio/cs/OLD/IBM943_OLD.java ! test/jdk/sun/nio/cs/OLD/IBM950_OLD.java ! test/jdk/sun/nio/cs/OLD/IBM970_OLD.java Changeset: f152abfd2751 Author: jlahoda Date: 2019-01-11 10:46 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/f152abfd2751 8215244: jdk/jshell/ToolBasicTest.java testHistoryReference failed Summary: Mark history entries from previous sessions with timestamp that is definitelly in the past. Reviewed-by: rfield ! src/jdk.jshell/share/classes/jdk/internal/jshell/tool/ConsoleIOContext.java Changeset: 672b629e1f72 Author: mdoerr Date: 2019-01-11 11:02 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/672b629e1f72 8216376: [PPC64] Possibly unreliable stack frame resizing in template interpreter Reviewed-by: goetz, gromero ! src/hotspot/cpu/ppc/stubGenerator_ppc.cpp ! src/hotspot/cpu/ppc/templateInterpreterGenerator_ppc.cpp Changeset: cb7fff9105a8 Author: eosterlund Date: 2019-01-11 13:15 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/cb7fff9105a8 8215754: ZGC: nmethod is not unlinked from Method before rendezvous handshake Reviewed-by: pliden, neliasso ! src/hotspot/share/code/nmethod.cpp ! src/hotspot/share/code/nmethod.hpp ! src/hotspot/share/gc/z/zBarrierSetNMethod.cpp ! src/hotspot/share/gc/z/zNMethodTable.cpp Changeset: f7491df4fd3a Author: roland Date: 2019-01-10 13:54 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/f7491df4fd3a 8216482: Shenandoah: typo in ShenandoahBarrierSetC2::clone_barrier_at_expansion() causes failed compilations Reviewed-by: thartmann, shade, rkennke ! src/hotspot/share/gc/shenandoah/c2/shenandoahBarrierSetC2.cpp Changeset: a995647f4911 Author: vromero Date: 2019-01-11 09:02 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/a995647f4911 8215648: remove equals and hashCode implementations from j.l.i.VarHandle Reviewed-by: mchung ! src/java.base/share/classes/java/lang/invoke/VarHandle.java ! src/java.base/share/classes/java/lang/invoke/X-VarHandle.java.template ! src/java.base/share/classes/java/lang/invoke/X-VarHandleByteArrayView.java.template ! test/jdk/java/lang/constant/CondyDescTest.java ! test/jdk/java/lang/invoke/VarHandles/VarHandleTestAccessBoolean.java ! test/jdk/java/lang/invoke/VarHandles/VarHandleTestAccessByte.java ! test/jdk/java/lang/invoke/VarHandles/VarHandleTestAccessChar.java ! test/jdk/java/lang/invoke/VarHandles/VarHandleTestAccessDouble.java ! test/jdk/java/lang/invoke/VarHandles/VarHandleTestAccessFloat.java ! test/jdk/java/lang/invoke/VarHandles/VarHandleTestAccessInt.java ! test/jdk/java/lang/invoke/VarHandles/VarHandleTestAccessLong.java ! test/jdk/java/lang/invoke/VarHandles/VarHandleTestAccessShort.java ! test/jdk/java/lang/invoke/VarHandles/VarHandleTestAccessString.java ! test/jdk/java/lang/invoke/VarHandles/VarHandleTestByteArrayAsChar.java ! test/jdk/java/lang/invoke/VarHandles/VarHandleTestByteArrayAsDouble.java ! test/jdk/java/lang/invoke/VarHandles/VarHandleTestByteArrayAsFloat.java ! test/jdk/java/lang/invoke/VarHandles/VarHandleTestByteArrayAsInt.java ! test/jdk/java/lang/invoke/VarHandles/VarHandleTestByteArrayAsLong.java ! test/jdk/java/lang/invoke/VarHandles/VarHandleTestByteArrayAsShort.java Changeset: fa2f191e72f5 Author: darcy Date: 2019-01-11 08:32 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/fa2f191e72f5 8213299: runtime/appcds/jigsaw/classpathtests/EmptyClassInBootClassPath.java failed with java.lang.NoSuchMethodException Reviewed-by: dholmes ! src/java.base/share/classes/java/lang/Class.java ! test/hotspot/jtreg/ProblemList.txt Changeset: 5fa71cce89eb Author: jjg Date: 2019-01-11 11:32 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/5fa71cce89eb 8210561: Command-line help wrong for javac --module Reviewed-by: darcy ! src/jdk.compiler/share/classes/com/sun/tools/javac/resources/javac.properties Changeset: 6b963dd96b5e Author: jjg Date: 2019-01-11 11:42 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/6b963dd96b5e Merge Changeset: f0490430ef7a Author: roland Date: 2019-01-11 10:03 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/f0490430ef7a 8216549: Mismatched unsafe access to non escaping object fails Reviewed-by: vlivanov, kvn, thartmann ! src/hotspot/share/opto/escape.cpp + test/hotspot/jtreg/compiler/unsafe/MismatchedUnsafeLoadFromNewObject.java Changeset: 5022a4915fe9 Author: xuelei Date: 2019-01-14 10:00 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/5022a4915fe9 8214418: half-closed SSLEngine status may cause application dead loop Reviewed-by: jnimeh, dfuchs, chegar ! src/java.base/share/classes/sun/security/ssl/Ciphertext.java ! src/java.base/share/classes/sun/security/ssl/SSLEngineImpl.java ! src/java.base/share/classes/sun/security/ssl/TransportContext.java Changeset: b685bc048276 Author: dnsimon Date: 2019-01-14 21:34 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/b685bc048276 8215313: [AOT] java/lang/String/Split.java fails with AOTed java.base Reviewed-by: kvn, never, dlong Contributed-by: Josef Haider ! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.asm.amd64/src/org/graalvm/compiler/asm/amd64/AMD64Assembler.java ! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.lir.amd64/src/org/graalvm/compiler/lir/amd64/AMD64ArrayIndexOfOp.java Changeset: 8f822a19309b Author: lancea Date: 2019-01-14 16:35 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/8f822a19309b 8216362: Better error message handling when there is an invalid Manifest Reviewed-by: lancea, rriggs, mullan Contributed-by: Philipp Kunz ! src/java.base/share/classes/java/util/jar/Manifest.java + test/jdk/java/util/jar/Manifest/IncludeInExceptionsTest.java Changeset: 8065db7231ae Author: kvn Date: 2019-01-14 13:45 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/8065db7231ae 8216151: [Graal] Module jdk.internal.vm.compiler.management has not been granted accessClassInPackage.org.graalvm.compiler.debug Summary: update default.policy based on latest changes in jdk.internal.vm.compiler.management Reviewed-by: thartmann, alanb, mchung ! src/java.base/share/lib/security/default.policy Changeset: 64049c8e7452 Author: jwilhelm Date: 2019-01-14 23:05 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/64049c8e7452 Merge ! src/hotspot/share/ci/ciEnv.hpp ! src/hotspot/share/ci/ciMethodData.hpp ! src/hotspot/share/ci/ciObjectFactory.hpp ! src/hotspot/share/code/nmethod.hpp ! src/hotspot/share/oops/instanceKlass.cpp ! src/hotspot/share/oops/methodData.hpp ! src/java.base/share/classes/java/lang/Class.java ! src/java.base/share/classes/java/util/jar/Manifest.java ! src/java.base/share/lib/security/default.policy ! test/hotspot/jtreg/ProblemList.txt Changeset: fcddd67f986f Author: manc Date: 2018-12-10 17:57 +0800 URL: http://hg.openjdk.java.net/panama/dev/rev/fcddd67f986f 8215114: Fix indent and dead code in GCPolicyCounters Summary: Clean up gcPolicyCounters.hpp Reviewed-by: tschatzl, sjohanss ! src/hotspot/share/gc/shared/gcPolicyCounters.hpp Changeset: 520f8e2041bb Author: shurailine Date: 2019-01-14 08:22 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/520f8e2041bb 8215729: Enhance makefiles to allow collecting code coverage with JCov Reviewed-by: erikj ! make/Main.gmk ! make/RunTests.gmk ! make/common/FindTests.gmk Changeset: de5564099c01 Author: shurailine Date: 2019-01-14 08:23 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/de5564099c01 Merge Changeset: d633be26e59b Author: dholmes Date: 2019-01-14 20:56 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/d633be26e59b 8217017: [TESTBUG] Tests fail to compile after JDK-8216265 Reviewed-by: kvn ! test/jdk/java/nio/channels/spi/SelectorProvider/inheritedChannel/InheritedChannelTest.java Changeset: bd9043ffaa2a Author: weijun Date: 2019-01-15 11:21 +0800 URL: http://hg.openjdk.java.net/panama/dev/rev/bd9043ffaa2a 8215922: jar spec is not precise when describing jar file re-signing Reviewed-by: lancea, mullan + test/jdk/sun/security/tools/jarsigner/SignedAgain.java Changeset: 0b2574a2a6c7 Author: stuefe Date: 2019-01-15 08:03 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/0b2574a2a6c7 8216982: Assertion poison page established too early Reviewed-by: mdoerr, dholmes ! src/hotspot/share/runtime/thread.cpp Changeset: 54aa3ea04fe8 Author: dfuchs Date: 2019-01-15 11:34 +0000 URL: http://hg.openjdk.java.net/panama/dev/rev/54aa3ea04fe8 8216974: HttpConnection not returned to the pool after 204 response Summary: MultiExchange now call nullBody() on Exchange after receiving 204 Reviewed-by: chegar ! src/java.net.http/share/classes/jdk/internal/net/http/Exchange.java ! src/java.net.http/share/classes/jdk/internal/net/http/ExchangeImpl.java ! src/java.net.http/share/classes/jdk/internal/net/http/Http1Exchange.java ! src/java.net.http/share/classes/jdk/internal/net/http/Http1Response.java ! src/java.net.http/share/classes/jdk/internal/net/http/HttpConnection.java ! src/java.net.http/share/classes/jdk/internal/net/http/MultiExchange.java ! test/jdk/java/net/httpclient/Response204.java Changeset: 50355c3d35c0 Author: rriggs Date: 2019-01-15 09:22 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/50355c3d35c0 8080569: java/lang/ProcessBuilder/DestroyTest.java fails with "Process terminated prematurely" Reviewed-by: lancea, bchristi ! test/jdk/java/lang/ProcessBuilder/DestroyTest.java Changeset: c58de85b30d2 Author: hseigel Date: 2019-01-15 14:55 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/c58de85b30d2 8216563: [TESTBUG] Change stressTime to default to 30 for nsk tests (part 2) Summary: Change the default from 60 seconds to 30 seconds. Reviewed-by: coleenp, mseledtsov ! test/hotspot/jtreg/vmTestbase/nsk/share/test/StressOptions.java ! test/hotspot/jtreg/vmTestbase/vm/runtime/defmeth/README ! test/hotspot/jtreg/vmTestbase/vm/share/options/package-info.java Changeset: 65a1d49d1718 Author: redestad Date: 2019-01-15 21:17 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/65a1d49d1718 8216995: Clean up JFR command line processing Reviewed-by: gziemski, mgronlun ! src/hotspot/share/jfr/recorder/jfrRecorder.cpp ! src/hotspot/share/runtime/globals_extension.hpp Changeset: 9e968a576dd2 Author: rriggs Date: 2019-01-15 15:56 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/9e968a576dd2 8202675: Replace process-wide terminology in serial filtering to be consistent Reviewed-by: alanb, lancea ! src/java.base/share/classes/java/io/ObjectInputFilter.java ! src/java.base/share/classes/java/io/ObjectInputStream.java ! src/java.base/share/classes/java/io/ObjectStreamConstants.java ! src/java.base/share/conf/security/java.security Changeset: d193d58ae79d Author: dholmes Date: 2019-01-15 16:40 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/d193d58ae79d 8213397: Stack dump should show more clearly when a thread is blocked on a class initialization monitor Reviewed-by: rehn, coleenp ! src/hotspot/share/oops/instanceKlass.cpp ! src/hotspot/share/runtime/thread.cpp ! src/hotspot/share/runtime/thread.hpp ! src/hotspot/share/runtime/thread.inline.hpp ! src/hotspot/share/runtime/vframe.cpp + test/hotspot/jtreg/runtime/Thread/TestThreadDumpClassInitMonitor.java Changeset: 8e260023fc53 Author: roland Date: 2019-01-14 15:07 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/8e260023fc53 8216135: C2 assert(!had_error) failed: bad dominance Reviewed-by: thartmann, kvn ! src/hotspot/share/opto/loopPredicate.cpp ! src/hotspot/share/opto/loopTransform.cpp ! src/hotspot/share/opto/loopnode.hpp ! test/hotspot/jtreg/ProblemList.txt Changeset: b5281bf751ea Author: eosterlund Date: 2019-01-15 09:44 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/b5281bf751ea 8216427: ciMethodData::load_extra_data() does not always unpack the last entry Reviewed-by: thartmann, kvn ! src/hotspot/share/ci/ciMethodData.cpp Changeset: 1884ecc20c38 Author: phedlin Date: 2018-12-18 10:12 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/1884ecc20c38 8210392: assert(Compile::current()->live_nodes() < Compile::current()->max_node_limit()) failed: Live Node limit exceeded limit Summary: Avoid excessive split-if. Reviewed-by: thartmann, neliasso ! src/hotspot/share/opto/loopnode.hpp ! src/hotspot/share/opto/loopopts.cpp + test/hotspot/jtreg/compiler/loopopts/Test8210392.java Changeset: 314c5b5d9369 Author: iveresov Date: 2019-01-15 10:40 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/314c5b5d9369 8196568: [Graal] LongMulOverflowTest.java fails with "runTestOverflow() did not overflow" Summary: Temporarily cripple j.l.Math.*Exact() instrinsics to pass TCK Reviewed-by: kvn, dlong, never ! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.replacements.test/src/org/graalvm/compiler/replacements/test/IntegerExactExceptionTest.java ! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.replacements.test/src/org/graalvm/compiler/replacements/test/IntegerExactFoldTest.java ! src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.replacements/src/org/graalvm/compiler/replacements/StandardGraphBuilderPlugins.java Changeset: 8ce4083fc831 Author: jjg Date: 2019-01-15 11:05 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/8ce4083fc831 8212233: javadoc fails on jdk12 with "The code being documented uses modules but the packages defined in $URL are in the unnamed module." Reviewed-by: hannesw, pmuthuswamy ! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/IndexRedirectWriter.java ! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/toolkit/util/Extern.java + test/langtools/jdk/javadoc/doclet/testLinkOption/TestLinkOptionWithAutomaticModule.java Changeset: 205d2db1dc25 Author: valeriep Date: 2018-06-14 23:47 +0000 URL: http://hg.openjdk.java.net/panama/dev/rev/205d2db1dc25 8203654: Improve cypher state updates Reviewed-by: ascarpino ! src/java.base/share/classes/javax/crypto/spec/GCMParameterSpec.java Changeset: ee7b0da99262 Author: bpb Date: 2018-07-19 07:02 -0700 URL: http://hg.openjdk.java.net/panama/dev/rev/ee7b0da99262 8206301: Improve NIO stability Reviewed-by: alanb, mschoene, rhalade ! src/java.base/windows/native/libnio/ch/DatagramDispatcher.c ! src/java.base/windows/native/libnio/ch/WindowsSelectorImpl.c Changeset: c66b192fe3b4 Author: dtitov Date: 2018-07-19 15:23 -0700 URL: http://hg.openjdk.java.net/panama/dev/rev/c66b192fe3b4 8205709: Proper allocation handling Reviewed-by: sspitsyn, mschoene, rhalade ! src/java.instrument/unix/native/libinstrument/FileSystemSupport_md.c ! src/java.instrument/windows/native/libinstrument/FileSystemSupport_md.c Changeset: 827db73f2d6a Author: psadhukhan Date: 2018-07-31 11:43 +0530 URL: http://hg.openjdk.java.net/panama/dev/rev/827db73f2d6a 8205360: Choose printer defaults Reviewed-by: prr, mschoene, rhalade ! src/java.desktop/windows/native/libawt/windows/WPrinterJob.cpp Changeset: 3d4e47348142 Author: dholmes Date: 2018-08-12 18:05 -0400 URL: http://hg.openjdk.java.net/panama/dev/rev/3d4e47348142 8205714: Initial class initialization Summary: ensure class is fully initialized before caching a resolved invokestatic, or patching the callsite Reviewed-by: acorn, coleenp, kvn, vlivanov ! src/hotspot/share/oops/cpCache.cpp ! src/hotspot/share/runtime/sharedRuntime.cpp Changeset: 066d2261108f Author: chegar Date: 2018-08-10 14:35 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/066d2261108f 8199166: Better interface lists Reviewed-by: igerasim, mschoene, michaelm, rhalade ! src/java.base/windows/native/libnet/NetworkInterface.c Changeset: 5deff84a4d48 Author: chegar Date: 2018-08-10 15:02 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/5deff84a4d48 8199156: Better route routing Reviewed-by: igerasim, mschoene, michaelm, rhalade ! src/java.base/unix/native/libnet/net_util_md.c Changeset: db202823fd94 Author: jnimeh Date: 2018-08-15 09:36 -0700 URL: http://hg.openjdk.java.net/panama/dev/rev/db202823fd94 8206295: More reliable p11 transactions Reviewed-by: valeriep, mschoene, rhalade ! src/jdk.crypto.cryptoki/windows/native/libj2pkcs11/p11_md.c Changeset: db7b41933563 Author: serb Date: 2018-08-16 10:22 -0700 URL: http://hg.openjdk.java.net/panama/dev/rev/db7b41933563 8204895: Better icon support Reviewed-by: prr, mschoene, aghaisas, rhalade ! src/java.desktop/unix/native/libawt_xawt/awt/awt_UNIXToolkit.c Changeset: 42037e059f2c Author: serb Date: 2018-08-21 13:57 -0700 URL: http://hg.openjdk.java.net/panama/dev/rev/42037e059f2c 8203955: Improve robot support Reviewed-by: prr, psadhukhan, rhalade, skoivu ! src/java.desktop/share/classes/java/awt/Robot.java Changeset: 2cdf4a989ee7 Author: sdama Date: 2018-08-28 13:22 +0530 URL: http://hg.openjdk.java.net/panama/dev/rev/2cdf4a989ee7 8203688: [testbug] Nashorn test test/nashorn/script/nosecurity/treeapi/diagnostic.js fails Summary: Updating the expected output for diagnostic.js to its correct value Reviewed-by: jlaskey, sundar Contributed-by: thejasvi.v.voniadka at oracle.com ! test/nashorn/script/nosecurity/treeapi/diagnostic.js.EXPECTED Changeset: 8ba5b3c76857 Author: coleenp Date: 2018-09-10 16:49 -0400 URL: http://hg.openjdk.java.net/panama/dev/rev/8ba5b3c76857 8210094: Better loading of classloader classes Reviewed-by: acorn, hseigel, ahgross, rhalade ! src/hotspot/share/classfile/classFileParser.cpp ! src/hotspot/share/classfile/classLoaderDataGraph.cpp ! src/hotspot/share/classfile/dictionary.cpp ! src/hotspot/share/classfile/dictionary.hpp ! src/hotspot/share/classfile/systemDictionary.cpp ! src/hotspot/share/classfile/verificationType.cpp ! src/hotspot/share/classfile/verifier.cpp ! src/hotspot/share/oops/constantPool.cpp ! src/hotspot/share/prims/jvm.cpp + test/hotspot/jtreg/runtime/ClassUnload/ConstantPoolDependsTest.java + test/hotspot/jtreg/runtime/ClassUnload/DictionaryDependsTest.java + test/hotspot/jtreg/runtime/ClassUnload/MyDiffClassLoader.java + test/hotspot/jtreg/runtime/ClassUnload/SuperDependsTest.java + test/hotspot/jtreg/runtime/ClassUnload/p2/c2.java Changeset: bc9faf59936d Author: coleenp Date: 2018-09-12 08:26 -0400 URL: http://hg.openjdk.java.net/panama/dev/rev/bc9faf59936d 8210624: Clean up better loading tests Reviewed-by: hseigel, jwilhelm ! test/hotspot/jtreg/runtime/ClassUnload/ConstantPoolDependsTest.java ! test/hotspot/jtreg/runtime/ClassUnload/DictionaryDependsTest.java ! test/hotspot/jtreg/runtime/ClassUnload/SuperDependsTest.java Changeset: cbb8341a127a Author: bpb Date: 2018-09-25 16:49 -0700 URL: http://hg.openjdk.java.net/panama/dev/rev/cbb8341a127a 8206290: Better FileChannel transfer performance Reviewed-by: alanb, rhalade, mschoene ! src/java.base/share/classes/sun/nio/ch/FileChannelImpl.java Changeset: d1ebdef71c73 Author: serb Date: 2018-09-27 12:54 -0700 URL: http://hg.openjdk.java.net/panama/dev/rev/d1ebdef71c73 8210598: Strengthen Windows Access Bridge Support Reviewed-by: prr, psadhukhan, rhalade, mschoene ! src/jdk.accessibility/windows/native/libwindowsaccessbridge/AccessBridgeMessageQueue.cpp Changeset: 0060e9d7c450 Author: weijun Date: 2018-09-29 10:08 +0800 URL: http://hg.openjdk.java.net/panama/dev/rev/0060e9d7c450 8210610: Improved LSA authentication Reviewed-by: valeriep, mschoene, rhalade ! src/java.security.jgss/windows/native/libw2k_lsa_auth/NativeCreds.c Changeset: 620b31ed8807 Author: bpb Date: 2018-08-22 15:55 -0700 URL: http://hg.openjdk.java.net/panama/dev/rev/620b31ed8807 8200659: Improve BigDecimal support Reviewed-by: darcy, rhalade, mschoene ! src/java.base/share/classes/java/math/BigDecimal.java ! src/java.base/share/classes/java/math/BigInteger.java ! test/jdk/java/math/BigDecimal/AddTests.java ! test/jdk/java/math/BigDecimal/Constructor.java + test/jdk/java/math/BigInteger/LargeValueExceptions.java Changeset: dff86e25073f Author: michaelm Date: 2018-10-05 08:54 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/dff86e25073f 8209094: Improve web server connections Reviewed-by: chegar, dfuchs, mschoene, igerasim ! make/lib/Lib-java.base.gmk ! src/java.base/share/classes/sun/net/www/protocol/http/ntlm/NTLMAuthenticationCallback.java ! src/java.base/share/conf/net.properties ! src/java.base/unix/classes/sun/net/www/protocol/http/ntlm/NTLMAuthentication.java ! src/java.base/windows/classes/sun/net/www/protocol/http/ntlm/NTLMAuthentication.java + src/java.base/windows/native/libnet/NTLMAuthentication.c Changeset: d845ee36da70 Author: prr Date: 2018-10-05 11:36 -0700 URL: http://hg.openjdk.java.net/panama/dev/rev/d845ee36da70 8210606: Improved data set handling Reviewed-by: serb, psadhukhan, mschoene, rhalade ! src/java.desktop/share/native/liblcms/cmscgats.c Changeset: e8bae92beee3 Author: weijun Date: 2018-10-08 12:55 +0800 URL: http://hg.openjdk.java.net/panama/dev/rev/e8bae92beee3 8210870: Libsunmscapi improved interactions Reviewed-by: valeriep, mschoene, rhalade ! src/jdk.crypto.mscapi/windows/classes/sun/security/mscapi/KeyStore.java ! src/jdk.crypto.mscapi/windows/native/libsunmscapi/security.cpp Changeset: b9149d907610 Author: prr Date: 2018-10-08 12:53 -0700 URL: http://hg.openjdk.java.net/panama/dev/rev/b9149d907610 8210866: Improve JPEG processing Reviewed-by: serb, psadhukhan, rhalade ! src/java.desktop/share/native/libjavajpeg/jmemmgr.c Changeset: ab474ef0a0ac Author: jwilhelm Date: 2018-09-13 01:49 +0200 URL: http://hg.openjdk.java.net/panama/dev/rev/ab474ef0a0ac 8199552: Update to build scripts Reviewed-by: jwilhelm, mschoene, rhalade Contributed-by: magnus.ihse.bursie at oracle.com ! make/autoconf/flags-cflags.m4 Changeset: fd6de53a0d6e Author: henryjen Date: 2018-12-13 11:47 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/fd6de53a0d6e Merge - make/scripts/pandoc-manpage-filter.js - make/scripts/pandoc-manpage-filter.sh.template ! src/hotspot/share/classfile/classFileParser.cpp - src/java.base/share/classes/com/sun/crypto/provider/HmacPKCS12PBESHA1.java - src/java.logging/share/classes/java/util/logging/package.html - src/java.prefs/share/classes/java/util/prefs/package.html - src/java.rmi/share/classes/java/rmi/activation/package.html - src/java.rmi/share/classes/java/rmi/dgc/package.html - src/java.rmi/share/classes/java/rmi/package.html - src/java.rmi/share/classes/java/rmi/registry/package.html - src/java.rmi/share/classes/java/rmi/server/package.html - src/java.rmi/share/classes/javax/rmi/ssl/package.html - src/java.security.jgss/share/classes/org/ietf/jgss/package.html - src/java.smartcardio/share/classes/javax/smartcardio/package.html - src/java.sql.rowset/share/classes/com/sun/rowset/package.html - src/java.sql.rowset/share/classes/com/sun/rowset/providers/package.html - src/java.sql.rowset/share/classes/javax/sql/rowset/serial/package.html - src/java.sql/share/classes/java/sql/package.html - src/java.sql/share/classes/javax/sql/package.html + src/jdk.crypto.mscapi/windows/classes/sun/security/mscapi/CKeyStore.java - src/jdk.crypto.mscapi/windows/classes/sun/security/mscapi/Key.java - src/jdk.crypto.mscapi/windows/classes/sun/security/mscapi/KeyStore.java - src/jdk.crypto.mscapi/windows/classes/sun/security/mscapi/RSACipher.java - src/jdk.crypto.mscapi/windows/classes/sun/security/mscapi/RSAKeyPair.java - src/jdk.crypto.mscapi/windows/classes/sun/security/mscapi/RSAKeyPairGenerator.java - src/jdk.crypto.mscapi/windows/classes/sun/security/mscapi/RSAPrivateKey.java - src/jdk.crypto.mscapi/windows/classes/sun/security/mscapi/RSAPublicKey.java - src/jdk.crypto.mscapi/windows/classes/sun/security/mscapi/RSASignature.java ! src/jdk.crypto.mscapi/windows/native/libsunmscapi/security.cpp - src/jdk.internal.le/share/classes/jdk/internal/jline/DefaultTerminal2.java - src/jdk.internal.le/share/classes/jdk/internal/jline/NoInterruptUnixTerminal.java - src/jdk.internal.le/share/classes/jdk/internal/jline/OSvTerminal.java - src/jdk.internal.le/share/classes/jdk/internal/jline/Terminal.java - src/jdk.internal.le/share/classes/jdk/internal/jline/Terminal2.java - src/jdk.internal.le/share/classes/jdk/internal/jline/TerminalFactory.java - src/jdk.internal.le/share/classes/jdk/internal/jline/TerminalSupport.java - src/jdk.internal.le/share/classes/jdk/internal/jline/UnixTerminal.java - src/jdk.internal.le/share/classes/jdk/internal/jline/UnsupportedTerminal.java - src/jdk.internal.le/share/classes/jdk/internal/jline/WindowsTerminal.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/ConsoleKeys.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/ConsoleReader.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/CursorBuffer.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/KeyMap.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/KillRing.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/Operation.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/UserInterruptException.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/WCWidth.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/completer/AggregateCompleter.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/completer/AnsiStringsCompleter.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/completer/ArgumentCompleter.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/completer/CandidateListCompletionHandler.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/completer/CandidateListCompletionHandler.properties - src/jdk.internal.le/share/classes/jdk/internal/jline/console/completer/Completer.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/completer/CompletionHandler.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/completer/EnumCompleter.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/completer/FileNameCompleter.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/completer/NullCompleter.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/completer/StringsCompleter.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/completer/package-info.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/history/FileHistory.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/history/History.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/history/MemoryHistory.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/history/PersistentHistory.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/history/package-info.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/internal/ConsoleReaderInputStream.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/internal/ConsoleRunner.java - src/jdk.internal.le/share/classes/jdk/internal/jline/console/package-info.java - src/jdk.internal.le/share/classes/jdk/internal/jline/extra/AnsiInterpretingOutputStream.java - src/jdk.internal.le/share/classes/jdk/internal/jline/extra/EditingHistory.java - src/jdk.internal.le/share/classes/jdk/internal/jline/internal/Ansi.java - src/jdk.internal.le/share/classes/jdk/internal/jline/internal/Configuration.java - src/jdk.internal.le/share/classes/jdk/internal/jline/internal/Curses.java - src/jdk.internal.le/share/classes/jdk/internal/jline/internal/InfoCmp.java - src/jdk.internal.le/share/classes/jdk/internal/jline/internal/InputStreamReader.java - src/jdk.internal.le/share/classes/jdk/internal/jline/internal/Log.java - src/jdk.internal.le/share/classes/jdk/internal/jline/internal/NonBlockingInputStream.java - src/jdk.internal.le/share/classes/jdk/internal/jline/internal/Nullable.java - src/jdk.internal.le/share/classes/jdk/internal/jline/internal/Preconditions.java - src/jdk.internal.le/share/classes/jdk/internal/jline/internal/ShutdownHooks.java - src/jdk.internal.le/share/classes/jdk/internal/jline/internal/TerminalLineSettings.java - src/jdk.internal.le/share/classes/jdk/internal/jline/internal/TestAccessible.java - src/jdk.internal.le/share/classes/jdk/internal/jline/internal/Urls.java - src/jdk.internal.le/share/classes/jdk/internal/jline/internal/package-info.java - src/jdk.internal.le/share/classes/jdk/internal/jline/package-info.java - src/jdk.internal.le/windows/native/lible/WindowsTerminal.cpp - src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/replacements/UnsafeAccess.java - src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/resources/jquery/images/ui-bg_flat_0_aaaaaa_40x100.png - src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/resources/jquery/images/ui-bg_flat_75_ffffff_40x100.png - src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/resources/jquery/images/ui-bg_glass_65_ffffff_1x400.png - test/jdk/jdk/internal/jline/console/StripAnsiTest.java - test/jdk/jdk/internal/jline/extra/AnsiInterpretingOutputStreamTest.java - test/jdk/jdk/internal/jline/extra/HistoryTest.java Changeset: b94283cb226b Author: henryjen Date: 2018-12-13 11:51 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/b94283cb226b Merge - src/hotspot/share/jfr/recorder/repository/jfrChunkSizeNotifier.cpp - src/hotspot/share/jfr/recorder/repository/jfrChunkSizeNotifier.hpp Changeset: 585902b2bfcb Author: henryjen Date: 2019-01-15 10:55 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/585902b2bfcb Merge ! make/autoconf/flags-cflags.m4 ! src/hotspot/share/classfile/dictionary.hpp ! src/hotspot/share/classfile/systemDictionary.cpp ! src/hotspot/share/prims/jvm.cpp ! src/java.base/unix/native/libnet/net_util_md.c - test/jdk/java/lang/String/AlignIndent.java - test/jdk/java/net/MulticastSocket/PromiscuousIPv6.java - test/jdk/java/nio/channels/DatagramChannel/PromiscuousIPv6.java - test/langtools/tools/javac/RawStringLiteralLang.java - test/langtools/tools/javac/RawStringLiteralLangAPI.java - test/langtools/tools/javac/diags/examples/RawStringLiteral.java Changeset: 36ca868f266f Author: vdeshpande Date: 2019-01-15 11:19 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/36ca868f266f 8216050: Superword optimization fails with assert(0 <= i && i < _len) failed: illegal index Summary: Fix for the crash by matching the operands by swapping to right positions. Reviewed-by: thartmann, kvn ! src/hotspot/cpu/x86/x86.ad ! src/hotspot/share/opto/loopopts.cpp ! src/hotspot/share/opto/mulnode.cpp ! src/hotspot/share/opto/mulnode.hpp ! src/hotspot/share/opto/superword.cpp Changeset: 464f2e1b0383 Author: cushon Date: 2018-12-07 16:56 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/464f2e1b0383 8198526: getAnnotatedOwnerType does not handle static nested classes correctly Reviewed-by: jfranck, vromero ! src/java.base/share/classes/sun/reflect/annotation/AnnotatedTypeFactory.java ! src/java.base/share/classes/sun/reflect/annotation/TypeAnnotation.java + test/jdk/java/lang/annotation/typeAnnotations/GetAnnotatedNestedSuperclass.java ! test/jdk/java/lang/annotation/typeAnnotations/GetAnnotatedOwnerType.java Changeset: 5afdd1100a20 Author: jwilhelm Date: 2019-01-15 22:54 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/5afdd1100a20 Merge ! make/autoconf/flags-cflags.m4 ! src/hotspot/share/classfile/classFileParser.cpp ! src/hotspot/share/classfile/dictionary.hpp ! src/hotspot/share/classfile/systemDictionary.cpp ! src/hotspot/share/opto/loopnode.hpp ! src/hotspot/share/opto/mulnode.hpp ! src/hotspot/share/prims/jvm.cpp ! src/java.base/share/classes/java/math/BigDecimal.java ! src/jdk.accessibility/windows/native/libwindowsaccessbridge/AccessBridgeMessageQueue.cpp ! test/hotspot/jtreg/ProblemList.txt Changeset: 550af62c5cbd Author: jjg Date: 2019-01-15 14:18 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/550af62c5cbd 8216319: Refactor JavadocTester to allow more on-by-default checkers; add A11YChecker Reviewed-by: hannesw + test/langtools/jdk/javadoc/lib/javadoc/tester/A11yChecker.java + test/langtools/jdk/javadoc/lib/javadoc/tester/HtmlChecker.java + test/langtools/jdk/javadoc/lib/javadoc/tester/HtmlParser.java ! test/langtools/jdk/javadoc/lib/javadoc/tester/JavadocTester.java + test/langtools/jdk/javadoc/lib/javadoc/tester/LinkChecker.java Changeset: 142b179dd60e Author: mbalao Date: 2019-01-15 19:24 -0300 URL: http://hg.openjdk.java.net/panama/dev/rev/142b179dd60e 8217088: Disable JDK-6913047 fix (SunPKCS11 memory leak) after JDK-8216597 (SIGBUS error in getNativeKeyInfo) Summary: Disable JDK-6913047 fix (SunPKCS11 memory leak) temporarily until JDK-8216597 (SIGBUS error in getNativeKeyInfo) is fixed. Reviewed-by: mullan ! src/jdk.crypto.cryptoki/share/classes/sun/security/pkcs11/P11Key.java Changeset: c79189826bbb Author: jjg Date: 2019-01-15 15:45 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/c79189826bbb 8217214: Recent new javadoc test needs to be updated Reviewed-by: mchung ! test/langtools/jdk/javadoc/doclet/testLinkOption/TestLinkOptionWithAutomaticModule.java Changeset: eabbb779d3eb Author: darcy Date: 2019-01-15 19:10 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/eabbb779d3eb 8217000: Refactor Class::methodToString Reviewed-by: smarks ! src/java.base/share/classes/java/lang/Class.java ! src/java.base/share/classes/java/lang/reflect/Executable.java Changeset: 07c09e65ca0f Author: pmuthuswamy Date: 2019-01-16 11:15 +0530 URL: http://hg.openjdk.java.net/panama/dev/rev/07c09e65ca0f 8202626: javadoc generates broken links to Reviewed-by: jjg ! src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/formats/html/PackageUseWriter.java ! test/langtools/jdk/javadoc/doclet/testPackageDeprecation/TestPackageDeprecation.java ! test/langtools/jdk/javadoc/doclet/testUseOption/TestUseOption.java Changeset: cfc839f28b89 Author: mdoerr Date: 2019-01-15 10:23 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/cfc839f28b89 8216426: Usage of array placement new may lead to memory corruption Reviewed-by: rehn, kbarrett, rkennke, eosterlund ! src/hotspot/share/utilities/concurrentHashTable.hpp ! src/hotspot/share/utilities/concurrentHashTable.inline.hpp Changeset: 91ab128a65a3 Author: mdoerr Date: 2019-01-16 10:16 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/91ab128a65a3 8216556: Unnecessary liveness computation with JVMTI Reviewed-by: redestad, dlong, kvn ! src/hotspot/share/ci/ciEnv.cpp ! src/hotspot/share/ci/ciEnv.hpp ! src/hotspot/share/ci/ciMethod.cpp Changeset: 61866ba87b31 Author: goetz Date: 2019-01-15 12:02 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/61866ba87b31 8217044: [aix] Launcher still adds old path to jli library to LIBPATH Reviewed-by: ihse, rriggs, dholmes ! src/java.base/unix/native/libjli/java_md_solinux.c Changeset: 6bd052801d02 Author: vromero Date: 2019-01-16 07:01 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/6bd052801d02 8216529: in case of a crash, javac should print out the parameters passed to it Reviewed-by: jjg, cushon ! src/jdk.compiler/share/classes/com/sun/tools/javac/main/Main.java ! src/jdk.compiler/share/classes/com/sun/tools/javac/resources/javac.properties Changeset: 331ba84b1e36 Author: zgu Date: 2019-01-09 19:05 -0500 URL: http://hg.openjdk.java.net/panama/dev/rev/331ba84b1e36 8215299: Remove G1CMTask::should_exit_termination()'s undesirable side-effect Reviewed-by: kbarrett, rkennke, tschatzl ! src/hotspot/share/gc/g1/g1ConcurrentMark.cpp ! src/hotspot/share/gc/g1/g1ConcurrentMark.hpp ! src/hotspot/share/gc/g1/g1ConcurrentMark.inline.hpp Changeset: d3aa93570779 Author: igerasim Date: 2019-01-16 10:12 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/d3aa93570779 8007606: Handle realloc() failure in unix/native/libnet/net_util_md.c correctly Reviewed-by: clanger, mbaesken ! src/java.base/unix/native/libnet/net_util_md.c Changeset: a47b8125b7cc Author: dfuchs Date: 2019-01-16 19:09 +0000 URL: http://hg.openjdk.java.net/panama/dev/rev/a47b8125b7cc 8217094: HttpClient SSL race if a socket IOException is raised before ALPN is available Summary: The patch makes suer that the SSLFlowDelegate's ALPN CF is always completed Reviewed-by: chegar ! src/java.net.http/share/classes/jdk/internal/net/http/common/SSLFlowDelegate.java ! src/java.net.http/share/classes/jdk/internal/net/http/common/SSLTube.java ! src/java.net.http/share/classes/jdk/internal/net/http/common/SubscriberWrapper.java + test/jdk/java/net/httpclient/ALPNFailureTest.java + test/jdk/java/net/httpclient/ALPNProxyFailureTest.java ! test/jdk/java/net/httpclient/DigestEchoServer.java Changeset: bdb29aa5fd31 Author: weijun Date: 2019-01-16 11:25 +0800 URL: http://hg.openjdk.java.net/panama/dev/rev/bdb29aa5fd31 8215694: keytool cannot generate RSASSA-PSS certificates Reviewed-by: xuelei ! src/java.base/share/classes/java/security/spec/PSSParameterSpec.java ! src/java.base/share/classes/sun/security/pkcs10/PKCS10.java ! src/java.base/share/classes/sun/security/rsa/PSSParameters.java ! src/java.base/share/classes/sun/security/tools/keytool/CertAndKeyGen.java ! src/java.base/share/classes/sun/security/tools/keytool/Main.java ! src/java.base/share/classes/sun/security/x509/AlgorithmId.java ! src/java.base/share/classes/sun/security/x509/X509CertImpl.java + test/jdk/sun/security/tools/keytool/PSS.java Changeset: ac431929db51 Author: never Date: 2019-01-15 22:59 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/ac431929db51 8215748: Application fails when executed with Graal Reviewed-by: iveresov, kvn, thartmann ! src/hotspot/share/jvmci/jvmciCompilerToVM.cpp ! test/hotspot/jtreg/compiler/jvmci/jdk.vm.ci.runtime.test/src/jdk/vm/ci/runtime/test/ResolvedJavaTypeResolveMethodTest.java Changeset: a6620d37728b Author: lucy Date: 2019-01-16 09:48 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/a6620d37728b 8216314: SIGILL in CodeHeapState::print_names() Reviewed-by: thartmann, kvn ! src/hotspot/share/code/codeHeapState.cpp ! src/hotspot/share/code/codeHeapState.hpp ! src/hotspot/share/compiler/compileBroker.cpp Changeset: 528207d2e73e Author: tschatzl Date: 2019-01-16 11:43 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/528207d2e73e 8216490: Spammy periodic GC log message contains random time stamp with periodic gc disabled Summary: Print periodic gc status at startup and only print regular messages if enabled. Reviewed-by: sangheki, kbarrett ! src/hotspot/share/gc/g1/g1YoungRemSetSamplingThread.cpp + test/hotspot/jtreg/gc/g1/TestPeriodicLogMessages.java Changeset: dbbe6654948d Author: roland Date: 2019-01-14 13:53 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/dbbe6654948d 8217043: Shenandoah: SIGSEGV in Type::meet_helper() at barrier expansion time Reviewed-by: shade, rkennke, thartmann ! src/hotspot/share/gc/shenandoah/c2/shenandoahSupport.cpp Changeset: 7c68a23014db Author: roland Date: 2019-01-11 14:27 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/7c68a23014db 8217042: Shenandoah: write barrier on backedge of strip mined loop causes c2 crash at expansion time Reviewed-by: rkennke, thartmann ! src/hotspot/share/gc/shenandoah/c2/shenandoahSupport.cpp Changeset: c52a37f40324 Author: vlivanov Date: 2019-01-15 16:41 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/c52a37f40324 8215757: C2: PhaseIdealLoop::create_new_if_for_predicate() computes wrong IDOM Reviewed-by: kvn, roland ! src/hotspot/share/opto/loopPredicate.cpp Changeset: 659b004b6a1b Author: mhalder Date: 2019-01-16 23:56 +0530 URL: http://hg.openjdk.java.net/panama/dev/rev/659b004b6a1b 8215280: Double click on titlebar not working for Frame with extended state set to MAXIMIZED_BOTH Reviewed-by: serb, kaddepalli ! src/java.desktop/macosx/classes/sun/lwawt/macosx/CPlatformWindow.java ! src/java.desktop/macosx/native/libawt_lwawt/awt/AWTWindow.m ! test/jdk/java/awt/Frame/UnfocusableMaximizedFrameResizablity/UnfocusableMaximizedFrameResizablity.java Changeset: cb4212fda8e4 Author: xuelei Date: 2019-01-16 11:19 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/cb4212fda8e4 8216045: The size of key_exchange may be wrong on FFDHE Reviewed-by: jnimeh ! src/java.base/share/classes/sun/security/ssl/DHKeyExchange.java Changeset: 58e25974ede4 Author: jwilhelm Date: 2019-01-16 20:53 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/58e25974ede4 Merge ! src/hotspot/share/code/codeHeapState.cpp ! src/hotspot/share/gc/g1/g1YoungRemSetSamplingThread.cpp ! src/java.desktop/macosx/classes/sun/lwawt/macosx/CPlatformWindow.java ! src/java.desktop/macosx/native/libawt_lwawt/awt/AWTWindow.m Changeset: 4a59f7042325 Author: tschatzl Date: 2019-01-16 21:37 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/4a59f7042325 8217203: Some more includes to .inline.hpp files in gc header files Reviewed-by: zgu, kbarrett, sangheki ! src/hotspot/share/gc/shared/parallelCleaning.hpp ! src/hotspot/share/gc/shared/workerManager.hpp ! src/hotspot/share/gc/shenandoah/shenandoahStringDedup.hpp From maurizio.cimadamore at oracle.com Wed Jan 16 21:10:44 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 16 Jan 2019 21:10:44 +0000 Subject: hg: panama/dev: Automatic merge with default Message-ID: <201901162110.x0GLAid0012247@aojmv0008.oracle.com> Changeset: 3b32ebedd86b Author: mcimadamore Date: 2019-01-16 22:10 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/3b32ebedd86b Automatic merge with default - test/jdk/java/lang/String/AlignIndent.java - test/langtools/jdk/javadoc/doclet/testHtmlLandmarkRegions/TestHtmlLankmarkRegions.java - test/langtools/tools/javac/RawStringLiteralLang.java - test/langtools/tools/javac/RawStringLiteralLangAPI.java - test/langtools/tools/javac/diags/examples/RawStringLiteral.java From maurizio.cimadamore at oracle.com Wed Jan 16 21:10:15 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 16 Jan 2019 21:10:15 +0000 Subject: hg: panama/dev: Automatic merge with default Message-ID: <201901162110.x0GLAFIZ011785@aojmv0008.oracle.com> Changeset: 60d97c39b669 Author: mcimadamore Date: 2019-01-16 22:10 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/60d97c39b669 Automatic merge with default ! make/RunTests.gmk ! make/autoconf/basics.m4 ! make/conf/jib-profiles.js ! src/hotspot/cpu/x86/assembler_x86.hpp ! src/hotspot/cpu/x86/macroAssembler_x86.hpp ! src/hotspot/share/logging/logTag.hpp - test/jdk/java/lang/String/AlignIndent.java - test/langtools/jdk/javadoc/doclet/testHtmlLandmarkRegions/TestHtmlLankmarkRegions.java - test/langtools/tools/javac/RawStringLiteralLang.java - test/langtools/tools/javac/RawStringLiteralLangAPI.java - test/langtools/tools/javac/diags/examples/RawStringLiteral.java From jbvernee at xs4all.nl Wed Jan 16 22:01:39 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Wed, 16 Jan 2019 23:01:39 +0100 Subject: [foreign] RFR 8217164: Enable jextract to handle function types in parameters Message-ID: <85cc47682511a5794ba8e3f1830fa410@xs4all.nl> Hi, Please review the following patch which allows jextract to handle function signatures with simple function types as parameter types e.g.: void func(void callback(int)) { // function taking a pointer to function taking an int // ... } Bug: https://bugs.openjdk.java.net/browse/JDK-8217164 Webrev: http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217164/webrev.00/ Thanks, Jorn From sundararajan.athijegannathan at oracle.com Thu Jan 17 04:56:15 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Thu, 17 Jan 2019 10:26:15 +0530 Subject: [foreign] RFR 8217246: jextract should parse single header file with clang In-Reply-To: References: <5C3F2A1F.5010906@oracle.com> Message-ID: <5C400AEF.7070801@oracle.com> Yes, you're right. 'distinct' call can be removed. Updated: https://cr.openjdk.java.net/~sundar/8217246/webrev.01/ All tests and samples fine on Mac. Tests fine on Linux. Thanks, -Sundar On 16/01/19, 11:25 PM, Maurizio Cimadamore wrote: > Looks good - I have one question: do we still need to call 'distinct' > in JextractTool::processHeaders ? > > That is, shouldn't it be the case that now the parser will only give > us a given cursor only once? I added this step because, since we were > doing multiple clang passes, it was sometimes necessary to filter out > duplicates (e.g. if two input headers both included time.h) - but now > that we unify the headers under a single clang compilation unit, > shouldn't the clang API already give us unique entities? > > [I'm saying this because I suspect that this distinct() call is > causing much of the overhead introduced by the recent jextract > refactoring] > > Maurizio > > On 16/01/2019 12:57, Sundararajan Athijegannathan wrote: >> Please review. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8217246 >> Webrev: https://cr.openjdk.java.net/~sundar/8217246/webrev.00/ >> >> Thanks, >> -Sundar From sundararajan.athijegannathan at oracle.com Thu Jan 17 06:48:48 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Thu, 17 Jan 2019 12:18:48 +0530 Subject: [foreign] RFR 8217164: Enable jextract to handle function types in parameters In-Reply-To: <85cc47682511a5794ba8e3f1830fa410@xs4all.nl> References: <85cc47682511a5794ba8e3f1830fa410@xs4all.nl> Message-ID: <5C402550.7020403@oracle.com> Looks good. Built & tested on Mac. Minor: test assumes testCB(int i) is reached and asserts the value. What if the test method was not reached? Perhaps reached boolean flag that is set inside the method testCB and assertion on field from the caller? -Sundar On 17/01/19, 3:31 AM, Jorn Vernee wrote: > Hi, > > Please review the following patch which allows jextract to handle > function signatures with simple function types as parameter types e.g.: > > void func(void callback(int)) { // function taking a pointer to > function taking an int > // ... > } > > Bug: https://bugs.openjdk.java.net/browse/JDK-8217164 > Webrev: > http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217164/webrev.00/ > > Thanks, > Jorn From jbvernee at xs4all.nl Thu Jan 17 07:54:29 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Thu, 17 Jan 2019 08:54:29 +0100 Subject: [foreign] RFR 8217246: jextract should parse single header file with clang In-Reply-To: <5C400AEF.7070801@oracle.com> References: <5C3F2A1F.5010906@oracle.com> <5C400AEF.7070801@oracle.com> Message-ID: <3c77c41c1ee933780027c0a1545515b3@xs4all.nl> Tests passing on Windows as well ?? Jorn Sundararajan Athijegannathan schreef op 2019-01-17 05:56: > Yes, you're right. 'distinct' call can be removed. > > Updated: https://cr.openjdk.java.net/~sundar/8217246/webrev.01/ > > All tests and samples fine on Mac. Tests fine on Linux. > > Thanks, > -Sundar > > On 16/01/19, 11:25 PM, Maurizio Cimadamore wrote: >> Looks good - I have one question: do we still need to call 'distinct' >> in JextractTool::processHeaders ? >> >> That is, shouldn't it be the case that now the parser will only give >> us a given cursor only once? I added this step because, since we were >> doing multiple clang passes, it was sometimes necessary to filter out >> duplicates (e.g. if two input headers both included time.h) - but now >> that we unify the headers under a single clang compilation unit, >> shouldn't the clang API already give us unique entities? >> >> [I'm saying this because I suspect that this distinct() call is >> causing much of the overhead introduced by the recent jextract >> refactoring] >> >> Maurizio >> >> On 16/01/2019 12:57, Sundararajan Athijegannathan wrote: >>> Please review. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8217246 >>> Webrev: https://cr.openjdk.java.net/~sundar/8217246/webrev.00/ >>> >>> Thanks, >>> -Sundar From jbvernee at xs4all.nl Thu Jan 17 08:19:40 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Thu, 17 Jan 2019 09:19:40 +0100 Subject: [foreign] RFR 8217164: Enable jextract to handle function types in parameters In-Reply-To: <5C402550.7020403@oracle.com> References: <85cc47682511a5794ba8e3f1830fa410@xs4all.nl> <5C402550.7020403@oracle.com> Message-ID: <99a17b9bf4c598e8ae0f6dddb42ec93b@xs4all.nl> Thanks, I added the check. Updated webrev: http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217164/webrev.01/ Jorn Sundararajan Athijegannathan schreef op 2019-01-17 07:48: > Looks good. Built & tested on Mac. > > Minor: test assumes testCB(int i) is reached and asserts the value. > What if the test method was not reached? Perhaps reached boolean flag > that is set inside the method testCB and assertion on field from the > caller? > > -Sundar > > On 17/01/19, 3:31 AM, Jorn Vernee wrote: >> Hi, >> >> Please review the following patch which allows jextract to handle >> function signatures with simple function types as parameter types >> e.g.: >> >> void func(void callback(int)) { // function taking a pointer to >> function taking an int >> // ... >> } >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8217164 >> Webrev: >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217164/webrev.00/ >> >> Thanks, >> Jorn From sundararajan.athijegannathan at oracle.com Thu Jan 17 10:18:10 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Thu, 17 Jan 2019 15:48:10 +0530 Subject: [foreign] RFR 8217164: Enable jextract to handle function types in parameters In-Reply-To: <99a17b9bf4c598e8ae0f6dddb42ec93b@xs4all.nl> References: <85cc47682511a5794ba8e3f1830fa410@xs4all.nl> <5C402550.7020403@oracle.com> <99a17b9bf4c598e8ae0f6dddb42ec93b@xs4all.nl> Message-ID: <5C405662.10808@oracle.com> Looks good. -Sundar On 17/01/19, 1:49 PM, Jorn Vernee wrote: > Thanks, I added the check. > > Updated webrev: > http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217164/webrev.01/ > > Jorn > > Sundararajan Athijegannathan schreef op 2019-01-17 07:48: >> Looks good. Built & tested on Mac. >> >> Minor: test assumes testCB(int i) is reached and asserts the value. >> What if the test method was not reached? Perhaps reached boolean flag >> that is set inside the method testCB and assertion on field from the >> caller? >> >> -Sundar >> >> On 17/01/19, 3:31 AM, Jorn Vernee wrote: >>> Hi, >>> >>> Please review the following patch which allows jextract to handle >>> function signatures with simple function types as parameter types e.g.: >>> >>> void func(void callback(int)) { // function taking a pointer to >>> function taking an int >>> // ... >>> } >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8217164 >>> Webrev: >>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217164/webrev.00/ >>> >>> Thanks, >>> Jorn From maurizio.cimadamore at oracle.com Thu Jan 17 11:01:46 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 17 Jan 2019 11:01:46 +0000 Subject: [foreign] RFR 8217246: jextract should parse single header file with clang In-Reply-To: <5C400AEF.7070801@oracle.com> References: <5C3F2A1F.5010906@oracle.com> <5C400AEF.7070801@oracle.com> Message-ID: <7b0a7a8a-4615-30d6-12a6-51be7e918c6f@oracle.com> +1 Maurizio On 17/01/2019 04:56, Sundararajan Athijegannathan wrote: > Yes, you're right. 'distinct' call can be removed. > > Updated: https://cr.openjdk.java.net/~sundar/8217246/webrev.01/ > > All tests and samples fine on Mac. Tests fine on Linux. > > Thanks, > -Sundar > > On 16/01/19, 11:25 PM, Maurizio Cimadamore wrote: >> Looks good - I have one question: do we still need to call 'distinct' >> in JextractTool::processHeaders ? >> >> That is, shouldn't it be the case that now the parser will only give >> us a given cursor only once? I added this step because, since we were >> doing multiple clang passes, it was sometimes necessary to filter out >> duplicates (e.g. if two input headers both included time.h) - but now >> that we unify the headers under a single clang compilation unit, >> shouldn't the clang API already give us unique entities? >> >> [I'm saying this because I suspect that this distinct() call is >> causing much of the overhead introduced by the recent jextract >> refactoring] >> >> Maurizio >> >> On 16/01/2019 12:57, Sundararajan Athijegannathan wrote: >>> Please review. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8217246 >>> Webrev: https://cr.openjdk.java.net/~sundar/8217246/webrev.00/ >>> >>> Thanks, >>> -Sundar From sundararajan.athijegannathan at oracle.com Thu Jan 17 11:21:15 2019 From: sundararajan.athijegannathan at oracle.com (sundararajan.athijegannathan at oracle.com) Date: Thu, 17 Jan 2019 11:21:15 +0000 Subject: hg: panama/dev: 8217246: jextract should parse single header file with clang Message-ID: <201901171121.x0HBLGMu025311@aojmv0008.oracle.com> Changeset: b67b1c30354d Author: sundar Date: 2019-01-17 16:49 +0530 URL: http://hg.openjdk.java.net/panama/dev/rev/b67b1c30354d 8217246: jextract should parse single header file with clang Reviewed-by: mcimadamore ! src/jdk.jextract/share/classes/com/sun/tools/jextract/Context.java ! src/jdk.jextract/share/classes/com/sun/tools/jextract/DuplicateDeclarationHandler.java ! src/jdk.jextract/share/classes/com/sun/tools/jextract/EmptyNameHandler.java ! src/jdk.jextract/share/classes/com/sun/tools/jextract/JextractTool.java ! src/jdk.jextract/share/classes/com/sun/tools/jextract/TreeFilter.java ! src/jdk.jextract/share/classes/com/sun/tools/jextract/TypedefHandler.java ! src/jdk.jextract/share/classes/com/sun/tools/jextract/parser/FindSymbol.java ! src/jdk.jextract/share/classes/com/sun/tools/jextract/parser/Parser.java ! src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/Tree.java From sundararajan.athijegannathan at oracle.com Thu Jan 17 11:22:53 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Thu, 17 Jan 2019 16:52:53 +0530 Subject: [foreign] RFR 8217245:Unify functions and layouts In-Reply-To: References: Message-ID: <5C40658D.7010903@oracle.com> Looks good. PS. I built/tested this patch on Mac. All tests fine. -Sundar On 16/01/19, 5:51 PM, Maurizio Cimadamore wrote: > Hi, > this is a followup for the RFC here [1]. The goal is to put both > Layout and Function under a common root, called Descriptor (this name > is very broad so we will change it at a later stage). > > I've put together a new webrev which addresses some of the comments > raised in [1]. This is an official request for review. > > Webrev: > > http://cr.openjdk.java.net/~mcimadamore/panama/layout%2bfunc_v3/ > > Cheers > Maurizio > > [1] - > https://mail.openjdk.java.net/pipermail/panama-dev/2018-November/003279.html > From zhuoren.wz at alibaba-inc.com Thu Jan 17 11:26:00 2019 From: zhuoren.wz at alibaba-inc.com (=?UTF-8?B?V2FuZyBaaHVvKFpodW9yZW4p?=) Date: Thu, 17 Jan 2019 19:26:00 +0800 Subject: =?UTF-8?B?UmU6IFtWZWN0b3JBUEldbWFza0Zyb21BcnJheSBJbnRyaW5zaWMgZmFpbHVyZQ==?= In-Reply-To: <4c086c97-ac8c-a07b-4535-40194338243e@oracle.com> References: <701fcf82-7ace-4e66-910b-3120753809a8.zhuoren.wz@alibaba-inc.com> <7b720dbb-91d8-4bef-e680-9474ead264f2@oracle.com> <77404c4b-cfa8-4f47-b58f-533ae0d9a1e6.zhuoren.wz@alibaba-inc.com>, <4c086c97-ac8c-a07b-4535-40194338243e@oracle.com> Message-ID: <7cb01cce-771f-49f5-bcc3-397a1e41a208.zhuoren.wz@alibaba-inc.com> > As a followup, it would be nice to refactor/cleanup Mask-related code on > > JDK-JVM boundary (VectorIntrinsics + library_call.cpp). For example, I > don't see a reason to pass int.class (and not boolean.class) as the > element type for mask-related operations. Currently int.class is used by load instrinsics to box/unbox mask vectors. Is it better to remove int.class and get neccesarry information from other parameters such as Int128Mask.class? Regards, Zhuo ------------------------------------------------------------------ Sender:Vladimir Ivanov Sent At:2019 Jan. 17 (Thu.) 04:40 Recipient:Sandler ; panama-dev Cc:li sanhong ; Kingsum Chow ; Jonathan Subject:Re: [VectorAPI]maskFromArray Intrinsic failure Looks good. Best regards, Vladimir Ivanov On 16/01/2019 04:46, Wang Zhuo(Zhuoren) wrote: > Webrev files have been uploaded to > http://cr.openjdk.java.net/~luchsh/mask_fix_zhuoren/. Please review. > > Regards, > Zhuoren > > ------------------------------------------------------------------ > From:Vladimir Ivanov > Sent At:2019 Jan. 16 (Wed.) 02:46 > To:Sandler ; panama-dev > > Cc:li sanhong ; Kingsum Chow > > Subject:Re: [VectorAPI]maskFromArray Intrinsic failure > > Good catch! > > src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Int128Vector.java: > > - bits, (((long) ix) << > ARRAY_SHIFT) + Unsafe.ARRAY_BOOLEAN_BASE_OFFSET, > > + bits, ((long) ix) + > Unsafe.ARRAY_BOOLEAN_BASE_OFFSET, > > It's better to replace ARRAY_SHIFT with Unsafe.ARRAY_BOOLEAN_INDEX_SCALE. > > > src/hotspot/share/opto/library_call.cpp: > > > + if (!arch_supports_vector(Op_LoadVector, num_elem, T_BOOLEAN, > VecMaskUseLoad)) { > > Why do you specify VecMaskUseLoad and not VecMaskNotUsed (as in other > cases)? > > Otherwise, looks good for now. Please, send complete webrev for review. > > As a followup, it would be nice to refactor/cleanup Mask-related code on > > JDK-JVM boundary (VectorIntrinsics + library_call.cpp). For example, I > don't see a reason to pass int.class (and not boolean.class) as the > element type for mask-related operations. > > Best regards, > Vladimir Ivanov > > On 15/01/2019 08:37, Wang Zhuo(Zhuoren) wrote: > > Hi, > > I found intrinsics for maskFromArray always failed. Here is a patch to fix this issue. > > Benchmarks containing heavy maskFromArray use can benifit from this patch. such as allTrue/anyTrue. > > > > This patch consists of two parts. In the JDK part, since boolean is only one byte, it is not correct to perform left shift ARRAY_SHIFT for array index. I listed only one file in the patch to avoid a long letter, but this modification should be applied to all types including the template file. > > return VectorIntrinsics.load(Int128Mask.class, int.class, LENGTH, > > - bits, (((long) ix) << ARRAY_SHIFT) + Unsafe.ARRAY_BOOLEAN_BASE_OFFSET, > > + bits, ((long) ix) + Unsafe.ARRAY_BOOLEAN_BASE_OFFSET, > > bits, ix, > > Another part is in hotspot, please check the below patch > > > > diff -r 890e996dfd1a src/hotspot/share/opto/library_call.cpp > > --- a/src/hotspot/share/opto/library_call.cpp Mon Jan 07 10:54:21 2019 +0800 > > +++ b/src/hotspot/share/opto/library_call.cpp Tue Jan 15 23:45:40 2019 +0800 > > @@ -7069,13 +7069,13 @@ > > > > // Now handle special case where load/store happens from/to byte array but element type is not byte. > > bool using_byte_array = arr_type != NULL && arr_type->elem()->array_element_basic_type() == T_BYTE && elem_bt != T_BYTE; > > - > > - // It must be the case that if it is not special byte array case, there is consistency between > > - // array and vector element types. > > - if (!using_byte_array && arr_type != NULL && elem_bt != arr_type->elem()->array_element_basic_type()) { > > + // Handle loading masks. > > + ciKlass* vbox_klass = vector_klass->const_oop()->as_instance()->java_lang_Class_klass(); > > + bool is_mask = vbox_klass->is_vectormask(); > > + // If there is no consistency between array and vector element types, it must be special byte array case or loading masks > > + if (!using_byte_array && arr_type != NULL && elem_bt != arr_type->elem()->array_element_basic_type() && !is_mask) { > > return false; > > } > > - > > // Since we are using byte array, we need to double check that the byte operations are supported by backend. > > if (using_byte_array) { > > int byte_num_elem = num_elem * type2aelembytes(elem_bt); > > @@ -7083,8 +7083,12 @@ > > return false; // not supported > > } > > } > > - > > - ciKlass* vbox_klass = vector_klass->const_oop()->as_instance()->java_lang_Class_klass(); > > + if (is_mask) { > > + if (!arch_supports_vector(Op_LoadVector, num_elem, T_BOOLEAN, VecMaskUseLoad)) { > > + return false; // not supported > > + } > > + } > > + > > const TypeInstPtr* vbox_type = TypeInstPtr::make_exact(TypePtr::NotNull, vbox_klass); > > > > if (is_store) { > > @@ -7114,9 +7118,15 @@ > > const TypeVect* to_vect_type = TypeVect::make(elem_bt, num_elem); > > vload = gvn().transform(new VectorReinterpretNode(vload, vload->bottom_type()->is_vect(), to_vect_type)); > > } else { > > - vload = gvn().transform(LoadVectorNode::make(0, control(), memory(addr), addr, addr_type, num_elem, elem_bt)); > > - } > > - > > + // Special handle for masks > > + if (is_mask) { > > + vload = gvn().transform(LoadVectorNode::make(0, control(), memory(addr), addr, addr_type, num_elem, T_BOOLEAN)); > > + const TypeVect* to_vect_type = TypeVect::make(elem_bt, num_elem); > > + vload = gvn().transform(new VectorLoadMaskNode(vload, to_vect_type)); > > + } else { > > + vload = gvn().transform(LoadVectorNode::make(0, control(), memory(addr), addr, addr_type, num_elem, elem_bt)); > > + } > > + } > > Node* box = box_vector(vload, vbox_type, elem_bt, num_elem); > > set_vector_result(box); > > } > > --- a/src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Int128Vector.java Mon Jan 07 10:54:21 2019 +0800 > > +++ b/src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Int128Vector.java Tue Jan 15 23:45:40 2019 +0800 > > @@ -1338,7 +1338,7 @@ > > Objects.requireNonNull(bits); > > ix = VectorIntrinsics.checkIndex(ix, bits.length, LENGTH); > > return VectorIntrinsics.load(Int128Mask.class, int.class, LENGTH, > > - bits, (((long) ix) << ARRAY_SHIFT) + Unsafe.ARRAY_BOOLEAN_BASE_OFFSET, > > + bits, ((long) ix) + Unsafe.ARRAY_BOOLEAN_BASE_OFFSET, > > bits, ix, > > (c, idx) -> opm(n -> c[idx + n])); > > > > Regards, > > Zhuoren > > > From jbvernee at xs4all.nl Thu Jan 17 11:39:20 2019 From: jbvernee at xs4all.nl (jbvernee at xs4all.nl) Date: Thu, 17 Jan 2019 11:39:20 +0000 Subject: hg: panama/dev: 8217164: jextract can not handle function prototypes as function arguments. Message-ID: <201901171139.x0HBdLs2004440@aojmv0008.oracle.com> Changeset: ff814631de1f Author: jvernee Date: 2019-01-17 12:38 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/ff814631de1f 8217164: jextract can not handle function prototypes as function arguments. Summary: Enable jextract to handle function types in parameters Reviewed-by: sundar ! src/jdk.jextract/share/classes/com/sun/tools/jextract/TypeDictionary.java ! src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/LayoutUtils.java + test/jdk/com/sun/tools/jextract/testSimpleFuncParam/SimpleFuncParamTest.java + test/jdk/com/sun/tools/jextract/testSimpleFuncParam/funcParam.h + test/jdk/com/sun/tools/jextract/testSimpleFuncParam/libFuncParam.c From jbvernee at xs4all.nl Thu Jan 17 13:04:50 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Thu, 17 Jan 2019 14:04:50 +0100 Subject: [foreign] RFC: Lazy symbol resolution Message-ID: Hi, Currently HeaderImplGenerator eagerly resolves symbols from the native header interface while generating the backing implementation. Sometimes the symbols defined in a single header are compiled into multiple shared libraries. All of those libraries have to be included when jextracting the header with `-l`, and all of them have to be loaded when generating the header interface implementation. Because symbols are eagerly resolved, every symbol in a header has to be loaded at the time of header generation. I've thrown together a patch which trades in the eager resolution for lazy resolution using dynamic constants that were recently added to the jdk. Using the flag java.foreign.binder.LAZY_RESOLUTION=true the bahaviour can be turned on. The main advantage is that not every symbol in a header interface has to be present at runtime, while symbols that _are_ present can still be used. Webrev: http://cr.openjdk.java.net/~jvernee/panama/webrevs/lazyresolution/webrev.00/ What do you think of this idea? Is lazy resolution desirable? Jorn From maurizio.cimadamore at oracle.com Thu Jan 17 13:55:35 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Thu, 17 Jan 2019 13:55:35 +0000 Subject: hg: panama/dev: 8217245: Unify functions and layouts Message-ID: <201901171355.x0HDtZrJ008437@aojmv0008.oracle.com> Changeset: b981c23cb71e Author: mcimadamore Date: 2019-01-17 13:54 +0000 URL: http://hg.openjdk.java.net/panama/dev/rev/b981c23cb71e 8217245: Unify functions and layouts Reviewed-by: sundar + src/java.base/share/classes/java/foreign/layout/AbstractDescriptor.java - src/java.base/share/classes/java/foreign/layout/AbstractLayout.java ! src/java.base/share/classes/java/foreign/layout/Address.java + src/java.base/share/classes/java/foreign/layout/Descriptor.java ! src/java.base/share/classes/java/foreign/layout/Function.java ! src/java.base/share/classes/java/foreign/layout/Group.java ! src/java.base/share/classes/java/foreign/layout/Layout.java ! src/java.base/share/classes/java/foreign/layout/Padding.java ! src/java.base/share/classes/java/foreign/layout/Sequence.java ! src/java.base/share/classes/java/foreign/layout/Unresolved.java ! src/java.base/share/classes/java/foreign/layout/Value.java ! src/java.base/share/classes/jdk/internal/foreign/HeaderImplGenerator.java ! src/java.base/share/classes/jdk/internal/foreign/Util.java ! src/java.base/share/classes/jdk/internal/foreign/memory/DescriptorParser.java ! test/jdk/com/sun/tools/jextract/testStruct/StructTest.java From maurizio.cimadamore at oracle.com Thu Jan 17 14:00:06 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 17 Jan 2019 14:00:06 +0000 Subject: [foreign] RFC: Lazy symbol resolution In-Reply-To: References: Message-ID: <95396d31-d510-527c-9c33-5051251af513@oracle.com> Hi Jorn, this is an option I also considered in the past, e.g. using an indy to implement the method. At the time I decided against it, because I think that, in this phase, it would be preferrable to have eager failures in case something went wrong either at extraction time or at binding time. And I think that would still be the case. If you don't mind, let's keep this in our backpocket - it is clearly where we need to be longer term, but I'm not sure that now is the right time to be adding this functionality, as it can hide test failures (even though I notice that the flag is disabled by default). Maurizio On 17/01/2019 13:04, Jorn Vernee wrote: > Hi, > > Currently HeaderImplGenerator eagerly resolves symbols from the native > header interface while generating the backing implementation. > Sometimes the symbols defined in a single header are compiled into > multiple shared libraries. All of those libraries have to be included > when jextracting the header with `-l`, and all of them have to be > loaded when generating the header interface implementation. Because > symbols are eagerly resolved, every symbol in a header has to be > loaded at the time of header generation. > > I've thrown together a patch which trades in the eager resolution for > lazy resolution using dynamic constants that were recently added to > the jdk. Using the flag java.foreign.binder.LAZY_RESOLUTION=true the > bahaviour can be turned on. The main advantage is that not every > symbol in a header interface has to be present at runtime, while > symbols that _are_ present can still be used. > > Webrev: > http://cr.openjdk.java.net/~jvernee/panama/webrevs/lazyresolution/webrev.00/ > > What do you think of this idea? Is lazy resolution desirable? > > Jorn From jbvernee at xs4all.nl Thu Jan 17 16:36:36 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Thu, 17 Jan 2019 17:36:36 +0100 Subject: [foreign] Instructions for using libraries on Windows Message-ID: Hi, I have updated panama_foreign.md with instructions for jextracting and running OpenGL and Python. I have also tried other Windows applicable libraries, here were the issues: LAPACK & BLAS: require libgfortran-3 as a dependency, which I couldn't find a binary distro for so quickly. (MINGW has a package for it, but only libgfortran-4) Tensorflow: is crashing jextract with a stack overflow error. ``` Exception in thread "main" java.lang.StackOverflowError at jdk.internal.clang/jdk.internal.clang.StructType.(StructType.java:37) at jdk.internal.clang/jdk.internal.clang.StructType.(StructType.java:32) at jdk.internal.clang/jdk.internal.clang.Type.(Type.java:30) at jdk.internal.clang/jdk.internal.clang.Type.canonicalType(Native Method) at jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) at jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) at jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) at jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) ... ``` FWIW, you made a note about needing to pass `-C -c -C c++` to jextract so it doesn't interpret some methods as varargs. I've tried without those flags and am observing the same vararg problem, but in that case extraction does succeed. The other libraries libproc, readline, and unistd don't seem applicable to Windows. I will be on the lookout for Windows specific examples that could be interesting. The `make update-build-doc` does not work on windows [1], so I ask that you generate the .html dos for me instead. Here is the updated panama_foreign.md: http://cr.openjdk.java.net/~jvernee/panama/panama_foreign.md Jorn [1] : https://mail.openjdk.java.net/pipermail/build-dev/2019-January/024666.html From jbvernee at xs4all.nl Thu Jan 17 16:48:34 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Thu, 17 Jan 2019 17:48:34 +0100 Subject: [foreign] Instructions for using libraries on Windows In-Reply-To: References: Message-ID: > The `make update-build-doc` does not work on windows [1], so I ask > that you generate the .html dos for me instead. Apologies. It seems to be a configuration problem on my end, and not necessarily a problem with Windows. Jorn Jorn Vernee schreef op 2019-01-17 17:36: > Hi, > > I have updated panama_foreign.md with instructions for jextracting and > running OpenGL and Python. I have also tried other Windows applicable > libraries, here were the issues: > > LAPACK & BLAS: require libgfortran-3 as a dependency, which I couldn't > find a binary distro for so quickly. (MINGW has a package for it, but > only libgfortran-4) > > Tensorflow: is crashing jextract with a stack overflow error. > > ``` > Exception in thread "main" java.lang.StackOverflowError > at > jdk.internal.clang/jdk.internal.clang.StructType.(StructType.java:37) > at > jdk.internal.clang/jdk.internal.clang.StructType.(StructType.java:32) > at > jdk.internal.clang/jdk.internal.clang.Type.(Type.java:30) > at > jdk.internal.clang/jdk.internal.clang.Type.canonicalType(Native > Method) > at > jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) > at > jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) > at > jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) > at > jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) > ... > ``` > > FWIW, you made a note about needing to pass `-C -c -C c++` to jextract > so it doesn't interpret some methods as varargs. I've tried without > those flags and am observing the same vararg problem, but in that case > extraction does succeed. > > The other libraries libproc, readline, and unistd don't seem > applicable to Windows. I will be on the lookout for Windows specific > examples that could be interesting. > > The `make update-build-doc` does not work on windows [1], so I ask > that you generate the .html dos for me instead. > > Here is the updated panama_foreign.md: > http://cr.openjdk.java.net/~jvernee/panama/panama_foreign.md > > Jorn > > [1] : > https://mail.openjdk.java.net/pipermail/build-dev/2019-January/024666.html From maurizio.cimadamore at oracle.com Thu Jan 17 17:20:53 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 17 Jan 2019 17:20:53 +0000 Subject: [foreign] Instructions for using libraries on Windows In-Reply-To: References: Message-ID: <86747435-1e9e-8a56-c4c7-60b99fec39c0@oracle.com> Let us know if you need help with pushing this Thanks for trying this out on Windows! Maurizio On 17/01/2019 16:48, Jorn Vernee wrote: >> The `make update-build-doc` does not work on windows [1], so I ask >> that you generate the .html dos for me instead. > > Apologies. It seems to be a configuration problem on my end, and not > necessarily a problem with Windows. > > Jorn > > Jorn Vernee schreef op 2019-01-17 17:36: >> Hi, >> >> I have updated panama_foreign.md with instructions for jextracting and >> running OpenGL and Python. I have also tried other Windows applicable >> libraries, here were the issues: >> >> LAPACK & BLAS: require libgfortran-3 as a dependency, which I couldn't >> find a binary distro for so quickly. (MINGW has a package for it, but >> only libgfortran-4) >> >> Tensorflow: is crashing jextract with a stack overflow error. >> >> ``` >> ??? Exception in thread "main" java.lang.StackOverflowError >> ??????? at >> jdk.internal.clang/jdk.internal.clang.StructType.(StructType.java:37) >> >> ??????? at >> jdk.internal.clang/jdk.internal.clang.StructType.(StructType.java:32) >> >> ??????? at >> jdk.internal.clang/jdk.internal.clang.Type.(Type.java:30) >> ??????? at >> jdk.internal.clang/jdk.internal.clang.Type.canonicalType(Native >> Method) >> ??????? at >> jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) >> >> ??????? at >> jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) >> >> ??????? at >> jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) >> >> ??????? at >> jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) >> >> ??????? ... >> ``` >> >> FWIW, you made a note about needing to pass `-C -c -C c++` to jextract >> so it doesn't interpret some methods as varargs. I've tried without >> those flags and am observing the same vararg problem, but in that case >> extraction does succeed. >> >> The other libraries libproc, readline, and unistd don't seem >> applicable to Windows. I will be on the lookout for Windows specific >> examples that could be interesting. >> >> The `make update-build-doc` does not work on windows [1], so I ask >> that you generate the .html dos for me instead. >> >> Here is the updated panama_foreign.md: >> http://cr.openjdk.java.net/~jvernee/panama/panama_foreign.md >> >> Jorn >> >> [1] : >> https://mail.openjdk.java.net/pipermail/build-dev/2019-January/024666.html From jbvernee at xs4all.nl Thu Jan 17 17:27:46 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Thu, 17 Jan 2019 18:27:46 +0100 Subject: [foreign] Instructions for using libraries on Windows In-Reply-To: <86747435-1e9e-8a56-c4c7-60b99fec39c0@oracle.com> References: <86747435-1e9e-8a56-c4c7-60b99fec39c0@oracle.com> Message-ID: I got the .html generation working with the help of Erik from build-dev, so I think I can manage myself. Want me to sent an RFR with an actual webrev as well? Jorn Maurizio Cimadamore schreef op 2019-01-17 18:20: > Let us know if you need help with pushing this > > Thanks for trying this out on Windows! > > Maurizio > > On 17/01/2019 16:48, Jorn Vernee wrote: >>> The `make update-build-doc` does not work on windows [1], so I ask >>> that you generate the .html dos for me instead. >> >> Apologies. It seems to be a configuration problem on my end, and not >> necessarily a problem with Windows. >> >> Jorn >> >> Jorn Vernee schreef op 2019-01-17 17:36: >>> Hi, >>> >>> I have updated panama_foreign.md with instructions for jextracting >>> and >>> running OpenGL and Python. I have also tried other Windows applicable >>> libraries, here were the issues: >>> >>> LAPACK & BLAS: require libgfortran-3 as a dependency, which I >>> couldn't >>> find a binary distro for so quickly. (MINGW has a package for it, but >>> only libgfortran-4) >>> >>> Tensorflow: is crashing jextract with a stack overflow error. >>> >>> ``` >>> ??? Exception in thread "main" java.lang.StackOverflowError >>> ??????? at >>> jdk.internal.clang/jdk.internal.clang.StructType.(StructType.java:37) >>> ??????? at >>> jdk.internal.clang/jdk.internal.clang.StructType.(StructType.java:32) >>> ??????? at >>> jdk.internal.clang/jdk.internal.clang.Type.(Type.java:30) >>> ??????? at >>> jdk.internal.clang/jdk.internal.clang.Type.canonicalType(Native >>> Method) >>> ??????? at >>> jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) >>> ??????? at >>> jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) >>> ??????? at >>> jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) >>> ??????? at >>> jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) >>> ??????? ... >>> ``` >>> >>> FWIW, you made a note about needing to pass `-C -c -C c++` to >>> jextract >>> so it doesn't interpret some methods as varargs. I've tried without >>> those flags and am observing the same vararg problem, but in that >>> case >>> extraction does succeed. >>> >>> The other libraries libproc, readline, and unistd don't seem >>> applicable to Windows. I will be on the lookout for Windows specific >>> examples that could be interesting. >>> >>> The `make update-build-doc` does not work on windows [1], so I ask >>> that you generate the .html dos for me instead. >>> >>> Here is the updated panama_foreign.md: >>> http://cr.openjdk.java.net/~jvernee/panama/panama_foreign.md >>> >>> Jorn >>> >>> [1] : >>> https://mail.openjdk.java.net/pipermail/build-dev/2019-January/024666.html From maurizio.cimadamore at oracle.com Thu Jan 17 18:14:44 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 17 Jan 2019 18:14:44 +0000 Subject: [foreign] Instructions for using libraries on Windows In-Reply-To: References: <86747435-1e9e-8a56-c4c7-60b99fec39c0@oracle.com> Message-ID: <1bfd7c81-4964-87e1-32a6-81745fc9abe6@oracle.com> The changes you made look good - only nit, in the Windows notes you mention "Visual Studios" (note the plural) a few times - I'd replace with the singular form. The rest is good Maurizio On 17/01/2019 17:27, Jorn Vernee wrote: > I got the .html generation working with the help of Erik from > build-dev, so I think I can manage myself. > > Want me to sent an RFR with an actual webrev as well? > > Jorn > > Maurizio Cimadamore schreef op 2019-01-17 18:20: >> Let us know if you need help with pushing this >> >> Thanks for trying this out on Windows! >> >> Maurizio >> >> On 17/01/2019 16:48, Jorn Vernee wrote: >>>> The `make update-build-doc` does not work on windows [1], so I ask >>>> that you generate the .html dos for me instead. >>> >>> Apologies. It seems to be a configuration problem on my end, and not >>> necessarily a problem with Windows. >>> >>> Jorn >>> >>> Jorn Vernee schreef op 2019-01-17 17:36: >>>> Hi, >>>> >>>> I have updated panama_foreign.md with instructions for jextracting and >>>> running OpenGL and Python. I have also tried other Windows applicable >>>> libraries, here were the issues: >>>> >>>> LAPACK & BLAS: require libgfortran-3 as a dependency, which I couldn't >>>> find a binary distro for so quickly. (MINGW has a package for it, but >>>> only libgfortran-4) >>>> >>>> Tensorflow: is crashing jextract with a stack overflow error. >>>> >>>> ``` >>>> ??? Exception in thread "main" java.lang.StackOverflowError >>>> ??????? at >>>> jdk.internal.clang/jdk.internal.clang.StructType.(StructType.java:37) >>>> ??????? at >>>> jdk.internal.clang/jdk.internal.clang.StructType.(StructType.java:32) >>>> ??????? at >>>> jdk.internal.clang/jdk.internal.clang.Type.(Type.java:30) >>>> ??????? at >>>> jdk.internal.clang/jdk.internal.clang.Type.canonicalType(Native >>>> Method) >>>> ??????? at >>>> jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) >>>> ??????? at >>>> jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) >>>> ??????? at >>>> jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) >>>> ??????? at >>>> jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) >>>> ??????? ... >>>> ``` >>>> >>>> FWIW, you made a note about needing to pass `-C -c -C c++` to jextract >>>> so it doesn't interpret some methods as varargs. I've tried without >>>> those flags and am observing the same vararg problem, but in that case >>>> extraction does succeed. >>>> >>>> The other libraries libproc, readline, and unistd don't seem >>>> applicable to Windows. I will be on the lookout for Windows specific >>>> examples that could be interesting. >>>> >>>> The `make update-build-doc` does not work on windows [1], so I ask >>>> that you generate the .html dos for me instead. >>>> >>>> Here is the updated panama_foreign.md: >>>> http://cr.openjdk.java.net/~jvernee/panama/panama_foreign.md >>>> >>>> Jorn >>>> >>>> [1] : >>>> https://mail.openjdk.java.net/pipermail/build-dev/2019-January/024666.html From jbvernee at xs4all.nl Thu Jan 17 18:30:06 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Thu, 17 Jan 2019 19:30:06 +0100 Subject: [foreign] Instructions for using libraries on Windows In-Reply-To: <1bfd7c81-4964-87e1-32a6-81745fc9abe6@oracle.com> References: <86747435-1e9e-8a56-c4c7-60b99fec39c0@oracle.com> <1bfd7c81-4964-87e1-32a6-81745fc9abe6@oracle.com> Message-ID: Good catch. Here is the updated webrev: http://cr.openjdk.java.net/~jvernee/panama/webrevs/instructions/webrev.01/ I've also added specific command lines for building/running on Windows (I was using intellij before). Jorn Maurizio Cimadamore schreef op 2019-01-17 19:14: > The changes you made look good - only nit, in the Windows notes you > mention "Visual Studios" (note the plural) a few times - I'd replace > with the singular form. > > The rest is good > > Maurizio > > On 17/01/2019 17:27, Jorn Vernee wrote: >> I got the .html generation working with the help of Erik from >> build-dev, so I think I can manage myself. >> >> Want me to sent an RFR with an actual webrev as well? >> >> Jorn >> >> Maurizio Cimadamore schreef op 2019-01-17 18:20: >>> Let us know if you need help with pushing this >>> >>> Thanks for trying this out on Windows! >>> >>> Maurizio >>> >>> On 17/01/2019 16:48, Jorn Vernee wrote: >>>>> The `make update-build-doc` does not work on windows [1], so I ask >>>>> that you generate the .html dos for me instead. >>>> >>>> Apologies. It seems to be a configuration problem on my end, and not >>>> necessarily a problem with Windows. >>>> >>>> Jorn >>>> >>>> Jorn Vernee schreef op 2019-01-17 17:36: >>>>> Hi, >>>>> >>>>> I have updated panama_foreign.md with instructions for jextracting >>>>> and >>>>> running OpenGL and Python. I have also tried other Windows >>>>> applicable >>>>> libraries, here were the issues: >>>>> >>>>> LAPACK & BLAS: require libgfortran-3 as a dependency, which I >>>>> couldn't >>>>> find a binary distro for so quickly. (MINGW has a package for it, >>>>> but >>>>> only libgfortran-4) >>>>> >>>>> Tensorflow: is crashing jextract with a stack overflow error. >>>>> >>>>> ``` >>>>> ??? Exception in thread "main" java.lang.StackOverflowError >>>>> ??????? at >>>>> jdk.internal.clang/jdk.internal.clang.StructType.(StructType.java:37) >>>>> ??????? at >>>>> jdk.internal.clang/jdk.internal.clang.StructType.(StructType.java:32) >>>>> ??????? at >>>>> jdk.internal.clang/jdk.internal.clang.Type.(Type.java:30) >>>>> ??????? at >>>>> jdk.internal.clang/jdk.internal.clang.Type.canonicalType(Native >>>>> Method) >>>>> ??????? at >>>>> jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) >>>>> ??????? at >>>>> jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) >>>>> ??????? at >>>>> jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) >>>>> ??????? at >>>>> jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) >>>>> ??????? ... >>>>> ``` >>>>> >>>>> FWIW, you made a note about needing to pass `-C -c -C c++` to >>>>> jextract >>>>> so it doesn't interpret some methods as varargs. I've tried without >>>>> those flags and am observing the same vararg problem, but in that >>>>> case >>>>> extraction does succeed. >>>>> >>>>> The other libraries libproc, readline, and unistd don't seem >>>>> applicable to Windows. I will be on the lookout for Windows >>>>> specific >>>>> examples that could be interesting. >>>>> >>>>> The `make update-build-doc` does not work on windows [1], so I ask >>>>> that you generate the .html dos for me instead. >>>>> >>>>> Here is the updated panama_foreign.md: >>>>> http://cr.openjdk.java.net/~jvernee/panama/panama_foreign.md >>>>> >>>>> Jorn >>>>> >>>>> [1] : >>>>> https://mail.openjdk.java.net/pipermail/build-dev/2019-January/024666.html From maurizio.cimadamore at oracle.com Thu Jan 17 18:37:30 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 17 Jan 2019 18:37:30 +0000 Subject: [foreign] Instructions for using libraries on Windows In-Reply-To: References: <86747435-1e9e-8a56-c4c7-60b99fec39c0@oracle.com> <1bfd7c81-4964-87e1-32a6-81745fc9abe6@oracle.com> Message-ID: Another note, we have separate sections for different platforms - I see that your changes create a single section for OpenGL for both Windows and Linux - would be better to tease them apart, and then just say that the example on Windows is the same (as done for other examples). Also, please separate installation from extraction, as it happens for the other sections. Cheers Maurizio On 17/01/2019 18:30, Jorn Vernee wrote: > Good catch. > > Here is the updated webrev: > http://cr.openjdk.java.net/~jvernee/panama/webrevs/instructions/webrev.01/ > > I've also added specific command lines for building/running on Windows > (I was using intellij before). > > Jorn > > Maurizio Cimadamore schreef op 2019-01-17 19:14: >> The changes you made look good - only nit, in the Windows notes you >> mention "Visual Studios" (note the plural) a few times - I'd replace >> with the singular form. >> >> The rest is good >> >> Maurizio >> >> On 17/01/2019 17:27, Jorn Vernee wrote: >>> I got the .html generation working with the help of Erik from >>> build-dev, so I think I can manage myself. >>> >>> Want me to sent an RFR with an actual webrev as well? >>> >>> Jorn >>> >>> Maurizio Cimadamore schreef op 2019-01-17 18:20: >>>> Let us know if you need help with pushing this >>>> >>>> Thanks for trying this out on Windows! >>>> >>>> Maurizio >>>> >>>> On 17/01/2019 16:48, Jorn Vernee wrote: >>>>>> The `make update-build-doc` does not work on windows [1], so I ask >>>>>> that you generate the .html dos for me instead. >>>>> >>>>> Apologies. It seems to be a configuration problem on my end, and >>>>> not necessarily a problem with Windows. >>>>> >>>>> Jorn >>>>> >>>>> Jorn Vernee schreef op 2019-01-17 17:36: >>>>>> Hi, >>>>>> >>>>>> I have updated panama_foreign.md with instructions for >>>>>> jextracting and >>>>>> running OpenGL and Python. I have also tried other Windows >>>>>> applicable >>>>>> libraries, here were the issues: >>>>>> >>>>>> LAPACK & BLAS: require libgfortran-3 as a dependency, which I >>>>>> couldn't >>>>>> find a binary distro for so quickly. (MINGW has a package for it, >>>>>> but >>>>>> only libgfortran-4) >>>>>> >>>>>> Tensorflow: is crashing jextract with a stack overflow error. >>>>>> >>>>>> ``` >>>>>> ??? Exception in thread "main" java.lang.StackOverflowError >>>>>> ??????? at >>>>>> jdk.internal.clang/jdk.internal.clang.StructType.(StructType.java:37) >>>>>> ??????? at >>>>>> jdk.internal.clang/jdk.internal.clang.StructType.(StructType.java:32) >>>>>> ??????? at >>>>>> jdk.internal.clang/jdk.internal.clang.Type.(Type.java:30) >>>>>> ??????? at >>>>>> jdk.internal.clang/jdk.internal.clang.Type.canonicalType(Native >>>>>> Method) >>>>>> ??????? at >>>>>> jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) >>>>>> ??????? at >>>>>> jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) >>>>>> ??????? at >>>>>> jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) >>>>>> ??????? at >>>>>> jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) >>>>>> ??????? ... >>>>>> ``` >>>>>> >>>>>> FWIW, you made a note about needing to pass `-C -c -C c++` to >>>>>> jextract >>>>>> so it doesn't interpret some methods as varargs. I've tried without >>>>>> those flags and am observing the same vararg problem, but in that >>>>>> case >>>>>> extraction does succeed. >>>>>> >>>>>> The other libraries libproc, readline, and unistd don't seem >>>>>> applicable to Windows. I will be on the lookout for Windows specific >>>>>> examples that could be interesting. >>>>>> >>>>>> The `make update-build-doc` does not work on windows [1], so I ask >>>>>> that you generate the .html dos for me instead. >>>>>> >>>>>> Here is the updated panama_foreign.md: >>>>>> http://cr.openjdk.java.net/~jvernee/panama/panama_foreign.md >>>>>> >>>>>> Jorn >>>>>> >>>>>> [1] : >>>>>> https://mail.openjdk.java.net/pipermail/build-dev/2019-January/024666.html From jbvernee at xs4all.nl Thu Jan 17 18:57:37 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Thu, 17 Jan 2019 19:57:37 +0100 Subject: [foreign] Instructions for using libraries on Windows In-Reply-To: References: <86747435-1e9e-8a56-c4c7-60b99fec39c0@oracle.com> <1bfd7c81-4964-87e1-32a6-81745fc9abe6@oracle.com> Message-ID: Ok, the format didn't seem super consistent so I just went with what seemed convenient. I've tried to follow the section format that was already present for each library now. Updated webrev: http://cr.openjdk.java.net/~jvernee/panama/webrevs/instructions/webrev.02/ Jorn Maurizio Cimadamore schreef op 2019-01-17 19:37: > Another note, we have separate sections for different platforms - I > see that your changes create a single section for OpenGL for both > Windows and Linux - would be better to tease them apart, and then just > say that the example on Windows is the same (as done for other > examples). > > Also, please separate installation from extraction, as it happens for > the other sections. > > Cheers > > Maurizio > > > On 17/01/2019 18:30, Jorn Vernee wrote: >> Good catch. >> >> Here is the updated webrev: >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/instructions/webrev.01/ >> >> I've also added specific command lines for building/running on Windows >> (I was using intellij before). >> >> Jorn >> >> Maurizio Cimadamore schreef op 2019-01-17 19:14: >>> The changes you made look good - only nit, in the Windows notes you >>> mention "Visual Studios" (note the plural) a few times - I'd replace >>> with the singular form. >>> >>> The rest is good >>> >>> Maurizio >>> >>> On 17/01/2019 17:27, Jorn Vernee wrote: >>>> I got the .html generation working with the help of Erik from >>>> build-dev, so I think I can manage myself. >>>> >>>> Want me to sent an RFR with an actual webrev as well? >>>> >>>> Jorn >>>> >>>> Maurizio Cimadamore schreef op 2019-01-17 18:20: >>>>> Let us know if you need help with pushing this >>>>> >>>>> Thanks for trying this out on Windows! >>>>> >>>>> Maurizio >>>>> >>>>> On 17/01/2019 16:48, Jorn Vernee wrote: >>>>>>> The `make update-build-doc` does not work on windows [1], so I >>>>>>> ask >>>>>>> that you generate the .html dos for me instead. >>>>>> >>>>>> Apologies. It seems to be a configuration problem on my end, and >>>>>> not necessarily a problem with Windows. >>>>>> >>>>>> Jorn >>>>>> >>>>>> Jorn Vernee schreef op 2019-01-17 17:36: >>>>>>> Hi, >>>>>>> >>>>>>> I have updated panama_foreign.md with instructions for >>>>>>> jextracting and >>>>>>> running OpenGL and Python. I have also tried other Windows >>>>>>> applicable >>>>>>> libraries, here were the issues: >>>>>>> >>>>>>> LAPACK & BLAS: require libgfortran-3 as a dependency, which I >>>>>>> couldn't >>>>>>> find a binary distro for so quickly. (MINGW has a package for it, >>>>>>> but >>>>>>> only libgfortran-4) >>>>>>> >>>>>>> Tensorflow: is crashing jextract with a stack overflow error. >>>>>>> >>>>>>> ``` >>>>>>> ??? Exception in thread "main" java.lang.StackOverflowError >>>>>>> ??????? at >>>>>>> jdk.internal.clang/jdk.internal.clang.StructType.(StructType.java:37) >>>>>>> ??????? at >>>>>>> jdk.internal.clang/jdk.internal.clang.StructType.(StructType.java:32) >>>>>>> ??????? at >>>>>>> jdk.internal.clang/jdk.internal.clang.Type.(Type.java:30) >>>>>>> ??????? at >>>>>>> jdk.internal.clang/jdk.internal.clang.Type.canonicalType(Native >>>>>>> Method) >>>>>>> ??????? at >>>>>>> jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) >>>>>>> ??????? at >>>>>>> jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) >>>>>>> ??????? at >>>>>>> jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) >>>>>>> ??????? at >>>>>>> jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) >>>>>>> ??????? ... >>>>>>> ``` >>>>>>> >>>>>>> FWIW, you made a note about needing to pass `-C -c -C c++` to >>>>>>> jextract >>>>>>> so it doesn't interpret some methods as varargs. I've tried >>>>>>> without >>>>>>> those flags and am observing the same vararg problem, but in that >>>>>>> case >>>>>>> extraction does succeed. >>>>>>> >>>>>>> The other libraries libproc, readline, and unistd don't seem >>>>>>> applicable to Windows. I will be on the lookout for Windows >>>>>>> specific >>>>>>> examples that could be interesting. >>>>>>> >>>>>>> The `make update-build-doc` does not work on windows [1], so I >>>>>>> ask >>>>>>> that you generate the .html dos for me instead. >>>>>>> >>>>>>> Here is the updated panama_foreign.md: >>>>>>> http://cr.openjdk.java.net/~jvernee/panama/panama_foreign.md >>>>>>> >>>>>>> Jorn >>>>>>> >>>>>>> [1] : >>>>>>> https://mail.openjdk.java.net/pipermail/build-dev/2019-January/024666.html From kishor.kharbas at intel.com Thu Jan 17 19:36:33 2019 From: kishor.kharbas at intel.com (Kharbas, Kishor) Date: Thu, 17 Jan 2019 19:36:33 +0000 Subject: [Vector] RFR: reshape, resize, rebracket, cast In-Reply-To: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A5012D@FMSMSX126.amr.corp.intel.com> References: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A5012D@FMSMSX126.amr.corp.intel.com> Message-ID: Hi, I have new webrev which modifies the tests to use the new methods - http://cr.openjdk.java.net/~kkharbas/vector-api/webrev-reshape_etc.01 Incremental webrev - http://cr.openjdk.java.net/~kkharbas/vector-api/webrev-reshape_etc.01_to_base To summarize the api changes made are: Old methods: class Species { /* E x S -> E' x S' */ ????Vector reshape(Vector v) /* E x S -> E' x S */ Vector rebracket(Vector v) /* E x S -> E x S' */ Vector resize(Vector v) /* E x S -> E' x S' */ Vector cast(Vector v) } New methods: class Vector { /* E x S -> E' x S' */ /* was reshape() and rebracket() before */ Vector reinterpret(Species s) /* E x S -> E x S' */ /* was resize() before */ Vector reshape(Species s) /* E x S -> E' x S' */ Vector cast(Species s) } Thanks, Kishor > -----Original Message----- > From: Viswanathan, Sandhya > Sent: Wednesday, January 16, 2019 12:33 PM > To: Kharbas, Kishor ; panama- > dev at openjdk.java.net; Brian Goetz ; John Rose > ; Vladimir Ivanov > > Cc: joe.darcy at oracle.com > Subject: RE: [Vector] RFR: reshape, resize, rebracket, cast > > > Adding Brian, John, Joe and Vladimir to the thread. Please do give your > feedback on the patch from Kishor > (http://cr.openjdk.java.net/~kkharbas/vector-api/webrev-reshape_etc) > implementing the reinterpret API as suggested by Brian and Vladimir in > thread https://mail.openjdk.java.net/pipermail/panama-dev/2018- > December/003365.html. > > Best Regards, > Sandhya > > > -----Original Message----- > From: panama-dev [mailto:panama-dev-bounces at openjdk.java.net] On > Behalf Of Kharbas, Kishor > Sent: Wednesday, January 16, 2019 12:20 AM > To: panama-dev at openjdk.java.net > Subject: [Vector] RFR: reshape, resize, rebracket, cast > > Hi, > I have a patch which refactors the above methods as discussed in this thread > - https://mail.openjdk.java.net/pipermail/panama-dev/2018- > December/003365.html. > > Please review the changes at - http://cr.openjdk.java.net/~kkharbas/vector- > api/webrev-reshape_etc > Generation of intrinsics and correct vector boxing elimination have been > verified. In the next patch I will add changes to jtreg tests. > > I wanted to bring forth one small issue(possibly) which programmers might > face with this change - > > We provide specialized types like IntVector, FloatVector, etc for users to > define their vectors and they would be able to write code like this with > previous methods, > FloatVector float256 = SPECIES_FLOAT256.cast(int256); Here FloatSpecies > would always return FloatVector. > > However with this change, since cast() is defined on a vector and takes > species of a generic type, it cannot return a specialized Vector like IntVector > or FloatVector. User has to explicitly cast the return vector from Vector > to specialized Vector or use a generic vector of corresponding element type. > For example, > FloatVector float256 = (FloatVector)int256.cast(SPECIES_FLOAT256); > or > Vector float256 = int256.cast(SPECIES_FLOAT256); > > I am not sure if this is even a problem, but I thought its worth mentioning. > > Thanks > Kishor From maurizio.cimadamore at oracle.com Thu Jan 17 20:50:54 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 17 Jan 2019 20:50:54 +0000 Subject: [foreign] Instructions for using libraries on Windows In-Reply-To: References: <86747435-1e9e-8a56-c4c7-60b99fec39c0@oracle.com> <1bfd7c81-4964-87e1-32a6-81745fc9abe6@oracle.com> Message-ID: Looks good, before pushing, please revert changes to building.html and testing.html Cheers Maurizio On 17/01/2019 18:57, Jorn Vernee wrote: > Ok, the format didn't seem super consistent so I just went with what > seemed convenient. I've tried to follow the section format that was > already present for each library now. > > Updated webrev: > http://cr.openjdk.java.net/~jvernee/panama/webrevs/instructions/webrev.02/ > > Jorn > > Maurizio Cimadamore schreef op 2019-01-17 19:37: >> Another note, we have separate sections for different platforms - I >> see that your changes create a single section for OpenGL for both >> Windows and Linux - would be better to tease them apart, and then just >> say that the example on Windows is the same (as done for other >> examples). >> >> Also, please separate installation from extraction, as it happens for >> the other sections. >> >> Cheers >> >> Maurizio >> >> >> On 17/01/2019 18:30, Jorn Vernee wrote: >>> Good catch. >>> >>> Here is the updated webrev: >>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/instructions/webrev.01/ >>> >>> I've also added specific command lines for building/running on >>> Windows (I was using intellij before). >>> >>> Jorn >>> >>> Maurizio Cimadamore schreef op 2019-01-17 19:14: >>>> The changes you made look good - only nit, in the Windows notes you >>>> mention "Visual Studios" (note the plural) a few times - I'd replace >>>> with the singular form. >>>> >>>> The rest is good >>>> >>>> Maurizio >>>> >>>> On 17/01/2019 17:27, Jorn Vernee wrote: >>>>> I got the .html generation working with the help of Erik from >>>>> build-dev, so I think I can manage myself. >>>>> >>>>> Want me to sent an RFR with an actual webrev as well? >>>>> >>>>> Jorn >>>>> >>>>> Maurizio Cimadamore schreef op 2019-01-17 18:20: >>>>>> Let us know if you need help with pushing this >>>>>> >>>>>> Thanks for trying this out on Windows! >>>>>> >>>>>> Maurizio >>>>>> >>>>>> On 17/01/2019 16:48, Jorn Vernee wrote: >>>>>>>> The `make update-build-doc` does not work on windows [1], so I ask >>>>>>>> that you generate the .html dos for me instead. >>>>>>> >>>>>>> Apologies. It seems to be a configuration problem on my end, and >>>>>>> not necessarily a problem with Windows. >>>>>>> >>>>>>> Jorn >>>>>>> >>>>>>> Jorn Vernee schreef op 2019-01-17 17:36: >>>>>>>> Hi, >>>>>>>> >>>>>>>> I have updated panama_foreign.md with instructions for >>>>>>>> jextracting and >>>>>>>> running OpenGL and Python. I have also tried other Windows >>>>>>>> applicable >>>>>>>> libraries, here were the issues: >>>>>>>> >>>>>>>> LAPACK & BLAS: require libgfortran-3 as a dependency, which I >>>>>>>> couldn't >>>>>>>> find a binary distro for so quickly. (MINGW has a package for >>>>>>>> it, but >>>>>>>> only libgfortran-4) >>>>>>>> >>>>>>>> Tensorflow: is crashing jextract with a stack overflow error. >>>>>>>> >>>>>>>> ``` >>>>>>>> ??? Exception in thread "main" java.lang.StackOverflowError >>>>>>>> ??????? at >>>>>>>> jdk.internal.clang/jdk.internal.clang.StructType.(StructType.java:37) >>>>>>>> ??????? at >>>>>>>> jdk.internal.clang/jdk.internal.clang.StructType.(StructType.java:32) >>>>>>>> ??????? at >>>>>>>> jdk.internal.clang/jdk.internal.clang.Type.(Type.java:30) >>>>>>>> ??????? at >>>>>>>> jdk.internal.clang/jdk.internal.clang.Type.canonicalType(Native >>>>>>>> Method) >>>>>>>> ??????? at >>>>>>>> jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) >>>>>>>> ??????? at >>>>>>>> jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) >>>>>>>> ??????? at >>>>>>>> jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) >>>>>>>> ??????? at >>>>>>>> jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) >>>>>>>> ??????? ... >>>>>>>> ``` >>>>>>>> >>>>>>>> FWIW, you made a note about needing to pass `-C -c -C c++` to >>>>>>>> jextract >>>>>>>> so it doesn't interpret some methods as varargs. I've tried >>>>>>>> without >>>>>>>> those flags and am observing the same vararg problem, but in >>>>>>>> that case >>>>>>>> extraction does succeed. >>>>>>>> >>>>>>>> The other libraries libproc, readline, and unistd don't seem >>>>>>>> applicable to Windows. I will be on the lookout for Windows >>>>>>>> specific >>>>>>>> examples that could be interesting. >>>>>>>> >>>>>>>> The `make update-build-doc` does not work on windows [1], so I ask >>>>>>>> that you generate the .html dos for me instead. >>>>>>>> >>>>>>>> Here is the updated panama_foreign.md: >>>>>>>> http://cr.openjdk.java.net/~jvernee/panama/panama_foreign.md >>>>>>>> >>>>>>>> Jorn >>>>>>>> >>>>>>>> [1] : >>>>>>>> https://mail.openjdk.java.net/pipermail/build-dev/2019-January/024666.html From jbvernee at xs4all.nl Thu Jan 17 20:57:18 2019 From: jbvernee at xs4all.nl (jbvernee at xs4all.nl) Date: Thu, 17 Jan 2019 20:57:18 +0000 Subject: hg: panama/dev: Summary: Adding instructions for running Python and OpenGL on Windows Message-ID: <201901172057.x0HKvI1l021731@aojmv0008.oracle.com> Changeset: 4bdede14e920 Author: jvernee Date: 2019-01-17 21:56 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/4bdede14e920 Summary: Adding instructions for running Python and OpenGL on Windows Reviewed-by: mcimadamore ! doc/panama_foreign.html ! doc/panama_foreign.md From jbvernee at xs4all.nl Thu Jan 17 21:13:09 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Thu, 17 Jan 2019 22:13:09 +0100 Subject: [foreign] Instructions for using libraries on Windows In-Reply-To: References: Message-ID: <40253cd459e1428db8fd90e6928de1e5@xs4all.nl> FWIW, I've investigated the stack overflow with Tensorflow, and it seems to be caused by a template type in the standard C++ library on Windows (since we're running in C++ mode). As a workaround we can manually fix the function prototypes in `c_api.h` needed for the example: TF_Version() -> TF_Version(void) TF_NewGraph() -> TF_NewGraph(void) TF_NewSessionOptions() -> TF_NewSessionOptions(void) TF_NewStatus() -> TF_NewStatus(void) After that jextract succeeds without using the extra Clang flags, and the example seems to work as well. Jorn Jorn Vernee schreef op 2019-01-17 17:36: > Hi, > > I have updated panama_foreign.md with instructions for jextracting and > running OpenGL and Python. I have also tried other Windows applicable > libraries, here were the issues: > > LAPACK & BLAS: require libgfortran-3 as a dependency, which I couldn't > find a binary distro for so quickly. (MINGW has a package for it, but > only libgfortran-4) > > Tensorflow: is crashing jextract with a stack overflow error. > > ``` > Exception in thread "main" java.lang.StackOverflowError > at > jdk.internal.clang/jdk.internal.clang.StructType.(StructType.java:37) > at > jdk.internal.clang/jdk.internal.clang.StructType.(StructType.java:32) > at > jdk.internal.clang/jdk.internal.clang.Type.(Type.java:30) > at > jdk.internal.clang/jdk.internal.clang.Type.canonicalType(Native > Method) > at > jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) > at > jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) > at > jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) > at > jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) > ... > ``` > > FWIW, you made a note about needing to pass `-C -c -C c++` to jextract > so it doesn't interpret some methods as varargs. I've tried without > those flags and am observing the same vararg problem, but in that case > extraction does succeed. > > The other libraries libproc, readline, and unistd don't seem > applicable to Windows. I will be on the lookout for Windows specific > examples that could be interesting. > > The `make update-build-doc` does not work on windows [1], so I ask > that you generate the .html dos for me instead. > > Here is the updated panama_foreign.md: > http://cr.openjdk.java.net/~jvernee/panama/panama_foreign.md > > Jorn > > [1] : > https://mail.openjdk.java.net/pipermail/build-dev/2019-January/024666.html From jbvernee at xs4all.nl Thu Jan 17 21:17:49 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Thu, 17 Jan 2019 22:17:49 +0100 Subject: [foreign] Instructions for using libraries on Windows In-Reply-To: <40253cd459e1428db8fd90e6928de1e5@xs4all.nl> References: <40253cd459e1428db8fd90e6928de1e5@xs4all.nl> Message-ID: <4b9dcdffcc91cdaaf86d8e09b1549c4d@xs4all.nl> This also has been fixed in the TensorFlow recently: https://github.com/tensorflow/tensorflow/commit/ceee617b3675037da8637997f673ab9a993ee5de So this should not be a problem anymore once this change makes it into the binary distributions. Jorn Jorn Vernee schreef op 2019-01-17 22:13: > FWIW, I've investigated the stack overflow with Tensorflow, and it > seems to be caused by a template type in the standard C++ library on > Windows (since we're running in C++ mode). > > As a workaround we can manually fix the function prototypes in > `c_api.h` needed for the example: > > TF_Version() -> TF_Version(void) > TF_NewGraph() -> TF_NewGraph(void) > TF_NewSessionOptions() -> TF_NewSessionOptions(void) > TF_NewStatus() -> TF_NewStatus(void) > > After that jextract succeeds without using the extra Clang flags, and > the example seems to work as well. > > Jorn > > Jorn Vernee schreef op 2019-01-17 17:36: >> Hi, >> >> I have updated panama_foreign.md with instructions for jextracting and >> running OpenGL and Python. I have also tried other Windows applicable >> libraries, here were the issues: >> >> LAPACK & BLAS: require libgfortran-3 as a dependency, which I couldn't >> find a binary distro for so quickly. (MINGW has a package for it, but >> only libgfortran-4) >> >> Tensorflow: is crashing jextract with a stack overflow error. >> >> ``` >> Exception in thread "main" java.lang.StackOverflowError >> at >> jdk.internal.clang/jdk.internal.clang.StructType.(StructType.java:37) >> at >> jdk.internal.clang/jdk.internal.clang.StructType.(StructType.java:32) >> at >> jdk.internal.clang/jdk.internal.clang.Type.(Type.java:30) >> at >> jdk.internal.clang/jdk.internal.clang.Type.canonicalType(Native >> Method) >> at >> jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) >> at >> jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) >> at >> jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) >> at >> jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) >> ... >> ``` >> >> FWIW, you made a note about needing to pass `-C -c -C c++` to jextract >> so it doesn't interpret some methods as varargs. I've tried without >> those flags and am observing the same vararg problem, but in that case >> extraction does succeed. >> >> The other libraries libproc, readline, and unistd don't seem >> applicable to Windows. I will be on the lookout for Windows specific >> examples that could be interesting. >> >> The `make update-build-doc` does not work on windows [1], so I ask >> that you generate the .html dos for me instead. >> >> Here is the updated panama_foreign.md: >> http://cr.openjdk.java.net/~jvernee/panama/panama_foreign.md >> >> Jorn >> >> [1] : >> https://mail.openjdk.java.net/pipermail/build-dev/2019-January/024666.html From maurizio.cimadamore at oracle.com Thu Jan 17 21:27:57 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 17 Jan 2019 21:27:57 +0000 Subject: [foreign] Instructions for using libraries on Windows In-Reply-To: <4b9dcdffcc91cdaaf86d8e09b1549c4d@xs4all.nl> References: <40253cd459e1428db8fd90e6928de1e5@xs4all.nl> <4b9dcdffcc91cdaaf86d8e09b1549c4d@xs4all.nl> Message-ID: Cool! Thanks for investigating... Maurizio On 17/01/2019 21:17, Jorn Vernee wrote: > This also has been fixed in the TensorFlow recently: > https://github.com/tensorflow/tensorflow/commit/ceee617b3675037da8637997f673ab9a993ee5de > > So this should not be a problem anymore once this change makes it into > the binary distributions. > > Jorn > > Jorn Vernee schreef op 2019-01-17 22:13: >> FWIW, I've investigated the stack overflow with Tensorflow, and it >> seems to be caused by a template type in the standard C++ library on >> Windows (since we're running in C++ mode). >> >> As a workaround we can manually fix the function prototypes in >> `c_api.h` needed for the example: >> >> ??? TF_Version() -> TF_Version(void) >> ??? TF_NewGraph() -> TF_NewGraph(void) >> ??? TF_NewSessionOptions() -> TF_NewSessionOptions(void) >> ??? TF_NewStatus() -> TF_NewStatus(void) >> >> After that jextract succeeds without using the extra Clang flags, and >> the example seems to work as well. >> >> Jorn >> >> Jorn Vernee schreef op 2019-01-17 17:36: >>> Hi, >>> >>> I have updated panama_foreign.md with instructions for jextracting and >>> running OpenGL and Python. I have also tried other Windows applicable >>> libraries, here were the issues: >>> >>> LAPACK & BLAS: require libgfortran-3 as a dependency, which I couldn't >>> find a binary distro for so quickly. (MINGW has a package for it, but >>> only libgfortran-4) >>> >>> Tensorflow: is crashing jextract with a stack overflow error. >>> >>> ``` >>> ??? Exception in thread "main" java.lang.StackOverflowError >>> ??????? at >>> jdk.internal.clang/jdk.internal.clang.StructType.(StructType.java:37) >>> >>> ??????? at >>> jdk.internal.clang/jdk.internal.clang.StructType.(StructType.java:32) >>> >>> ??????? at >>> jdk.internal.clang/jdk.internal.clang.Type.(Type.java:30) >>> ??????? at >>> jdk.internal.clang/jdk.internal.clang.Type.canonicalType(Native >>> Method) >>> ??????? at >>> jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) >>> >>> ??????? at >>> jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) >>> >>> ??????? at >>> jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) >>> >>> ??????? at >>> jdk.jextract/com.sun.tools.jextract.TypeDictionary.getInternal(TypeDictionary.java:134) >>> >>> ??????? ... >>> ``` >>> >>> FWIW, you made a note about needing to pass `-C -c -C c++` to jextract >>> so it doesn't interpret some methods as varargs. I've tried without >>> those flags and am observing the same vararg problem, but in that case >>> extraction does succeed. >>> >>> The other libraries libproc, readline, and unistd don't seem >>> applicable to Windows. I will be on the lookout for Windows specific >>> examples that could be interesting. >>> >>> The `make update-build-doc` does not work on windows [1], so I ask >>> that you generate the .html dos for me instead. >>> >>> Here is the updated panama_foreign.md: >>> http://cr.openjdk.java.net/~jvernee/panama/panama_foreign.md >>> >>> Jorn >>> >>> [1] : >>> https://mail.openjdk.java.net/pipermail/build-dev/2019-January/024666.html From vladimir.x.ivanov at oracle.com Thu Jan 17 22:11:23 2019 From: vladimir.x.ivanov at oracle.com (vladimir.x.ivanov at oracle.com) Date: Thu, 17 Jan 2019 22:11:23 +0000 Subject: hg: panama/dev: 3 new changesets Message-ID: <201901172211.x0HMBOqS024462@aojmv0008.oracle.com> Changeset: 7d834373b72d Author: vlivanov Date: 2019-01-17 09:37 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/7d834373b72d Manual merge with default ! src/hotspot/cpu/aarch64/assembler_aarch64.hpp ! src/hotspot/cpu/aarch64/globals_aarch64.hpp ! src/hotspot/cpu/x86/assembler_x86.hpp ! src/hotspot/cpu/x86/macroAssembler_x86.hpp ! src/hotspot/cpu/x86/stubRoutines_x86.hpp ! src/hotspot/cpu/x86/vmStructs_x86.hpp ! src/hotspot/cpu/x86/vm_version_x86.hpp ! src/hotspot/cpu/x86/x86.ad ! src/hotspot/share/ci/ciMethod.cpp ! src/hotspot/share/ci/ciMethod.hpp ! src/hotspot/share/ci/ciType.hpp ! src/hotspot/share/classfile/vmSymbols.hpp ! src/hotspot/share/code/location.hpp ! src/hotspot/share/opto/addnode.hpp ! src/hotspot/share/opto/c2_globals.hpp ! src/hotspot/share/opto/callGenerator.hpp ! src/hotspot/share/opto/callnode.cpp ! src/hotspot/share/opto/callnode.hpp ! src/hotspot/share/opto/compile.cpp ! src/hotspot/share/opto/compile.hpp ! src/hotspot/share/opto/escape.cpp ! src/hotspot/share/opto/graphKit.hpp ! src/hotspot/share/opto/macro.hpp ! src/hotspot/share/opto/matcher.hpp ! src/hotspot/share/opto/movenode.hpp ! src/hotspot/share/opto/node.hpp ! src/hotspot/share/opto/phase.hpp ! src/hotspot/share/opto/phasetype.hpp ! src/hotspot/share/opto/runtime.hpp ! src/hotspot/share/opto/subnode.hpp ! src/hotspot/share/opto/superword.cpp ! src/hotspot/share/opto/type.hpp ! src/hotspot/share/opto/vectornode.hpp ! src/hotspot/share/runtime/globals.hpp ! src/hotspot/share/runtime/sharedRuntime.hpp ! src/hotspot/share/runtime/stackValue.hpp ! src/hotspot/share/runtime/stubRoutines.hpp ! src/hotspot/share/utilities/globalDefinitions.hpp - test/jdk/java/lang/String/AlignIndent.java - test/langtools/jdk/javadoc/doclet/testHtmlLandmarkRegions/TestHtmlLankmarkRegions.java - test/langtools/tools/javac/RawStringLiteralLang.java - test/langtools/tools/javac/RawStringLiteralLangAPI.java - test/langtools/tools/javac/diags/examples/RawStringLiteral.java Changeset: 0c2e5845179c Author: vlivanov Date: 2019-01-17 09:37 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/0c2e5845179c Vector.rearrange() breaks vector box elimination due to lambda capturing in default implementation ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Byte128Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Byte256Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Byte512Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Byte64Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/ByteMaxVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Double128Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Double256Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Double512Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Double64Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/DoubleMaxVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Float128Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Float256Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Float512Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Float64Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/FloatMaxVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Int128Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Int256Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Int512Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Int64Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/IntMaxVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Long128Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Long256Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Long512Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Long64Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/LongMaxVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Short128Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Short256Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Short512Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Short64Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/ShortMaxVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/X-VectorBits.java.template Changeset: a8516a4be714 Author: vlivanov Date: 2019-01-17 09:37 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/a8516a4be714 Fix memory access intrinsics for Vector.Mask Contributed-by: Wang Zhuo ! src/hotspot/share/opto/library_call.cpp ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Byte128Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Byte256Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Byte512Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Byte64Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/ByteMaxVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Double128Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Double256Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Double512Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Double64Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/DoubleMaxVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Float128Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Float256Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Float512Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Float64Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/FloatMaxVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Int128Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Int256Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Int512Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Int64Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/IntMaxVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Long128Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Long256Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Long512Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Long64Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/LongMaxVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Short128Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Short256Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Short512Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Short64Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/ShortMaxVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/X-VectorBits.java.template From maurizio.cimadamore at oracle.com Thu Jan 17 22:15:28 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Thu, 17 Jan 2019 22:15:28 +0000 Subject: hg: panama/dev: Automatic merge with vectorIntrinsics Message-ID: <201901172215.x0HMFSK7026129@aojmv0008.oracle.com> Changeset: 7b8b6bbc9e03 Author: mcimadamore Date: 2019-01-17 23:15 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/7b8b6bbc9e03 Automatic merge with vectorIntrinsics ! make/RunTests.gmk ! make/autoconf/basics.m4 ! make/conf/jib-profiles.js ! src/hotspot/cpu/x86/assembler_x86.hpp ! src/hotspot/cpu/x86/macroAssembler_x86.hpp ! src/hotspot/share/logging/logTag.hpp - test/jdk/java/lang/String/AlignIndent.java - test/langtools/jdk/javadoc/doclet/testHtmlLandmarkRegions/TestHtmlLankmarkRegions.java - test/langtools/tools/javac/RawStringLiteralLang.java - test/langtools/tools/javac/RawStringLiteralLangAPI.java - test/langtools/tools/javac/diags/examples/RawStringLiteral.java From maurizio.cimadamore at oracle.com Thu Jan 17 22:15:47 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Thu, 17 Jan 2019 22:15:47 +0000 Subject: hg: panama/dev: Automatic merge with foreign Message-ID: <201901172215.x0HMFmZx026397@aojmv0008.oracle.com> Changeset: f805dc372f4c Author: mcimadamore Date: 2019-01-17 23:15 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/f805dc372f4c Automatic merge with foreign ! src/hotspot/cpu/x86/assembler_x86.hpp ! src/hotspot/cpu/x86/macroAssembler_x86.hpp - src/java.base/share/classes/java/foreign/layout/AbstractLayout.java From sundararajan.athijegannathan at oracle.com Fri Jan 18 08:46:55 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Fri, 18 Jan 2019 14:16:55 +0530 Subject: [foreign] RFR 8217369: jextract macro parser could avoid javac for simple macros Message-ID: <5C41927F.1000506@oracle.com> Please review. Bug: https://bugs.openjdk.java.net/browse/JDK-8217369 Webrev: https://cr.openjdk.java.net/~sundar/8217369/webrev.00/ 3 to 4s improvement on Python sample (13s to 9/10s on Mac). About 45% are simple int valued macros. Thanks, -Sundar From maurizio.cimadamore at oracle.com Fri Jan 18 11:09:37 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Fri, 18 Jan 2019 11:09:37 +0000 Subject: [foreign] RFR 8217369: jextract macro parser could avoid javac for simple macros In-Reply-To: <5C41927F.1000506@oracle.com> References: <5C41927F.1000506@oracle.com> Message-ID: <67372a00-9686-7d0b-baf6-0f58fb279d6b@oracle.com> Hi Sundar, the patch looks good - I'm not super super that splitting the code in this way is worth - in the sense that having a unique path to process macro seems generally more regular and less error prone (albeit more inefficient). Also, I believe there could be some differences with respect to the old behavior - for instance, what happens if a number doesn't fit into an Integer? The javac-based approach will return a long, not sure what will happen in this case - likely the decode() method will throw and you will fall back to use javac again. But I thought it was worth asking. Maurizio On 18/01/2019 08:46, Sundararajan Athijegannathan wrote: > Please review. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8217369 > Webrev: https://cr.openjdk.java.net/~sundar/8217369/webrev.00/ > > 3 to 4s improvement on Python sample (13s to 9/10s on Mac). About 45% > are simple int valued macros. > > Thanks, > -Sundar From sundararajan.athijegannathan at oracle.com Fri Jan 18 11:19:39 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Fri, 18 Jan 2019 16:49:39 +0530 Subject: [foreign] RFR 8217369: jextract macro parser could avoid javac for simple macros In-Reply-To: <67372a00-9686-7d0b-baf6-0f58fb279d6b@oracle.com> References: <5C41927F.1000506@oracle.com> <67372a00-9686-7d0b-baf6-0f58fb279d6b@oracle.com> Message-ID: <5C41B64B.3060707@oracle.com> Hi, Yes, Integer.decode throws NumberFormatException if number won't fit as int - and the jextract control will fall back to use javac. BTW, even javac approach would require user to use "L" suffix (for larger int that won't fit as int), right? -Sundar On 18/01/19, 4:39 PM, Maurizio Cimadamore wrote: > Hi Sundar, > the patch looks good - I'm not super super that splitting the code in > this way is worth - in the sense that having a unique path to process > macro seems generally more regular and less error prone (albeit more > inefficient). > > Also, I believe there could be some differences with respect to the > old behavior - for instance, what happens if a number doesn't fit into > an Integer? The javac-based approach will return a long, not sure what > will happen in this case - likely the decode() method will throw and > you will fall back to use javac again. But I thought it was worth asking. > > Maurizio > > On 18/01/2019 08:46, Sundararajan Athijegannathan wrote: >> Please review. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8217369 >> Webrev: https://cr.openjdk.java.net/~sundar/8217369/webrev.00/ >> >> 3 to 4s improvement on Python sample (13s to 9/10s on Mac). About 45% >> are simple int valued macros. >> >> Thanks, >> -Sundar From maurizio.cimadamore at oracle.com Fri Jan 18 11:22:18 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Fri, 18 Jan 2019 11:22:18 +0000 Subject: [foreign] RFR 8217369: jextract macro parser could avoid javac for simple macros In-Reply-To: <5C41B64B.3060707@oracle.com> References: <5C41927F.1000506@oracle.com> <67372a00-9686-7d0b-baf6-0f58fb279d6b@oracle.com> <5C41B64B.3060707@oracle.com> Message-ID: On 18/01/2019 11:19, Sundararajan Athijegannathan wrote: > Hi, > > Yes, Integer.decode throws NumberFormatException if number won't fit > as int - and the jextract control will fall back to use javac. BTW, > even javac approach would require user to use "L" suffix (for larger > int that won't fit as int), right? Sure, long constants need L suffix. Ok, I was more checking that we did not truncate the value as an Integer. I'm happy with the change, and it's relatively harmless - so let's try it out. Maurizio > > -Sundar > > On 18/01/19, 4:39 PM, Maurizio Cimadamore wrote: >> Hi Sundar, >> the patch looks good - I'm not super super that splitting the code in >> this way is worth - in the sense that having a unique path to process >> macro seems generally more regular and less error prone (albeit more >> inefficient). >> >> Also, I believe there could be some differences with respect to the >> old behavior - for instance, what happens if a number doesn't fit >> into an Integer? The javac-based approach will return a long, not >> sure what will happen in this case - likely the decode() method will >> throw and you will fall back to use javac again. But I thought it was >> worth asking. >> >> Maurizio >> >> On 18/01/2019 08:46, Sundararajan Athijegannathan wrote: >>> Please review. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8217369 >>> Webrev: https://cr.openjdk.java.net/~sundar/8217369/webrev.00/ >>> >>> 3 to 4s improvement on Python sample (13s to 9/10s on Mac). About >>> 45% are simple int valued macros. >>> >>> Thanks, >>> -Sundar From sundararajan.athijegannathan at oracle.com Fri Jan 18 11:27:09 2019 From: sundararajan.athijegannathan at oracle.com (sundararajan.athijegannathan at oracle.com) Date: Fri, 18 Jan 2019 11:27:09 +0000 Subject: hg: panama/dev: 8217369: jextract macro parser could avoid javac for simple macros Message-ID: <201901181127.x0IBR9mE002601@aojmv0008.oracle.com> Changeset: eaca2d16b80b Author: sundar Date: 2019-01-18 16:57 +0530 URL: http://hg.openjdk.java.net/panama/dev/rev/eaca2d16b80b 8217369: jextract macro parser could avoid javac for simple macros Reviewed-by: mcimadamore ! src/jdk.jextract/share/classes/com/sun/tools/jextract/parser/MacroParser.java From maurizio.cimadamore at oracle.com Fri Jan 18 11:30:11 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Fri, 18 Jan 2019 11:30:11 +0000 Subject: hg: panama/dev: Automatic merge with foreign Message-ID: <201901181130.x0IBUBND003877@aojmv0008.oracle.com> Changeset: a89db750f8ba Author: mcimadamore Date: 2019-01-18 12:30 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/a89db750f8ba Automatic merge with foreign From vladimir.x.ivanov at oracle.com Fri Jan 18 18:53:24 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 18 Jan 2019 10:53:24 -0800 Subject: CFV: New panama Committer: Wang Zhuo Message-ID: <9e892bc8-9a8b-f66a-0edd-3b3707fe0786@oracle.com> I hereby nominate Wang Zhuo (email: zhuoren.wz at alibaba-inc.com) to Committer role in Project Panama. Zhuoren works on Vector API at Alibaba and has already made 3 contributions [1] to the implementation. Votes are due by January, 25, 2019. Only current panama Committers [2] are eligible to vote on this nomination. Votes must be cast in the open by replying to this mailing list. For Lazy Consensus voting instructions, see [3]. Best regards, Vladimir Ivanov [1] http://hg.openjdk.java.net/panama/dev/rev/a511da8eef00 http://hg.openjdk.java.net/panama/dev/rev/a8516a4be714 http://hg.openjdk.java.net/panama/dev/rev/dfe1c1d8a316 [2] http://openjdk.java.net/census#panama [3] http://openjdk.java.net/projects#committer-vote From vladimir.x.ivanov at oracle.com Fri Jan 18 18:53:48 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 18 Jan 2019 10:53:48 -0800 Subject: CFV: New panama Committer: Wang Zhuo In-Reply-To: <9e892bc8-9a8b-f66a-0edd-3b3707fe0786@oracle.com> References: <9e892bc8-9a8b-f66a-0edd-3b3707fe0786@oracle.com> Message-ID: <3d882742-1527-554c-9dea-cef031822831@oracle.com> Vote: yes Best regards, Vladimir Ivanov On 18/01/2019 10:53, Vladimir Ivanov wrote: > I hereby nominate Wang Zhuo (email: zhuoren.wz at alibaba-inc.com) to > Committer role in Project Panama. From john.r.rose at oracle.com Fri Jan 18 20:57:09 2019 From: john.r.rose at oracle.com (John Rose) Date: Fri, 18 Jan 2019 12:57:09 -0800 Subject: CFV: New panama Committer: Wang Zhuo In-Reply-To: <9e892bc8-9a8b-f66a-0edd-3b3707fe0786@oracle.com> References: <9e892bc8-9a8b-f66a-0edd-3b3707fe0786@oracle.com> Message-ID: <3416CBCE-1141-49BB-B85F-DBFBD91B92E8@oracle.com> Vote: yes From john.r.rose at oracle.com Fri Jan 18 21:12:01 2019 From: john.r.rose at oracle.com (John Rose) Date: Fri, 18 Jan 2019 13:12:01 -0800 Subject: Toward distributions of native libraries In-Reply-To: References: <51e5d7d3-6f95-9b33-1858-645bc388e17e@gmail.com> <479f6425-f3f2-e4ec-abe4-9303996dcd82@oracle.com> Message-ID: On Jan 15, 2019, at 9:56 PM, Samuel Audet wrote: > > On 1/14/19 3:04 PM, John Rose wrote: >> On Jan 12, 2019, at 7:42 PM, Samuel Audet wrote: >>> >>> Thanks for the feedback! If it were a priority, and if it were me, I would start by ensuring that the high-level API can also be optimized in the case of AOT, but this isn't happening at the moment. I just shortened it this way because I consider AOT to be just as important as JIT. How do you feel I should phrase this then? That it isn't a priority /only in the case of AOT/? I'll edit it that way if that's accurate! If not, I would welcome additional explanations, for example, see https://github.com/oracle/graal/issues/885 . Thanks >> Performance is important, period. The discussion on #885 peters out with you saying, "but can it work efficiently?". That's a complicated question, to which I believe the answer is almost certainly "yes". We are, fact, planning to make such things "work efficiently", and "such things" include JIT, AOT, and SVM compilation of Panama apps. >> So I agree with Maurizio's comment, that this bit of your blog post (a very good one!) would better reflect our intentions by omitting the comment about performance. If you wish, you could satisfy our observations by saying something straightforward like "Although the Panama people claim to value high performance, I don't think they can achieve it, especially for AOT." If that's true for you, in which case, wait and see how it all turns out. >> To put it in a nutshell: You misrepresent our priorities (a set of mental states Maurizio and I can and do authoritatively explain) by reasoning about their feasibility (which reasonable people can disagree about). > > I would rather learn to disagree than to ignore each other, so thank you for the feedback! > > IMO, the question I asked to the SVM team isn't a complicated one. I asked in the simple case of getpid() whether or not we will be able to achieve the same speed as "org.graalvm.nativeimage.c", and although you may believe the answer is "yes", the fact is that it looks like we don't know. So the answer is "probably, but we don't know for sure". What if it turns out to be "no"? Are you prepared to refactor the whole high-level API to something else entirely and maybe even bring back static methods as an option if we're really up against the wall in one or more corner cases where they would help performance? If that's the way you guys work, that's fine too, but let's be clear about it! Life is risky, and sometimes you hit a dead end. You are asking "what if there's a dead end here for a non-static API w.r.t. performance", forcing us to refactor the whole thing. "What if" indeed. I suppose we differ about the probability of a disruptive refactor. I think it is vanishingly unlikely, given the large bag of tricks the JVM has to optimize dynamic code as if it were static. So the "what if" question is uninteresting to me. > Anyway, I like to stay objective, so I've rephrased the passage on http://bytedeco.org/news/2019/01/11/importance-of-a-distribution/ without any implications about what you consider important or not: > > "numbers showing faster-than-JNI performance for both JIT and AOT compilers are still lacking" > > Sounds good? I'm happy to hear at least that AOT is of importance too! Thanks, Samuel! ? John From sandhya.viswanathan at intel.com Fri Jan 18 21:59:04 2019 From: sandhya.viswanathan at intel.com (Viswanathan, Sandhya) Date: Fri, 18 Jan 2019 21:59:04 +0000 Subject: New panama Committer: Wang Zhuo In-Reply-To: <9e892bc8-9a8b-f66a-0edd-3b3707fe0786@oracle.com> References: <9e892bc8-9a8b-f66a-0edd-3b3707fe0786@oracle.com> Message-ID: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A512E5@FMSMSX126.amr.corp.intel.com> Vote: yes Best regards, Sandhya -----Original Message----- From: panama-dev [mailto:panama-dev-bounces at openjdk.java.net] On Behalf Of Vladimir Ivanov Sent: Friday, January 18, 2019 10:53 AM To: 'panama-dev at openjdk.java.net' Subject: CFV: New panama Committer: Wang Zhuo I hereby nominate Wang Zhuo (email: zhuoren.wz at alibaba-inc.com) to Committer role in Project Panama. Zhuoren works on Vector API at Alibaba and has already made 3 contributions [1] to the implementation. Votes are due by January, 25, 2019. Only current panama Committers [2] are eligible to vote on this nomination. Votes must be cast in the open by replying to this mailing list. For Lazy Consensus voting instructions, see [3]. Best regards, Vladimir Ivanov [1] http://hg.openjdk.java.net/panama/dev/rev/a511da8eef00 http://hg.openjdk.java.net/panama/dev/rev/a8516a4be714 http://hg.openjdk.java.net/panama/dev/rev/dfe1c1d8a316 [2] http://openjdk.java.net/census#panama [3] http://openjdk.java.net/projects#committer-vote From jbvernee at xs4all.nl Sat Jan 19 15:16:49 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Sat, 19 Jan 2019 16:16:49 +0100 Subject: [foreign] RFR 8217380: LayoutType::ofStruct should try to resolve Struct layout Message-ID: <73f856cdb194769699234ec1c13921e1@xs4all.nl> Hi, Please review the following patch. Bug: https://bugs.openjdk.java.net/browse/JDK-8217380 Webrev: http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217380/webrev.00/ Thanks, Jorn From richard at openkappa.co.uk Sat Jan 19 19:28:40 2019 From: richard at openkappa.co.uk (Richard Startin) Date: Sat, 19 Jan 2019 19:28:40 +0000 Subject: Allocation caused by vector rebracketing twice Message-ID: It's great to see that there is now a shiftR method because this was the missing link to make vector bit counts possible. With great excitement, I tried this at 54348:a8516a4be714 but it doesn't work very well yet. @Benchmark public int vectorBitCount() { int bitCount = 0; var lookupPos = YMM_BYTE.fromArray(LOOKUP_POS, 0); var lookupNeg = YMM_BYTE.fromArray(LOOKUP_NEG, 0); var lowMask = YMM_BYTE.broadcast((byte)0x0F); for (int i = 0; i < data.length; i+= 4) { var bytes = (ByteVector)YMM_LONG.fromArray(data, i).rebracket(YMM_BYTE); bitCount += (int)((LongVector)lookupPos.rearrange(bytes.and(lowMask).toShuffle()) .add(lookupNeg.rearrange(bytes.shiftR(4).and(lowMask).toShuffle())) .rebracket(YMM_LONG)).addAll(); } return bitCount; } JMH -prof gc shows high allocation rates: Iteration 1: 0.008 ops/us ?gc.alloc.rate: 1112.997 MB/sec ?gc.alloc.rate.norm: 219264.051 B/op ?gc.churn.G1_Eden_Space: 1042.296 MB/sec ?gc.churn.G1_Eden_Space.norm: 205335.753 B/op ?gc.churn.G1_Old_Gen: 0.001 MB/sec ?gc.churn.G1_Old_Gen.norm: 0.258 B/op ?gc.count: 6.000 counts ?gc.time: 6.000 ms Stripping the code down until negligible allocation rates are observed, the smallest reproducer is where the vector is rebracketed and then the reverse rebracket is performed: @Benchmark public int vectorBitCount() { int bitCount = 0; for (int i = 0; i < data.length; i+= 4) { bitCount += (int)((LongVector)YMM_LONG.fromArray(data, i).rebracket(YMM_BYTE).rebracket(YMM_LONG)).addAll(); } return bitCount; } -prof gc: Iteration 1: 0.310 ops/us ?gc.alloc.rate: 3223.539 MB/sec ?gc.alloc.rate.norm: 16384.001 B/op ?gc.churn.G1_Eden_Space: 3084.887 MB/sec ?gc.churn.G1_Eden_Space.norm: 15679.287 B/op ?gc.churn.G1_Old_Gen: 0.005 MB/sec ?gc.churn.G1_Old_Gen.norm: 0.026 B/op ?gc.count: 8.000 counts ?gc.time: 9.000 ms Without reversing the rebracket I see negligible allocation @Benchmark public int vectorBitCount() { int bitCount = 0; for (int i = 0; i < data.length; i+= 4) { bitCount += (int)((ByteVector)YMM_LONG.fromArray(data, i).rebracket(YMM_BYTE)).addAll(); } return bitCount; } Iteration 1: 1.456 ops/us ?gc.alloc.rate: ? 10?? MB/sec ?gc.alloc.rate.norm: ? 10?? B/op ?gc.count: ? 0 counts Thanks, Richard From maurizio.cimadamore at oracle.com Mon Jan 21 10:52:32 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 21 Jan 2019 10:52:32 +0000 Subject: [foreign] RFR 8217380: LayoutType::ofStruct should try to resolve Struct layout In-Reply-To: <73f856cdb194769699234ec1c13921e1@xs4all.nl> References: <73f856cdb194769699234ec1c13921e1@xs4all.nl> Message-ID: Hi, I've been wanting to solve this in the past, but I stumbled upon the fact that we could not always ensure 'eager' resolution when calling LayoutType.ofStruct. This is something that eventually should be fixed - the situations where such resolution cannot happen are always caused by the fact that jextract is generating cross-header symbolic references - this is another problem that could be addressed by switching to a library-per-extraction approach, where then all symbolic references will become self-contained. In the meantime, your idea of adding a 'tryResolve' is a good one; I'll do some more tests on my side, to make sure that everything is ok, and then I'll approve Cheers Maurizio On 19/01/2019 15:16, Jorn Vernee wrote: > Hi, > > Please review the following patch. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8217380 > Webrev: > http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217380/webrev.00/ > > Thanks, > Jorn From jbvernee at xs4all.nl Mon Jan 21 11:11:55 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Mon, 21 Jan 2019 12:11:55 +0100 Subject: [foreign] RFR 8217380: LayoutType::ofStruct should try to resolve Struct layout In-Reply-To: References: <73f856cdb194769699234ec1c13921e1@xs4all.nl> Message-ID: <6814b0d1e0c80e252e49e13a03f430d0@xs4all.nl> Ok thanks, FWIW, I thought it was best to try to do the resolution as early as possible to avoid having to do it multiple times later on. We need the carrier type (which has the annotations) to do the resolution, and the earliest time carrier and layout come together seems to be in LayoutType. The other place where we could try and resolve the layout is in Scope, right before the allocation, since that's when resolution is needed. After sending the RFR email I have been thinking that we might want to have some check there either way to see if the type being allocated is actually complete. e.g.: if(type.isPartial()) { throw new IllegalArgumentException("Can not allocate incomplete type: " + type); } However, the `isPartial()` check does not only check if a type is incomplete. For instance for a pointer; the Address layout isPartial() method is inherited from Value which delegates to it's `content`, if it has one, to do the `isPartial()` check [1]. This makes sense if we have just a Value, but imho Address should override isPartial() to always return `true`, since we can always allocate a pointer (just maybe not what it points to). What do you think? Jorn [1] : http://hg.openjdk.java.net/panama/dev/file/tip/src/java.base/share/classes/java/foreign/layout/Value.java#l100 Maurizio Cimadamore schreef op 2019-01-21 11:52: > Hi, > I've been wanting to solve this in the past, but I stumbled upon the > fact that we could not always ensure 'eager' resolution when calling > LayoutType.ofStruct. This is something that eventually should be fixed > - the situations where such resolution cannot happen are always caused > by the fact that jextract is generating cross-header symbolic > references - this is another problem that could be addressed by > switching to a library-per-extraction approach, where then all > symbolic references will become self-contained. > > In the meantime, your idea of adding a 'tryResolve' is a good one; > I'll do some more tests on my side, to make sure that everything is > ok, and then I'll approve > > Cheers > Maurizio > > On 19/01/2019 15:16, Jorn Vernee wrote: >> Hi, >> >> Please review the following patch. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8217380 >> Webrev: >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217380/webrev.00/ >> >> Thanks, >> Jorn From jbvernee at xs4all.nl Mon Jan 21 11:21:49 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Mon, 21 Jan 2019 12:21:49 +0100 Subject: [foreign] RFR 8217380: LayoutType::ofStruct should try to resolve Struct layout In-Reply-To: <6814b0d1e0c80e252e49e13a03f430d0@xs4all.nl> References: <73f856cdb194769699234ec1c13921e1@xs4all.nl> <6814b0d1e0c80e252e49e13a03f430d0@xs4all.nl> Message-ID: <4bdf74890a5f514bba4c9ff6032c9f43@xs4all.nl> > but imho Address should override isPartial() to always return `true` I mean `false` here. i.e. an Address is never partial, it is always a complete type. Jorn Jorn Vernee schreef op 2019-01-21 12:11: > Ok thanks, > > FWIW, I thought it was best to try to do the resolution as early as > possible to avoid having to do it multiple times later on. We need the > carrier type (which has the annotations) to do the resolution, and the > earliest time carrier and layout come together seems to be in > LayoutType. > > The other place where we could try and resolve the layout is in Scope, > right before the allocation, since that's when resolution is needed. > After sending the RFR email I have been thinking that we might want to > have some check there either way to see if the type being allocated is > actually complete. e.g.: > > if(type.isPartial()) { > throw new IllegalArgumentException("Can not allocate > incomplete type: " + type); > } > > However, the `isPartial()` check does not only check if a type is > incomplete. For instance for a pointer; the Address layout isPartial() > method is inherited from Value which delegates to it's `content`, if > it has one, to do the `isPartial()` check [1]. This makes sense if we > have just a Value, but imho Address should override isPartial() to > always return `true`, since we can always allocate a pointer (just > maybe not what it points to). > > What do you think? > > Jorn > > [1] : > http://hg.openjdk.java.net/panama/dev/file/tip/src/java.base/share/classes/java/foreign/layout/Value.java#l100 > > Maurizio Cimadamore schreef op 2019-01-21 11:52: >> Hi, >> I've been wanting to solve this in the past, but I stumbled upon the >> fact that we could not always ensure 'eager' resolution when calling >> LayoutType.ofStruct. This is something that eventually should be fixed >> - the situations where such resolution cannot happen are always caused >> by the fact that jextract is generating cross-header symbolic >> references - this is another problem that could be addressed by >> switching to a library-per-extraction approach, where then all >> symbolic references will become self-contained. >> >> In the meantime, your idea of adding a 'tryResolve' is a good one; >> I'll do some more tests on my side, to make sure that everything is >> ok, and then I'll approve >> >> Cheers >> Maurizio >> >> On 19/01/2019 15:16, Jorn Vernee wrote: >>> Hi, >>> >>> Please review the following patch. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8217380 >>> Webrev: >>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217380/webrev.00/ >>> >>> Thanks, >>> Jorn From maurizio.cimadamore at oracle.com Mon Jan 21 11:36:57 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 21 Jan 2019 11:36:57 +0000 Subject: [foreign] RFR 8217380: LayoutType::ofStruct should try to resolve Struct layout In-Reply-To: <4bdf74890a5f514bba4c9ff6032c9f43@xs4all.nl> References: <73f856cdb194769699234ec1c13921e1@xs4all.nl> <6814b0d1e0c80e252e49e13a03f430d0@xs4all.nl> <4bdf74890a5f514bba4c9ff6032c9f43@xs4all.nl> Message-ID: <6c5b9ca6-35f7-753f-bc08-e04264d8683c@oracle.com> On 21/01/2019 11:21, Jorn Vernee wrote: >> but imho Address should override isPartial() to always return `true` > > I mean `false` here. i.e. an Address is never partial, it is always a > complete type. It depends on the point of view - right now the API consider partial any layout that has some unresolved layout in its guts. But, as you observe, not all partial layouts are created equals - and there are two possible things you might want to do with them: 1) ask for size 2) dereference If you just need (1), then excluding addresses from the isPartial business (as you suggest) is the right thing; but if you need (2), that is not enough. Looking at how the code uses 'isPartial', especially in LayoutResolver, I think (1) might be the best semantics; note that if I have a partial layout according to the (2) definition, and I resolve it using a LayoutResolver, nothing will happen, as LayoutResolver will only affect Sequence, Group and Unresolved, leaving Address alone. So I think changing isPartial to reflect that would lead to an improvement. We can also update the javadoc of Descriptor::isPartial to specify exactly what the semantics is. Maurizio > > Jorn > > Jorn Vernee schreef op 2019-01-21 12:11: >> Ok thanks, >> >> FWIW, I thought it was best to try to do the resolution as early as >> possible to avoid having to do it multiple times later on. We need the >> carrier type (which has the annotations) to do the resolution, and the >> earliest time carrier and layout come together seems to be in >> LayoutType. >> >> The other place where we could try and resolve the layout is in Scope, >> right before the allocation, since that's when resolution is needed. >> After sending the RFR email I have been thinking that we might want to >> have some check there either way to see if the type being allocated is >> actually complete. e.g.: >> >> ??? if(type.isPartial()) { >> ??????? throw new IllegalArgumentException("Can not allocate >> incomplete type: " + type); >> ??? } >> >> However, the `isPartial()` check does not only check if a type is >> incomplete. For instance for a pointer; the Address layout isPartial() >> method is inherited from Value which delegates to it's `content`, if >> it has one, to do the `isPartial()` check [1]. This makes sense if we >> have just a Value, but imho Address should override isPartial() to >> always return `true`, since we can always allocate a pointer (just >> maybe not what it points to). >> >> What do you think? >> >> Jorn >> >> [1] : >> http://hg.openjdk.java.net/panama/dev/file/tip/src/java.base/share/classes/java/foreign/layout/Value.java#l100 >> >> >> Maurizio Cimadamore schreef op 2019-01-21 11:52: >>> Hi, >>> I've been wanting to solve this in the past, but I stumbled upon the >>> fact that we could not always ensure 'eager' resolution when calling >>> LayoutType.ofStruct. This is something that eventually should be fixed >>> - the situations where such resolution cannot happen are always caused >>> by the fact that jextract is generating cross-header symbolic >>> references - this is another problem that could be addressed by >>> switching to a library-per-extraction approach, where then all >>> symbolic references will become self-contained. >>> >>> In the meantime, your idea of adding a 'tryResolve' is a good one; >>> I'll do some more tests on my side, to make sure that everything is >>> ok, and then I'll approve >>> >>> Cheers >>> Maurizio >>> >>> On 19/01/2019 15:16, Jorn Vernee wrote: >>>> Hi, >>>> >>>> Please review the following patch. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8217380 >>>> Webrev: >>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217380/webrev.00/ >>>> >>>> Thanks, >>>> Jorn From jbvernee at xs4all.nl Mon Jan 21 12:00:32 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Mon, 21 Jan 2019 13:00:32 +0100 Subject: [foreign] RFR 8217414: Remove bounds check in BoundedPointer constructor Message-ID: Hi, I have a patch addressing 2 vacuously passing tests (api/PointerTest and api/ArrayTest). The patch fixes the tests, and to make that happen removes the bounds check in BoundedPointer's constructor. As discussed offline; this check is not needed since we already de a check when dereferencing a pointer in BoundedMemoryRegion, and the check in the constructor is overly restrictive since it should be fine to create out-of-bounds pointers as long as it is not dereferenced. This does mean the tests needed to be tweaked since now an access exception is throw at a later point. Please review the following. Bug: https://bugs.openjdk.java.net/browse/JDK-8217414 Webrev: http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217414/webrev.00 Thanks, Jorn From tomasz.kowalczewski at gmail.com Mon Jan 21 12:35:28 2019 From: tomasz.kowalczewski at gmail.com (Tomasz Kowalczewski) Date: Mon, 21 Jan 2019 13:35:28 +0100 Subject: [vector api] Vector elements shift confusion Message-ID: Hi, I started this email having problems using rotateER on vectors. I have not found any way to call this method without getting an exception :). More investigation revealed that shift-ing vector elements does not work in accordance with javadoc. I realised this email will be quite long so I split it up and will describe just shift issues. If my suspicions are correct I will follow up with rotations. First shiftEL operation describes itself "as if rotating left the lane elements by i [...] zero value is placed into the result vector at lane indexes less than i % this.length()." *I suspect that the rotation is described in the wrong direction*. Lets try to confirm it but just looking at zeroed lane indices (using get() which "Gets the lane element at lane index i"): @Test public void shouldShiftElementsLeft() { // Given IntVector intVector = IntVector.species(Vector.Shape.S_128_BIT).fromArray(new int[]{ 1, 2, 3, 4 }, 0); // When intVector = intVector.shiftEL(2); // Then assertThat(intVector.get(0)).isEqualTo(0); assertThat(intVector.get(1)).isEqualTo(0); } So after the shift of "2" lane indexes less than "2" should be zero. This test fails as the rotation is done in the opposite (to me - correct) direction: @Test public void shouldShiftElementsLeft() { // Given IntVector intVector = IntVector.species(Vector.Shape.S_128_BIT).fromArray(new int[]{ 1, 2, 3, 4 }, 0); // When intVector = intVector.shiftEL(2); // Then assertThat(intVector.toArray()).containsExactly(3, 4, 0, 0); } Same case is with shiftER. Java doc says that "zero value is placed into the result vector at lane indexes greater or equal to this.length() - (i % this.length()).". But zeroed are indexes < i % this.length(). Looks like implementations are correct but javadoc-s are swapped (I hope I am not making fool of myself). Please advise, Tomasz From maurizio.cimadamore at oracle.com Mon Jan 21 12:53:32 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 21 Jan 2019 12:53:32 +0000 Subject: [foreign] RFR 8217380: LayoutType::ofStruct should try to resolve Struct layout In-Reply-To: References: <73f856cdb194769699234ec1c13921e1@xs4all.nl> Message-ID: On 21/01/2019 10:52, Maurizio Cimadamore wrote: > Hi, > I've been wanting to solve this in the past, but I stumbled upon the > fact that we could not always ensure 'eager' resolution when calling > LayoutType.ofStruct. This is something that eventually should be fixed > - the situations where such resolution cannot happen are always caused > by the fact that jextract is generating cross-header symbolic > references - this is another problem that could be addressed by > switching to a library-per-extraction approach, where then all > symbolic references will become self-contained. > > In the meantime, your idea of adding a 'tryResolve' is a good one; > I'll do some more tests on my side, to make sure that everything is > ok, and then I'll approve All tests look good - this can go ahead Maurizio > > Cheers > Maurizio > > On 19/01/2019 15:16, Jorn Vernee wrote: >> Hi, >> >> Please review the following patch. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8217380 >> Webrev: >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217380/webrev.00/ >> >> Thanks, >> Jorn From maurizio.cimadamore at oracle.com Mon Jan 21 12:56:23 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 21 Jan 2019 12:56:23 +0000 Subject: [foreign] RFR 8217414: Remove bounds check in BoundedPointer constructor In-Reply-To: References: Message-ID: <027a2038-77f8-5213-bf44-ae678c8a07af@oracle.com> I agree that the check should removed from the constructor - but we should make sure that checks happen when the pointer is used. Dereference is one situation, function call is another. I guess dereference is already covered (after all, a memory read will fail if outside the boundaries), but I don't think passing an out of bound pointer to a function is currently being checked, and it should if we remove the constructor eager check Maurizio On 21/01/2019 12:00, Jorn Vernee wrote: > Hi, > > I have a patch addressing 2 vacuously passing tests (api/PointerTest > and api/ArrayTest). The patch fixes the tests, and to make that happen > removes the bounds check in BoundedPointer's constructor. As discussed > offline; this check is not needed since we already de a check when > dereferencing a pointer in BoundedMemoryRegion, and the check in the > constructor is overly restrictive since it should be fine to create > out-of-bounds pointers as long as it is not dereferenced. This does > mean the tests needed to be tweaked since now an access exception is > throw at a later point. > > Please review the following. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8217414 > Webrev: > http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217414/webrev.00 > > Thanks, > Jorn From sundararajan.athijegannathan at oracle.com Mon Jan 21 12:56:51 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Mon, 21 Jan 2019 18:26:51 +0530 Subject: [foreign] RFR 8217414: Remove bounds check in BoundedPointer constructor In-Reply-To: References: Message-ID: <5C45C193.3050809@oracle.com> This change looks good. PS. All tests run fine on Mac. -Sundar On 21/01/19, 5:30 PM, Jorn Vernee wrote: > Hi, > > I have a patch addressing 2 vacuously passing tests (api/PointerTest > and api/ArrayTest). The patch fixes the tests, and to make that happen > removes the bounds check in BoundedPointer's constructor. As discussed > offline; this check is not needed since we already de a check when > dereferencing a pointer in BoundedMemoryRegion, and the check in the > constructor is overly restrictive since it should be fine to create > out-of-bounds pointers as long as it is not dereferenced. This does > mean the tests needed to be tweaked since now an access exception is > throw at a later point. > > Please review the following. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8217414 > Webrev: > http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217414/webrev.00 > > Thanks, > Jorn From jbvernee at xs4all.nl Mon Jan 21 13:00:43 2019 From: jbvernee at xs4all.nl (jbvernee at xs4all.nl) Date: Mon, 21 Jan 2019 13:00:43 +0000 Subject: hg: panama/dev: 8217380: Nested struct layouts not resolved before allocating Message-ID: <201901211300.x0LD0iQ7029803@aojmv0008.oracle.com> Changeset: 23d87d583011 Author: jvernee Date: 2019-01-21 14:00 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/23d87d583011 8217380: Nested struct layouts not resolved before allocating Summary: LayoutType::ofStruct should try to resolve Struct layout Reviewed-by: mcimadamore ! src/java.base/share/classes/java/foreign/memory/LayoutType.java ! src/java.base/share/classes/jdk/internal/foreign/LayoutResolver.java ! src/java.base/share/classes/jdk/internal/foreign/ScopeImpl.java ! src/java.base/share/classes/jdk/internal/foreign/Util.java + test/jdk/com/sun/tools/jextract/unresolved/TestResolve.java + test/jdk/com/sun/tools/jextract/unresolved/test.h From sundararajan.athijegannathan at oracle.com Mon Jan 21 13:03:54 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Mon, 21 Jan 2019 18:33:54 +0530 Subject: [foreign] RFR 8217414: Remove bounds check in BoundedPointer constructor In-Reply-To: <027a2038-77f8-5213-bf44-ae678c8a07af@oracle.com> References: <027a2038-77f8-5213-bf44-ae678c8a07af@oracle.com> Message-ID: <5C45C33A.5000605@oracle.com> Yes - I forgot that we discussed this offline. Also, we could have a test for struct with a pointer field - i.e., we check it on assignment to fields (so that if out of bound pointer is assigned to a struct field -- and the the struct is passed to a native function won't be an issue). -Sundar On 21/01/19, 6:26 PM, Maurizio Cimadamore wrote: > I agree that the check should removed from the constructor - but we > should make sure that checks happen when the pointer is used. > Dereference is one situation, function call is another. I guess > dereference is already covered (after all, a memory read will fail if > outside the boundaries), but I don't think passing an out of bound > pointer to a function is currently being checked, and it should if we > remove the constructor eager check > > Maurizio > > On 21/01/2019 12:00, Jorn Vernee wrote: >> Hi, >> >> I have a patch addressing 2 vacuously passing tests (api/PointerTest >> and api/ArrayTest). The patch fixes the tests, and to make that >> happen removes the bounds check in BoundedPointer's constructor. As >> discussed offline; this check is not needed since we already de a >> check when dereferencing a pointer in BoundedMemoryRegion, and the >> check in the constructor is overly restrictive since it should be >> fine to create out-of-bounds pointers as long as it is not >> dereferenced. This does mean the tests needed to be tweaked since now >> an access exception is throw at a later point. >> >> Please review the following. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8217414 >> Webrev: >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217414/webrev.00 >> >> Thanks, >> Jorn From maurizio.cimadamore at oracle.com Mon Jan 21 13:05:14 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Mon, 21 Jan 2019 13:05:14 +0000 Subject: hg: panama/dev: Automatic merge with foreign Message-ID: <201901211305.x0LD5Exh003261@aojmv0008.oracle.com> Changeset: a328d9b09716 Author: mcimadamore Date: 2019-01-21 14:05 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/a328d9b09716 Automatic merge with foreign From jbvernee at xs4all.nl Mon Jan 21 13:12:42 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Mon, 21 Jan 2019 14:12:42 +0100 Subject: [foreign] RFR 8217414: Remove bounds check in BoundedPointer constructor In-Reply-To: <027a2038-77f8-5213-bf44-ae678c8a07af@oracle.com> References: <027a2038-77f8-5213-bf44-ae678c8a07af@oracle.com> Message-ID: <3e13e71e99bb5f4a6d7b5aeb0d80d3f7@xs4all.nl> There is a `checkAlive()` that gets called when a pointer is unboxed [1], I thought you were talking about that check. But, I don't think we should do a bounds check before passing a pointer to native code. The "one past the end of a container" pointer idiom is common and some APIs might expect to be passed such a pointer. It seems too restrictive to require that pointers passed to native code have to be in bounds (tbh I have my doubts about the liveliness check as well). Jorn [1] : http://hg.openjdk.java.net/panama/dev/file/tip/src/java.base/share/classes/jdk/internal/foreign/memory/BoundedPointer.java#l75 Maurizio Cimadamore schreef op 2019-01-21 13:56: > I agree that the check should removed from the constructor - but we > should make sure that checks happen when the pointer is used. > Dereference is one situation, function call is another. I guess > dereference is already covered (after all, a memory read will fail if > outside the boundaries), but I don't think passing an out of bound > pointer to a function is currently being checked, and it should if we > remove the constructor eager check > > Maurizio > > On 21/01/2019 12:00, Jorn Vernee wrote: >> Hi, >> >> I have a patch addressing 2 vacuously passing tests (api/PointerTest >> and api/ArrayTest). The patch fixes the tests, and to make that happen >> removes the bounds check in BoundedPointer's constructor. As discussed >> offline; this check is not needed since we already de a check when >> dereferencing a pointer in BoundedMemoryRegion, and the check in the >> constructor is overly restrictive since it should be fine to create >> out-of-bounds pointers as long as it is not dereferenced. This does >> mean the tests needed to be tweaked since now an access exception is >> throw at a later point. >> >> Please review the following. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8217414 >> Webrev: >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217414/webrev.00 >> >> Thanks, >> Jorn From maurizio.cimadamore at oracle.com Mon Jan 21 13:42:30 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 21 Jan 2019 13:42:30 +0000 Subject: [foreign] RFR 8217414: Remove bounds check in BoundedPointer constructor In-Reply-To: <3e13e71e99bb5f4a6d7b5aeb0d80d3f7@xs4all.nl> References: <027a2038-77f8-5213-bf44-ae678c8a07af@oracle.com> <3e13e71e99bb5f4a6d7b5aeb0d80d3f7@xs4all.nl> Message-ID: On 21/01/2019 13:12, Jorn Vernee wrote: > There is a `checkAlive()` that gets called when a pointer is unboxed > [1], I thought you were talking about that check. But, I don't think > we should do a bounds check before passing a pointer to native code. > The "one past the end of a container" pointer idiom is common and some > APIs might expect to be passed such a pointer. It seems too > restrictive to require that pointers passed to native code have to be > in bounds (tbh I have my doubts about the liveliness check as well). The whole point of the Panama API is to provide enriched safety. Now, if users want to opt out of that, we might (in the future) provide knobs to do so, but I see no point for passing a pointer that is not alive around (given it is known to contain garbage). Similarly, I see very little point in passing a pointer that is out of bounds - granted there might be few use cases for that (e.g. pass a 'limit' pointer to a library), but I'd prefer to stick with stricter semantics and loosen that up on a use case by use case basis. I think the main point of this patch should be to _delay_ the check that is currently done on the constructor, not to remove it completely. It is ok if some piece of code temporarily produces an out of bound pointers (e.g. when iterating), but I think we should limit the exposure of that weird pointer to the rest of the machinery. Maurizio > > Jorn > > [1] : > http://hg.openjdk.java.net/panama/dev/file/tip/src/java.base/share/classes/jdk/internal/foreign/memory/BoundedPointer.java#l75 > > Maurizio Cimadamore schreef op 2019-01-21 13:56: >> I agree that the check should removed from the constructor - but we >> should make sure that checks happen when the pointer is used. >> Dereference is one situation, function call is another. I guess >> dereference is already covered (after all, a memory read will fail if >> outside the boundaries), but I don't think passing an out of bound >> pointer to a function is currently being checked, and it should if we >> remove the constructor eager check >> >> Maurizio >> >> On 21/01/2019 12:00, Jorn Vernee wrote: >>> Hi, >>> >>> I have a patch addressing 2 vacuously passing tests (api/PointerTest >>> and api/ArrayTest). The patch fixes the tests, and to make that >>> happen removes the bounds check in BoundedPointer's constructor. As >>> discussed offline; this check is not needed since we already de a >>> check when dereferencing a pointer in BoundedMemoryRegion, and the >>> check in the constructor is overly restrictive since it should be >>> fine to create out-of-bounds pointers as long as it is not >>> dereferenced. This does mean the tests needed to be tweaked since >>> now an access exception is throw at a later point. >>> >>> Please review the following. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8217414 >>> Webrev: >>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217414/webrev.00 >>> >>> Thanks, >>> Jorn From jbvernee at xs4all.nl Mon Jan 21 13:54:44 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Mon, 21 Jan 2019 14:54:44 +0100 Subject: [foreign] RFR 8217414: Remove bounds check in BoundedPointer constructor In-Reply-To: References: <027a2038-77f8-5213-bf44-ae678c8a07af@oracle.com> <3e13e71e99bb5f4a6d7b5aeb0d80d3f7@xs4all.nl> Message-ID: <3e6569c13ba5e1a6dea145146a3099ea@xs4all.nl> Ok, I see your point. Plus, if a user finds the API too restrictive they could always fall back on native allocation APIs like malloc & free, or possibly custom LayoutTypes when those are introduced. I think it will be sufficient to move the check to `References.OfPointer` and `References.OfCallback`. But I will add some tests to make sure as well. Jorn Maurizio Cimadamore schreef op 2019-01-21 14:42: > On 21/01/2019 13:12, Jorn Vernee wrote: >> There is a `checkAlive()` that gets called when a pointer is unboxed >> [1], I thought you were talking about that check. But, I don't think >> we should do a bounds check before passing a pointer to native code. >> The "one past the end of a container" pointer idiom is common and some >> APIs might expect to be passed such a pointer. It seems too >> restrictive to require that pointers passed to native code have to be >> in bounds (tbh I have my doubts about the liveliness check as well). > > The whole point of the Panama API is to provide enriched safety. Now, > if users want to opt out of that, we might (in the future) provide > knobs to do so, but I see no point for passing a pointer that is not > alive around (given it is known to contain garbage). Similarly, I see > very little point in passing a pointer that is out of bounds - granted > there might be few use cases for that (e.g. pass a 'limit' pointer to > a library), but I'd prefer to stick with stricter semantics and loosen > that up on a use case by use case basis. > > I think the main point of this patch should be to _delay_ the check > that is currently done on the constructor, not to remove it > completely. It is ok if some piece of code temporarily produces an out > of bound pointers (e.g. when iterating), but I think we should limit > the exposure of that weird pointer to the rest of the machinery. > > Maurizio > >> >> Jorn >> >> [1] : >> http://hg.openjdk.java.net/panama/dev/file/tip/src/java.base/share/classes/jdk/internal/foreign/memory/BoundedPointer.java#l75 >> >> Maurizio Cimadamore schreef op 2019-01-21 13:56: >>> I agree that the check should removed from the constructor - but we >>> should make sure that checks happen when the pointer is used. >>> Dereference is one situation, function call is another. I guess >>> dereference is already covered (after all, a memory read will fail if >>> outside the boundaries), but I don't think passing an out of bound >>> pointer to a function is currently being checked, and it should if we >>> remove the constructor eager check >>> >>> Maurizio >>> >>> On 21/01/2019 12:00, Jorn Vernee wrote: >>>> Hi, >>>> >>>> I have a patch addressing 2 vacuously passing tests (api/PointerTest >>>> and api/ArrayTest). The patch fixes the tests, and to make that >>>> happen removes the bounds check in BoundedPointer's constructor. As >>>> discussed offline; this check is not needed since we already de a >>>> check when dereferencing a pointer in BoundedMemoryRegion, and the >>>> check in the constructor is overly restrictive since it should be >>>> fine to create out-of-bounds pointers as long as it is not >>>> dereferenced. This does mean the tests needed to be tweaked since >>>> now an access exception is throw at a later point. >>>> >>>> Please review the following. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8217414 >>>> Webrev: >>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217414/webrev.00 >>>> >>>> Thanks, >>>> Jorn From jbvernee at xs4all.nl Mon Jan 21 14:29:44 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Mon, 21 Jan 2019 15:29:44 +0100 Subject: [foreign] RFR 8217414: Remove bounds check in BoundedPointer constructor In-Reply-To: <3e6569c13ba5e1a6dea145146a3099ea@xs4all.nl> References: <027a2038-77f8-5213-bf44-ae678c8a07af@oracle.com> <3e13e71e99bb5f4a6d7b5aeb0d80d3f7@xs4all.nl> <3e6569c13ba5e1a6dea145146a3099ea@xs4all.nl> Message-ID: Updated webrev: http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217414/webrev.01 I ended up not adding the check for Callbacks since AFAICT there is currently no public API to create a Callback with an OOB pointer. Should I add the check any ways? Jorn Jorn Vernee schreef op 2019-01-21 14:54: > Ok, I see your point. Plus, if a user finds the API too restrictive > they could always fall back on native allocation APIs like malloc & > free, or possibly custom LayoutTypes when those are introduced. > > I think it will be sufficient to move the check to > `References.OfPointer` and `References.OfCallback`. But I will add > some tests to make sure as well. > > Jorn > > Maurizio Cimadamore schreef op 2019-01-21 14:42: >> On 21/01/2019 13:12, Jorn Vernee wrote: >>> There is a `checkAlive()` that gets called when a pointer is unboxed >>> [1], I thought you were talking about that check. But, I don't think >>> we should do a bounds check before passing a pointer to native code. >>> The "one past the end of a container" pointer idiom is common and >>> some APIs might expect to be passed such a pointer. It seems too >>> restrictive to require that pointers passed to native code have to be >>> in bounds (tbh I have my doubts about the liveliness check as well). >> >> The whole point of the Panama API is to provide enriched safety. Now, >> if users want to opt out of that, we might (in the future) provide >> knobs to do so, but I see no point for passing a pointer that is not >> alive around (given it is known to contain garbage). Similarly, I see >> very little point in passing a pointer that is out of bounds - granted >> there might be few use cases for that (e.g. pass a 'limit' pointer to >> a library), but I'd prefer to stick with stricter semantics and loosen >> that up on a use case by use case basis. >> >> I think the main point of this patch should be to _delay_ the check >> that is currently done on the constructor, not to remove it >> completely. It is ok if some piece of code temporarily produces an out >> of bound pointers (e.g. when iterating), but I think we should limit >> the exposure of that weird pointer to the rest of the machinery. >> >> Maurizio >> >>> >>> Jorn >>> >>> [1] : >>> http://hg.openjdk.java.net/panama/dev/file/tip/src/java.base/share/classes/jdk/internal/foreign/memory/BoundedPointer.java#l75 >>> >>> Maurizio Cimadamore schreef op 2019-01-21 13:56: >>>> I agree that the check should removed from the constructor - but we >>>> should make sure that checks happen when the pointer is used. >>>> Dereference is one situation, function call is another. I guess >>>> dereference is already covered (after all, a memory read will fail >>>> if >>>> outside the boundaries), but I don't think passing an out of bound >>>> pointer to a function is currently being checked, and it should if >>>> we >>>> remove the constructor eager check >>>> >>>> Maurizio >>>> >>>> On 21/01/2019 12:00, Jorn Vernee wrote: >>>>> Hi, >>>>> >>>>> I have a patch addressing 2 vacuously passing tests >>>>> (api/PointerTest and api/ArrayTest). The patch fixes the tests, and >>>>> to make that happen removes the bounds check in BoundedPointer's >>>>> constructor. As discussed offline; this check is not needed since >>>>> we already de a check when dereferencing a pointer in >>>>> BoundedMemoryRegion, and the check in the constructor is overly >>>>> restrictive since it should be fine to create out-of-bounds >>>>> pointers as long as it is not dereferenced. This does mean the >>>>> tests needed to be tweaked since now an access exception is throw >>>>> at a later point. >>>>> >>>>> Please review the following. >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8217414 >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217414/webrev.00 >>>>> >>>>> Thanks, >>>>> Jorn From maurizio.cimadamore at oracle.com Mon Jan 21 15:15:11 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 21 Jan 2019 15:15:11 +0000 Subject: [foreign] RFR 8217414: Remove bounds check in BoundedPointer constructor In-Reply-To: References: <027a2038-77f8-5213-bf44-ae678c8a07af@oracle.com> <3e13e71e99bb5f4a6d7b5aeb0d80d3f7@xs4all.nl> <3e6569c13ba5e1a6dea145146a3099ea@xs4all.nl> Message-ID: <80ad827c-faed-8364-781c-1fa9e572d1b3@oracle.com> On 21/01/2019 14:29, Jorn Vernee wrote: > Updated webrev: > http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217414/webrev.01 > > I ended up not adding the check for Callbacks since AFAICT there is > currently no public API to create a Callback with an OOB pointer. > Should I add the check any ways? That's fine, but I'm pretty sure that the added test will fail with the direct invoker. You might want to tweak this method in DirectSignatureShuffler: private static long pointerToLong(Pointer value) throws IllegalAccessException { ??????? return value.addr(); ??? } Another option, would be to always throw when somebody calls addr() and the pointer is out of bounds - this could actually be a better solution (which would also subsume the changes in References.OfPointer) ? Maurizio > > Jorn > > Jorn Vernee schreef op 2019-01-21 14:54: >> Ok, I see your point. Plus, if a user finds the API too restrictive >> they could always fall back on native allocation APIs like malloc & >> free, or possibly custom LayoutTypes when those are introduced. >> >> I think it will be sufficient to move the check to >> `References.OfPointer` and `References.OfCallback`. But I will add >> some tests to make sure as well. >> >> Jorn >> >> Maurizio Cimadamore schreef op 2019-01-21 14:42: >>> On 21/01/2019 13:12, Jorn Vernee wrote: >>>> There is a `checkAlive()` that gets called when a pointer is >>>> unboxed [1], I thought you were talking about that check. But, I >>>> don't think we should do a bounds check before passing a pointer to >>>> native code. The "one past the end of a container" pointer idiom is >>>> common and some APIs might expect to be passed such a pointer. It >>>> seems too restrictive to require that pointers passed to native >>>> code have to be in bounds (tbh I have my doubts about the >>>> liveliness check as well). >>> >>> The whole point of the Panama API is to provide enriched safety. Now, >>> if users want to opt out of that, we might (in the future) provide >>> knobs to do so, but I see no point for passing a pointer that is not >>> alive around (given it is known to contain garbage). Similarly, I see >>> very little point in passing a pointer that is out of bounds - granted >>> there might be few use cases for that (e.g. pass a 'limit' pointer to >>> a library), but I'd prefer to stick with stricter semantics and loosen >>> that up on a use case by use case basis. >>> >>> I think the main point of this patch should be to _delay_ the check >>> that is currently done on the constructor, not to remove it >>> completely. It is ok if some piece of code temporarily produces an out >>> of bound pointers (e.g. when iterating), but I think we should limit >>> the exposure of that weird pointer to the rest of the machinery. >>> >>> Maurizio >>> >>>> >>>> Jorn >>>> >>>> [1] : >>>> http://hg.openjdk.java.net/panama/dev/file/tip/src/java.base/share/classes/jdk/internal/foreign/memory/BoundedPointer.java#l75 >>>> >>>> Maurizio Cimadamore schreef op 2019-01-21 13:56: >>>>> I agree that the check should removed from the constructor - but we >>>>> should make sure that checks happen when the pointer is used. >>>>> Dereference is one situation, function call is another. I guess >>>>> dereference is already covered (after all, a memory read will fail if >>>>> outside the boundaries), but I don't think passing an out of bound >>>>> pointer to a function is currently being checked, and it should if we >>>>> remove the constructor eager check >>>>> >>>>> Maurizio >>>>> >>>>> On 21/01/2019 12:00, Jorn Vernee wrote: >>>>>> Hi, >>>>>> >>>>>> I have a patch addressing 2 vacuously passing tests >>>>>> (api/PointerTest and api/ArrayTest). The patch fixes the tests, >>>>>> and to make that happen removes the bounds check in >>>>>> BoundedPointer's constructor. As discussed offline; this check is >>>>>> not needed since we already de a check when dereferencing a >>>>>> pointer in BoundedMemoryRegion, and the check in the constructor >>>>>> is overly restrictive since it should be fine to create >>>>>> out-of-bounds pointers as long as it is not dereferenced. This >>>>>> does mean the tests needed to be tweaked since now an access >>>>>> exception is throw at a later point. >>>>>> >>>>>> Please review the following. >>>>>> >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8217414 >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217414/webrev.00 >>>>>> >>>>>> Thanks, >>>>>> Jorn From jbvernee at xs4all.nl Mon Jan 21 15:17:38 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Mon, 21 Jan 2019 16:17:38 +0100 Subject: [foreign] RFR 8217414: Remove bounds check in BoundedPointer constructor In-Reply-To: References: <027a2038-77f8-5213-bf44-ae678c8a07af@oracle.com> <3e13e71e99bb5f4a6d7b5aeb0d80d3f7@xs4all.nl> <3e6569c13ba5e1a6dea145146a3099ea@xs4all.nl> Message-ID: <4124d1a13b3d4b4d2cf9ed258dbb944f@xs4all.nl> Sorry, I forgot to add a test for struct field set as well. Updated webrev: http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217414/webrev.02 I also went ahead and implemented the check for Callbacks as well by delegating to the pointer unboxing code in the Callback unboxing code. There is no test for the Callback case since there was no public API to create a OOB Callback. Jorn Jorn Vernee schreef op 2019-01-21 15:29: > Updated webrev: > http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217414/webrev.01 > > I ended up not adding the check for Callbacks since AFAICT there is > currently no public API to create a Callback with an OOB pointer. > Should I add the check any ways? > > Jorn > > Jorn Vernee schreef op 2019-01-21 14:54: >> Ok, I see your point. Plus, if a user finds the API too restrictive >> they could always fall back on native allocation APIs like malloc & >> free, or possibly custom LayoutTypes when those are introduced. >> >> I think it will be sufficient to move the check to >> `References.OfPointer` and `References.OfCallback`. But I will add >> some tests to make sure as well. >> >> Jorn >> >> Maurizio Cimadamore schreef op 2019-01-21 14:42: >>> On 21/01/2019 13:12, Jorn Vernee wrote: >>>> There is a `checkAlive()` that gets called when a pointer is unboxed >>>> [1], I thought you were talking about that check. But, I don't think >>>> we should do a bounds check before passing a pointer to native code. >>>> The "one past the end of a container" pointer idiom is common and >>>> some APIs might expect to be passed such a pointer. It seems too >>>> restrictive to require that pointers passed to native code have to >>>> be in bounds (tbh I have my doubts about the liveliness check as >>>> well). >>> >>> The whole point of the Panama API is to provide enriched safety. Now, >>> if users want to opt out of that, we might (in the future) provide >>> knobs to do so, but I see no point for passing a pointer that is not >>> alive around (given it is known to contain garbage). Similarly, I see >>> very little point in passing a pointer that is out of bounds - >>> granted >>> there might be few use cases for that (e.g. pass a 'limit' pointer to >>> a library), but I'd prefer to stick with stricter semantics and >>> loosen >>> that up on a use case by use case basis. >>> >>> I think the main point of this patch should be to _delay_ the check >>> that is currently done on the constructor, not to remove it >>> completely. It is ok if some piece of code temporarily produces an >>> out >>> of bound pointers (e.g. when iterating), but I think we should limit >>> the exposure of that weird pointer to the rest of the machinery. >>> >>> Maurizio >>> >>>> >>>> Jorn >>>> >>>> [1] : >>>> http://hg.openjdk.java.net/panama/dev/file/tip/src/java.base/share/classes/jdk/internal/foreign/memory/BoundedPointer.java#l75 >>>> >>>> Maurizio Cimadamore schreef op 2019-01-21 13:56: >>>>> I agree that the check should removed from the constructor - but we >>>>> should make sure that checks happen when the pointer is used. >>>>> Dereference is one situation, function call is another. I guess >>>>> dereference is already covered (after all, a memory read will fail >>>>> if >>>>> outside the boundaries), but I don't think passing an out of bound >>>>> pointer to a function is currently being checked, and it should if >>>>> we >>>>> remove the constructor eager check >>>>> >>>>> Maurizio >>>>> >>>>> On 21/01/2019 12:00, Jorn Vernee wrote: >>>>>> Hi, >>>>>> >>>>>> I have a patch addressing 2 vacuously passing tests >>>>>> (api/PointerTest and api/ArrayTest). The patch fixes the tests, >>>>>> and to make that happen removes the bounds check in >>>>>> BoundedPointer's constructor. As discussed offline; this check is >>>>>> not needed since we already de a check when dereferencing a >>>>>> pointer in BoundedMemoryRegion, and the check in the constructor >>>>>> is overly restrictive since it should be fine to create >>>>>> out-of-bounds pointers as long as it is not dereferenced. This >>>>>> does mean the tests needed to be tweaked since now an access >>>>>> exception is throw at a later point. >>>>>> >>>>>> Please review the following. >>>>>> >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8217414 >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217414/webrev.00 >>>>>> >>>>>> Thanks, >>>>>> Jorn From jbvernee at xs4all.nl Mon Jan 21 15:27:09 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Mon, 21 Jan 2019 16:27:09 +0100 Subject: [foreign] RFR 8217414: Remove bounds check in BoundedPointer constructor In-Reply-To: <80ad827c-faed-8364-781c-1fa9e572d1b3@oracle.com> References: <027a2038-77f8-5213-bf44-ae678c8a07af@oracle.com> <3e13e71e99bb5f4a6d7b5aeb0d80d3f7@xs4all.nl> <3e6569c13ba5e1a6dea145146a3099ea@xs4all.nl> <80ad827c-faed-8364-781c-1fa9e572d1b3@oracle.com> Message-ID: <145bff1522eb74198d4f4545e78c1693@xs4all.nl> Maurizio Cimadamore schreef op 2019-01-21 16:15: > On 21/01/2019 14:29, Jorn Vernee wrote: >> Updated webrev: >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217414/webrev.01 >> >> I ended up not adding the check for Callbacks since AFAICT there is >> currently no public API to create a Callback with an OOB pointer. >> Should I add the check any ways? > > That's fine, but I'm pretty sure that the added test will fail with > the direct invoker. Ah OK, I guess the direct invoker just side-steps LayoutTypes altogether (I have not spent that much time with that code). > You might want to tweak this method in DirectSignatureShuffler: > > private static long pointerToLong(Pointer value) throws > IllegalAccessException { > ??????? return value.addr(); > ??? } > > Another option, would be to always throw when somebody calls addr() > and the pointer is out of bounds - this could actually be a better > solution (which would also subsume the changes in > References.OfPointer) ? We need to be able to retrieve the address as part of BoundedPointer::equals, which is needed to make the iterate code work, so doing the check in `addr()` would still be too early. I'll have a look at the direct invoker. Jorn > Maurizio > >> >> Jorn >> >> Jorn Vernee schreef op 2019-01-21 14:54: >>> Ok, I see your point. Plus, if a user finds the API too restrictive >>> they could always fall back on native allocation APIs like malloc & >>> free, or possibly custom LayoutTypes when those are introduced. >>> >>> I think it will be sufficient to move the check to >>> `References.OfPointer` and `References.OfCallback`. But I will add >>> some tests to make sure as well. >>> >>> Jorn >>> >>> Maurizio Cimadamore schreef op 2019-01-21 14:42: >>>> On 21/01/2019 13:12, Jorn Vernee wrote: >>>>> There is a `checkAlive()` that gets called when a pointer is >>>>> unboxed [1], I thought you were talking about that check. But, I >>>>> don't think we should do a bounds check before passing a pointer to >>>>> native code. The "one past the end of a container" pointer idiom is >>>>> common and some APIs might expect to be passed such a pointer. It >>>>> seems too restrictive to require that pointers passed to native >>>>> code have to be in bounds (tbh I have my doubts about the >>>>> liveliness check as well). >>>> >>>> The whole point of the Panama API is to provide enriched safety. >>>> Now, >>>> if users want to opt out of that, we might (in the future) provide >>>> knobs to do so, but I see no point for passing a pointer that is not >>>> alive around (given it is known to contain garbage). Similarly, I >>>> see >>>> very little point in passing a pointer that is out of bounds - >>>> granted >>>> there might be few use cases for that (e.g. pass a 'limit' pointer >>>> to >>>> a library), but I'd prefer to stick with stricter semantics and >>>> loosen >>>> that up on a use case by use case basis. >>>> >>>> I think the main point of this patch should be to _delay_ the check >>>> that is currently done on the constructor, not to remove it >>>> completely. It is ok if some piece of code temporarily produces an >>>> out >>>> of bound pointers (e.g. when iterating), but I think we should limit >>>> the exposure of that weird pointer to the rest of the machinery. >>>> >>>> Maurizio >>>> >>>>> >>>>> Jorn >>>>> >>>>> [1] : >>>>> http://hg.openjdk.java.net/panama/dev/file/tip/src/java.base/share/classes/jdk/internal/foreign/memory/BoundedPointer.java#l75 >>>>> >>>>> Maurizio Cimadamore schreef op 2019-01-21 13:56: >>>>>> I agree that the check should removed from the constructor - but >>>>>> we >>>>>> should make sure that checks happen when the pointer is used. >>>>>> Dereference is one situation, function call is another. I guess >>>>>> dereference is already covered (after all, a memory read will fail >>>>>> if >>>>>> outside the boundaries), but I don't think passing an out of bound >>>>>> pointer to a function is currently being checked, and it should if >>>>>> we >>>>>> remove the constructor eager check >>>>>> >>>>>> Maurizio >>>>>> >>>>>> On 21/01/2019 12:00, Jorn Vernee wrote: >>>>>>> Hi, >>>>>>> >>>>>>> I have a patch addressing 2 vacuously passing tests >>>>>>> (api/PointerTest and api/ArrayTest). The patch fixes the tests, >>>>>>> and to make that happen removes the bounds check in >>>>>>> BoundedPointer's constructor. As discussed offline; this check is >>>>>>> not needed since we already de a check when dereferencing a >>>>>>> pointer in BoundedMemoryRegion, and the check in the constructor >>>>>>> is overly restrictive since it should be fine to create >>>>>>> out-of-bounds pointers as long as it is not dereferenced. This >>>>>>> does mean the tests needed to be tweaked since now an access >>>>>>> exception is throw at a later point. >>>>>>> >>>>>>> Please review the following. >>>>>>> >>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8217414 >>>>>>> Webrev: >>>>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217414/webrev.00 >>>>>>> >>>>>>> Thanks, >>>>>>> Jorn From maurizio.cimadamore at oracle.com Mon Jan 21 15:31:14 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 21 Jan 2019 15:31:14 +0000 Subject: [foreign] RFR 8217414: Remove bounds check in BoundedPointer constructor In-Reply-To: <145bff1522eb74198d4f4545e78c1693@xs4all.nl> References: <027a2038-77f8-5213-bf44-ae678c8a07af@oracle.com> <3e13e71e99bb5f4a6d7b5aeb0d80d3f7@xs4all.nl> <3e6569c13ba5e1a6dea145146a3099ea@xs4all.nl> <80ad827c-faed-8364-781c-1fa9e572d1b3@oracle.com> <145bff1522eb74198d4f4545e78c1693@xs4all.nl> Message-ID: <42d20409-9b16-d7eb-93fc-b7dfef832204@oracle.com> On 21/01/2019 15:27, Jorn Vernee wrote: > Maurizio Cimadamore schreef op 2019-01-21 16:15: >> On 21/01/2019 14:29, Jorn Vernee wrote: >>> Updated webrev: >>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217414/webrev.01 >>> >>> I ended up not adding the check for Callbacks since AFAICT there is >>> currently no public API to create a Callback with an OOB pointer. >>> Should I add the check any ways? >> >> That's fine, but I'm pretty sure that the added test will fail with >> the direct invoker. > > Ah OK, I guess the direct invoker just side-steps LayoutTypes > altogether (I have not spent that much time with that code). > >> You might want to tweak this method in DirectSignatureShuffler: >> >> private static long pointerToLong(Pointer value) throws >> IllegalAccessException { >> ??????? return value.addr(); >> ??? } >> >> Another option, would be to always throw when somebody calls addr() >> and the pointer is out of bounds - this could actually be a better >> solution (which would also subsume the changes in >> References.OfPointer) ? > > We need to be able to retrieve the address as part of > BoundedPointer::equals, which is needed to make the iterate code work, > so doing the check in `addr()` would still be too early. That seems a non-issue - e.g. internally we could have two addr() functions, a checked one (exposed by the API) and an unchecked one, which can be used as an impl aid for Pointer::equals, no? Maurizio > > I'll have a look at the direct invoker. > > Jorn > >> Maurizio >> >>> >>> Jorn >>> >>> Jorn Vernee schreef op 2019-01-21 14:54: >>>> Ok, I see your point. Plus, if a user finds the API too restrictive >>>> they could always fall back on native allocation APIs like malloc & >>>> free, or possibly custom LayoutTypes when those are introduced. >>>> >>>> I think it will be sufficient to move the check to >>>> `References.OfPointer` and `References.OfCallback`. But I will add >>>> some tests to make sure as well. >>>> >>>> Jorn >>>> >>>> Maurizio Cimadamore schreef op 2019-01-21 14:42: >>>>> On 21/01/2019 13:12, Jorn Vernee wrote: >>>>>> There is a `checkAlive()` that gets called when a pointer is >>>>>> unboxed [1], I thought you were talking about that check. But, I >>>>>> don't think we should do a bounds check before passing a pointer >>>>>> to native code. The "one past the end of a container" pointer >>>>>> idiom is common and some APIs might expect to be passed such a >>>>>> pointer. It seems too restrictive to require that pointers passed >>>>>> to native code have to be in bounds (tbh I have my doubts about >>>>>> the liveliness check as well). >>>>> >>>>> The whole point of the Panama API is to provide enriched safety. Now, >>>>> if users want to opt out of that, we might (in the future) provide >>>>> knobs to do so, but I see no point for passing a pointer that is not >>>>> alive around (given it is known to contain garbage). Similarly, I see >>>>> very little point in passing a pointer that is out of bounds - >>>>> granted >>>>> there might be few use cases for that (e.g. pass a 'limit' pointer to >>>>> a library), but I'd prefer to stick with stricter semantics and >>>>> loosen >>>>> that up on a use case by use case basis. >>>>> >>>>> I think the main point of this patch should be to _delay_ the check >>>>> that is currently done on the constructor, not to remove it >>>>> completely. It is ok if some piece of code temporarily produces an >>>>> out >>>>> of bound pointers (e.g. when iterating), but I think we should limit >>>>> the exposure of that weird pointer to the rest of the machinery. >>>>> >>>>> Maurizio >>>>> >>>>>> >>>>>> Jorn >>>>>> >>>>>> [1] : >>>>>> http://hg.openjdk.java.net/panama/dev/file/tip/src/java.base/share/classes/jdk/internal/foreign/memory/BoundedPointer.java#l75 >>>>>> >>>>>> Maurizio Cimadamore schreef op 2019-01-21 13:56: >>>>>>> I agree that the check should removed from the constructor - but we >>>>>>> should make sure that checks happen when the pointer is used. >>>>>>> Dereference is one situation, function call is another. I guess >>>>>>> dereference is already covered (after all, a memory read will >>>>>>> fail if >>>>>>> outside the boundaries), but I don't think passing an out of bound >>>>>>> pointer to a function is currently being checked, and it should >>>>>>> if we >>>>>>> remove the constructor eager check >>>>>>> >>>>>>> Maurizio >>>>>>> >>>>>>> On 21/01/2019 12:00, Jorn Vernee wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> I have a patch addressing 2 vacuously passing tests >>>>>>>> (api/PointerTest and api/ArrayTest). The patch fixes the tests, >>>>>>>> and to make that happen removes the bounds check in >>>>>>>> BoundedPointer's constructor. As discussed offline; this check >>>>>>>> is not needed since we already de a check when dereferencing a >>>>>>>> pointer in BoundedMemoryRegion, and the check in the >>>>>>>> constructor is overly restrictive since it should be fine to >>>>>>>> create out-of-bounds pointers as long as it is not >>>>>>>> dereferenced. This does mean the tests needed to be tweaked >>>>>>>> since now an access exception is throw at a later point. >>>>>>>> >>>>>>>> Please review the following. >>>>>>>> >>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8217414 >>>>>>>> Webrev: >>>>>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217414/webrev.00 >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Jorn From jbvernee at xs4all.nl Mon Jan 21 15:48:01 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Mon, 21 Jan 2019 16:48:01 +0100 Subject: [foreign] RFR 8217414: Remove bounds check in BoundedPointer constructor In-Reply-To: <42d20409-9b16-d7eb-93fc-b7dfef832204@oracle.com> References: <027a2038-77f8-5213-bf44-ae678c8a07af@oracle.com> <3e13e71e99bb5f4a6d7b5aeb0d80d3f7@xs4all.nl> <3e6569c13ba5e1a6dea145146a3099ea@xs4all.nl> <80ad827c-faed-8364-781c-1fa9e572d1b3@oracle.com> <145bff1522eb74198d4f4545e78c1693@xs4all.nl> <42d20409-9b16-d7eb-93fc-b7dfef832204@oracle.com> Message-ID: <612faf056431f3ef511bc9ed3db707c6@xs4all.nl> Maurizio Cimadamore schreef op 2019-01-21 16:31: > On 21/01/2019 15:27, Jorn Vernee wrote: >> Maurizio Cimadamore schreef op 2019-01-21 16:15: >>> On 21/01/2019 14:29, Jorn Vernee wrote: >>>> Updated webrev: >>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217414/webrev.01 >>>> >>>> I ended up not adding the check for Callbacks since AFAICT there is >>>> currently no public API to create a Callback with an OOB pointer. >>>> Should I add the check any ways? >>> >>> That's fine, but I'm pretty sure that the added test will fail with >>> the direct invoker. >> >> Ah OK, I guess the direct invoker just side-steps LayoutTypes >> altogether (I have not spent that much time with that code). >> >>> You might want to tweak this method in DirectSignatureShuffler: >>> >>> private static long pointerToLong(Pointer value) throws >>> IllegalAccessException { >>> ??????? return value.addr(); >>> ??? } >>> >>> Another option, would be to always throw when somebody calls addr() >>> and the pointer is out of bounds - this could actually be a better >>> solution (which would also subsume the changes in >>> References.OfPointer) ? >> >> We need to be able to retrieve the address as part of >> BoundedPointer::equals, which is needed to make the iterate code work, >> so doing the check in `addr()` would still be too early. > > That seems a non-issue - e.g. internally we could have two addr() > functions, a checked one (exposed by the API) and an unchecked one, > which can be used as an impl aid for Pointer::equals, no? Yeah I guess that works as well. hashCode() would also need to be updated in that case. But, I feel that it weakens the semantics for Pointer a bit; the address can be used for equals and hashCode while a pointer is OOB, but we can't look at it's address (even if we don't want to use the address). Doing the check at the point where we are actually unboxing the pointer seems like the more natural choice to me. Jorn > Maurizio > >> >> I'll have a look at the direct invoker. >> >> Jorn >> >>> Maurizio >>> >>>> >>>> Jorn >>>> >>>> Jorn Vernee schreef op 2019-01-21 14:54: >>>>> Ok, I see your point. Plus, if a user finds the API too restrictive >>>>> they could always fall back on native allocation APIs like malloc & >>>>> free, or possibly custom LayoutTypes when those are introduced. >>>>> >>>>> I think it will be sufficient to move the check to >>>>> `References.OfPointer` and `References.OfCallback`. But I will add >>>>> some tests to make sure as well. >>>>> >>>>> Jorn >>>>> >>>>> Maurizio Cimadamore schreef op 2019-01-21 14:42: >>>>>> On 21/01/2019 13:12, Jorn Vernee wrote: >>>>>>> There is a `checkAlive()` that gets called when a pointer is >>>>>>> unboxed [1], I thought you were talking about that check. But, I >>>>>>> don't think we should do a bounds check before passing a pointer >>>>>>> to native code. The "one past the end of a container" pointer >>>>>>> idiom is common and some APIs might expect to be passed such a >>>>>>> pointer. It seems too restrictive to require that pointers passed >>>>>>> to native code have to be in bounds (tbh I have my doubts about >>>>>>> the liveliness check as well). >>>>>> >>>>>> The whole point of the Panama API is to provide enriched safety. >>>>>> Now, >>>>>> if users want to opt out of that, we might (in the future) provide >>>>>> knobs to do so, but I see no point for passing a pointer that is >>>>>> not >>>>>> alive around (given it is known to contain garbage). Similarly, I >>>>>> see >>>>>> very little point in passing a pointer that is out of bounds - >>>>>> granted >>>>>> there might be few use cases for that (e.g. pass a 'limit' pointer >>>>>> to >>>>>> a library), but I'd prefer to stick with stricter semantics and >>>>>> loosen >>>>>> that up on a use case by use case basis. >>>>>> >>>>>> I think the main point of this patch should be to _delay_ the >>>>>> check >>>>>> that is currently done on the constructor, not to remove it >>>>>> completely. It is ok if some piece of code temporarily produces an >>>>>> out >>>>>> of bound pointers (e.g. when iterating), but I think we should >>>>>> limit >>>>>> the exposure of that weird pointer to the rest of the machinery. >>>>>> >>>>>> Maurizio >>>>>> >>>>>>> >>>>>>> Jorn >>>>>>> >>>>>>> [1] : >>>>>>> http://hg.openjdk.java.net/panama/dev/file/tip/src/java.base/share/classes/jdk/internal/foreign/memory/BoundedPointer.java#l75 >>>>>>> >>>>>>> Maurizio Cimadamore schreef op 2019-01-21 13:56: >>>>>>>> I agree that the check should removed from the constructor - but >>>>>>>> we >>>>>>>> should make sure that checks happen when the pointer is used. >>>>>>>> Dereference is one situation, function call is another. I guess >>>>>>>> dereference is already covered (after all, a memory read will >>>>>>>> fail if >>>>>>>> outside the boundaries), but I don't think passing an out of >>>>>>>> bound >>>>>>>> pointer to a function is currently being checked, and it should >>>>>>>> if we >>>>>>>> remove the constructor eager check >>>>>>>> >>>>>>>> Maurizio >>>>>>>> >>>>>>>> On 21/01/2019 12:00, Jorn Vernee wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> I have a patch addressing 2 vacuously passing tests >>>>>>>>> (api/PointerTest and api/ArrayTest). The patch fixes the tests, >>>>>>>>> and to make that happen removes the bounds check in >>>>>>>>> BoundedPointer's constructor. As discussed offline; this check >>>>>>>>> is not needed since we already de a check when dereferencing a >>>>>>>>> pointer in BoundedMemoryRegion, and the check in the >>>>>>>>> constructor is overly restrictive since it should be fine to >>>>>>>>> create out-of-bounds pointers as long as it is not >>>>>>>>> dereferenced. This does mean the tests needed to be tweaked >>>>>>>>> since now an access exception is throw at a later point. >>>>>>>>> >>>>>>>>> Please review the following. >>>>>>>>> >>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8217414 >>>>>>>>> Webrev: >>>>>>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217414/webrev.00 >>>>>>>>> Thanks, >>>>>>>>> Jorn From maurizio.cimadamore at oracle.com Mon Jan 21 15:54:46 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 21 Jan 2019 15:54:46 +0000 Subject: [foreign] RFR 8217414: Remove bounds check in BoundedPointer constructor In-Reply-To: <612faf056431f3ef511bc9ed3db707c6@xs4all.nl> References: <027a2038-77f8-5213-bf44-ae678c8a07af@oracle.com> <3e13e71e99bb5f4a6d7b5aeb0d80d3f7@xs4all.nl> <3e6569c13ba5e1a6dea145146a3099ea@xs4all.nl> <80ad827c-faed-8364-781c-1fa9e572d1b3@oracle.com> <145bff1522eb74198d4f4545e78c1693@xs4all.nl> <42d20409-9b16-d7eb-93fc-b7dfef832204@oracle.com> <612faf056431f3ef511bc9ed3db707c6@xs4all.nl> Message-ID: On 21/01/2019 15:48, Jorn Vernee wrote: > Maurizio Cimadamore schreef op 2019-01-21 16:31: >> On 21/01/2019 15:27, Jorn Vernee wrote: >>> Maurizio Cimadamore schreef op 2019-01-21 16:15: >>>> On 21/01/2019 14:29, Jorn Vernee wrote: >>>>> Updated webrev: >>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217414/webrev.01 >>>>> >>>>> I ended up not adding the check for Callbacks since AFAICT there >>>>> is currently no public API to create a Callback with an OOB >>>>> pointer. Should I add the check any ways? >>>> >>>> That's fine, but I'm pretty sure that the added test will fail with >>>> the direct invoker. >>> >>> Ah OK, I guess the direct invoker just side-steps LayoutTypes >>> altogether (I have not spent that much time with that code). >>> >>>> You might want to tweak this method in DirectSignatureShuffler: >>>> >>>> private static long pointerToLong(Pointer value) throws >>>> IllegalAccessException { >>>> ??????? return value.addr(); >>>> ??? } >>>> >>>> Another option, would be to always throw when somebody calls addr() >>>> and the pointer is out of bounds - this could actually be a better >>>> solution (which would also subsume the changes in >>>> References.OfPointer) ? >>> >>> We need to be able to retrieve the address as part of >>> BoundedPointer::equals, which is needed to make the iterate code >>> work, so doing the check in `addr()` would still be too early. >> >> That seems a non-issue - e.g. internally we could have two addr() >> functions, a checked one (exposed by the API) and an unchecked one, >> which can be used as an impl aid for Pointer::equals, no? > > Yeah I guess that works as well. hashCode() would also need to be > updated in that case. But, I feel that it weakens the semantics for > Pointer a bit; the address can be used for equals and hashCode while a > pointer is OOB, but we can't look at it's address (even if we don't > want to use the address). Doing the check at the point where we are > actually unboxing the pointer seems like the more natural choice to me. Note that it is essentially an accident of the current impl that addr() does something in the first place. I can imagine cases where users do not want their pointers to reveal real memory addresses - so addr() should always be taken as something that 'can throw' as the signature implies right now. Also, addr() is also where the liveness check takes place, so that's already how the code works. Maurizio > > Jorn > >> Maurizio >> >>> >>> I'll have a look at the direct invoker. >>> >>> Jorn >>> >>>> Maurizio >>>> >>>>> >>>>> Jorn >>>>> >>>>> Jorn Vernee schreef op 2019-01-21 14:54: >>>>>> Ok, I see your point. Plus, if a user finds the API too restrictive >>>>>> they could always fall back on native allocation APIs like malloc & >>>>>> free, or possibly custom LayoutTypes when those are introduced. >>>>>> >>>>>> I think it will be sufficient to move the check to >>>>>> `References.OfPointer` and `References.OfCallback`. But I will add >>>>>> some tests to make sure as well. >>>>>> >>>>>> Jorn >>>>>> >>>>>> Maurizio Cimadamore schreef op 2019-01-21 14:42: >>>>>>> On 21/01/2019 13:12, Jorn Vernee wrote: >>>>>>>> There is a `checkAlive()` that gets called when a pointer is >>>>>>>> unboxed [1], I thought you were talking about that check. But, >>>>>>>> I don't think we should do a bounds check before passing a >>>>>>>> pointer to native code. The "one past the end of a container" >>>>>>>> pointer idiom is common and some APIs might expect to be passed >>>>>>>> such a pointer. It seems too restrictive to require that >>>>>>>> pointers passed to native code have to be in bounds (tbh I have >>>>>>>> my doubts about the liveliness check as well). >>>>>>> >>>>>>> The whole point of the Panama API is to provide enriched safety. >>>>>>> Now, >>>>>>> if users want to opt out of that, we might (in the future) provide >>>>>>> knobs to do so, but I see no point for passing a pointer that is >>>>>>> not >>>>>>> alive around (given it is known to contain garbage). Similarly, >>>>>>> I see >>>>>>> very little point in passing a pointer that is out of bounds - >>>>>>> granted >>>>>>> there might be few use cases for that (e.g. pass a 'limit' >>>>>>> pointer to >>>>>>> a library), but I'd prefer to stick with stricter semantics and >>>>>>> loosen >>>>>>> that up on a use case by use case basis. >>>>>>> >>>>>>> I think the main point of this patch should be to _delay_ the check >>>>>>> that is currently done on the constructor, not to remove it >>>>>>> completely. It is ok if some piece of code temporarily produces >>>>>>> an out >>>>>>> of bound pointers (e.g. when iterating), but I think we should >>>>>>> limit >>>>>>> the exposure of that weird pointer to the rest of the machinery. >>>>>>> >>>>>>> Maurizio >>>>>>> >>>>>>>> >>>>>>>> Jorn >>>>>>>> >>>>>>>> [1] : >>>>>>>> http://hg.openjdk.java.net/panama/dev/file/tip/src/java.base/share/classes/jdk/internal/foreign/memory/BoundedPointer.java#l75 >>>>>>>> >>>>>>>> Maurizio Cimadamore schreef op 2019-01-21 13:56: >>>>>>>>> I agree that the check should removed from the constructor - >>>>>>>>> but we >>>>>>>>> should make sure that checks happen when the pointer is used. >>>>>>>>> Dereference is one situation, function call is another. I guess >>>>>>>>> dereference is already covered (after all, a memory read will >>>>>>>>> fail if >>>>>>>>> outside the boundaries), but I don't think passing an out of >>>>>>>>> bound >>>>>>>>> pointer to a function is currently being checked, and it >>>>>>>>> should if we >>>>>>>>> remove the constructor eager check >>>>>>>>> >>>>>>>>> Maurizio >>>>>>>>> >>>>>>>>> On 21/01/2019 12:00, Jorn Vernee wrote: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> I have a patch addressing 2 vacuously passing tests >>>>>>>>>> (api/PointerTest and api/ArrayTest). The patch fixes the >>>>>>>>>> tests, and to make that happen removes the bounds check in >>>>>>>>>> BoundedPointer's constructor. As discussed offline; this >>>>>>>>>> check is not needed since we already de a check when >>>>>>>>>> dereferencing a pointer in BoundedMemoryRegion, and the check >>>>>>>>>> in the constructor is overly restrictive since it should be >>>>>>>>>> fine to create out-of-bounds pointers as long as it is not >>>>>>>>>> dereferenced. This does mean the tests needed to be tweaked >>>>>>>>>> since now an access exception is throw at a later point. >>>>>>>>>> >>>>>>>>>> Please review the following. >>>>>>>>>> >>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8217414 >>>>>>>>>> Webrev: >>>>>>>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217414/webrev.00 >>>>>>>>>> Thanks, >>>>>>>>>> Jorn From maurizio.cimadamore at oracle.com Mon Jan 21 16:12:47 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 21 Jan 2019 16:12:47 +0000 Subject: [foreign] RFR 8217414: Remove bounds check in BoundedPointer constructor In-Reply-To: References: <027a2038-77f8-5213-bf44-ae678c8a07af@oracle.com> <3e13e71e99bb5f4a6d7b5aeb0d80d3f7@xs4all.nl> <3e6569c13ba5e1a6dea145146a3099ea@xs4all.nl> <80ad827c-faed-8364-781c-1fa9e572d1b3@oracle.com> <145bff1522eb74198d4f4545e78c1693@xs4all.nl> <42d20409-9b16-d7eb-93fc-b7dfef832204@oracle.com> <612faf056431f3ef511bc9ed3db707c6@xs4all.nl> Message-ID: <170819f5-9f6b-05a2-12bd-e1abbd231610@oracle.com> On 21/01/2019 15:54, Maurizio Cimadamore wrote: > Also, addr() is also where the liveness check takes place, so that's > already how the code works. Also, there doesn't seem to be much point in restricting the passing of a dead/out of bound pointer to a native function when the user can simply workaround by calling addr() on that pointer and pass the resulting long. I'm not saying that we should make the API bullet proof (with native code I think that would be hard to achieve), but we should at least try to make it hard for people to use the API in ways it is not intended to. Maurizio From jbvernee at xs4all.nl Mon Jan 21 16:39:12 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Mon, 21 Jan 2019 17:39:12 +0100 Subject: [foreign] RFR 8217414: Remove bounds check in BoundedPointer constructor In-Reply-To: <170819f5-9f6b-05a2-12bd-e1abbd231610@oracle.com> References: <027a2038-77f8-5213-bf44-ae678c8a07af@oracle.com> <3e13e71e99bb5f4a6d7b5aeb0d80d3f7@xs4all.nl> <3e6569c13ba5e1a6dea145146a3099ea@xs4all.nl> <80ad827c-faed-8364-781c-1fa9e572d1b3@oracle.com> <145bff1522eb74198d4f4545e78c1693@xs4all.nl> <42d20409-9b16-d7eb-93fc-b7dfef832204@oracle.com> <612faf056431f3ef511bc9ed3db707c6@xs4all.nl> <170819f5-9f6b-05a2-12bd-e1abbd231610@oracle.com> Message-ID: All good points, thanks. I moved the checkAlive() and checkInBounds() calls to addr() and have equals/hashCode/isNull use effectiveAddress() which doesn't do the checks. So, I'm now allowing dead pointers to be used in those 3 methods as well, since it seemed cleaner to either do all checks in addr() or none of the checks in effectiveAddress(), instead of doing just the bounds check in addr() and the liveliness check in effectiveAddress(). Is that okay? Updated webrev: http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217414/webrev.03/ Jorn Maurizio Cimadamore schreef op 2019-01-21 17:12: > On 21/01/2019 15:54, Maurizio Cimadamore wrote: >> Also, addr() is also where the liveness check takes place, so that's >> already how the code works. > > Also, there doesn't seem to be much point in restricting the passing > of a dead/out of bound pointer to a native function when the user can > simply workaround by calling addr() on that pointer and pass the > resulting long. > > I'm not saying that we should make the API bullet proof (with native > code I think that would be hard to achieve), but we should at least > try to make it hard for people to use the API in ways it is not > intended to. > > Maurizio From maurizio.cimadamore at oracle.com Mon Jan 21 16:52:36 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 21 Jan 2019 16:52:36 +0000 Subject: [foreign] RFR 8217453: add support for member annotation scheme Message-ID: Hi, this patch implements the member-based annotation scheme described here: http://cr.openjdk.java.net/~mcimadamore/panama/panama-annos.html (see end of document) This patch diverges from the proposal in two minor aspects: * @NativeCallback is left in place (we have bigger plans for allowing callback types to be referenced by name) * unresolved layout grammar: the layout expression used braces - e.g. ${foo}, not $(foo), to minimize confusion with layout annotations I've also took care of a couple of minor issues: * I added exclusion for the SystemHeadersTest which fails on Windows if Visual Studio is not installed * I fixed a bunch of javadoc issues (esp. in Pointer) which prevented javadoc to be generated correctly Webrev: http://cr.openjdk.java.net/~mcimadamore/panama/8217453/ Thanks Maurizio From maurizio.cimadamore at oracle.com Mon Jan 21 17:10:04 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 21 Jan 2019 17:10:04 +0000 Subject: [foreign] RFR 8217414: Remove bounds check in BoundedPointer constructor In-Reply-To: References: <027a2038-77f8-5213-bf44-ae678c8a07af@oracle.com> <3e13e71e99bb5f4a6d7b5aeb0d80d3f7@xs4all.nl> <3e6569c13ba5e1a6dea145146a3099ea@xs4all.nl> <80ad827c-faed-8364-781c-1fa9e572d1b3@oracle.com> <145bff1522eb74198d4f4545e78c1693@xs4all.nl> <42d20409-9b16-d7eb-93fc-b7dfef832204@oracle.com> <612faf056431f3ef511bc9ed3db707c6@xs4all.nl> <170819f5-9f6b-05a2-12bd-e1abbd231610@oracle.com> Message-ID: On 21/01/2019 16:39, Jorn Vernee wrote: > All good points, thanks. > > I moved the checkAlive() and checkInBounds() calls to addr() and have > equals/hashCode/isNull use effectiveAddress() which doesn't do the > checks. So, I'm now allowing dead pointers to be used in those 3 > methods as well, since it seemed cleaner to either do all checks in > addr() or none of the checks in effectiveAddress(), instead of doing > just the bounds check in addr() and the liveliness check in > effectiveAddress(). Is that okay? That looks like a good approach > > Updated webrev: > http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217414/webrev.03/ Looks good - let's also update the javadoc for Pointer::addr? (e.g. to document exactly when addr() will throw) Maurizio > > Jorn > > Maurizio Cimadamore schreef op 2019-01-21 17:12: >> On 21/01/2019 15:54, Maurizio Cimadamore wrote: >>> Also, addr() is also where the liveness check takes place, so that's >>> already how the code works. >> >> Also, there doesn't seem to be much point in restricting the passing >> of a dead/out of bound pointer to a native function when the user can >> simply workaround by calling addr() on that pointer and pass the >> resulting long. >> >> I'm not saying that we should make the API bullet proof (with native >> code I think that would be hard to achieve), but we should at least >> try to make it hard for people to use the API in ways it is not >> intended to. >> >> Maurizio From maurizio.cimadamore at oracle.com Mon Jan 21 17:30:21 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 21 Jan 2019 17:30:21 +0000 Subject: [foreign] RFR 8217453: add support for member annotation scheme In-Reply-To: References: Message-ID: <19a3a7d0-da9f-89d8-9750-d3882f17a395@oracle.com> Forgot to mention: the patch passes tests on all platforms. Maurizio On 21/01/2019 16:52, Maurizio Cimadamore wrote: > Hi, > this patch implements the member-based annotation scheme described here: > > http://cr.openjdk.java.net/~mcimadamore/panama/panama-annos.html > > (see end of document) > > This patch diverges from the proposal in two minor aspects: > > * @NativeCallback is left in place (we have bigger plans for allowing > callback types to be referenced by name) > * unresolved layout grammar: the layout expression used braces - e.g. > ${foo}, not $(foo), to minimize confusion with layout annotations > > I've also took care of a couple of minor issues: > > * I added exclusion for the SystemHeadersTest which fails on Windows > if Visual Studio is not installed > * I fixed a bunch of javadoc issues (esp. in Pointer) which prevented > javadoc to be generated correctly > > Webrev: > > http://cr.openjdk.java.net/~mcimadamore/panama/8217453/ > > Thanks > Maurizio > From jbvernee at xs4all.nl Mon Jan 21 18:00:51 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Mon, 21 Jan 2019 19:00:51 +0100 Subject: [foreign] RFR 8217414: Remove bounds check in BoundedPointer constructor In-Reply-To: References: <027a2038-77f8-5213-bf44-ae678c8a07af@oracle.com> <3e13e71e99bb5f4a6d7b5aeb0d80d3f7@xs4all.nl> <3e6569c13ba5e1a6dea145146a3099ea@xs4all.nl> <80ad827c-faed-8364-781c-1fa9e572d1b3@oracle.com> <145bff1522eb74198d4f4545e78c1693@xs4all.nl> <42d20409-9b16-d7eb-93fc-b7dfef832204@oracle.com> <612faf056431f3ef511bc9ed3db707c6@xs4all.nl> <170819f5-9f6b-05a2-12bd-e1abbd231610@oracle.com> Message-ID: <3a4a1957327825ee4d98311e1a6f8019@xs4all.nl> Updated an pushed. Thanks for the reviews! Jorn Maurizio Cimadamore schreef op 2019-01-21 18:10: > On 21/01/2019 16:39, Jorn Vernee wrote: >> All good points, thanks. >> >> I moved the checkAlive() and checkInBounds() calls to addr() and have >> equals/hashCode/isNull use effectiveAddress() which doesn't do the >> checks. So, I'm now allowing dead pointers to be used in those 3 >> methods as well, since it seemed cleaner to either do all checks in >> addr() or none of the checks in effectiveAddress(), instead of doing >> just the bounds check in addr() and the liveliness check in >> effectiveAddress(). Is that okay? > That looks like a good approach >> >> Updated webrev: >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217414/webrev.03/ > > Looks good - let's also update the javadoc for Pointer::addr? (e.g. to > document exactly when addr() will throw) > > Maurizio > > >> >> Jorn >> >> Maurizio Cimadamore schreef op 2019-01-21 17:12: >>> On 21/01/2019 15:54, Maurizio Cimadamore wrote: >>>> Also, addr() is also where the liveness check takes place, so that's >>>> already how the code works. >>> >>> Also, there doesn't seem to be much point in restricting the passing >>> of a dead/out of bound pointer to a native function when the user can >>> simply workaround by calling addr() on that pointer and pass the >>> resulting long. >>> >>> I'm not saying that we should make the API bullet proof (with native >>> code I think that would be hard to achieve), but we should at least >>> try to make it hard for people to use the API in ways it is not >>> intended to. >>> >>> Maurizio From jbvernee at xs4all.nl Mon Jan 21 17:59:15 2019 From: jbvernee at xs4all.nl (jbvernee at xs4all.nl) Date: Mon, 21 Jan 2019 17:59:15 +0000 Subject: hg: panama/dev: 8217414: api/PointerTest and api/ArrayTest are vacuously passing Message-ID: <201901211759.x0LHxFCA017998@aojmv0008.oracle.com> Changeset: e75f2c93b4d1 Author: jvernee Date: 2019-01-21 18:58 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/e75f2c93b4d1 8217414: api/PointerTest and api/ArrayTest are vacuously passing Summary: Remove bounds check in BoundedPointer constructor Reviewed-by: mcimadamore ! src/java.base/share/classes/java/foreign/memory/Pointer.java ! src/java.base/share/classes/jdk/internal/foreign/memory/BoundedPointer.java + test/jdk/java/foreign/OutOfBoundsTest.java ! test/jdk/java/foreign/api/ArrayTest.java ! test/jdk/java/foreign/api/PointerTest.java + test/jdk/java/foreign/libOutOfBounds.c ! test/jdk/java/foreign/types/PointerTest.java From maurizio.cimadamore at oracle.com Mon Jan 21 18:05:18 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Mon, 21 Jan 2019 18:05:18 +0000 Subject: hg: panama/dev: Automatic merge with foreign Message-ID: <201901211805.x0LI5IDo020358@aojmv0008.oracle.com> Changeset: 2a2a6c0bc17f Author: mcimadamore Date: 2019-01-21 19:05 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/2a2a6c0bc17f Automatic merge with foreign From jbvernee at xs4all.nl Mon Jan 21 18:13:52 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Mon, 21 Jan 2019 19:13:52 +0100 Subject: [foreign] RFR 8217420: Have UniversalNativeInvoker and UniversalUpcallHandler create (and close) the scopes used by boxing code Message-ID: <60c37cbd463ff8a5ab7ab9899565ab47@xs4all.nl> Hi, another patch :) From the bug description: Currently, when (un-)boxing code needs to do an allocation a Scope is created for that, and this Scope is then leaked. For an up-/down-call there are 2 kinds of scopes; 1) for values being passed as arguments by value, which are valid for the duration of the call. 2) for values being returned, which are up to the caller to manage. We can do better than the current scheme of always leaking, by having UniversalUpcallHandler and UniversalNativeInvoker create both of these types of scopes, pass them to boxing code, and then close scopes of type 1. when the call is finished. This will reduce the amount of memory being leaked, and could also help identify bugs where references to temporary objects are leaked. --- After implementing this, it turns out that UniversalUpcallHandler only needs a Scope for the arguments being passed to the java method being called. For returns there should be no allocations; either the value is being passed in registers, or written to a pointer passed in by the caller. I did discover a problem with this patch in the TestJextractFFI patch code: public Stream children() { final ArrayList ar = new ArrayList<>(); // FIXME: need a way to pass ar down as user data d try (Scope sc = Scope.newNativeScope()) { LibClang.lib.clang_visitChildren(cursor, sc.allocateCallback((c, p, d) -> { ar.add(new Cursor(c)); return LibClang.lib.CXChildVisit_Continue(); }), Pointer.nullPointer()); return ar.stream(); } } According to C semantics, the argument to the lambda `c`, which is a CXCursor struct being passed in by-value, is only valid for the duration of the call. The Windows ABI depends on this fact, because it allocates the struct on the stack of the caller, and then passes a pointer to the callee (the lambda in this case). I think on SysV a by-value struct is always passed in register + stack? But, the `c` argument is being leaked from the lambda, by wrapping it in a Cursor and storing it in a list in the enclosing scope. When the call to the lambda ends, the struct is deallocated and the pointer that is stored on the Java side is now dangling. I've initially solved the crash this was causing by defensively copying the struct in the Windows boxing code (instead of just wrapping the passed pointer), but I realized this also means that a user would have to manually release a struct passed into an upcall to avoid a memory leak. We have 2 options here: 1.) Go with C style. Struct arguments are valid only during an upcall, and we have to manually copy them if we want to 'leak' them from the scope of the upcall. 2.) Go with status quo. Struct doesn't need to be copied manually when saving it outside the scope of the upcall. The copy is defensively done in Windows boxing code instead, and sometimes redundant. Only on Windows (for now), struct arguments to an upcall have to be manually released to avoid a memory leak. I think 1 is the better option here since this adheres to the expectations native code has when calling into Java. I've reflected that in the patch as well, but of course I'm interested in hearing other opinions as well :) Note that the crash I was seeing before no longer happens since the Scope used in the boxing code is now closed after the call, and we get an exception instead. Bug: https://bugs.openjdk.java.net/browse/JDK-8217420 Webrev: http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217420/webrev.00/ Cheers, Jorn From jbvernee at xs4all.nl Mon Jan 21 18:31:11 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Mon, 21 Jan 2019 19:31:11 +0100 Subject: [foreign] RFC: Jextract -l options ignored by SymbolFilter when -L is not specified Message-ID: <698464e8f7692e12a7062155c7e599d9@xs4all.nl> Hi, I've recently updated the instructions for using libraries on Windows. For python the jextract example I gave was: jextract -l python27 -o "python.jar" -t "org.python" C:\Python27\include\Python.h I'm lacking an `-L` option here (for specifying library directories) since the contents of PATH seems to be added to java.library.path by default, and this is presumably also how jextract is able to load the library. But, since I'm not using an `-L` option, SymbolFilter is not checking if the symbols are in the python27.dll [1] private void initSymChecker(List linkCheckPaths) { if (!libraryNames.isEmpty() && !linkCheckPaths.isEmpty()) { // ... init symChecker } else { symChecker = null; } } (linkCheckPaths comes from the -L option values) This behaviour is somewhat unexpected. At least a warning that missing an `-L` option will turn off symbol checking would be nice. We could also add the paths in `java.library.path` to the list of link check paths in jextract [2]. That would mean that the symbol checker would run for the example command. What do you think? Jorn [1] : http://hg.openjdk.java.net/panama/dev/file/eaca2d16b80b/src/jdk.jextract/share/classes/com/sun/tools/jextract/SymbolFilter.java#l89 [2] : http://cr.openjdk.java.net/~jvernee/panama/webrevs/jlibpath/webrev.01/ From jbvernee at xs4all.nl Mon Jan 21 19:12:53 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Mon, 21 Jan 2019 20:12:53 +0100 Subject: hg: panama/dev: 8217414: api/PointerTest and api/ArrayTest are vacuously passing In-Reply-To: <201901211759.x0LHxFCA017998@aojmv0008.oracle.com> References: <201901211759.x0LHxFCA017998@aojmv0008.oracle.com> Message-ID: Looks like the new OutOfBoundsTest in this Changeset is failing. I'm not sure what's going on, as I ran the tests multiple times with passing results (and I can verify that as well by looking at my console logs). Maybe it's some tooling problem, or maybe I did something silly, idk. Any ways, I think I've found the problem, will send another update soon. Sorry about that, Jorn jbvernee at xs4all.nl schreef op 2019-01-21 18:59: > Changeset: e75f2c93b4d1 > Author: jvernee > Date: 2019-01-21 18:58 +0100 > URL: http://hg.openjdk.java.net/panama/dev/rev/e75f2c93b4d1 > > 8217414: api/PointerTest and api/ArrayTest are vacuously passing > Summary: Remove bounds check in BoundedPointer constructor > Reviewed-by: mcimadamore > > ! src/java.base/share/classes/java/foreign/memory/Pointer.java > ! > src/java.base/share/classes/jdk/internal/foreign/memory/BoundedPointer.java > + test/jdk/java/foreign/OutOfBoundsTest.java > ! test/jdk/java/foreign/api/ArrayTest.java > ! test/jdk/java/foreign/api/PointerTest.java > + test/jdk/java/foreign/libOutOfBounds.c > ! test/jdk/java/foreign/types/PointerTest.java From vladimir.x.ivanov at oracle.com Mon Jan 21 19:27:10 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Mon, 21 Jan 2019 11:27:10 -0800 Subject: Allocation caused by vector rebracketing twice In-Reply-To: References: Message-ID: I think what you are seeing is a consequence of missing support for LongVector.addAll() on pre-AVX512 CPUs [1]: in the latter case (1 rebracket) ByteVector.addAll() is called. You can verify that by looking at -XX:+PrintInlining output: 3106 166 Benchmark::vectorBitCount (54 bytes) ... @ 40 jdk.incubator.vector.Long256Vector::addAll (19 bytes) force inline by annotation @ 10 java.lang.invoke.Invokers$Holder::linkToTargetMethod (8 bytes) force inline by annotation @ 4 java.lang.invoke.LambdaForm$MH/0x00000008011cb440::invoke (8 bytes) force inline by annotation @ 15 jdk.incubator.vector.VectorIntrinsics::reductionCoerced (16 bytes) failed to inline (intrinsic) Best regards, Vladimir Ivanov [1] http://hg.openjdk.java.net/panama/dev/file/a8516a4be714/src/hotspot/cpu/x86/x86.ad#l1421 On 19/01/2019 11:28, Richard Startin wrote: > It's great to see that there is now a shiftR method because this was the missing link to make vector bit counts possible. With great excitement, I tried this at 54348:a8516a4be714 but it doesn't work very well yet. > > > @Benchmark > public int vectorBitCount() { > int bitCount = 0; > var lookupPos = YMM_BYTE.fromArray(LOOKUP_POS, 0); > var lookupNeg = YMM_BYTE.fromArray(LOOKUP_NEG, 0); > var lowMask = YMM_BYTE.broadcast((byte)0x0F); > for (int i = 0; i < data.length; i+= 4) { > var bytes = (ByteVector)YMM_LONG.fromArray(data, i).rebracket(YMM_BYTE); > bitCount += (int)((LongVector)lookupPos.rearrange(bytes.and(lowMask).toShuffle()) > .add(lookupNeg.rearrange(bytes.shiftR(4).and(lowMask).toShuffle())) > .rebracket(YMM_LONG)).addAll(); > } > return bitCount; > } > > > JMH -prof gc shows high allocation rates: > > > Iteration 1: 0.008 ops/us > ?gc.alloc.rate: 1112.997 MB/sec > ?gc.alloc.rate.norm: 219264.051 B/op > ?gc.churn.G1_Eden_Space: 1042.296 MB/sec > ?gc.churn.G1_Eden_Space.norm: 205335.753 B/op > ?gc.churn.G1_Old_Gen: 0.001 MB/sec > ?gc.churn.G1_Old_Gen.norm: 0.258 B/op > ?gc.count: 6.000 counts > ?gc.time: 6.000 ms > > Stripping the code down until negligible allocation rates are observed, the smallest reproducer is where the vector is rebracketed and then the reverse rebracket is performed: > > @Benchmark > public int vectorBitCount() { > int bitCount = 0; > for (int i = 0; i < data.length; i+= 4) { > bitCount += (int)((LongVector)YMM_LONG.fromArray(data, i).rebracket(YMM_BYTE).rebracket(YMM_LONG)).addAll(); > } > return bitCount; > } > > -prof gc: > > Iteration 1: 0.310 ops/us > ?gc.alloc.rate: 3223.539 MB/sec > ?gc.alloc.rate.norm: 16384.001 B/op > ?gc.churn.G1_Eden_Space: 3084.887 MB/sec > ?gc.churn.G1_Eden_Space.norm: 15679.287 B/op > ?gc.churn.G1_Old_Gen: 0.005 MB/sec > ?gc.churn.G1_Old_Gen.norm: 0.026 B/op > ?gc.count: 8.000 counts > ?gc.time: 9.000 ms > > Without reversing the rebracket I see negligible allocation > > @Benchmark > public int vectorBitCount() { > int bitCount = 0; > for (int i = 0; i < data.length; i+= 4) { > bitCount += (int)((ByteVector)YMM_LONG.fromArray(data, i).rebracket(YMM_BYTE)).addAll(); > } > return bitCount; > } > > Iteration 1: 1.456 ops/us > ?gc.alloc.rate: ? 10?? MB/sec > ?gc.alloc.rate.norm: ? 10?? B/op > ?gc.count: ? 0 counts > > Thanks, > Richard > > > From sandhya.viswanathan at intel.com Mon Jan 21 19:29:16 2019 From: sandhya.viswanathan at intel.com (Viswanathan, Sandhya) Date: Mon, 21 Jan 2019 19:29:16 +0000 Subject: Allocation caused by vector rebracketing twice In-Reply-To: References: Message-ID: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A51F11@FMSMSX126.amr.corp.intel.com> Hi Richard, Thanks a lot for your feedback. I analyzed this further and it looks like the high allocation rate is happening because the Long addAll reduction intrinsic is not in place for AVX < 3 and so boxing is happening to call the Java implementation. I will send out a patch shortly which should fix this. Best Regards, Sandhya -----Original Message----- From: panama-dev [mailto:panama-dev-bounces at openjdk.java.net] On Behalf Of Richard Startin Sent: Saturday, January 19, 2019 11:29 AM To: panama-dev at openjdk.java.net Subject: Allocation caused by vector rebracketing twice It's great to see that there is now a shiftR method because this was the missing link to make vector bit counts possible. With great excitement, I tried this at 54348:a8516a4be714 but it doesn't work very well yet. @Benchmark public int vectorBitCount() { int bitCount = 0; var lookupPos = YMM_BYTE.fromArray(LOOKUP_POS, 0); var lookupNeg = YMM_BYTE.fromArray(LOOKUP_NEG, 0); var lowMask = YMM_BYTE.broadcast((byte)0x0F); for (int i = 0; i < data.length; i+= 4) { var bytes = (ByteVector)YMM_LONG.fromArray(data, i).rebracket(YMM_BYTE); bitCount += (int)((LongVector)lookupPos.rearrange(bytes.and(lowMask).toShuffle()) .add(lookupNeg.rearrange(bytes.shiftR(4).and(lowMask).toShuffle())) .rebracket(YMM_LONG)).addAll(); } return bitCount; } JMH -prof gc shows high allocation rates: Iteration 1: 0.008 ops/us ?gc.alloc.rate: 1112.997 MB/sec ?gc.alloc.rate.norm: 219264.051 B/op ?gc.churn.G1_Eden_Space: 1042.296 MB/sec ?gc.churn.G1_Eden_Space.norm: 205335.753 B/op ?gc.churn.G1_Old_Gen: 0.001 MB/sec ?gc.churn.G1_Old_Gen.norm: 0.258 B/op ?gc.count: 6.000 counts ?gc.time: 6.000 ms Stripping the code down until negligible allocation rates are observed, the smallest reproducer is where the vector is rebracketed and then the reverse rebracket is performed: @Benchmark public int vectorBitCount() { int bitCount = 0; for (int i = 0; i < data.length; i+= 4) { bitCount += (int)((LongVector)YMM_LONG.fromArray(data, i).rebracket(YMM_BYTE).rebracket(YMM_LONG)).addAll(); } return bitCount; } -prof gc: Iteration 1: 0.310 ops/us ?gc.alloc.rate: 3223.539 MB/sec ?gc.alloc.rate.norm: 16384.001 B/op ?gc.churn.G1_Eden_Space: 3084.887 MB/sec ?gc.churn.G1_Eden_Space.norm: 15679.287 B/op ?gc.churn.G1_Old_Gen: 0.005 MB/sec ?gc.churn.G1_Old_Gen.norm: 0.026 B/op ?gc.count: 8.000 counts ?gc.time: 9.000 ms Without reversing the rebracket I see negligible allocation @Benchmark public int vectorBitCount() { int bitCount = 0; for (int i = 0; i < data.length; i+= 4) { bitCount += (int)((ByteVector)YMM_LONG.fromArray(data, i).rebracket(YMM_BYTE)).addAll(); } return bitCount; } Iteration 1: 1.456 ops/us ?gc.alloc.rate: ? 10?? MB/sec ?gc.alloc.rate.norm: ? 10?? B/op ?gc.count: ? 0 counts Thanks, Richard From sandhya.viswanathan at intel.com Mon Jan 21 19:32:50 2019 From: sandhya.viswanathan at intel.com (Viswanathan, Sandhya) Date: Mon, 21 Jan 2019 19:32:50 +0000 Subject: [vector] Support long addAll() for AVX < 3 Message-ID: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A51F25@FMSMSX126.amr.corp.intel.com> Hi All, Please find below a small patch supporting long addAll for AVX < 3: http://cr.openjdk.java.net/~sviswanathan/vectorIntrinsics/longAddReduction/webrev.00/ Best Regards, Sandhya From vladimir.x.ivanov at oracle.com Mon Jan 21 19:43:43 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Mon, 21 Jan 2019 11:43:43 -0800 Subject: [vector] Support long addAll() for AVX < 3 In-Reply-To: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A51F25@FMSMSX126.amr.corp.intel.com> References: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A51F25@FMSMSX126.amr.corp.intel.com> Message-ID: src/hotspot/cpu/x86/x86.ad: #ifdef _LP64 instruct rvadd2L_reduction_reg(rRegL dst, rRegL src1, vecX src2, vecX tmp, vecX tmp2) %{ - predicate(UseAVX > 2); + predicate(UseSSE >= 2); There's no need in the check, since SSE2 support is a requirement on x64. BTW it would be nice to move 64-bit specific declarations into x86_64.ad. instruct rvadd4L_reduction_reg(rRegL dst, rRegL src1, vecY src2, vecY tmp, vecY tmp2) %{ - predicate(UseAVX > 2); + predicate(UseAVX >= 1); Are you sure about AVX1 support? As I see in the docs, 256-bit pshufd variant is available since AVX2. (On a side note, I'm in favor of exclusive range checks: UseAVX > 1 vs UseAVX >= 2. The former reads better in x86.ad.) Best regards, Vladimir Ivanov On 21/01/2019 11:32, Viswanathan, Sandhya wrote: > Hi All, > > Please find below a small patch supporting long addAll for AVX < 3: > > http://cr.openjdk.java.net/~sviswanathan/vectorIntrinsics/longAddReduction/webrev.00/ > > Best Regards, > Sandhya > From jbvernee at xs4all.nl Mon Jan 21 19:51:04 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Mon, 21 Jan 2019 20:51:04 +0100 Subject: [foreign] RFR 8217463: Fix for failing test from 54355 (e75f2c93b4d1) Message-ID: Hi, Please review the following patch which is a small fix for the failing test mentioned earlier [1] Bug: https://bugs.openjdk.java.net/browse/JDK-8217463 Webrev: http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217463/webrev.00/ Thanks, Jorn [1] : https://mail.openjdk.java.net/pipermail/panama-dev/2019-January/003886.html From sandhya.viswanathan at intel.com Mon Jan 21 20:56:33 2019 From: sandhya.viswanathan at intel.com (Viswanathan, Sandhya) Date: Mon, 21 Jan 2019 20:56:33 +0000 Subject: [vector] Support long addAll() for AVX < 3 In-Reply-To: References: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A51F25@FMSMSX126.amr.corp.intel.com> Message-ID: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A51F53@FMSMSX126.amr.corp.intel.com> Hi Vladimir, Thanks a lot for a quick review. >> + predicate(UseSSE >= 2); >> There's no need in the check, since SSE2 support is a requirement on x64. Will do. >> Are you sure about AVX1 support? As I see in the docs, 256-bit pshufd variant is available since AVX2. You are right, I will make this change. >> BTW it would be nice to move 64-bit specific declarations into x86_64.ad. The base rules for AddReductionVL are coming from mainline and it has been under LP64 check. Also there are many instances of LP64 check in x86.ad file as traditionally x86.ad held all the vectorization related rules. May be that needs to be a separate refactor at some point starting from mainline JDK. Please let me know what you think and I will send an updated webrev accordingly. Best Regards, Sandhya -----Original Message----- From: Vladimir Ivanov [mailto:vladimir.x.ivanov at oracle.com] Sent: Monday, January 21, 2019 11:44 AM To: Viswanathan, Sandhya ; panama-dev at openjdk.java.net Subject: Re: [vector] Support long addAll() for AVX < 3 src/hotspot/cpu/x86/x86.ad: #ifdef _LP64 instruct rvadd2L_reduction_reg(rRegL dst, rRegL src1, vecX src2, vecX tmp, vecX tmp2) %{ - predicate(UseAVX > 2); + predicate(UseSSE >= 2); There's no need in the check, since SSE2 support is a requirement on x64. BTW it would be nice to move 64-bit specific declarations into x86_64.ad. instruct rvadd4L_reduction_reg(rRegL dst, rRegL src1, vecY src2, vecY tmp, vecY tmp2) %{ - predicate(UseAVX > 2); + predicate(UseAVX >= 1); Are you sure about AVX1 support? As I see in the docs, 256-bit pshufd variant is available since AVX2. (On a side note, I'm in favor of exclusive range checks: UseAVX > 1 vs UseAVX >= 2. The former reads better in x86.ad.) Best regards, Vladimir Ivanov On 21/01/2019 11:32, Viswanathan, Sandhya wrote: > Hi All, > > Please find below a small patch supporting long addAll for AVX < 3: > > http://cr.openjdk.java.net/~sviswanathan/vectorIntrinsics/longAddReduction/webrev.00/ > > Best Regards, > Sandhya > From maurizio.cimadamore at oracle.com Mon Jan 21 21:02:55 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 21 Jan 2019 21:02:55 +0000 Subject: [foreign] RFR 8217463: Fix for failing test from 54355 (e75f2c93b4d1) In-Reply-To: References: Message-ID: <5e0ba700-c7ad-42fa-f151-91457026764f@oracle.com> Looks good, not sure why the failure hasn't been picked up when I ran the tests on the first version of your patch... Maurizio On 21/01/2019 19:51, Jorn Vernee wrote: > Hi, > > Please review the following patch which is a small fix for the failing > test mentioned earlier [1] > > Bug: https://bugs.openjdk.java.net/browse/JDK-8217463 > Webrev: > http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217463/webrev.00/ > > Thanks, > Jorn > > [1] : > https://mail.openjdk.java.net/pipermail/panama-dev/2019-January/003886.html From jbvernee at xs4all.nl Mon Jan 21 21:11:35 2019 From: jbvernee at xs4all.nl (jbvernee at xs4all.nl) Date: Mon, 21 Jan 2019 21:11:35 +0000 Subject: hg: panama/dev: 8217463: Recently added OutOfBoundsTest fails Message-ID: <201901212111.x0LLBZ2I008280@aojmv0008.oracle.com> Changeset: acd918a2bbb0 Author: jvernee Date: 2019-01-21 22:10 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/acd918a2bbb0 8217463: Recently added OutOfBoundsTest fails Summary: Fix for failing test from 54355 (e75f2c93b4d1) Reviewed-by: mcimadamore ! test/jdk/java/foreign/OutOfBoundsTest.java From jbvernee at xs4all.nl Mon Jan 21 21:12:02 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Mon, 21 Jan 2019 22:12:02 +0100 Subject: [foreign] RFR 8217463: Fix for failing test from 54355 (e75f2c93b4d1) In-Reply-To: <5e0ba700-c7ad-42fa-f151-91457026764f@oracle.com> References: <5e0ba700-c7ad-42fa-f151-91457026764f@oracle.com> Message-ID: On your side as well? Well, that's a mystery... Thanks for the review! Pushed. Jorn Maurizio Cimadamore schreef op 2019-01-21 22:02: > Looks good, not sure why the failure hasn't been picked up when I ran > the tests on the first version of your patch... > > Maurizio > > On 21/01/2019 19:51, Jorn Vernee wrote: >> Hi, >> >> Please review the following patch which is a small fix for the failing >> test mentioned earlier [1] >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8217463 >> Webrev: >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217463/webrev.00/ >> >> Thanks, >> Jorn >> >> [1] : >> https://mail.openjdk.java.net/pipermail/panama-dev/2019-January/003886.html From maurizio.cimadamore at oracle.com Mon Jan 21 21:15:15 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Mon, 21 Jan 2019 21:15:15 +0000 Subject: hg: panama/dev: Automatic merge with foreign Message-ID: <201901212115.x0LLFGFp009721@aojmv0008.oracle.com> Changeset: 49851164e508 Author: mcimadamore Date: 2019-01-21 22:15 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/49851164e508 Automatic merge with foreign From maurizio.cimadamore at oracle.com Mon Jan 21 21:16:10 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 21 Jan 2019 21:16:10 +0000 Subject: [foreign] RFR 8217420: Have UniversalNativeInvoker and UniversalUpcallHandler create (and close) the scopes used by boxing code In-Reply-To: <60c37cbd463ff8a5ab7ab9899565ab47@xs4all.nl> References: <60c37cbd463ff8a5ab7ab9899565ab47@xs4all.nl> Message-ID: <6d221129-4997-e993-a126-7168e7adf885@oracle.com> Hi Jorn, I think I agree with the spirit of the fix. I like that this makes the scopes stricter - which ended up uncovering some issues. Again, I have to point out that this fix only restrict things for universal invoker mode - I believe that when using the direct invocation scheme, parameters to upcalls will be passed in leaky scopes, which will not reveal the clang-related failure. I think I'd like to see a specific test for lifecycle of callback args, so that we can easily run it with all possible invoker modes and make sure it passes. Maurizio On 21/01/2019 18:13, Jorn Vernee wrote: > Hi, another patch :) > > From the bug description: > > Currently, when (un-)boxing code needs to do an allocation a Scope is > created for that, and this Scope is then leaked. > > For an up-/down-call there are 2 kinds of scopes; 1) for values being > passed as arguments by value, which are valid for the duration of the > call. 2) for values being returned, which are up to the caller to manage. > > We can do better than the current scheme of always leaking, by having > UniversalUpcallHandler and UniversalNativeInvoker create both of these > types of scopes, pass them to boxing code, and then close scopes of > type 1. when the call is finished. > > This will reduce the amount of memory being leaked, and could also > help identify bugs where references to temporary objects are leaked. > > --- > > After implementing this, it turns out that UniversalUpcallHandler only > needs a Scope for the arguments being passed to the java method being > called. For returns there should be no allocations; either the value > is being passed in registers, or written to a pointer passed in by the > caller. > > I did discover a problem with this patch in the TestJextractFFI patch > code: > > ??? public Stream children() { > ??????? final ArrayList ar = new ArrayList<>(); > ??????? // FIXME: need a way to pass ar down as user data d > ??????? try (Scope sc = Scope.newNativeScope()) { > ??????????? LibClang.lib.clang_visitChildren(cursor, > sc.allocateCallback((c, p, d) -> { > ??????????????? ar.add(new Cursor(c)); > ??????????????? return LibClang.lib.CXChildVisit_Continue(); > ??????????? }), Pointer.nullPointer()); > ??????????? return ar.stream(); > ??????? } > ??? } > > According to C semantics, the argument to the lambda `c`, which is a > CXCursor struct being passed in by-value, is only valid for the > duration of the call. The Windows ABI depends on this fact, because it > allocates the struct on the stack of the caller, and then passes a > pointer to the callee (the lambda in this case). I think on SysV a > by-value struct is always passed in register + stack? > > But, the `c` argument is being leaked from the lambda, by wrapping it > in a Cursor and storing it in a list in the enclosing scope. When the > call to the lambda ends, the struct is deallocated and the pointer > that is stored on the Java side is now dangling. I've initially solved > the crash this was causing by defensively copying the struct in the > Windows boxing code (instead of just wrapping the passed pointer), but > I realized this also means that a user would have to manually release > a struct passed into an upcall to avoid a memory leak. > > We have 2 options here: > > 1.) Go with C style. Struct arguments are valid only during an upcall, > and we have to manually copy them if we want to 'leak' them from the > scope of the upcall. > 2.) Go with status quo. Struct doesn't need to be copied manually when > saving it outside the scope of the upcall. The copy is defensively > done in Windows boxing code instead, and sometimes redundant. Only on > Windows (for now), struct arguments to an upcall have to be manually > released to avoid a memory leak. > > I think 1 is the better option here since this adheres to the > expectations native code has when calling into Java. I've reflected > that in the patch as well, but of course I'm interested in hearing > other opinions as well :) Note that the crash I was seeing before no > longer happens since the Scope used in the boxing code is now closed > after the call, and we get an exception instead. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8217420 > Webrev: > http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217420/webrev.00/ > > Cheers, > Jorn From maurizio.cimadamore at oracle.com Mon Jan 21 21:18:16 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 21 Jan 2019 21:18:16 +0000 Subject: [foreign] RFR 8217463: Fix for failing test from 54355 (e75f2c93b4d1) In-Reply-To: References: <5e0ba700-c7ad-42fa-f151-91457026764f@oracle.com> Message-ID: Sorry for the confusion - I tested 8217380 not 8217414 (which caused the issue). Maurizio On 21/01/2019 21:12, Jorn Vernee wrote: > On your side as well? Well, that's a mystery... > > Thanks for the review! Pushed. > > Jorn > > Maurizio Cimadamore schreef op 2019-01-21 22:02: >> Looks good, not sure why the failure hasn't been picked up when I ran >> the tests on the first version of your patch... >> >> Maurizio >> >> On 21/01/2019 19:51, Jorn Vernee wrote: >>> Hi, >>> >>> Please review the following patch which is a small fix for the >>> failing test mentioned earlier [1] >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8217463 >>> Webrev: >>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217463/webrev.00/ >>> >>> Thanks, >>> Jorn >>> >>> [1] : >>> https://mail.openjdk.java.net/pipermail/panama-dev/2019-January/003886.html From jbvernee at xs4all.nl Mon Jan 21 21:31:38 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Mon, 21 Jan 2019 22:31:38 +0100 Subject: [foreign] RFR 8217420: Have UniversalNativeInvoker and UniversalUpcallHandler create (and close) the scopes used by boxing code In-Reply-To: <6d221129-4997-e993-a126-7168e7adf885@oracle.com> References: <60c37cbd463ff8a5ab7ab9899565ab47@xs4all.nl> <6d221129-4997-e993-a126-7168e7adf885@oracle.com> Message-ID: <6ba6da2fa7c519b57f871b8f84110b2d@xs4all.nl> Maurizio Cimadamore schreef op 2019-01-21 22:16: > Hi Jorn, > I think I agree with the spirit of the fix. I like that this makes the > scopes stricter - which ended up uncovering some issues. > > Again, I have to point out that this fix only restrict things for > universal invoker mode - I believe that when using the direct > invocation scheme, parameters to upcalls will be passed in leaky > scopes, which will not reveal the clang-related failure. Sorry about that, I wasn't sure before if I was supposed to make changes to the direct invoker as well, since I would have no way of testing those changes locally. (I've put it on my checklist for the future). > I think I'd like to see a specific test for lifecycle of callback > args, so that we can easily run it with all possible invoker modes and > make sure it passes. I added a test for that (CallScopeTest), is there something else you'd want to have tested? AFAICT this lifecycle problem currently only exists with structs. Jorn > Maurizio > > On 21/01/2019 18:13, Jorn Vernee wrote: >> Hi, another patch :) >> >> From the bug description: >> >> Currently, when (un-)boxing code needs to do an allocation a Scope is >> created for that, and this Scope is then leaked. >> >> For an up-/down-call there are 2 kinds of scopes; 1) for values being >> passed as arguments by value, which are valid for the duration of the >> call. 2) for values being returned, which are up to the caller to >> manage. >> >> We can do better than the current scheme of always leaking, by having >> UniversalUpcallHandler and UniversalNativeInvoker create both of these >> types of scopes, pass them to boxing code, and then close scopes of >> type 1. when the call is finished. >> >> This will reduce the amount of memory being leaked, and could also >> help identify bugs where references to temporary objects are leaked. >> >> --- >> >> After implementing this, it turns out that UniversalUpcallHandler only >> needs a Scope for the arguments being passed to the java method being >> called. For returns there should be no allocations; either the value >> is being passed in registers, or written to a pointer passed in by the >> caller. >> >> I did discover a problem with this patch in the TestJextractFFI patch >> code: >> >> ??? public Stream children() { >> ??????? final ArrayList ar = new ArrayList<>(); >> ??????? // FIXME: need a way to pass ar down as user data d >> ??????? try (Scope sc = Scope.newNativeScope()) { >> ??????????? LibClang.lib.clang_visitChildren(cursor, >> sc.allocateCallback((c, p, d) -> { >> ??????????????? ar.add(new Cursor(c)); >> ??????????????? return LibClang.lib.CXChildVisit_Continue(); >> ??????????? }), Pointer.nullPointer()); >> ??????????? return ar.stream(); >> ??????? } >> ??? } >> >> According to C semantics, the argument to the lambda `c`, which is a >> CXCursor struct being passed in by-value, is only valid for the >> duration of the call. The Windows ABI depends on this fact, because it >> allocates the struct on the stack of the caller, and then passes a >> pointer to the callee (the lambda in this case). I think on SysV a >> by-value struct is always passed in register + stack? >> >> But, the `c` argument is being leaked from the lambda, by wrapping it >> in a Cursor and storing it in a list in the enclosing scope. When the >> call to the lambda ends, the struct is deallocated and the pointer >> that is stored on the Java side is now dangling. I've initially solved >> the crash this was causing by defensively copying the struct in the >> Windows boxing code (instead of just wrapping the passed pointer), but >> I realized this also means that a user would have to manually release >> a struct passed into an upcall to avoid a memory leak. >> >> We have 2 options here: >> >> 1.) Go with C style. Struct arguments are valid only during an upcall, >> and we have to manually copy them if we want to 'leak' them from the >> scope of the upcall. >> 2.) Go with status quo. Struct doesn't need to be copied manually when >> saving it outside the scope of the upcall. The copy is defensively >> done in Windows boxing code instead, and sometimes redundant. Only on >> Windows (for now), struct arguments to an upcall have to be manually >> released to avoid a memory leak. >> >> I think 1 is the better option here since this adheres to the >> expectations native code has when calling into Java. I've reflected >> that in the patch as well, but of course I'm interested in hearing >> other opinions as well :) Note that the crash I was seeing before no >> longer happens since the Scope used in the boxing code is now closed >> after the call, and we get an exception instead. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8217420 >> Webrev: >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217420/webrev.00/ >> >> Cheers, >> Jorn From maurizio.cimadamore at oracle.com Mon Jan 21 21:48:12 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 21 Jan 2019 21:48:12 +0000 Subject: [foreign] RFR 8217420: Have UniversalNativeInvoker and UniversalUpcallHandler create (and close) the scopes used by boxing code In-Reply-To: <6ba6da2fa7c519b57f871b8f84110b2d@xs4all.nl> References: <60c37cbd463ff8a5ab7ab9899565ab47@xs4all.nl> <6d221129-4997-e993-a126-7168e7adf885@oracle.com> <6ba6da2fa7c519b57f871b8f84110b2d@xs4all.nl> Message-ID: <5901905f-da21-6aca-c184-291cf27a4afd@oracle.com> On 21/01/2019 21:31, Jorn Vernee wrote: > Maurizio Cimadamore schreef op 2019-01-21 22:16: >> Hi Jorn, >> I think I agree with the spirit of the fix. I like that this makes the >> scopes stricter - which ended up uncovering some issues. >> >> Again, I have to point out that this fix only restrict things for >> universal invoker mode - I believe that when using the direct >> invocation scheme, parameters to upcalls will be passed in leaky >> scopes, which will not reveal the clang-related failure. > > Sorry about that, I wasn't sure before if I was supposed to make > changes to the direct invoker as well, since I would have no way of > testing those changes locally. (I've put it on my checklist for the > future). > >> I think I'd like to see a specific test for lifecycle of callback >> args, so that we can easily run it with all possible invoker modes and >> make sure it passes. > > I added a test for that (CallScopeTest), is there something else you'd > want to have tested? AFAICT this lifecycle problem currently only > exists with structs. That's good - I missed that earlier. I guess we need to make sure that the same semantics is applied on all platforms; I'm a bit uncertain on how that's gonna work for the direct invoker (or the linkToNative invoker, for what it's worth) - as these invokers are based on the assumption that arguments can be converted into their native form on the fly - I'm not sure in those invoker it would be easy to do something such as creating a call-wide scope (in either direction). I can try and take a look. I guess I could use the MethodHandles.tryFinally combinator to make sure the scope is closed after the call (and also inject a Scope prefix parameter on all adapter handles), but I'd also have to check what happens to performances. Maurizio > > Jorn > >> Maurizio >> >> On 21/01/2019 18:13, Jorn Vernee wrote: >>> Hi, another patch :) >>> >>> From the bug description: >>> >>> Currently, when (un-)boxing code needs to do an allocation a Scope >>> is created for that, and this Scope is then leaked. >>> >>> For an up-/down-call there are 2 kinds of scopes; 1) for values >>> being passed as arguments by value, which are valid for the duration >>> of the call. 2) for values being returned, which are up to the >>> caller to manage. >>> >>> We can do better than the current scheme of always leaking, by >>> having UniversalUpcallHandler and UniversalNativeInvoker create both >>> of these types of scopes, pass them to boxing code, and then close >>> scopes of type 1. when the call is finished. >>> >>> This will reduce the amount of memory being leaked, and could also >>> help identify bugs where references to temporary objects are leaked. >>> >>> --- >>> >>> After implementing this, it turns out that UniversalUpcallHandler >>> only needs a Scope for the arguments being passed to the java method >>> being called. For returns there should be no allocations; either the >>> value is being passed in registers, or written to a pointer passed >>> in by the caller. >>> >>> I did discover a problem with this patch in the TestJextractFFI >>> patch code: >>> >>> ??? public Stream children() { >>> ??????? final ArrayList ar = new ArrayList<>(); >>> ??????? // FIXME: need a way to pass ar down as user data d >>> ??????? try (Scope sc = Scope.newNativeScope()) { >>> ??????????? LibClang.lib.clang_visitChildren(cursor, >>> sc.allocateCallback((c, p, d) -> { >>> ??????????????? ar.add(new Cursor(c)); >>> ??????????????? return LibClang.lib.CXChildVisit_Continue(); >>> ??????????? }), Pointer.nullPointer()); >>> ??????????? return ar.stream(); >>> ??????? } >>> ??? } >>> >>> According to C semantics, the argument to the lambda `c`, which is a >>> CXCursor struct being passed in by-value, is only valid for the >>> duration of the call. The Windows ABI depends on this fact, because >>> it allocates the struct on the stack of the caller, and then passes >>> a pointer to the callee (the lambda in this case). I think on SysV a >>> by-value struct is always passed in register + stack? >>> >>> But, the `c` argument is being leaked from the lambda, by wrapping >>> it in a Cursor and storing it in a list in the enclosing scope. When >>> the call to the lambda ends, the struct is deallocated and the >>> pointer that is stored on the Java side is now dangling. I've >>> initially solved the crash this was causing by defensively copying >>> the struct in the Windows boxing code (instead of just wrapping the >>> passed pointer), but I realized this also means that a user would >>> have to manually release a struct passed into an upcall to avoid a >>> memory leak. >>> >>> We have 2 options here: >>> >>> 1.) Go with C style. Struct arguments are valid only during an >>> upcall, and we have to manually copy them if we want to 'leak' them >>> from the scope of the upcall. >>> 2.) Go with status quo. Struct doesn't need to be copied manually >>> when saving it outside the scope of the upcall. The copy is >>> defensively done in Windows boxing code instead, and sometimes >>> redundant. Only on Windows (for now), struct arguments to an upcall >>> have to be manually released to avoid a memory leak. >>> >>> I think 1 is the better option here since this adheres to the >>> expectations native code has when calling into Java. I've reflected >>> that in the patch as well, but of course I'm interested in hearing >>> other opinions as well :) Note that the crash I was seeing before no >>> longer happens since the Scope used in the boxing code is now closed >>> after the call, and we get an exception instead. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8217420 >>> Webrev: >>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217420/webrev.00/ >>> >>> Cheers, >>> Jorn From jbvernee at xs4all.nl Mon Jan 21 21:51:59 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Mon, 21 Jan 2019 22:51:59 +0100 Subject: [foreign] RFR 8217420: Have UniversalNativeInvoker and UniversalUpcallHandler create (and close) the scopes used by boxing code In-Reply-To: <6ba6da2fa7c519b57f871b8f84110b2d@xs4all.nl> References: <60c37cbd463ff8a5ab7ab9899565ab47@xs4all.nl> <6d221129-4997-e993-a126-7168e7adf885@oracle.com> <6ba6da2fa7c519b57f871b8f84110b2d@xs4all.nl> Message-ID: <26d601d2a56ac88ff719dbe4bba4e3b4@xs4all.nl> >> I think I'd like to see a specific test for lifecycle of callback >> args, so that we can easily run it with all possible invoker modes and >> make sure it passes. > > I added a test for that (CallScopeTest), is there something else you'd > want to have tested? AFAICT this lifecycle problem currently only > exists with structs. Actually, now that you mention it, this is only testing for a fits-in-register struct, not for one being passed by pointer, which is the case I was trying to solve. I'll take a look at adding more tests tomorrow (getting tired). Jorn Jorn Vernee schreef op 2019-01-21 22:31: > Maurizio Cimadamore schreef op 2019-01-21 22:16: >> Hi Jorn, >> I think I agree with the spirit of the fix. I like that this makes the >> scopes stricter - which ended up uncovering some issues. >> >> Again, I have to point out that this fix only restrict things for >> universal invoker mode - I believe that when using the direct >> invocation scheme, parameters to upcalls will be passed in leaky >> scopes, which will not reveal the clang-related failure. > > Sorry about that, I wasn't sure before if I was supposed to make > changes to the direct invoker as well, since I would have no way of > testing those changes locally. (I've put it on my checklist for the > future). > >> I think I'd like to see a specific test for lifecycle of callback >> args, so that we can easily run it with all possible invoker modes and >> make sure it passes. > > I added a test for that (CallScopeTest), is there something else you'd > want to have tested? AFAICT this lifecycle problem currently only > exists with structs. > > Jorn > >> Maurizio >> >> On 21/01/2019 18:13, Jorn Vernee wrote: >>> Hi, another patch :) >>> >>> From the bug description: >>> >>> Currently, when (un-)boxing code needs to do an allocation a Scope is >>> created for that, and this Scope is then leaked. >>> >>> For an up-/down-call there are 2 kinds of scopes; 1) for values being >>> passed as arguments by value, which are valid for the duration of the >>> call. 2) for values being returned, which are up to the caller to >>> manage. >>> >>> We can do better than the current scheme of always leaking, by having >>> UniversalUpcallHandler and UniversalNativeInvoker create both of >>> these types of scopes, pass them to boxing code, and then close >>> scopes of type 1. when the call is finished. >>> >>> This will reduce the amount of memory being leaked, and could also >>> help identify bugs where references to temporary objects are leaked. >>> >>> --- >>> >>> After implementing this, it turns out that UniversalUpcallHandler >>> only needs a Scope for the arguments being passed to the java method >>> being called. For returns there should be no allocations; either the >>> value is being passed in registers, or written to a pointer passed in >>> by the caller. >>> >>> I did discover a problem with this patch in the TestJextractFFI patch >>> code: >>> >>> ??? public Stream children() { >>> ??????? final ArrayList ar = new ArrayList<>(); >>> ??????? // FIXME: need a way to pass ar down as user data d >>> ??????? try (Scope sc = Scope.newNativeScope()) { >>> ??????????? LibClang.lib.clang_visitChildren(cursor, >>> sc.allocateCallback((c, p, d) -> { >>> ??????????????? ar.add(new Cursor(c)); >>> ??????????????? return LibClang.lib.CXChildVisit_Continue(); >>> ??????????? }), Pointer.nullPointer()); >>> ??????????? return ar.stream(); >>> ??????? } >>> ??? } >>> >>> According to C semantics, the argument to the lambda `c`, which is a >>> CXCursor struct being passed in by-value, is only valid for the >>> duration of the call. The Windows ABI depends on this fact, because >>> it allocates the struct on the stack of the caller, and then passes a >>> pointer to the callee (the lambda in this case). I think on SysV a >>> by-value struct is always passed in register + stack? >>> >>> But, the `c` argument is being leaked from the lambda, by wrapping it >>> in a Cursor and storing it in a list in the enclosing scope. When the >>> call to the lambda ends, the struct is deallocated and the pointer >>> that is stored on the Java side is now dangling. I've initially >>> solved the crash this was causing by defensively copying the struct >>> in the Windows boxing code (instead of just wrapping the passed >>> pointer), but I realized this also means that a user would have to >>> manually release a struct passed into an upcall to avoid a memory >>> leak. >>> >>> We have 2 options here: >>> >>> 1.) Go with C style. Struct arguments are valid only during an >>> upcall, and we have to manually copy them if we want to 'leak' them >>> from the scope of the upcall. >>> 2.) Go with status quo. Struct doesn't need to be copied manually >>> when saving it outside the scope of the upcall. The copy is >>> defensively done in Windows boxing code instead, and sometimes >>> redundant. Only on Windows (for now), struct arguments to an upcall >>> have to be manually released to avoid a memory leak. >>> >>> I think 1 is the better option here since this adheres to the >>> expectations native code has when calling into Java. I've reflected >>> that in the patch as well, but of course I'm interested in hearing >>> other opinions as well :) Note that the crash I was seeing before no >>> longer happens since the Scope used in the boxing code is now closed >>> after the call, and we get an exception instead. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8217420 >>> Webrev: >>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217420/webrev.00/ >>> >>> Cheers, >>> Jorn From vladimir.x.ivanov at oracle.com Mon Jan 21 21:57:36 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Mon, 21 Jan 2019 13:57:36 -0800 Subject: [vector] Support long addAll() for AVX < 3 In-Reply-To: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A51F53@FMSMSX126.amr.corp.intel.com> References: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A51F25@FMSMSX126.amr.corp.intel.com> <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A51F53@FMSMSX126.amr.corp.intel.com> Message-ID: >>> BTW it would be nice to move 64-bit specific declarations into x86_64.ad. > The base rules for AddReductionVL are coming from mainline and it has been under LP64 check. Also there are many instances of LP64 check in x86.ad file as traditionally x86.ad held all the vectorization related rules. May be that needs to be a separate refactor at some point starting from mainline JDK. Please let me know what you think and I will send an updated webrev accordingly. Sure, I'm perfectly fine with addressing it separately. Best regards, Vladimir Ivanov > -----Original Message----- > From: Vladimir Ivanov [mailto:vladimir.x.ivanov at oracle.com] > Sent: Monday, January 21, 2019 11:44 AM > To: Viswanathan, Sandhya ; panama-dev at openjdk.java.net > Subject: Re: [vector] Support long addAll() for AVX < 3 > > src/hotspot/cpu/x86/x86.ad: > #ifdef _LP64 > instruct rvadd2L_reduction_reg(rRegL dst, rRegL src1, vecX src2, vecX tmp, vecX tmp2) %{ > - predicate(UseAVX > 2); > + predicate(UseSSE >= 2); > > There's no need in the check, since SSE2 support is a requirement on x64. > > BTW it would be nice to move 64-bit specific declarations into x86_64.ad. > > > instruct rvadd4L_reduction_reg(rRegL dst, rRegL src1, vecY src2, vecY > tmp, vecY tmp2) %{ > - predicate(UseAVX > 2); > + predicate(UseAVX >= 1); > > Are you sure about AVX1 support? As I see in the docs, 256-bit pshufd > variant is available since AVX2. > > (On a side note, I'm in favor of exclusive range checks: UseAVX > 1 vs > UseAVX >= 2. The former reads better in x86.ad.) > > Best regards, > Vladimir Ivanov > > On 21/01/2019 11:32, Viswanathan, Sandhya wrote: >> Hi All, >> >> Please find below a small patch supporting long addAll for AVX < 3: >> >> http://cr.openjdk.java.net/~sviswanathan/vectorIntrinsics/longAddReduction/webrev.00/ >> >> Best Regards, >> Sandhya >> From sandhya.viswanathan at intel.com Mon Jan 21 23:29:43 2019 From: sandhya.viswanathan at intel.com (Viswanathan, Sandhya) Date: Mon, 21 Jan 2019 23:29:43 +0000 Subject: [vector] Support long addAll() for AVX < 3 In-Reply-To: References: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A51F25@FMSMSX126.amr.corp.intel.com> <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A51F53@FMSMSX126.amr.corp.intel.com> Message-ID: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A5205D@FMSMSX126.amr.corp.intel.com> Please find the updated patch at: http://cr.openjdk.java.net/~sviswanathan/vectorIntrinsics/longAddReduction/webrev.01/ Changes vs previous patch: 1) instruct rvadd2L_reduction_reg: Remove predicate(UseSSE >= 2) as SSE2 is a requirement for LP64. 2) instruct rvadd4L_reduction_reg: The predicate should be UseAVX > 1. Best Regards, Sandhya -----Original Message----- From: Vladimir Ivanov [mailto:vladimir.x.ivanov at oracle.com] Sent: Monday, January 21, 2019 1:58 PM To: Viswanathan, Sandhya ; panama-dev at openjdk.java.net Subject: Re: [vector] Support long addAll() for AVX < 3 >>> BTW it would be nice to move 64-bit specific declarations into x86_64.ad. > The base rules for AddReductionVL are coming from mainline and it has been under LP64 check. Also there are many instances of LP64 check in x86.ad file as traditionally x86.ad held all the vectorization related rules. May be that needs to be a separate refactor at some point starting from mainline JDK. Please let me know what you think and I will send an updated webrev accordingly. Sure, I'm perfectly fine with addressing it separately. Best regards, Vladimir Ivanov > -----Original Message----- > From: Vladimir Ivanov [mailto:vladimir.x.ivanov at oracle.com] > Sent: Monday, January 21, 2019 11:44 AM > To: Viswanathan, Sandhya ; panama-dev at openjdk.java.net > Subject: Re: [vector] Support long addAll() for AVX < 3 > > src/hotspot/cpu/x86/x86.ad: > #ifdef _LP64 > instruct rvadd2L_reduction_reg(rRegL dst, rRegL src1, vecX src2, vecX tmp, vecX tmp2) %{ > - predicate(UseAVX > 2); > + predicate(UseSSE >= 2); > > There's no need in the check, since SSE2 support is a requirement on x64. > > BTW it would be nice to move 64-bit specific declarations into x86_64.ad. > > > instruct rvadd4L_reduction_reg(rRegL dst, rRegL src1, vecY src2, vecY > tmp, vecY tmp2) %{ > - predicate(UseAVX > 2); > + predicate(UseAVX >= 1); > > Are you sure about AVX1 support? As I see in the docs, 256-bit pshufd > variant is available since AVX2. > > (On a side note, I'm in favor of exclusive range checks: UseAVX > 1 vs > UseAVX >= 2. The former reads better in x86.ad.) > > Best regards, > Vladimir Ivanov > > On 21/01/2019 11:32, Viswanathan, Sandhya wrote: >> Hi All, >> >> Please find below a small patch supporting long addAll for AVX < 3: >> >> http://cr.openjdk.java.net/~sviswanathan/vectorIntrinsics/longAddReduction/webrev.00/ >> >> Best Regards, >> Sandhya >> From vladimir.x.ivanov at oracle.com Mon Jan 21 23:49:10 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Mon, 21 Jan 2019 15:49:10 -0800 Subject: [vector] Support long addAll() for AVX < 3 In-Reply-To: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A5205D@FMSMSX126.amr.corp.intel.com> References: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A51F25@FMSMSX126.amr.corp.intel.com> <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A51F53@FMSMSX126.amr.corp.intel.com> <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A5205D@FMSMSX126.amr.corp.intel.com> Message-ID: <905af405-9757-6e14-fc45-f1ac1184e4a5@oracle.com> Looks good. Best regards, Vladimir Ivanov On 21/01/2019 15:29, Viswanathan, Sandhya wrote: > Please find the updated patch at: > > http://cr.openjdk.java.net/~sviswanathan/vectorIntrinsics/longAddReduction/webrev.01/ > > Changes vs previous patch: > 1) instruct rvadd2L_reduction_reg: Remove predicate(UseSSE >= 2) as SSE2 is a requirement for LP64. > 2) instruct rvadd4L_reduction_reg: The predicate should be UseAVX > 1. > > Best Regards, > Sandhya > > -----Original Message----- > From: Vladimir Ivanov [mailto:vladimir.x.ivanov at oracle.com] > Sent: Monday, January 21, 2019 1:58 PM > To: Viswanathan, Sandhya ; panama-dev at openjdk.java.net > Subject: Re: [vector] Support long addAll() for AVX < 3 > > >>>> BTW it would be nice to move 64-bit specific declarations into x86_64.ad. >> The base rules for AddReductionVL are coming from mainline and it has been under LP64 check. Also there are many instances of LP64 check in x86.ad file as traditionally x86.ad held all the vectorization related rules. May be that needs to be a separate refactor at some point starting from mainline JDK. Please let me know what you think and I will send an updated webrev accordingly. > > Sure, I'm perfectly fine with addressing it separately. > > Best regards, > Vladimir Ivanov > >> -----Original Message----- >> From: Vladimir Ivanov [mailto:vladimir.x.ivanov at oracle.com] >> Sent: Monday, January 21, 2019 11:44 AM >> To: Viswanathan, Sandhya ; panama-dev at openjdk.java.net >> Subject: Re: [vector] Support long addAll() for AVX < 3 >> >> src/hotspot/cpu/x86/x86.ad: >> #ifdef _LP64 >> instruct rvadd2L_reduction_reg(rRegL dst, rRegL src1, vecX src2, vecX tmp, vecX tmp2) %{ >> - predicate(UseAVX > 2); >> + predicate(UseSSE >= 2); >> >> There's no need in the check, since SSE2 support is a requirement on x64. >> >> BTW it would be nice to move 64-bit specific declarations into x86_64.ad. >> >> >> instruct rvadd4L_reduction_reg(rRegL dst, rRegL src1, vecY src2, vecY >> tmp, vecY tmp2) %{ >> - predicate(UseAVX > 2); >> + predicate(UseAVX >= 1); >> >> Are you sure about AVX1 support? As I see in the docs, 256-bit pshufd >> variant is available since AVX2. >> >> (On a side note, I'm in favor of exclusive range checks: UseAVX > 1 vs >> UseAVX >= 2. The former reads better in x86.ad.) >> >> Best regards, >> Vladimir Ivanov >> >> On 21/01/2019 11:32, Viswanathan, Sandhya wrote: >>> Hi All, >>> >>> Please find below a small patch supporting long addAll for AVX < 3: >>> >>> http://cr.openjdk.java.net/~sviswanathan/vectorIntrinsics/longAddReduction/webrev.00/ >>> >>> Best Regards, >>> Sandhya >>> From john.r.rose at oracle.com Tue Jan 22 00:47:36 2019 From: john.r.rose at oracle.com (John Rose) Date: Mon, 21 Jan 2019 16:47:36 -0800 Subject: [foreign] RFR 8217420: Have UniversalNativeInvoker and UniversalUpcallHandler create (and close) the scopes used by boxing code In-Reply-To: <60c37cbd463ff8a5ab7ab9899565ab47@xs4all.nl> References: <60c37cbd463ff8a5ab7ab9899565ab47@xs4all.nl> Message-ID: <15EB2E57-A975-41AF-B0C2-621FB0633641@oracle.com> On Jan 21, 2019, at 10:13 AM, Jorn Vernee wrote: > > We have 2 options here: > > 1.) Go with C style. Struct arguments are valid only during an upcall, and we have to manually copy them if we want to 'leak' them from the scope of the upcall. > 2.) Go with status quo. Struct doesn't need to be copied manually when saving it outside the scope of the upcall. The copy is defensively done in Windows boxing code instead, and sometimes redundant. Only on Windows (for now), struct arguments to an upcall have to be manually released to avoid a memory leak. I wouldn't call #1 C style, but rather ABI style. C style allows by-value passes of structs in all directions, with zero concern for pointer scoping, because the pointers are kept invisible. I think #2 is closer to C style, properly understood, but it's not perfect either if it requires manual deallocation. So, calling #1 ABI style, we then also have: 3.) Go with C style. Struct arguments (as opposed to pointers to structs) are passed by value and all appearances in Java exhibit value-like semantics, which includes no lifetime constraints. The struct is copied onto the Java heap, defensively, and is preserved there with no writability. If we had true Valhalla value types, it could be a value object instance, but it should be at least a value-based class. There's still a trade-off here, as usual. Putting the struct bit-image on the Java heap means it is safe to share across threads and across scopes, and will be GC-ed when no longer in use. That's all good. The downside is you can only use it for further by-value actions, such as passing it as an argument later on to a down-call or storing it bitwise into a container. If you need to pass an address for the struct data to a C function, that address needs to point to a scoped off-heap copy of the struct value. Internally, at the ABI level, the by-value calling sequence will almost certainly need to recopy the struct into registers or into a stack buffer: But that's what C does anyway, so that copy doesn't really bear on the decision to store struct values on-heap or off-heap. Also, design issues for arguments to up-calls are often paralleled with returns from down-calls, and it's probably true in this case. Whatever policy we go with for by-value structs should apply equally to up-call arguments and down-call returns. ? John From Ningsheng.Jian at arm.com Tue Jan 22 02:18:13 2019 From: Ningsheng.Jian at arm.com (Ningsheng Jian (Arm Technology China)) Date: Tue, 22 Jan 2019 02:18:13 +0000 Subject: New panama Committer: Wang Zhuo In-Reply-To: <9e892bc8-9a8b-f66a-0edd-3b3707fe0786@oracle.com> References: <9e892bc8-9a8b-f66a-0edd-3b3707fe0786@oracle.com> Message-ID: Vote: yes Thanks, Ningsheng > -----Original Message----- > From: panama-dev On Behalf Of > Vladimir Ivanov > Sent: Saturday, January 19, 2019 2:53 AM > To: 'panama-dev at openjdk.java.net' > Subject: CFV: New panama Committer: Wang Zhuo > > I hereby nominate Wang Zhuo (email: zhuoren.wz at alibaba-inc.com) to > Committer role in Project Panama. > > Zhuoren works on Vector API at Alibaba and has already made 3 contributions [1] > to the implementation. > > Votes are due by January, 25, 2019. > > Only current panama Committers [2] are eligible to vote on this nomination. > Votes must be cast in the open by replying to this mailing list. > > For Lazy Consensus voting instructions, see [3]. > > Best regards, > Vladimir Ivanov > > [1] http://hg.openjdk.java.net/panama/dev/rev/a511da8eef00 > http://hg.openjdk.java.net/panama/dev/rev/a8516a4be714 > http://hg.openjdk.java.net/panama/dev/rev/dfe1c1d8a316 > > [2] http://openjdk.java.net/census#panama > [3] http://openjdk.java.net/projects#committer-vote From sundararajan.athijegannathan at oracle.com Tue Jan 22 04:28:13 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Tue, 22 Jan 2019 09:58:13 +0530 Subject: [foreign] RFR 8208081: jextract throws "java.lang.IllegalStateException: Partially used storage unit" Message-ID: <5C469BDD.50909@oracle.com> Please review. Bug: https://bugs.openjdk.java.net/browse/JDK-8208081 Webrev: https://cr.openjdk.java.net/~sundar/8208081/webrev.00/ Thanks, -Sundar From sundararajan.athijegannathan at oracle.com Tue Jan 22 05:34:36 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Tue, 22 Jan 2019 11:04:36 +0530 Subject: [foreign] RFR 8217453: add support for member annotation scheme In-Reply-To: <19a3a7d0-da9f-89d8-9750-d3882f17a395@oracle.com> References: <19a3a7d0-da9f-89d8-9750-d3882f17a395@oracle.com> Message-ID: <5C46AB6C.9040009@oracle.com> * test/jdk/com/sun/tools/jextract/staticForwarder/StaticForwarderTest.java Unused import for NativeSetter? * NativeGetter and NativeSetter annotations allow target to be ElementType.TYPE as well. We need only METHOD, right? or am I missing something? * test/jdk/java/foreign/OutOfBoundsTest.java (latest test added) still uses older 'declarations' field in NativeHeader annotation. PS. I can jextract and run lapack example on Mac (this sample used to fail in jextract step). -Sundar On 21/01/19, 11:00 PM, Maurizio Cimadamore wrote: > Forgot to mention: the patch passes tests on all platforms. > > Maurizio > > On 21/01/2019 16:52, Maurizio Cimadamore wrote: >> Hi, >> this patch implements the member-based annotation scheme described here: >> >> http://cr.openjdk.java.net/~mcimadamore/panama/panama-annos.html >> >> (see end of document) >> >> This patch diverges from the proposal in two minor aspects: >> >> * @NativeCallback is left in place (we have bigger plans for allowing >> callback types to be referenced by name) >> * unresolved layout grammar: the layout expression used braces - e.g. >> ${foo}, not $(foo), to minimize confusion with layout annotations >> >> I've also took care of a couple of minor issues: >> >> * I added exclusion for the SystemHeadersTest which fails on Windows >> if Visual Studio is not installed >> * I fixed a bunch of javadoc issues (esp. in Pointer) which prevented >> javadoc to be generated correctly >> >> Webrev: >> >> http://cr.openjdk.java.net/~mcimadamore/panama/8217453/ >> >> Thanks >> Maurizio >> From sundararajan.athijegannathan at oracle.com Tue Jan 22 05:41:55 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Tue, 22 Jan 2019 11:11:55 +0530 Subject: [foreign] RFC: Jextract -l options ignored by SymbolFilter when -L is not specified In-Reply-To: <698464e8f7692e12a7062155c7e599d9@xs4all.nl> References: <698464e8f7692e12a7062155c7e599d9@xs4all.nl> Message-ID: <5C46AD23.9000207@oracle.com> I don't think it is a bug - afaik it is as per design. The primary use of "-l" is to record the library in annotation of the generated jar - so that binder can auto-load the library (either from java.library.path configuration or -rpath value recorded in annotation). It is okay to record name of the shared object alone and leave the library path configuration to java.library.path setting. "-L" option is added feature to perform missing symbols checking. "-rpath" option is to add a path for library search - so that binder can locate the shared object in the specific directory. If no -rpath is specified, "-L" is used for runtime search as well. -Sundar On 22/01/19, 12:01 AM, Jorn Vernee wrote: > Hi, > > I've recently updated the instructions for using libraries on Windows. > For python the jextract example I gave was: > > jextract -l python27 -o "python.jar" -t "org.python" > C:\Python27\include\Python.h > > I'm lacking an `-L` option here (for specifying library directories) > since the contents of PATH seems to be added to java.library.path by > default, and this is presumably also how jextract is able to load the > library. But, since I'm not using an `-L` option, SymbolFilter is not > checking if the symbols are in the python27.dll [1] > > private void initSymChecker(List linkCheckPaths) { > if (!libraryNames.isEmpty() && !linkCheckPaths.isEmpty()) { > // ... init symChecker > } else { > symChecker = null; > } > } > > (linkCheckPaths comes from the -L option values) > > This behaviour is somewhat unexpected. At least a warning that missing > an `-L` option will turn off symbol checking would be nice. > > We could also add the paths in `java.library.path` to the list of link > check paths in jextract [2]. That would mean that the symbol checker > would run for the example command. > > What do you think? > > Jorn > > [1] : > http://hg.openjdk.java.net/panama/dev/file/eaca2d16b80b/src/jdk.jextract/share/classes/com/sun/tools/jextract/SymbolFilter.java#l89 > [2] : > http://cr.openjdk.java.net/~jvernee/panama/webrevs/jlibpath/webrev.01/ From samuel.audet at gmail.com Tue Jan 22 09:01:09 2019 From: samuel.audet at gmail.com (Samuel Audet) Date: Tue, 22 Jan 2019 18:01:09 +0900 Subject: Toward distributions of native libraries In-Reply-To: References: <51e5d7d3-6f95-9b33-1858-645bc388e17e@gmail.com> <479f6425-f3f2-e4ec-abe4-9303996dcd82@oracle.com> Message-ID: <10417618-1542-cdb8-5f6f-d3da32718166@gmail.com> On 1/19/19 6:12 AM, John Rose wrote: > Life is risky, and sometimes you hit a dead end. ?You are asking "what > if there's a dead end here for a non-static API w.r.t. performance", > forcing us to refactor the whole thing. ?"What if" indeed. ?I suppose we > differ about the probability of a disruptive refactor. ?I think it?is > vanishingly unlikely, given the large bag of tricks the JVM has to > optimize dynamic code as if it were static. ?So the "what if" question > is uninteresting to me. JIT, sure, but AOT? Well, I'm starting to believe you :) I did a bit of benchmarking here: > JNI time = 28 > C time = 10 > dummy time = 10 > native time = 7 https://github.com/oracle/graal/issues/885#issuecomment-456299353 Samuel From jbvernee at xs4all.nl Tue Jan 22 11:15:19 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Tue, 22 Jan 2019 12:15:19 +0100 Subject: [foreign] RFR 8217420: Have UniversalNativeInvoker and UniversalUpcallHandler create (and close) the scopes used by boxing code In-Reply-To: <15EB2E57-A975-41AF-B0C2-621FB0633641@oracle.com> References: <60c37cbd463ff8a5ab7ab9899565ab47@xs4all.nl> <15EB2E57-A975-41AF-B0C2-621FB0633641@oracle.com> Message-ID: <9081cf0c47d723ca64b887cfadb9288c@xs4all.nl> Comments inline... John Rose schreef op 2019-01-22 01:47: > On Jan 21, 2019, at 10:13 AM, Jorn Vernee wrote: > >> We have 2 options here: >> >> 1.) Go with C style. Struct arguments are valid only during an >> upcall, and we have to manually copy them if we want to 'leak' them >> from the scope of the upcall. >> 2.) Go with status quo. Struct doesn't need to be copied manually >> when saving it outside the scope of the upcall. The copy is >> defensively done in Windows boxing code instead, and sometimes >> redundant. Only on Windows (for now), struct arguments to an upcall >> have to be manually released to avoid a memory leak. > > I wouldn't call #1 C style, but rather ABI style. C style allows > by-value > passes of structs in all directions, with zero concern for pointer > scoping, > because the pointers are kept invisible. > > I think #2 is closer to C style, properly understood, but it's not > perfect > either if it requires manual deallocation. > > So, calling #1 ABI style, we then also have: > > 3.) Go with C style. Struct arguments (as opposed to pointers to > structs) > are passed by value and all appearances in Java exhibit value-like > semantics, > which includes no lifetime constraints. The struct is copied onto the > Java > heap, defensively, and is preserved there with no writability. If we > had > true Valhalla value types, it could be a value object instance, but it > should > be at least a value-based class. I have not been keeping up with Valhalla lately. Having value-like semantics in Java would mean embedding the bit-image directly into the value-type object right? I thought the tech was not there yet for this. I think you'd need some kind of fixed-length long[] that is fused into the struct object and gets copied along with it? This has the same problem that a user would need to create an off-heap copy if they wanted to pass a pointer to it to a C function (that you mentioned later). Or could there be support for direct pointers to value types, as alternative to the Object + offset scheme in Unsafe right now? (since they don't get moved by the GC, right?). > There's still a trade-off here, as usual. Putting the struct > bit-image > on the Java heap means it is safe to share across threads and across > scopes, and will be GC-ed when no longer in use. That's all good. > The downside is you can only use it for further by-value actions, > such as passing it as an argument later on to a down-call or storing > it bitwise into a container. If you need to pass an address for the > struct data to a C function, that address needs to point to a scoped > off-heap copy of the struct value. Internally, at the ABI level, the > by-value calling sequence will almost certainly need to recopy the > struct into registers or into a stack buffer: But that's what C does > anyway, so that copy doesn't really bear on the decision to store > struct values on-heap or off-heap. > > Also, design issues for arguments to up-calls are often paralleled > with returns from down-calls, and it's probably true in this case. > Whatever policy we go with for by-value structs should apply > equally to up-call arguments and down-call returns. The way I think this would work for down-call returns is that; a by-value struct return constitutes a transfer of resources to the caller. In native code this is also true, but the returned value immediately gets automatic storage duration, because it's a value type. But, on the Java side the value resides (currently) off-heap, and _someone_ has to manage this resource. So I feel we'd need to teach users that they have to keep that in mind (maybe with some @ResourceTransfer annotation). In the patch this is asymmetrical with upcalls, and I didn't find a way of doing something automatic for struct by-value returns yet. I experimented with tying the returned value's Scope to the lifetime of the returned Struct object using a Cleaner. But, I realized this also means that whenever you want to pass a pointer to the struct to a C function it's up to the caller to keep the object alive, and if the passing of this pointer is meant as a resource-transfer then you'd really want a way to detach the object from the Cleaner again (which I don't think there is?). Having symmetry for down-/up-calls seems important to me as well for simplicity's sake, and I'm starting to see the appeal of the status quo more and more (i.e. defensive copy of up-call args). If we go with that I think it's important to tell the developer when we're transferring a resource to them, and it's up to them to manage it. Jorn From jbvernee at xs4all.nl Tue Jan 22 11:55:04 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Tue, 22 Jan 2019 12:55:04 +0100 Subject: [foreign] RFC: Jextract -l options ignored by SymbolFilter when -L is not specified In-Reply-To: <5C46AD23.9000207@oracle.com> References: <698464e8f7692e12a7062155c7e599d9@xs4all.nl> <5C46AD23.9000207@oracle.com> Message-ID: I see, that paints a clear picture thanks. So as far as libraries go: "-l" : record library name in @NativeHeader::libraries "-rpath" : record runtime library path in @NativeHeader::libraryPaths "-L" : do symbol checking using the libraries specified by "-l" "--infer-rpath" : infer "-rpath" values from "-L" values The picture I had was: "-l" : record library name in @NativeHeader::libraries AND do symbol checking with this library "-rpath" : record runtime library path in @NativeHeader::libraryPaths "-L" : provide lookup paths to load libraries for "-l" "--infer-rpath" : infer "-rpath" values from "-L" values The current "-L" semantics seem a little unintuitive to me to be honest, since "-L" also provide the lookup paths for "-l". If depending on java.library.path to load libraries with "-l" is intended, I would propose replacing the "-L" option with something like a "--filter-symbols" option, and have users use java.library.path to provide the paths used to load the libraries for "-l", and then have "--infer-rpath" copy the paths from java.library.path instead. That way we have only one source of library paths (java.library.path). What do you think? Jorn Sundararajan Athijegannathan schreef op 2019-01-22 06:41: > I don't think it is a bug - afaik it is as per design. The primary use > of "-l" is to record the library in annotation of the generated jar - > so that binder can auto-load the library (either from > java.library.path configuration or -rpath value recorded in > annotation). It is okay to record name of the shared object alone and > leave the library path configuration to java.library.path setting. > > "-L" option is added feature to perform missing symbols checking. > "-rpath" option is to add a path for library search - so that binder > can locate the shared object in the specific directory. If no -rpath > is specified, "-L" is used for runtime search as well. > > -Sundar > > On 22/01/19, 12:01 AM, Jorn Vernee wrote: >> Hi, >> >> I've recently updated the instructions for using libraries on Windows. >> For python the jextract example I gave was: >> >> jextract -l python27 -o "python.jar" -t "org.python" >> C:\Python27\include\Python.h >> >> I'm lacking an `-L` option here (for specifying library directories) >> since the contents of PATH seems to be added to java.library.path by >> default, and this is presumably also how jextract is able to load the >> library. But, since I'm not using an `-L` option, SymbolFilter is not >> checking if the symbols are in the python27.dll [1] >> >> private void initSymChecker(List linkCheckPaths) { >> if (!libraryNames.isEmpty() && !linkCheckPaths.isEmpty()) { >> // ... init symChecker >> } else { >> symChecker = null; >> } >> } >> >> (linkCheckPaths comes from the -L option values) >> >> This behaviour is somewhat unexpected. At least a warning that missing >> an `-L` option will turn off symbol checking would be nice. >> >> We could also add the paths in `java.library.path` to the list of link >> check paths in jextract [2]. That would mean that the symbol checker >> would run for the example command. >> >> What do you think? >> >> Jorn >> >> [1] : >> http://hg.openjdk.java.net/panama/dev/file/eaca2d16b80b/src/jdk.jextract/share/classes/com/sun/tools/jextract/SymbolFilter.java#l89 >> [2] : >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/jlibpath/webrev.01/ From maurizio.cimadamore at oracle.com Tue Jan 22 11:58:21 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 22 Jan 2019 11:58:21 +0000 Subject: [foreign] RFC: Jextract -l options ignored by SymbolFilter when -L is not specified In-Reply-To: <5C46AD23.9000207@oracle.com> References: <698464e8f7692e12a7062155c7e599d9@xs4all.nl> <5C46AD23.9000207@oracle.com> Message-ID: <4089850d-fec6-40cf-c036-e666169a84b3@oracle.com> Looking at this, I remember being confused about this too. Let me try to see if we can find a better stacking for the existing options - as Sundar said, we currently have: * -l This option is used to specify library _names_. The main goal of this option is to alter the contents of the @NativeHeader annotation (by adding the library name) but there are, as we shall see, other subtle side-effects. * -L + -l When both -L and -l are specified, the so called "missing symbols check" will kick in,? that is, jextract will check that all symbols in the library are indeed defined in the header files being extracted. A subtle side-effect of that check, is that when -l and -L are specified together, and the missing symbol check is enabled, jextract will warn for symbols not found and _it will exclude them_ from the extracted classfile (w/o need for --include-symbols or --exclude-symbols). * -L + -l + -infer-rpath When -L and -l are used together, and the -infer-rpath option is given, a runtime library path will be inferred from the contents of -L, and will be stored in @NativeHeader, so that the binder can use it. I think the status quo is a bit confusing - because -L has multiple functions (it serves up the library paths to be used as inferred rpaths, and it also serves up the library paths to be used for the missing symbol check). I think a more consistent stacking could be something like this: -l --> used to specify library _names_; only side-effect is contents of @NativeHeader -L --> used to specify _custom_ library _paths_; no side-effects -exclude-missing -> must be used in conjunction with -l and -L ; enables the missing symbol check and auto-exclusion -infer-rpath -> must be used in conjunction with -l and -L ; enables rpath inference (rpath inferred with paths specified in -L) Thoughts? Maurizio On 22/01/2019 05:41, Sundararajan Athijegannathan wrote: > I don't think it is a bug - afaik it is as per design. The primary use > of "-l" is to record the library in annotation of the generated jar - > so that binder can auto-load the library (either from > java.library.path configuration or -rpath value recorded in > annotation).? It is okay to record name of the shared object alone and > leave the library path configuration to java.library.path setting. > > "-L" option is added feature to perform missing symbols checking. > "-rpath" option is to add a path for library search - so that binder > can locate the shared object in the specific directory. If no -rpath > is specified, "-L" is used for runtime search as well. > > -Sundar > > On 22/01/19, 12:01 AM, Jorn Vernee wrote: >> Hi, >> >> I've recently updated the instructions for using libraries on >> Windows. For python the jextract example I gave was: >> >> ??? jextract -l python27 -o "python.jar" -t "org.python" >> C:\Python27\include\Python.h >> >> I'm lacking an `-L` option here (for specifying library directories) >> since the contents of PATH seems to be added to java.library.path by >> default, and this is presumably also how jextract is able to load the >> library. But, since I'm not using an `-L` option, SymbolFilter is not >> checking if the symbols are in the python27.dll [1] >> >> ??? private void initSymChecker(List linkCheckPaths) { >> ??????? if (!libraryNames.isEmpty() && !linkCheckPaths.isEmpty()) { >> ??????????? // ... init symChecker >> ??????? } else { >> ??????????? symChecker = null; >> ??????? } >> ??? } >> >> (linkCheckPaths comes from the -L option values) >> >> This behaviour is somewhat unexpected. At least a warning that >> missing an `-L` option will turn off symbol checking would be nice. >> >> We could also add the paths in `java.library.path` to the list of >> link check paths in jextract [2]. That would mean that the symbol >> checker would run for the example command. >> >> What do you think? >> >> Jorn >> >> [1] : >> http://hg.openjdk.java.net/panama/dev/file/eaca2d16b80b/src/jdk.jextract/share/classes/com/sun/tools/jextract/SymbolFilter.java#l89 >> [2] : >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/jlibpath/webrev.01/ From maurizio.cimadamore at oracle.com Tue Jan 22 12:01:02 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 22 Jan 2019 12:01:02 +0000 Subject: [foreign] RFR 8217453: add support for member annotation scheme In-Reply-To: <5C46AB6C.9040009@oracle.com> References: <19a3a7d0-da9f-89d8-9750-d3882f17a395@oracle.com> <5C46AB6C.9040009@oracle.com> Message-ID: <305f98ea-b108-974d-3e0e-98ddd24a5ee1@oracle.com> Thanks, updated webrev http://cr.openjdk.java.net/~mcimadamore/panama/8217453_v2/ Maurizio On 22/01/2019 05:34, Sundararajan Athijegannathan wrote: > > > * > test/jdk/com/sun/tools/jextract/staticForwarder/StaticForwarderTest.java > > Unused import for NativeSetter? > > * NativeGetter and NativeSetter annotations allow target to be > ElementType.TYPE as well.? We need only METHOD, right? or am I missing > something? > > * test/jdk/java/foreign/OutOfBoundsTest.java (latest test added) still > uses older 'declarations' field in NativeHeader annotation. > > PS. I can jextract and run lapack example on Mac (this sample used to > fail in jextract step). > > -Sundar > > On 21/01/19, 11:00 PM, Maurizio Cimadamore wrote: >> Forgot to mention: the patch passes tests on all platforms. >> >> Maurizio >> >> On 21/01/2019 16:52, Maurizio Cimadamore wrote: >>> Hi, >>> this patch implements the member-based annotation scheme described >>> here: >>> >>> http://cr.openjdk.java.net/~mcimadamore/panama/panama-annos.html >>> >>> (see end of document) >>> >>> This patch diverges from the proposal in two minor aspects: >>> >>> * @NativeCallback is left in place (we have bigger plans for >>> allowing callback types to be referenced by name) >>> * unresolved layout grammar: the layout expression used braces - >>> e.g. ${foo}, not $(foo), to minimize confusion with layout annotations >>> >>> I've also took care of a couple of minor issues: >>> >>> * I added exclusion for the SystemHeadersTest which fails on Windows >>> if Visual Studio is not installed >>> * I fixed a bunch of javadoc issues (esp. in Pointer) which >>> prevented javadoc to be generated correctly >>> >>> Webrev: >>> >>> http://cr.openjdk.java.net/~mcimadamore/panama/8217453/ >>> >>> Thanks >>> Maurizio >>> From jbvernee at xs4all.nl Tue Jan 22 12:09:44 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Tue, 22 Jan 2019 13:09:44 +0100 Subject: [foreign] RFC: Jextract -l options ignored by SymbolFilter when -L is not specified In-Reply-To: <4089850d-fec6-40cf-c036-e666169a84b3@oracle.com> References: <698464e8f7692e12a7062155c7e599d9@xs4all.nl> <5C46AD23.9000207@oracle.com> <4089850d-fec6-40cf-c036-e666169a84b3@oracle.com> Message-ID: <30080ab5a99a1a9abf34a89a6c70b8b8@xs4all.nl> This sounds good, I really like the idea of a separate option to enable the symbol filtering. But can you share what you think the role of java.library.path should be as well? Jorn Maurizio Cimadamore schreef op 2019-01-22 12:58: > Looking at this, I remember being confused about this too. > > Let me try to see if we can find a better stacking for the existing > options - as Sundar said, we currently have: > > * -l > > This option is used to specify library _names_. > > The main goal of this option is to alter the contents of the > @NativeHeader annotation (by adding the library name) but there are, > as we shall see, other subtle side-effects. > > * -L + -l > > When both -L and -l are specified, the so called "missing symbols > check" will kick in,? that is, jextract will check that all symbols in > the library are indeed defined in the header files being extracted. A > subtle side-effect of that check, is that when -l and -L are specified > together, and the missing symbol check is enabled, jextract will warn > for symbols not found and _it will exclude them_ from the extracted > classfile (w/o need for --include-symbols or --exclude-symbols). > > * -L + -l + -infer-rpath > > When -L and -l are used together, and the -infer-rpath option is > given, a runtime library path will be inferred from the contents of > -L, and will be stored in @NativeHeader, so that the binder can use > it. > > > I think the status quo is a bit confusing - because -L has multiple > functions (it serves up the library paths to be used as inferred > rpaths, and it also serves up the library paths to be used for the > missing symbol check). I think a more consistent stacking could be > something like this: > > -l --> used to specify library _names_; only side-effect is contents > of @NativeHeader > > -L --> used to specify _custom_ library _paths_; no side-effects > > -exclude-missing -> must be used in conjunction with -l and -L ; > enables the missing symbol check and auto-exclusion > > -infer-rpath -> must be used in conjunction with -l and -L ; enables > rpath inference (rpath inferred with paths specified in -L) > > > Thoughts? > > Maurizio > > > On 22/01/2019 05:41, Sundararajan Athijegannathan wrote: >> I don't think it is a bug - afaik it is as per design. The primary use >> of "-l" is to record the library in annotation of the generated jar - >> so that binder can auto-load the library (either from >> java.library.path configuration or -rpath value recorded in >> annotation).? It is okay to record name of the shared object alone and >> leave the library path configuration to java.library.path setting. >> >> "-L" option is added feature to perform missing symbols checking. >> "-rpath" option is to add a path for library search - so that binder >> can locate the shared object in the specific directory. If no -rpath >> is specified, "-L" is used for runtime search as well. >> >> -Sundar >> >> On 22/01/19, 12:01 AM, Jorn Vernee wrote: >>> Hi, >>> >>> I've recently updated the instructions for using libraries on >>> Windows. For python the jextract example I gave was: >>> >>> ??? jextract -l python27 -o "python.jar" -t "org.python" >>> C:\Python27\include\Python.h >>> >>> I'm lacking an `-L` option here (for specifying library directories) >>> since the contents of PATH seems to be added to java.library.path by >>> default, and this is presumably also how jextract is able to load the >>> library. But, since I'm not using an `-L` option, SymbolFilter is not >>> checking if the symbols are in the python27.dll [1] >>> >>> ??? private void initSymChecker(List linkCheckPaths) { >>> ??????? if (!libraryNames.isEmpty() && !linkCheckPaths.isEmpty()) { >>> ??????????? // ... init symChecker >>> ??????? } else { >>> ??????????? symChecker = null; >>> ??????? } >>> ??? } >>> >>> (linkCheckPaths comes from the -L option values) >>> >>> This behaviour is somewhat unexpected. At least a warning that >>> missing an `-L` option will turn off symbol checking would be nice. >>> >>> We could also add the paths in `java.library.path` to the list of >>> link check paths in jextract [2]. That would mean that the symbol >>> checker would run for the example command. >>> >>> What do you think? >>> >>> Jorn >>> >>> [1] : >>> http://hg.openjdk.java.net/panama/dev/file/eaca2d16b80b/src/jdk.jextract/share/classes/com/sun/tools/jextract/SymbolFilter.java#l89 >>> [2] : >>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/jlibpath/webrev.01/ From sundararajan.athijegannathan at oracle.com Tue Jan 22 12:10:19 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Tue, 22 Jan 2019 17:40:19 +0530 Subject: [foreign] RFR 8217453: add support for member annotation scheme In-Reply-To: <305f98ea-b108-974d-3e0e-98ddd24a5ee1@oracle.com> References: <19a3a7d0-da9f-89d8-9750-d3882f17a395@oracle.com> <5C46AB6C.9040009@oracle.com> <305f98ea-b108-974d-3e0e-98ddd24a5ee1@oracle.com> Message-ID: <5C47082B.7000903@oracle.com> Looks good. -Sundar On 22/01/19, 5:31 PM, Maurizio Cimadamore wrote: > Thanks, > updated webrev > > http://cr.openjdk.java.net/~mcimadamore/panama/8217453_v2/ > > Maurizio > > On 22/01/2019 05:34, Sundararajan Athijegannathan wrote: >> >> >> * >> test/jdk/com/sun/tools/jextract/staticForwarder/StaticForwarderTest.java >> >> Unused import for NativeSetter? >> >> * NativeGetter and NativeSetter annotations allow target to be >> ElementType.TYPE as well. We need only METHOD, right? or am I >> missing something? >> >> * test/jdk/java/foreign/OutOfBoundsTest.java (latest test added) >> still uses older 'declarations' field in NativeHeader annotation. >> >> PS. I can jextract and run lapack example on Mac (this sample used to >> fail in jextract step). >> >> -Sundar >> >> On 21/01/19, 11:00 PM, Maurizio Cimadamore wrote: >>> Forgot to mention: the patch passes tests on all platforms. >>> >>> Maurizio >>> >>> On 21/01/2019 16:52, Maurizio Cimadamore wrote: >>>> Hi, >>>> this patch implements the member-based annotation scheme described >>>> here: >>>> >>>> http://cr.openjdk.java.net/~mcimadamore/panama/panama-annos.html >>>> >>>> (see end of document) >>>> >>>> This patch diverges from the proposal in two minor aspects: >>>> >>>> * @NativeCallback is left in place (we have bigger plans for >>>> allowing callback types to be referenced by name) >>>> * unresolved layout grammar: the layout expression used braces - >>>> e.g. ${foo}, not $(foo), to minimize confusion with layout annotations >>>> >>>> I've also took care of a couple of minor issues: >>>> >>>> * I added exclusion for the SystemHeadersTest which fails on >>>> Windows if Visual Studio is not installed >>>> * I fixed a bunch of javadoc issues (esp. in Pointer) which >>>> prevented javadoc to be generated correctly >>>> >>>> Webrev: >>>> >>>> http://cr.openjdk.java.net/~mcimadamore/panama/8217453/ >>>> >>>> Thanks >>>> Maurizio >>>> From maurizio.cimadamore at oracle.com Tue Jan 22 12:31:11 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 22 Jan 2019 12:31:11 +0000 Subject: [foreign] RFC: Jextract -l options ignored by SymbolFilter when -L is not specified In-Reply-To: <30080ab5a99a1a9abf34a89a6c70b8b8@xs4all.nl> References: <698464e8f7692e12a7062155c7e599d9@xs4all.nl> <5C46AD23.9000207@oracle.com> <4089850d-fec6-40cf-c036-e666169a84b3@oracle.com> <30080ab5a99a1a9abf34a89a6c70b8b8@xs4all.nl> Message-ID: <4de2ba3b-4971-95d3-cb3b-9a0d478656d9@oracle.com> On 22/01/2019 12:09, Jorn Vernee wrote: > This sounds good, I really like the idea of a separate option to > enable the symbol filtering. But can you share what you think the role > of java.library.path should be as well? I think using java.library.path as a default for the missing symbol check could be ok. But I don't think it would be ok to use it as a basis for infer-rpath. That is, I don't want static properties (e.g. valid at extraction time) to spill onto the runtime. If the user really wants to set some dynamic property, it has to use an explicit flag to do so (e.g. -L). Maurizio > > Jorn > > Maurizio Cimadamore schreef op 2019-01-22 12:58: >> Looking at this, I remember being confused about this too. >> >> Let me try to see if we can find a better stacking for the existing >> options - as Sundar said, we currently have: >> >> * -l >> >> This option is used to specify library _names_. >> >> The main goal of this option is to alter the contents of the >> @NativeHeader annotation (by adding the library name) but there are, >> as we shall see, other subtle side-effects. >> >> * -L + -l >> >> When both -L and -l are specified, the so called "missing symbols >> check" will kick in,? that is, jextract will check that all symbols in >> the library are indeed defined in the header files being extracted. A >> subtle side-effect of that check, is that when -l and -L are specified >> together, and the missing symbol check is enabled, jextract will warn >> for symbols not found and _it will exclude them_ from the extracted >> classfile (w/o need for --include-symbols or --exclude-symbols). >> >> * -L + -l + -infer-rpath >> >> When -L and -l are used together, and the -infer-rpath option is >> given, a runtime library path will be inferred from the contents of >> -L, and will be stored in @NativeHeader, so that the binder can use >> it. >> >> >> I think the status quo is a bit confusing - because -L has multiple >> functions (it serves up the library paths to be used as inferred >> rpaths, and it also serves up the library paths to be used for the >> missing symbol check). I think a more consistent stacking could be >> something like this: >> >> -l --> used to specify library _names_; only side-effect is contents >> of @NativeHeader >> >> -L --> used to specify _custom_ library _paths_; no side-effects >> >> -exclude-missing -> must be used in conjunction with -l and -L ; >> enables the missing symbol check and auto-exclusion >> >> -infer-rpath -> must be used in conjunction with -l and -L ; enables >> rpath inference (rpath inferred with paths specified in -L) >> >> >> Thoughts? >> >> Maurizio >> >> >> On 22/01/2019 05:41, Sundararajan Athijegannathan wrote: >>> I don't think it is a bug - afaik it is as per design. The primary >>> use of "-l" is to record the library in annotation of the generated >>> jar - so that binder can auto-load the library (either from >>> java.library.path configuration or -rpath value recorded in >>> annotation).? It is okay to record name of the shared object alone >>> and leave the library path configuration to java.library.path setting. >>> >>> "-L" option is added feature to perform missing symbols checking. >>> "-rpath" option is to add a path for library search - so that binder >>> can locate the shared object in the specific directory. If no -rpath >>> is specified, "-L" is used for runtime search as well. >>> >>> -Sundar >>> >>> On 22/01/19, 12:01 AM, Jorn Vernee wrote: >>>> Hi, >>>> >>>> I've recently updated the instructions for using libraries on >>>> Windows. For python the jextract example I gave was: >>>> >>>> ??? jextract -l python27 -o "python.jar" -t "org.python" >>>> C:\Python27\include\Python.h >>>> >>>> I'm lacking an `-L` option here (for specifying library >>>> directories) since the contents of PATH seems to be added to >>>> java.library.path by default, and this is presumably also how >>>> jextract is able to load the library. But, since I'm not using an >>>> `-L` option, SymbolFilter is not checking if the symbols are in the >>>> python27.dll [1] >>>> >>>> ??? private void initSymChecker(List linkCheckPaths) { >>>> ??????? if (!libraryNames.isEmpty() && !linkCheckPaths.isEmpty()) { >>>> ??????????? // ... init symChecker >>>> ??????? } else { >>>> ??????????? symChecker = null; >>>> ??????? } >>>> ??? } >>>> >>>> (linkCheckPaths comes from the -L option values) >>>> >>>> This behaviour is somewhat unexpected. At least a warning that >>>> missing an `-L` option will turn off symbol checking would be nice. >>>> >>>> We could also add the paths in `java.library.path` to the list of >>>> link check paths in jextract [2]. That would mean that the symbol >>>> checker would run for the example command. >>>> >>>> What do you think? >>>> >>>> Jorn >>>> >>>> [1] : >>>> http://hg.openjdk.java.net/panama/dev/file/eaca2d16b80b/src/jdk.jextract/share/classes/com/sun/tools/jextract/SymbolFilter.java#l89 >>>> [2] : >>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/jlibpath/webrev.01/ From maurizio.cimadamore at oracle.com Tue Jan 22 12:28:49 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Tue, 22 Jan 2019 12:28:49 +0000 Subject: hg: panama/dev: 8217453: add support for member annotation scheme Message-ID: <201901221228.x0MCSnVk025011@aojmv0008.oracle.com> Changeset: 9c4e3cc4ce5e Author: mcimadamore Date: 2019-01-22 12:28 +0000 URL: http://hg.openjdk.java.net/panama/dev/rev/9c4e3cc4ce5e 8217453: add support for member annotation scheme Reviewed-by: sundar ! src/java.base/share/classes/java/foreign/NativeMethodType.java ! src/java.base/share/classes/java/foreign/Scope.java + src/java.base/share/classes/java/foreign/annotations/NativeAddressof.java + src/java.base/share/classes/java/foreign/annotations/NativeFunction.java + src/java.base/share/classes/java/foreign/annotations/NativeGetter.java ! src/java.base/share/classes/java/foreign/annotations/NativeHeader.java + src/java.base/share/classes/java/foreign/annotations/NativeSetter.java ! src/java.base/share/classes/java/foreign/layout/Unresolved.java ! src/java.base/share/classes/java/foreign/memory/DoubleComplex.java ! src/java.base/share/classes/java/foreign/memory/FloatComplex.java ! src/java.base/share/classes/java/foreign/memory/LongDoubleComplex.java ! src/java.base/share/classes/java/foreign/memory/Pointer.java ! src/java.base/share/classes/jdk/internal/foreign/BinderClassGenerator.java ! src/java.base/share/classes/jdk/internal/foreign/BinderClassWriter.java ! src/java.base/share/classes/jdk/internal/foreign/HeaderImplGenerator.java ! src/java.base/share/classes/jdk/internal/foreign/LayoutResolver.java ! src/java.base/share/classes/jdk/internal/foreign/StructImplGenerator.java ! src/java.base/share/classes/jdk/internal/foreign/memory/DescriptorParser.java ! src/jdk.jextract/share/classes/com/sun/tools/jextract/AsmCodeFactory.java ! src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/LayoutUtils.java ! test/jdk/com/sun/tools/jextract/JextractToolProviderTest.java ! test/jdk/com/sun/tools/jextract/Runner.java ! test/jdk/com/sun/tools/jextract/TestDowncall.java ! test/jdk/com/sun/tools/jextract/TestUpcall.java ! test/jdk/com/sun/tools/jextract/compare/TypedefAnonStruct.java ! test/jdk/com/sun/tools/jextract/compare/bitfields.java ! test/jdk/com/sun/tools/jextract/compare/globalFuncPointer.java ! test/jdk/com/sun/tools/jextract/compare/pad.java ! test/jdk/com/sun/tools/jextract/compare/recursive.java ! test/jdk/com/sun/tools/jextract/compare/simple.java ! test/jdk/com/sun/tools/jextract/compare/windows/bitfields.java ! test/jdk/com/sun/tools/jextract/compare/windows/simple.java ! test/jdk/com/sun/tools/jextract/staticForwarder/StaticForwarderTest.java ! test/jdk/com/sun/tools/jextract/systemHeaders/SystemHeadersTest.java ! test/jdk/com/sun/tools/jextract/testStruct/StructTest.java ! test/jdk/java/foreign/BindLookupTest.java ! test/jdk/java/foreign/DuplicateStructs.java ! test/jdk/java/foreign/EmptyLayoutNameTest.java ! test/jdk/java/foreign/GlobalVariable.java ! test/jdk/java/foreign/Hello.java ! test/jdk/java/foreign/LongDoubleTest.java ! test/jdk/java/foreign/OutOfBoundsTest.java ! test/jdk/java/foreign/PaddedStructTest.java ! test/jdk/java/foreign/RegisterStructTest.java ! test/jdk/java/foreign/SignatureMismatchTest.java ! test/jdk/java/foreign/StdLibTest.java ! test/jdk/java/foreign/StructByValueTest.java ! test/jdk/java/foreign/System/UnixSystem.java - test/jdk/java/foreign/TestHeaderDeclarations.java ! test/jdk/java/foreign/Upcall/CallbackSort.java ! test/jdk/java/foreign/Upcall/DoubleUpcall.java ! test/jdk/java/foreign/Upcall/StructUpcall.java ! test/jdk/java/foreign/Upcall/Upcall.java ! test/jdk/java/foreign/getpid/unistd.java ! test/jdk/java/foreign/null/NullTest.java ! test/jdk/java/foreign/printf/stdio.java ! test/jdk/java/foreign/qsort/stdlib.java ! test/jdk/java/foreign/security/BindTest.java ! test/jdk/java/foreign/types/ArraysInFunctionsTest.java ! test/jdk/java/foreign/types/BitfieldsTest.java ! test/jdk/java/foreign/types/DeepAssignTest.java ! test/jdk/java/foreign/types/DescriptorTest.java ! test/jdk/java/foreign/types/FunctionAccessTest.java ! test/jdk/java/foreign/types/PointerScopeTest.java ! test/jdk/java/foreign/types/PointerTest.java ! test/jdk/java/foreign/types/StructTest.java From maurizio.cimadamore at oracle.com Tue Jan 22 12:35:16 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Tue, 22 Jan 2019 12:35:16 +0000 Subject: hg: panama/dev: Automatic merge with foreign Message-ID: <201901221235.x0MCZGbP027636@aojmv0008.oracle.com> Changeset: 33a7cf4c5eeb Author: mcimadamore Date: 2019-01-22 13:35 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/33a7cf4c5eeb Automatic merge with foreign - test/jdk/java/foreign/TestHeaderDeclarations.java From sundararajan.athijegannathan at oracle.com Tue Jan 22 13:05:12 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Tue, 22 Jan 2019 18:35:12 +0530 Subject: [foreign] RFR 8208081: jextract throws "java.lang.IllegalStateException: Partially used storage unit" In-Reply-To: <5C469BDD.50909@oracle.com> References: <5C469BDD.50909@oracle.com> Message-ID: <5C471508.8020700@oracle.com> * Cleaned-up the change in LayoutUtils.java a bit. * Added another test for struct type field followed by bitfield case Updated: https://cr.openjdk.java.net/~sundar/8208081/webrev.01/ Thanks -Sundar On 22/01/19, 9:58 AM, Sundararajan Athijegannathan wrote: > Please review. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8208081 > Webrev: https://cr.openjdk.java.net/~sundar/8208081/webrev.00/ > > Thanks, > -Sundar > From maurizio.cimadamore at oracle.com Tue Jan 22 13:20:41 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 22 Jan 2019 13:20:41 +0000 Subject: [foreign] RFR 8208081: jextract throws "java.lang.IllegalStateException: Partially used storage unit" In-Reply-To: <5C471508.8020700@oracle.com> References: <5C469BDD.50909@oracle.com> <5C471508.8020700@oracle.com> Message-ID: Looks good - and fixes the exception I was getting while extracting Python. Maurizio On 22/01/2019 13:05, Sundararajan Athijegannathan wrote: > * Cleaned-up the change in LayoutUtils.java a bit. > * Added another test for struct type field followed by bitfield case > > Updated: https://cr.openjdk.java.net/~sundar/8208081/webrev.01/ > > Thanks > -Sundar > > On 22/01/19, 9:58 AM, Sundararajan Athijegannathan wrote: >> Please review. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8208081 >> Webrev: https://cr.openjdk.java.net/~sundar/8208081/webrev.00/ >> >> Thanks, >> -Sundar >> From jbvernee at xs4all.nl Tue Jan 22 13:37:03 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Tue, 22 Jan 2019 14:37:03 +0100 Subject: [foreign] RFR 8208081: jextract throws "java.lang.IllegalStateException: Partially used storage unit" In-Reply-To: <5C471508.8020700@oracle.com> References: <5C469BDD.50909@oracle.com> <5C471508.8020700@oracle.com> Message-ID: Tests look good on Windows as well. Also fixes this exception when extracting Windows.h. Jorn Sundararajan Athijegannathan schreef op 2019-01-22 14:05: > * Cleaned-up the change in LayoutUtils.java a bit. > * Added another test for struct type field followed by bitfield case > > Updated: https://cr.openjdk.java.net/~sundar/8208081/webrev.01/ > > Thanks > -Sundar > > On 22/01/19, 9:58 AM, Sundararajan Athijegannathan wrote: >> Please review. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8208081 >> Webrev: https://cr.openjdk.java.net/~sundar/8208081/webrev.00/ >> >> Thanks, >> -Sundar >> From sundararajan.athijegannathan at oracle.com Tue Jan 22 14:04:31 2019 From: sundararajan.athijegannathan at oracle.com (sundararajan.athijegannathan at oracle.com) Date: Tue, 22 Jan 2019 14:04:31 +0000 Subject: hg: panama/dev: 8208081: jextract throws "java.lang.IllegalStateException: Partially used storage unit" Message-ID: <201901221404.x0ME4VWb011880@aojmv0008.oracle.com> Changeset: 9f79decafc06 Author: sundar Date: 2019-01-22 19:34 +0530 URL: http://hg.openjdk.java.net/panama/dev/rev/9f79decafc06 8208081: jextract throws "java.lang.IllegalStateException: Partially used storage unit" Reviewed-by: mcimadamore ! src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/LayoutUtils.java ! test/jdk/com/sun/tools/jextract/bitfields.h ! test/jdk/com/sun/tools/jextract/compare/bitfields.java ! test/jdk/com/sun/tools/jextract/compare/windows/bitfields.java ! test/jdk/java/foreign/OutOfBoundsTest.java From sundararajan.athijegannathan at oracle.com Tue Jan 22 14:06:24 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Tue, 22 Jan 2019 19:36:24 +0530 Subject: [foreign] RFR 8208081: jextract throws "java.lang.IllegalStateException: Partially used storage unit" In-Reply-To: References: <5C469BDD.50909@oracle.com> <5C471508.8020700@oracle.com> Message-ID: <5C472360.50508@oracle.com> Thanks for testing on Windows - nice to know it fixed an issue with jextracting Windows.h! -Sundar On 22/01/19, 7:07 PM, Jorn Vernee wrote: > Tests look good on Windows as well. > > Also fixes this exception when extracting Windows.h. > > Jorn > > Sundararajan Athijegannathan schreef op 2019-01-22 14:05: >> * Cleaned-up the change in LayoutUtils.java a bit. >> * Added another test for struct type field followed by bitfield case >> >> Updated: https://cr.openjdk.java.net/~sundar/8208081/webrev.01/ >> >> Thanks >> -Sundar >> >> On 22/01/19, 9:58 AM, Sundararajan Athijegannathan wrote: >>> Please review. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8208081 >>> Webrev: https://cr.openjdk.java.net/~sundar/8208081/webrev.00/ >>> >>> Thanks, >>> -Sundar >>> From maurizio.cimadamore at oracle.com Tue Jan 22 14:10:15 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Tue, 22 Jan 2019 14:10:15 +0000 Subject: hg: panama/dev: Automatic merge with foreign Message-ID: <201901221410.x0MEAGpc016366@aojmv0008.oracle.com> Changeset: 0fb48276c665 Author: mcimadamore Date: 2019-01-22 15:10 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/0fb48276c665 Automatic merge with foreign From maurizio.cimadamore at oracle.com Tue Jan 22 16:27:09 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 22 Jan 2019 16:27:09 +0000 Subject: scopes writeup Message-ID: <6fbff2eb-b100-0c03-7663-c732a3904302@oracle.com> Hi, at the end of last year I tried coming up with a proposal [1] which aimed at finding the basic concepts behind Panama scopes - I've been iterating (with the help of some internal feedback) on that proposal, and put it back together in a different form which, all things considered, it's not too far off from where we are now. https://cr.openjdk.java.net/~mcimadamore/panama/scopes.html That is, we still have a Scope abstraction - memory regions are still hidden under Scopes, and not part of the public API (in an attempt to limit the number of concepts exposed by the API - at least for now). Pointer is an immutable cursor into a memory region. Resources are just things with a scope (turns out, a native library is another such thing, since a native library needs a library-wide scope for allocating its own globals, etc.). On top of what we have now, this writeup pushes forward an improved ownership model, as well as some discussions concerning thread-confinement of scopes. I think this tweaked API proposal has the potential of fixing the current lifecycle issues, w/o making the API too complex to use. Additional abstraction (such as MemoryRegion) can be added back to the public API on a by need basis (e.g. provided relevant use cases). An experimental patch implementing the proposed approach can be found here: http://cr.openjdk.java.net/~mcimadamore/panama/scope-ownership_v2/ Comments welcome! Maurizio [1] - https://mail.openjdk.java.net/pipermail/panama-dev/2018-December/003605.html From henry.jen at oracle.com Tue Jan 22 16:35:26 2019 From: henry.jen at oracle.com (Henry Jen) Date: Tue, 22 Jan 2019 08:35:26 -0800 Subject: [foreign] RFC: Jextract -l options ignored by SymbolFilter when -L is not specified In-Reply-To: References: <698464e8f7692e12a7062155c7e599d9@xs4all.nl> <5C46AD23.9000207@oracle.com> Message-ID: > On Jan 22, 2019, at 3:55 AM, Jorn Vernee wrote: > > I see, that paints a clear picture thanks. So as far as libraries go: > > "-l" : record library name in @NativeHeader::libraries > "-rpath" : record runtime library path in @NativeHeader::libraryPaths > "-L" : do symbol checking using the libraries specified by "-l" > "--infer-rpath" : infer "-rpath" values from "-L" values > > The picture I had was: > > "-l" : record library name in @NativeHeader::libraries AND do symbol checking with this library > "-rpath" : record runtime library path in @NativeHeader::libraryPaths > "-L" : provide lookup paths to load libraries for "-l" > "--infer-rpath" : infer "-rpath" values from "-L? values > I am actually with Jorn on this. Cheers, Henry > The current "-L" semantics seem a little unintuitive to me to be honest, since "-L" also provide the lookup paths for "-l?. > > If depending on java.library.path to load libraries with "-l" is intended, I would propose replacing the "-L" option with something like a "--filter-symbols" option, and have users use java.library.path to provide the paths used to load the libraries for "-l", and then have "--infer-rpath" copy the paths from java.library.path instead. That way we have only one source of library paths (java.library.path). > > What do you think? > > Jorn > > Sundararajan Athijegannathan schreef op 2019-01-22 06:41: >> I don't think it is a bug - afaik it is as per design. The primary use >> of "-l" is to record the library in annotation of the generated jar - >> so that binder can auto-load the library (either from >> java.library.path configuration or -rpath value recorded in >> annotation). It is okay to record name of the shared object alone and >> leave the library path configuration to java.library.path setting. >> "-L" option is added feature to perform missing symbols checking. >> "-rpath" option is to add a path for library search - so that binder >> can locate the shared object in the specific directory. If no -rpath >> is specified, "-L" is used for runtime search as well. >> -Sundar >> On 22/01/19, 12:01 AM, Jorn Vernee wrote: >>> Hi, >>> I've recently updated the instructions for using libraries on Windows. For python the jextract example I gave was: >>> jextract -l python27 -o "python.jar" -t "org.python" C:\Python27\include\Python.h >>> I'm lacking an `-L` option here (for specifying library directories) since the contents of PATH seems to be added to java.library.path by default, and this is presumably also how jextract is able to load the library. But, since I'm not using an `-L` option, SymbolFilter is not checking if the symbols are in the python27.dll [1] >>> private void initSymChecker(List linkCheckPaths) { >>> if (!libraryNames.isEmpty() && !linkCheckPaths.isEmpty()) { >>> // ... init symChecker >>> } else { >>> symChecker = null; >>> } >>> } >>> (linkCheckPaths comes from the -L option values) >>> This behaviour is somewhat unexpected. At least a warning that missing an `-L` option will turn off symbol checking would be nice. >>> We could also add the paths in `java.library.path` to the list of link check paths in jextract [2]. That would mean that the symbol checker would run for the example command. >>> What do you think? >>> Jorn >>> [1] : http://hg.openjdk.java.net/panama/dev/file/eaca2d16b80b/src/jdk.jextract/share/classes/com/sun/tools/jextract/SymbolFilter.java#l89 >>> [2] : http://cr.openjdk.java.net/~jvernee/panama/webrevs/jlibpath/webrev.01/ From henry.jen at oracle.com Tue Jan 22 16:40:41 2019 From: henry.jen at oracle.com (Henry Jen) Date: Tue, 22 Jan 2019 08:40:41 -0800 Subject: [foreign] RFC: Jextract -l options ignored by SymbolFilter when -L is not specified In-Reply-To: <4de2ba3b-4971-95d3-cb3b-9a0d478656d9@oracle.com> References: <698464e8f7692e12a7062155c7e599d9@xs4all.nl> <5C46AD23.9000207@oracle.com> <4089850d-fec6-40cf-c036-e666169a84b3@oracle.com> <30080ab5a99a1a9abf34a89a6c70b8b8@xs4all.nl> <4de2ba3b-4971-95d3-cb3b-9a0d478656d9@oracle.com> Message-ID: <036DADBC-851D-494F-8FC0-094505BE5696@oracle.com> It is preferred to keep options compatible with cc when applicable. -L is only to provide the path for library at tooling time, that?s jextract. ?infer-path is similar to -R, will record the path for searching at runtime, that?s is, the path specified with -L will be added into search path of library. As symbol check, it should be enabled with -l. -L is simply provide extra path to search for the library, without -L, it will simply search in java.library.path. Cheers, Henry > On Jan 22, 2019, at 4:31 AM, Maurizio Cimadamore wrote: > > > On 22/01/2019 12:09, Jorn Vernee wrote: >> This sounds good, I really like the idea of a separate option to enable the symbol filtering. But can you share what you think the role of java.library.path should be as well? > > I think using java.library.path as a default for the missing symbol check could be ok. But I don't think it would be ok to use it as a basis for infer-rpath. That is, I don't want static properties (e.g. valid at extraction time) to spill onto the runtime. If the user really wants to set some dynamic property, it has to use an explicit flag to do so (e.g. -L). > > Maurizio > >> >> Jorn >> >> Maurizio Cimadamore schreef op 2019-01-22 12:58: >>> Looking at this, I remember being confused about this too. >>> >>> Let me try to see if we can find a better stacking for the existing >>> options - as Sundar said, we currently have: >>> >>> * -l >>> >>> This option is used to specify library _names_. >>> >>> The main goal of this option is to alter the contents of the >>> @NativeHeader annotation (by adding the library name) but there are, >>> as we shall see, other subtle side-effects. >>> >>> * -L + -l >>> >>> When both -L and -l are specified, the so called "missing symbols >>> check" will kick in, that is, jextract will check that all symbols in >>> the library are indeed defined in the header files being extracted. A >>> subtle side-effect of that check, is that when -l and -L are specified >>> together, and the missing symbol check is enabled, jextract will warn >>> for symbols not found and _it will exclude them_ from the extracted >>> classfile (w/o need for --include-symbols or --exclude-symbols). >>> >>> * -L + -l + -infer-rpath >>> >>> When -L and -l are used together, and the -infer-rpath option is >>> given, a runtime library path will be inferred from the contents of >>> -L, and will be stored in @NativeHeader, so that the binder can use >>> it. >>> >>> >>> I think the status quo is a bit confusing - because -L has multiple >>> functions (it serves up the library paths to be used as inferred >>> rpaths, and it also serves up the library paths to be used for the >>> missing symbol check). I think a more consistent stacking could be >>> something like this: >>> >>> -l --> used to specify library _names_; only side-effect is contents >>> of @NativeHeader >>> >>> -L --> used to specify _custom_ library _paths_; no side-effects >>> >>> -exclude-missing -> must be used in conjunction with -l and -L ; >>> enables the missing symbol check and auto-exclusion >>> >>> -infer-rpath -> must be used in conjunction with -l and -L ; enables >>> rpath inference (rpath inferred with paths specified in -L) >>> >>> >>> Thoughts? >>> >>> Maurizio >>> >>> >>> On 22/01/2019 05:41, Sundararajan Athijegannathan wrote: >>>> I don't think it is a bug - afaik it is as per design. The primary use of "-l" is to record the library in annotation of the generated jar - so that binder can auto-load the library (either from java.library.path configuration or -rpath value recorded in annotation). It is okay to record name of the shared object alone and leave the library path configuration to java.library.path setting. >>>> >>>> "-L" option is added feature to perform missing symbols checking. "-rpath" option is to add a path for library search - so that binder can locate the shared object in the specific directory. If no -rpath is specified, "-L" is used for runtime search as well. >>>> >>>> -Sundar >>>> >>>> On 22/01/19, 12:01 AM, Jorn Vernee wrote: >>>>> Hi, >>>>> >>>>> I've recently updated the instructions for using libraries on Windows. For python the jextract example I gave was: >>>>> >>>>> jextract -l python27 -o "python.jar" -t "org.python" C:\Python27\include\Python.h >>>>> >>>>> I'm lacking an `-L` option here (for specifying library directories) since the contents of PATH seems to be added to java.library.path by default, and this is presumably also how jextract is able to load the library. But, since I'm not using an `-L` option, SymbolFilter is not checking if the symbols are in the python27.dll [1] >>>>> >>>>> private void initSymChecker(List linkCheckPaths) { >>>>> if (!libraryNames.isEmpty() && !linkCheckPaths.isEmpty()) { >>>>> // ... init symChecker >>>>> } else { >>>>> symChecker = null; >>>>> } >>>>> } >>>>> >>>>> (linkCheckPaths comes from the -L option values) >>>>> >>>>> This behaviour is somewhat unexpected. At least a warning that missing an `-L` option will turn off symbol checking would be nice. >>>>> >>>>> We could also add the paths in `java.library.path` to the list of link check paths in jextract [2]. That would mean that the symbol checker would run for the example command. >>>>> >>>>> What do you think? >>>>> >>>>> Jorn >>>>> >>>>> [1] : http://hg.openjdk.java.net/panama/dev/file/eaca2d16b80b/src/jdk.jextract/share/classes/com/sun/tools/jextract/SymbolFilter.java#l89 >>>>> [2] : http://cr.openjdk.java.net/~jvernee/panama/webrevs/jlibpath/webrev.01/ From jbvernee at xs4all.nl Tue Jan 22 19:20:43 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Tue, 22 Jan 2019 20:20:43 +0100 Subject: [foreign] RFC: Jextract -l options ignored by SymbolFilter when -L is not specified In-Reply-To: <036DADBC-851D-494F-8FC0-094505BE5696@oracle.com> References: <698464e8f7692e12a7062155c7e599d9@xs4all.nl> <5C46AD23.9000207@oracle.com> <4089850d-fec6-40cf-c036-e666169a84b3@oracle.com> <30080ab5a99a1a9abf34a89a6c70b8b8@xs4all.nl> <4de2ba3b-4971-95d3-cb3b-9a0d478656d9@oracle.com> <036DADBC-851D-494F-8FC0-094505BE5696@oracle.com> Message-ID: <2f86de9e0e566913c1130e89556c541e@xs4all.nl> This also seems the most natural to me, since it follows what the linker flags do. -L is to specify additional linker directories. We would consider java.library.path to be the "default"/"system" directories. This is also what I tried to do in the patch [1] (minor update). With the addition of emitting a warning that symbol filtering is disabled when -l is used but no library paths are available (either in -L options or in java.library.path). That said, I think having an extra option to explicitly turn on, or off, the symbol checking is a good idea as well. Jorn [1]: http://cr.openjdk.java.net/~jvernee/panama/webrevs/jlibpath/webrev.02/ Henry Jen schreef op 2019-01-22 17:40: > It is preferred to keep options compatible with cc when applicable. > > -L is only to provide the path for library at tooling time, that?s > jextract. > ?infer-path is similar to -R, will record the path for searching at > runtime, that?s is, the path specified with -L will be added into > search path of library. > > As symbol check, it should be enabled with -l. -L is simply provide > extra path to search for the library, without -L, it will simply > search in java.library.path. > > Cheers, > Henry > > >> On Jan 22, 2019, at 4:31 AM, Maurizio Cimadamore >> wrote: >> >> >> On 22/01/2019 12:09, Jorn Vernee wrote: >>> This sounds good, I really like the idea of a separate option to >>> enable the symbol filtering. But can you share what you think the >>> role of java.library.path should be as well? >> >> I think using java.library.path as a default for the missing symbol >> check could be ok. But I don't think it would be ok to use it as a >> basis for infer-rpath. That is, I don't want static properties (e.g. >> valid at extraction time) to spill onto the runtime. If the user >> really wants to set some dynamic property, it has to use an explicit >> flag to do so (e.g. -L). >> >> Maurizio >> >>> >>> Jorn >>> >>> Maurizio Cimadamore schreef op 2019-01-22 12:58: >>>> Looking at this, I remember being confused about this too. >>>> >>>> Let me try to see if we can find a better stacking for the existing >>>> options - as Sundar said, we currently have: >>>> >>>> * -l >>>> >>>> This option is used to specify library _names_. >>>> >>>> The main goal of this option is to alter the contents of the >>>> @NativeHeader annotation (by adding the library name) but there are, >>>> as we shall see, other subtle side-effects. >>>> >>>> * -L + -l >>>> >>>> When both -L and -l are specified, the so called "missing symbols >>>> check" will kick in, that is, jextract will check that all symbols >>>> in >>>> the library are indeed defined in the header files being extracted. >>>> A >>>> subtle side-effect of that check, is that when -l and -L are >>>> specified >>>> together, and the missing symbol check is enabled, jextract will >>>> warn >>>> for symbols not found and _it will exclude them_ from the extracted >>>> classfile (w/o need for --include-symbols or --exclude-symbols). >>>> >>>> * -L + -l + -infer-rpath >>>> >>>> When -L and -l are used together, and the -infer-rpath option is >>>> given, a runtime library path will be inferred from the contents of >>>> -L, and will be stored in @NativeHeader, so that the binder can use >>>> it. >>>> >>>> >>>> I think the status quo is a bit confusing - because -L has multiple >>>> functions (it serves up the library paths to be used as inferred >>>> rpaths, and it also serves up the library paths to be used for the >>>> missing symbol check). I think a more consistent stacking could be >>>> something like this: >>>> >>>> -l --> used to specify library _names_; only side-effect is contents >>>> of @NativeHeader >>>> >>>> -L --> used to specify _custom_ library _paths_; no side-effects >>>> >>>> -exclude-missing -> must be used in conjunction with -l and -L ; >>>> enables the missing symbol check and auto-exclusion >>>> >>>> -infer-rpath -> must be used in conjunction with -l and -L ; enables >>>> rpath inference (rpath inferred with paths specified in -L) >>>> >>>> >>>> Thoughts? >>>> >>>> Maurizio >>>> >>>> >>>> On 22/01/2019 05:41, Sundararajan Athijegannathan wrote: >>>>> I don't think it is a bug - afaik it is as per design. The primary >>>>> use of "-l" is to record the library in annotation of the generated >>>>> jar - so that binder can auto-load the library (either from >>>>> java.library.path configuration or -rpath value recorded in >>>>> annotation). It is okay to record name of the shared object alone >>>>> and leave the library path configuration to java.library.path >>>>> setting. >>>>> >>>>> "-L" option is added feature to perform missing symbols checking. >>>>> "-rpath" option is to add a path for library search - so that >>>>> binder can locate the shared object in the specific directory. If >>>>> no -rpath is specified, "-L" is used for runtime search as well. >>>>> >>>>> -Sundar >>>>> >>>>> On 22/01/19, 12:01 AM, Jorn Vernee wrote: >>>>>> Hi, >>>>>> >>>>>> I've recently updated the instructions for using libraries on >>>>>> Windows. For python the jextract example I gave was: >>>>>> >>>>>> jextract -l python27 -o "python.jar" -t "org.python" >>>>>> C:\Python27\include\Python.h >>>>>> >>>>>> I'm lacking an `-L` option here (for specifying library >>>>>> directories) since the contents of PATH seems to be added to >>>>>> java.library.path by default, and this is presumably also how >>>>>> jextract is able to load the library. But, since I'm not using an >>>>>> `-L` option, SymbolFilter is not checking if the symbols are in >>>>>> the python27.dll [1] >>>>>> >>>>>> private void initSymChecker(List linkCheckPaths) { >>>>>> if (!libraryNames.isEmpty() && !linkCheckPaths.isEmpty()) >>>>>> { >>>>>> // ... init symChecker >>>>>> } else { >>>>>> symChecker = null; >>>>>> } >>>>>> } >>>>>> >>>>>> (linkCheckPaths comes from the -L option values) >>>>>> >>>>>> This behaviour is somewhat unexpected. At least a warning that >>>>>> missing an `-L` option will turn off symbol checking would be >>>>>> nice. >>>>>> >>>>>> We could also add the paths in `java.library.path` to the list of >>>>>> link check paths in jextract [2]. That would mean that the symbol >>>>>> checker would run for the example command. >>>>>> >>>>>> What do you think? >>>>>> >>>>>> Jorn >>>>>> >>>>>> [1] : >>>>>> http://hg.openjdk.java.net/panama/dev/file/eaca2d16b80b/src/jdk.jextract/share/classes/com/sun/tools/jextract/SymbolFilter.java#l89 >>>>>> [2] : >>>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/jlibpath/webrev.01/ From jbvernee at xs4all.nl Tue Jan 22 19:48:12 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Tue, 22 Jan 2019 20:48:12 +0100 Subject: [foreign] RFR 8217462: Add a jextract option to not generate typedef annotation interfaces Message-ID: Hi, From the bug description: Currently jextract generates annotation interfaces for typedefs. But, using these annotations is not required to call the native functions defined in a library. Rather, they are there to add additional metadata to java carrier types. Since use of these annotations interfaces is optional, it would be nice to have a jextract option to turn of their generation, in order to simplify generated classes (especially useful when viewing them with a decompiler/javap) Please review the following. Bug: https://bugs.openjdk.java.net/browse/JDK-8217462 Webrev: http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217462/webrev.00/ Thanks, Jorn From maurizio.cimadamore at oracle.com Tue Jan 22 21:10:00 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 22 Jan 2019 21:10:00 +0000 Subject: [foreign] RFR 8217462: Add a jextract option to not generate typedef annotation interfaces In-Reply-To: References: Message-ID: <158402ae-9f55-9dd7-66ad-8805ef550e33@oracle.com> Hi Jorn, while I understand where you are coming from, I'd like not to fragment the space of the jextract options too much - as discussed earlier today, there is already enough confusion over existing options, and we should clear that before adding new ones :-) Also, if we go down this path, I'm afraid that this will turn into having a full menu of options to support this or that generation tweak - this sort of things tend to add up, the options will likely be (ab)used in any possible configuration, with the result that the jextract output will be perceived as something not stable. I have also a sense that your request is coming in part from issues that were pointed out few days ago - e.g. jextract is currently generating too much stuff - even things that have nothing to do with the library being extracted (but which is recursively included by the headers). As we have said then, the solution for this problem is to adopt a library-centric approach, not adding custom flags to selectively switch off extraction features. Maurizio On 22/01/2019 19:48, Jorn Vernee wrote: > Hi, > > From the bug description: > > Currently jextract generates annotation interfaces for typedefs. But, > using these annotations is not required to call the native functions > defined in a library. Rather, they are there to add additional > metadata to java carrier types. > > Since use of these annotations interfaces is optional, it would be > nice to have a jextract option to turn of their generation, in order > to simplify generated classes (especially useful when viewing them > with a decompiler/javap) > > Please review the following. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8217462 > Webrev: > http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217462/webrev.00/ > > Thanks, > Jorn From john.r.rose at oracle.com Tue Jan 22 21:21:07 2019 From: john.r.rose at oracle.com (John Rose) Date: Tue, 22 Jan 2019 13:21:07 -0800 Subject: access modes for pointers and memory regions Message-ID: <60B00FE8-1B85-4C97-9ECC-CB4DD106F90C@oracle.com> `Pointer.AccessMode` tells whether a pointer is allowed to read and/or write the memory behind the pointer. http://hg.openjdk.java.net/panama/dev/file/9c4e3cc4ce5e/src/java.base/share/classes/java/foreign/memory/Pointer.java#l233 The access mode of a pointer is not meant to provide unambiguous information about the memory behind the pointer, but rather to restrict the operations on that memory *via that pointer*. The pointer acts as a capability to read and/or write the memory, but there may be other capabilities that can do things the pointer cannot. Holding a pointer with mode `AccessMode.READ` does not prevent another pointer from writing the same memory. I'm belaboring the obvious here, but it's for a good cause? A `BoundedMemoryRegion` has a mode flag also: http://hg.openjdk.java.net/panama/dev/file/9081e5f050d7/src/java.base/share/classes/jdk/internal/foreign/memory/BoundedMemoryRegion.java#l46 The meaning of this mode flag, like that of the pointer's mode flag, restricts the capability represented by the memory region. It is not an absolute statement about the memory referred to by the region. For example, a queue head might be implemented with a segment of memory, which one end of the queue writes and the other reads, and both pointers and memory regions might refer to that memory, but with disjoint capabilities; one side reads only and one side writes only, but the memory itself, apart from any capabilities that access it, is best described as `AccessMode.READ_WRITE`. If a memory region is marked as `READ` there is nothing to prevent other parties, with write permissions, from writing to the same memory. This is a useful quality, as in the case of queues, or with divide-and-conquer parallel algorithms, where each thread can read a large region of memory but only one thread may write some assigned sub-region which the other threads may only read. The absolute status of a segment of memory may be read/write, read-only, or write-only. That last category makes sense only in limited settings, for interprocess shared memory. But the first two are widespread. Examples of absolutely read/write memory are obvious, starting with any Java array or malloc buffer, unless either is protected somehow against writes. Standard examples of read-only memory include the characters of a Java string or of a C char array placed in (virtual) ROM. Thus we may use modes like `AccessMode.READ` either in an absolute or in a relative sense, and there is some wiggle room for the same object having differing modes in the two senses. Some combinations probably don't make sense, like a read/write pointer into a read-only memory segment. Here are the useful combinations, I think: R/RO: I can access this immutable segment. R/RW: I can read but not write this writable segment. W/RW: I can write but not read this writable segment. RW/RW: I have full access to this segment. The only combination not representable with `AccessMode` is the first (R/RO). One way to elevate it as an explicit property would be like this: ``` enum AccessMode { /** * A read access mode. * May be aliased to writable content. */ READ(1 << 0), /** * A write access mode. * May be aliased to readable content. */ WRITE(1 << 1), /** * A read access mode. * Never aliased to writable content. * Only content with this access * is guaranteed to be immutable. */ IMMUTABLE(READ.value | (1 << 2)), /** * A read and write access mode. * May be aliased with read or write modes. */ READ_WRITE(READ.value | WRITE.value); } ``` Or pointers and memory regions could be given semi-independent queries `isAccessibleFor(m)` and `memorySegmentIsAccessibleFor(m)`. Here's why I'm shining a light on this corner case: Unless there is a formalized and explicit indication of when memory is absolutely read-only, we will not be able to extend Java's safe publication guarantees to objects which are created by Panama-based factories. If you build a data structure and then share it with other threads, there are several things that can go badly wrong. But if your VM is willing to keep track of which memory segments are shareable, you can catch bugs which otherwise could go undetected. Of course, the kind of "keeping track" I mean here includes the following (none of which are new ideas): - thread-confined mutable data (scopes, etc.) - thread-shareable mutable data (lockable owner metadata) - thread-shareable frozen data (strings, ROM, etc.) - defensive copies of input data The last one, defensive copies, should be part of the API of pointers. A defensive copy should convert a mutable segment into a freshly copied immutable or confined private copy. (There are two ways to do defensive copies: We might call them "clone" and "freeze" in the setting of Java objects.) The defensive copy should be idempotent on frozen objects, so that repeated freezing is only as expensive as the making of the first copy. Getting the details right on this will allow native data to interoperate better with Java's multi-threaded APIs. Getting the details right requires a little extra attention to the the difference between a read-only capability and an absolutely read-only memory segment. Here's a sketch of the sort of thing I mean. http://cr.openjdk.java.net/~jrose/panama/immutable-sketch The sketch also includes better bounds checking for on-heap memory regions. ? John From jbvernee at xs4all.nl Tue Jan 22 21:31:30 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Tue, 22 Jan 2019 22:31:30 +0100 Subject: [foreign] RFR 8217462: Add a jextract option to not generate typedef annotation interfaces In-Reply-To: <158402ae-9f55-9dd7-66ad-8805ef550e33@oracle.com> References: <158402ae-9f55-9dd7-66ad-8805ef550e33@oracle.com> Message-ID: <9b2b224b0e4cc96735cb72fae5d3c5aa@xs4all.nl> Hi Maurizio, Thanks for the feedback. Your arguments are compelling (as always). Indeed, part of the problem is that typedefs are not being filtered based on library currently. Maybe I'm wrong, but I expect there is an intent to add the typedef annotations to get generated function signatures as well? At that point having a --no-typdef option to stop the annotations from being generated might be more useful. Otherwise, while having a full menu of options in stock jextract might not be desirable, maybe extra tree phases could be added through some plugin API? Or is jextract intended to be kept simpler than that? Jorn Maurizio Cimadamore schreef op 2019-01-22 22:10: > Hi Jorn, > while I understand where you are coming from, I'd like not to fragment > the space of the jextract options too much - as discussed earlier > today, there is already enough confusion over existing options, and we > should clear that before adding new ones :-) > > Also, if we go down this path, I'm afraid that this will turn into > having a full menu of options to support this or that generation tweak > - this sort of things tend to add up, the options will likely be > (ab)used in any possible configuration, with the result that the > jextract output will be perceived as something not stable. > > I have also a sense that your request is coming in part from issues > that were pointed out few days ago - e.g. jextract is currently > generating too much stuff - even things that have nothing to do with > the library being extracted (but which is recursively included by the > headers). As we have said then, the solution for this problem is to > adopt a library-centric approach, not adding custom flags to > selectively switch off extraction features. > > Maurizio > > On 22/01/2019 19:48, Jorn Vernee wrote: >> Hi, >> >> From the bug description: >> >> Currently jextract generates annotation interfaces for typedefs. But, >> using these annotations is not required to call the native functions >> defined in a library. Rather, they are there to add additional >> metadata to java carrier types. >> >> Since use of these annotations interfaces is optional, it would be >> nice to have a jextract option to turn of their generation, in order >> to simplify generated classes (especially useful when viewing them >> with a decompiler/javap) >> >> Please review the following. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8217462 >> Webrev: >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217462/webrev.00/ >> >> Thanks, >> Jorn From maurizio.cimadamore at oracle.com Tue Jan 22 21:56:44 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 22 Jan 2019 21:56:44 +0000 Subject: access modes for pointers and memory regions In-Reply-To: <60B00FE8-1B85-4C97-9ECC-CB4DD106F90C@oracle.com> References: <60B00FE8-1B85-4C97-9ECC-CB4DD106F90C@oracle.com> Message-ID: <796b0698-c080-a62b-c922-ce054e56c09e@oracle.com> Thanks for the great analysis. I have few questions... You say that we have AccessFlags on both Pointer and MemoryRegion, but in reality pointer does not really have such a flag. When you do Pointer::asReadOnly, all you get is a new pointer which points to a memory region that is the copy of the existing one, but with read-only access. So, I'd say the current model is essentially that of having a single access flag on memory region. The fact that memory region is not exposed in the API is one of the reasons as to why the 'asReadOnly' methods ended up in pointer. I agree that, in principle, we might have R/W/RW on pointers too - where perhaps what happens is that they affect dereference operations (e.g. Pointer::set will fail on a R pointer, regardless of the underlying memory being RW). To be honest, I don't like where the code is right now, and I'm also not a fan of the 'immutable' access flag. The problem being that I think access mode should be a property of the pointer, as you say in the very first line of your email. So, getting a read-only view of a pointer should, I believe, _have no effects_ on the underlying memory region - it should just return a new pointer with the new capability. That is, we should stop this business of recreating the memory region only to update an access flag - this is, I think, what gets in the way of offering a truly 'non-writeable' memory region (which you attempt to solve with the 'immutable' flag). If memory regions had a _non-upgradeable_ R/W/RW property, then you could really allocate a memory region that is read-only, if you wanted to. Any attempt to get a write-capable pointer out of that will fail. (in other words,, you can get a pointer with access X on a memory region with access Y only if X is equal or less strict than Y). So, I think that, of the two options you consider, I'm more for the quasi-independent access queries for pointer/memory region, as that seems a cleaner cut to me. But I agree with your line of reasoning - having some support for immutable memory regions seems required in order to support things like returning struct as value in the heap from native functions. Maurizio On 22/01/2019 21:21, John Rose wrote: > `Pointer.AccessMode` tells whether a pointer is allowed to read > and/or write the memory behind the pointer. > > http://hg.openjdk.java.net/panama/dev/file/9c4e3cc4ce5e/src/java.base/share/classes/java/foreign/memory/Pointer.java#l233 > > The access mode of a pointer is not meant to provide unambiguous > information about the memory behind the pointer, but rather to > restrict the operations on that memory *via that pointer*. The > pointer acts as a capability to read and/or write the memory, > but there may be other capabilities that can do things the > pointer cannot. Holding a pointer with mode `AccessMode.READ` > does not prevent another pointer from writing the same memory. > I'm belaboring the obvious here, but it's for a good cause? > > A `BoundedMemoryRegion` has a mode flag also: > > http://hg.openjdk.java.net/panama/dev/file/9081e5f050d7/src/java.base/share/classes/jdk/internal/foreign/memory/BoundedMemoryRegion.java#l46 > > The meaning of this mode flag, like that of the pointer's > mode flag, restricts the capability represented by the memory > region. It is not an absolute statement about the memory > referred to by the region. For example, a queue head > might be implemented with a segment of memory, which > one end of the queue writes and the other reads, and > both pointers and memory regions might refer to that > memory, but with disjoint capabilities; one side reads > only and one side writes only, but the memory itself, > apart from any capabilities that access it, is best > described as `AccessMode.READ_WRITE`. > > If a memory region is marked as `READ` there is > nothing to prevent other parties, with write permissions, > from writing to the same memory. > > This is a useful quality, as in the case of queues, > or with divide-and-conquer parallel algorithms, > where each thread can read a large region of memory > but only one thread may write some assigned sub-region > which the other threads may only read. > > The absolute status of a segment of memory may be > read/write, read-only, or write-only. That last > category makes sense only in limited settings, for > interprocess shared memory. But the first two are > widespread. > > Examples of absolutely read/write memory are obvious, > starting with any Java array or malloc buffer, unless either > is protected somehow against writes. Standard > examples of read-only memory include the > characters of a Java string or of a C char array > placed in (virtual) ROM. > > Thus we may use modes like `AccessMode.READ` > either in an absolute or in a relative sense, and > there is some wiggle room for the same object > having differing modes in the two senses. > Some combinations probably don't make sense, > like a read/write pointer into a read-only memory > segment. > > Here are the useful combinations, I think: > > R/RO: I can access this immutable segment. > R/RW: I can read but not write this writable segment. > W/RW: I can write but not read this writable segment. > RW/RW: I have full access to this segment. > > The only combination not representable with > `AccessMode` is the first (R/RO). One way to > elevate it as an explicit property would be like > this: > > ``` > enum AccessMode { > /** > * A read access mode. > * May be aliased to writable content. > */ > READ(1 << 0), > /** > * A write access mode. > * May be aliased to readable content. > */ > WRITE(1 << 1), > /** > * A read access mode. > * Never aliased to writable content. > * Only content with this access > * is guaranteed to be immutable. > */ > IMMUTABLE(READ.value | (1 << 2)), > /** > * A read and write access mode. > * May be aliased with read or write modes. > */ > READ_WRITE(READ.value | WRITE.value); > } > ``` > > Or pointers and memory regions could be given > semi-independent queries `isAccessibleFor(m)` > and `memorySegmentIsAccessibleFor(m)`. > > Here's why I'm shining a light on this corner case: > Unless there is a formalized and explicit indication > of when memory is absolutely read-only, we will > not be able to extend Java's safe publication guarantees > to objects which are created by Panama-based factories. > If you build a data structure and then share it with > other threads, there are several things that can go > badly wrong. But if your VM is willing to keep track > of which memory segments are shareable, you can > catch bugs which otherwise could go undetected. > > Of course, the kind of "keeping track" I mean here > includes the following (none of which are new ideas): > > - thread-confined mutable data (scopes, etc.) > - thread-shareable mutable data (lockable owner metadata) > - thread-shareable frozen data (strings, ROM, etc.) > - defensive copies of input data > > The last one, defensive copies, should be part of the > API of pointers. A defensive copy should convert > a mutable segment into a freshly copied immutable > or confined private copy. (There are two ways to > do defensive copies: We might call them "clone" > and "freeze" in the setting of Java objects.) The > defensive copy should be idempotent on frozen > objects, so that repeated freezing is only as expensive > as the making of the first copy. > > Getting the details right on this will allow native > data to interoperate better with Java's multi-threaded > APIs. Getting the details right requires a little extra > attention to the the difference between a read-only > capability and an absolutely read-only memory segment. > > Here's a sketch of the sort of thing I mean. > > http://cr.openjdk.java.net/~jrose/panama/immutable-sketch > > The sketch also includes better bounds checking for > on-heap memory regions. > > ? John From maurizio.cimadamore at oracle.com Tue Jan 22 21:59:54 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 22 Jan 2019 21:59:54 +0000 Subject: [foreign] RFR 8217462: Add a jextract option to not generate typedef annotation interfaces In-Reply-To: <9b2b224b0e4cc96735cb72fae5d3c5aa@xs4all.nl> References: <158402ae-9f55-9dd7-66ad-8805ef550e33@oracle.com> <9b2b224b0e4cc96735cb72fae5d3c5aa@xs4all.nl> Message-ID: <20cc6683-cd63-717d-9bcd-d6dd7327236e@oracle.com> On 22/01/2019 21:31, Jorn Vernee wrote: > Hi Maurizio, > > Thanks for the feedback. > > Your arguments are compelling (as always). Indeed, part of the problem > is that typedefs are not being filtered based on library currently. > > Maybe I'm wrong, but I expect there is an intent to add the typedef > annotations to get generated function signatures as well? At that > point having a --no-typdef option to stop the annotations from being > generated might be more useful. Yeah - the typedef annos should be generated there too as these annotations can be inspected with the IDE. > > Otherwise, while having a full menu of options in stock jextract might > not be desirable, maybe extra tree phases could be added through some > plugin API? Or is jextract intended to be kept simpler than that? Some plugin story would be desirable at some point, as that would pave the way to do customizations (e.g. some of the things under the 'civilization' umbrella, for instance). But plugins is the last piece of the puzzle - that is, first we need to come up with a jextract workflow that is satisfactory - when we're there we can start thinking about opening up pieces of the jextract pipeline. Maurizio > > Jorn > > Maurizio Cimadamore schreef op 2019-01-22 22:10: >> Hi Jorn, >> while I understand where you are coming from, I'd like not to fragment >> the space of the jextract options too much - as discussed earlier >> today, there is already enough confusion over existing options, and we >> should clear that before adding new ones :-) >> >> Also, if we go down this path, I'm afraid that this will turn into >> having a full menu of options to support this or that generation tweak >> - this sort of things tend to add up, the options will likely be >> (ab)used in any possible configuration, with the result that the >> jextract output will be perceived as something not stable. >> >> I have also a sense that your request is coming in part from issues >> that were pointed out few days ago - e.g. jextract is currently >> generating too much stuff - even things that have nothing to do with >> the library being extracted (but which is recursively included by the >> headers). As we have said then, the solution for this problem is to >> adopt a library-centric approach, not adding custom flags to >> selectively switch off extraction features. >> >> Maurizio >> >> On 22/01/2019 19:48, Jorn Vernee wrote: >>> Hi, >>> >>> From the bug description: >>> >>> Currently jextract generates annotation interfaces for typedefs. >>> But, using these annotations is not required to call the native >>> functions defined in a library. Rather, they are there to add >>> additional metadata to java carrier types. >>> >>> Since use of these annotations interfaces is optional, it would be >>> nice to have a jextract option to turn of their generation, in order >>> to simplify generated classes (especially useful when viewing them >>> with a decompiler/javap) >>> >>> Please review the following. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8217462 >>> Webrev: >>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217462/webrev.00/ >>> >>> Thanks, >>> Jorn From sandhya.viswanathan at intel.com Tue Jan 22 22:24:04 2019 From: sandhya.viswanathan at intel.com (sandhya.viswanathan at intel.com) Date: Tue, 22 Jan 2019 22:24:04 +0000 Subject: hg: panama/dev: Summary: Support long addAll() for AVX < 3 Message-ID: <201901222224.x0MMO5Zu001993@aojmv0008.oracle.com> Changeset: 0447b2b7d707 Author: sviswanathan Date: 2019-01-21 16:12 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/0447b2b7d707 Summary: Support long addAll() for AVX < 3 Reviewed-by: vlivanov ! src/hotspot/cpu/x86/x86.ad From maurizio.cimadamore at oracle.com Tue Jan 22 22:30:22 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Tue, 22 Jan 2019 22:30:22 +0000 Subject: hg: panama/dev: Automatic merge with vectorIntrinsics Message-ID: <201901222230.x0MMUNJ6004931@aojmv0008.oracle.com> Changeset: 4a749e5fc8ce Author: mcimadamore Date: 2019-01-22 23:30 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/4a749e5fc8ce Automatic merge with vectorIntrinsics From john.r.rose at oracle.com Tue Jan 22 22:45:18 2019 From: john.r.rose at oracle.com (John Rose) Date: Tue, 22 Jan 2019 14:45:18 -0800 Subject: access modes for pointers and memory regions In-Reply-To: <796b0698-c080-a62b-c922-ce054e56c09e@oracle.com> References: <60B00FE8-1B85-4C97-9ECC-CB4DD106F90C@oracle.com> <796b0698-c080-a62b-c922-ce054e56c09e@oracle.com> Message-ID: <8E6C9030-8E00-45BF-AA8F-294DBDFBB212@oracle.com> On Jan 22, 2019, at 1:56 PM, Maurizio Cimadamore wrote: > > Thanks for the great analysis. > > I have few questions... > > You say that we have AccessFlags on both Pointer and MemoryRegion, but in reality pointer does not really have such a flag. When you do Pointer::asReadOnly, all you get is a new pointer which points to a memory region that is the copy of the existing one, but with read-only access. > > So, I'd say the current model is essentially that of having a single access flag on memory region. The fact that memory region is not exposed in the API is one of the reasons as to why the 'asReadOnly' methods ended up in pointer. Yes, the mode is (a) exposed only indirectly, and (b) deferred to the memory region by a BoundedPointer. If the pointer exposed its MR as a first-class entity, then it would be clear that the mode is on the MR only. As it is, the AM type is coupled to both APIs, even though it appears in only one implementation. The more important point is that AMs are always relative to a capability, never absolute on a memory segment, and this is where the disconnect is when we try to reason about safe immutability. In terms of C-like languages, "immutable" is not the same as "const". > I agree that, in principle, we might have R/W/RW on pointers too - where perhaps what happens is that they affect dereference operations (e.g. Pointer::set will fail on a R pointer, regardless of the underlying memory being RW). I'm not proposing that; what I really want is a way to make and check immutable copies. And that seems to require a fourth access type, when you do the math. I could see painting that bikeshed several ways: IMMUTABLE, GLOBAL_READ, ABSOLUTE_READ, READ_ONLY, etc. Also there could be (as I suggested) a relative R/W/RW mode flag on view objects as distinct from the absolute R/RW mode flag on content objects. But the concept of content object is surprisingly slippery, so I prefer (at the moment) the idea of folding the immutable state into the views. It also allows the owner of a view to query that view, asking whether the view is immutable. This query is (as we all know) a key building block for efficiently managing defensive copies. > To be honest, I don't like where the code is right now, and I'm also not a fan of the 'immutable' access flag. Yeah, I'm not surprised. It's certainly different from how we think about file or virtual memory permissions. But see the D language, which uses this concept pretty well, by distinguishing between immutable and const pointers. We are not bound to primarily C or C++, but rather to the goal of making foreign code and data interoperate with VM code and data. And immutability is growing to be a first-class concept in Java, even if it is second-class in C. You might try to put immutability "behind" the view objects and see how that factors out. I did try that in my present sketch, since it seemed more principled. I ended up with the webrev code, which surfaces it in the view access bits. But maybe you can find a better way to submerge it again. > The problem being that I think access mode should be a property of the pointer, as you say in the very first line of your email. So, getting a read-only view of a pointer should, I believe, _have no effects_ on the underlying memory region - it should just return a new pointer with the new capability. I agree. A pointer is a capability to do stuff on a segment of memory. As an eventual value type, it has no state, and must be recopied (not modified) to get related capabilities. > That is, we should stop this business of recreating the memory region only to update an access flag - this is, I think, what gets in the way of offering a truly 'non-writeable' memory region (which you attempt to solve with the 'immutable' flag). I don't see that this point follows from the previous one. The real issue here is what is an MR and whether we need it. I think it's a view, and we need it as distinct from Pointers and Arrays. > > If memory regions had a _non-upgradeable_ R/W/RW property, then you could really allocate a memory region that is read-only, if you wanted to. Any attempt to get a write-capable pointer out of that will fail. (in other words,, you can get a pointer with access X on a memory region with access Y only if X is equal or less strict than Y). So you want MR to "pull its weight" by serving as a descriptor of a unique (unaliasable) segment of memory, not a view. If it says READ_ONLY, nobody will ever write anything into it, ever, even via tricky pointer aliases. Maybe that can work, but I'm dubious at present; like I said, the concept of a content object is very slippery. My money is on views, and the question is whether P+A+MR is too many different views or not. Can we get away with two or even one? If so, how does it all factor out? > So, I think that, of the two options you consider, I'm more for the quasi-independent access queries for pointer/memory region, as that seems a cleaner cut to me. > > But I agree with your line of reasoning - having some support for immutable memory regions seems required in order to support things like returning struct as value in the heap from native functions. Yes, that's what prompted me to do the deep dive on these issues. I don't want us to get caught with our hands empty when we need something like that, because the alternatives (user-managed returned values, mutable return values, explicit return buffers) are known to be buggy and unpleasant. ? John From samuel.audet at gmail.com Wed Jan 23 01:03:46 2019 From: samuel.audet at gmail.com (Samuel Audet) Date: Wed, 23 Jan 2019 10:03:46 +0900 Subject: scopes writeup In-Reply-To: <6fbff2eb-b100-0c03-7663-c732a3904302@oracle.com> References: <6fbff2eb-b100-0c03-7663-c732a3904302@oracle.com> Message-ID: <0e23525d-aefb-8484-8f46-d2e4897ee93d@gmail.com> Hi, Maurizio, I see there is an "interface Resource", but it doesn't allow us to provide custom allocators and deallocators. How could we, for example, manage all resources including those that are not pointers, structs, etc with a single Scope? Do we need to provide a new implementation of Scope for each type of Resource? Samuel On 1/23/19 1:27 AM, Maurizio Cimadamore wrote: > Hi, > at the end of last year I tried coming up with a proposal [1] which > aimed at finding the basic concepts behind Panama scopes - I've been > iterating (with the help of some internal feedback) on that proposal, > and put it back together in a different form which, all things > considered, it's not too far off from where we are now. > > https://cr.openjdk.java.net/~mcimadamore/panama/scopes.html > > That is, we still have a Scope abstraction - memory regions are still > hidden under Scopes, and not part of the public API (in an attempt to > limit the number of concepts exposed by the API - at least for now). > Pointer is an immutable cursor into a memory region. Resources are just > things with a scope (turns out, a native library is another such thing, > since a native library needs a library-wide scope for allocating its own > globals, etc.). > > On top of what we have now, this writeup pushes forward an improved > ownership model, as well as some discussions concerning > thread-confinement of scopes. > > I think this tweaked API proposal has the potential of fixing the > current lifecycle issues, w/o making the API too complex to use. > Additional abstraction (such as MemoryRegion) can be added back to the > public API on a by need basis (e.g. provided relevant use cases). > > An experimental patch implementing the proposed approach can be found here: > > http://cr.openjdk.java.net/~mcimadamore/panama/scope-ownership_v2/ > > Comments welcome! > > Maurizio > > [1] - > https://mail.openjdk.java.net/pipermail/panama-dev/2018-December/003605.html > > From sundararajan.athijegannathan at oracle.com Wed Jan 23 02:01:41 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Wed, 23 Jan 2019 07:31:41 +0530 Subject: [foreign] RFC: Jextract -l options ignored by SymbolFilter when -L is not specified In-Reply-To: <2f86de9e0e566913c1130e89556c541e@xs4all.nl> References: <698464e8f7692e12a7062155c7e599d9@xs4all.nl> <5C46AD23.9000207@oracle.com> <4089850d-fec6-40cf-c036-e666169a84b3@oracle.com> <30080ab5a99a1a9abf34a89a6c70b8b8@xs4all.nl> <4de2ba3b-4971-95d3-cb3b-9a0d478656d9@oracle.com> <036DADBC-851D-494F-8FC0-094505BE5696@oracle.com> <2f86de9e0e566913c1130e89556c541e@xs4all.nl> Message-ID: <5C47CB05.6060701@oracle.com> java.library.path contains only the JRE's own directories + system specific java dirs. This does not include directories like /usr/lib or /usr/local/lib (where you're likely to find shared objects of C libraries that are installed). In other words, the default value of java.library.path system property does not get you too far. If user has to pass java.library.path for jextract, s/he has to use jextract -J-Djava.library.path=... -J is needed to set pass JVM option to jdk/bin tools other than "java". That is not super friendly. S/he might as well use -L. So I'm not sure I agree with using "java.library.path" as fallback. -Sundar On 23/01/19, 12:50 AM, Jorn Vernee wrote: > This also seems the most natural to me, since it follows what the > linker flags do. > > -L is to specify additional linker directories. We would consider > java.library.path to be the "default"/"system" directories. > > This is also what I tried to do in the patch [1] (minor update). With > the addition of emitting a warning that symbol filtering is disabled > when -l is used but no library paths are available (either in -L > options or in java.library.path). > > That said, I think having an extra option to explicitly turn on, or > off, the symbol checking is a good idea as well. > > Jorn > > [1]: > http://cr.openjdk.java.net/~jvernee/panama/webrevs/jlibpath/webrev.02/ > > Henry Jen schreef op 2019-01-22 17:40: >> It is preferred to keep options compatible with cc when applicable. >> >> -L is only to provide the path for library at tooling time, that?s >> jextract. >> ?infer-path is similar to -R, will record the path for searching at >> runtime, that?s is, the path specified with -L will be added into >> search path of library. >> >> As symbol check, it should be enabled with -l. -L is simply provide >> extra path to search for the library, without -L, it will simply >> search in java.library.path. >> >> Cheers, >> Henry >> >> >>> On Jan 22, 2019, at 4:31 AM, Maurizio Cimadamore >>> wrote: >>> >>> >>> On 22/01/2019 12:09, Jorn Vernee wrote: >>>> This sounds good, I really like the idea of a separate option to >>>> enable the symbol filtering. But can you share what you think the >>>> role of java.library.path should be as well? >>> >>> I think using java.library.path as a default for the missing symbol >>> check could be ok. But I don't think it would be ok to use it as a >>> basis for infer-rpath. That is, I don't want static properties (e.g. >>> valid at extraction time) to spill onto the runtime. If the user >>> really wants to set some dynamic property, it has to use an explicit >>> flag to do so (e.g. -L). >>> >>> Maurizio >>> >>>> >>>> Jorn >>>> >>>> Maurizio Cimadamore schreef op 2019-01-22 12:58: >>>>> Looking at this, I remember being confused about this too. >>>>> >>>>> Let me try to see if we can find a better stacking for the existing >>>>> options - as Sundar said, we currently have: >>>>> >>>>> * -l >>>>> >>>>> This option is used to specify library _names_. >>>>> >>>>> The main goal of this option is to alter the contents of the >>>>> @NativeHeader annotation (by adding the library name) but there are, >>>>> as we shall see, other subtle side-effects. >>>>> >>>>> * -L + -l >>>>> >>>>> When both -L and -l are specified, the so called "missing symbols >>>>> check" will kick in, that is, jextract will check that all >>>>> symbols in >>>>> the library are indeed defined in the header files being extracted. A >>>>> subtle side-effect of that check, is that when -l and -L are >>>>> specified >>>>> together, and the missing symbol check is enabled, jextract will warn >>>>> for symbols not found and _it will exclude them_ from the extracted >>>>> classfile (w/o need for --include-symbols or --exclude-symbols). >>>>> >>>>> * -L + -l + -infer-rpath >>>>> >>>>> When -L and -l are used together, and the -infer-rpath option is >>>>> given, a runtime library path will be inferred from the contents of >>>>> -L, and will be stored in @NativeHeader, so that the binder can use >>>>> it. >>>>> >>>>> >>>>> I think the status quo is a bit confusing - because -L has multiple >>>>> functions (it serves up the library paths to be used as inferred >>>>> rpaths, and it also serves up the library paths to be used for the >>>>> missing symbol check). I think a more consistent stacking could be >>>>> something like this: >>>>> >>>>> -l --> used to specify library _names_; only side-effect is contents >>>>> of @NativeHeader >>>>> >>>>> -L --> used to specify _custom_ library _paths_; no side-effects >>>>> >>>>> -exclude-missing -> must be used in conjunction with -l and -L ; >>>>> enables the missing symbol check and auto-exclusion >>>>> >>>>> -infer-rpath -> must be used in conjunction with -l and -L ; enables >>>>> rpath inference (rpath inferred with paths specified in -L) >>>>> >>>>> >>>>> Thoughts? >>>>> >>>>> Maurizio >>>>> >>>>> >>>>> On 22/01/2019 05:41, Sundararajan Athijegannathan wrote: >>>>>> I don't think it is a bug - afaik it is as per design. The >>>>>> primary use of "-l" is to record the library in annotation of the >>>>>> generated jar - so that binder can auto-load the library (either >>>>>> from java.library.path configuration or -rpath value recorded in >>>>>> annotation). It is okay to record name of the shared object >>>>>> alone and leave the library path configuration to >>>>>> java.library.path setting. >>>>>> >>>>>> "-L" option is added feature to perform missing symbols checking. >>>>>> "-rpath" option is to add a path for library search - so that >>>>>> binder can locate the shared object in the specific directory. If >>>>>> no -rpath is specified, "-L" is used for runtime search as well. >>>>>> >>>>>> -Sundar >>>>>> >>>>>> On 22/01/19, 12:01 AM, Jorn Vernee wrote: >>>>>>> Hi, >>>>>>> >>>>>>> I've recently updated the instructions for using libraries on >>>>>>> Windows. For python the jextract example I gave was: >>>>>>> >>>>>>> jextract -l python27 -o "python.jar" -t "org.python" >>>>>>> C:\Python27\include\Python.h >>>>>>> >>>>>>> I'm lacking an `-L` option here (for specifying library >>>>>>> directories) since the contents of PATH seems to be added to >>>>>>> java.library.path by default, and this is presumably also how >>>>>>> jextract is able to load the library. But, since I'm not using >>>>>>> an `-L` option, SymbolFilter is not checking if the symbols are >>>>>>> in the python27.dll [1] >>>>>>> >>>>>>> private void initSymChecker(List linkCheckPaths) { >>>>>>> if (!libraryNames.isEmpty() && !linkCheckPaths.isEmpty()) { >>>>>>> // ... init symChecker >>>>>>> } else { >>>>>>> symChecker = null; >>>>>>> } >>>>>>> } >>>>>>> >>>>>>> (linkCheckPaths comes from the -L option values) >>>>>>> >>>>>>> This behaviour is somewhat unexpected. At least a warning that >>>>>>> missing an `-L` option will turn off symbol checking would be nice. >>>>>>> >>>>>>> We could also add the paths in `java.library.path` to the list >>>>>>> of link check paths in jextract [2]. That would mean that the >>>>>>> symbol checker would run for the example command. >>>>>>> >>>>>>> What do you think? >>>>>>> >>>>>>> Jorn >>>>>>> >>>>>>> [1] : >>>>>>> http://hg.openjdk.java.net/panama/dev/file/eaca2d16b80b/src/jdk.jextract/share/classes/com/sun/tools/jextract/SymbolFilter.java#l89 >>>>>>> [2] : >>>>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/jlibpath/webrev.01/ >>>>>>> From sundararajan.athijegannathan at oracle.com Wed Jan 23 02:06:42 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Wed, 23 Jan 2019 07:36:42 +0530 Subject: [foreign] RFR 8217462: Add a jextract option to not generate typedef annotation interfaces In-Reply-To: <158402ae-9f55-9dd7-66ad-8805ef550e33@oracle.com> References: <158402ae-9f55-9dd7-66ad-8805ef550e33@oracle.com> Message-ID: <5C47CC32.7010101@oracle.com> I agree. We should stabilize/regularize existing options first. -Sundar On 23/01/19, 2:40 AM, Maurizio Cimadamore wrote: > Hi Jorn, > while I understand where you are coming from, I'd like not to fragment > the space of the jextract options too much - as discussed earlier > today, there is already enough confusion over existing options, and we > should clear that before adding new ones :-) > > Also, if we go down this path, I'm afraid that this will turn into > having a full menu of options to support this or that generation tweak > - this sort of things tend to add up, the options will likely be > (ab)used in any possible configuration, with the result that the > jextract output will be perceived as something not stable. > > I have also a sense that your request is coming in part from issues > that were pointed out few days ago - e.g. jextract is currently > generating too much stuff - even things that have nothing to do with > the library being extracted (but which is recursively included by the > headers). As we have said then, the solution for this problem is to > adopt a library-centric approach, not adding custom flags to > selectively switch off extraction features. > > Maurizio > > On 22/01/2019 19:48, Jorn Vernee wrote: >> Hi, >> >> From the bug description: >> >> Currently jextract generates annotation interfaces for typedefs. But, >> using these annotations is not required to call the native functions >> defined in a library. Rather, they are there to add additional >> metadata to java carrier types. >> >> Since use of these annotations interfaces is optional, it would be >> nice to have a jextract option to turn of their generation, in order >> to simplify generated classes (especially useful when viewing them >> with a decompiler/javap) >> >> Please review the following. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8217462 >> Webrev: >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217462/webrev.00/ >> >> Thanks, >> Jorn From henry.jen at oracle.com Wed Jan 23 02:49:00 2019 From: henry.jen at oracle.com (Henry Jen) Date: Tue, 22 Jan 2019 18:49:00 -0800 Subject: [foreign] RFC: Jextract -l options ignored by SymbolFilter when -L is not specified In-Reply-To: <5C47CB05.6060701@oracle.com> References: <698464e8f7692e12a7062155c7e599d9@xs4all.nl> <5C46AD23.9000207@oracle.com> <4089850d-fec6-40cf-c036-e666169a84b3@oracle.com> <30080ab5a99a1a9abf34a89a6c70b8b8@xs4all.nl> <4de2ba3b-4971-95d3-cb3b-9a0d478656d9@oracle.com> <036DADBC-851D-494F-8FC0-094505BE5696@oracle.com> <2f86de9e0e566913c1130e89556c541e@xs4all.nl> <5C47CB05.6060701@oracle.com> Message-ID: <19941C48-DF18-414B-AC02-83F1C62F6712@oracle.com> What we are suggesting is that, -L is supplement to java.library.path. that -L is optional while -l is required for validating again the shared library. In fact, -L is pretty much like a shortcut to prepend more search path for java.library.path. Cheers, Henry > On Jan 22, 2019, at 6:01 PM, Sundararajan Athijegannathan wrote: > > java.library.path contains only the JRE's own directories + system specific java dirs. This does not include directories like /usr/lib or /usr/local/lib (where you're likely to find shared objects of C libraries that are installed). In other words, the default value of java.library.path system property does not get you too far. > > If user has to pass java.library.path for jextract, s/he has to use > > jextract -J-Djava.library.path=... > > -J is needed to set pass JVM option to jdk/bin tools other than "java". That is not super friendly. S/he might as well use -L. So I'm not sure I agree with using "java.library.path" as fallback. > > -Sundar > > On 23/01/19, 12:50 AM, Jorn Vernee wrote: >> This also seems the most natural to me, since it follows what the linker flags do. >> >> -L is to specify additional linker directories. We would consider java.library.path to be the "default"/"system" directories. >> >> This is also what I tried to do in the patch [1] (minor update). With the addition of emitting a warning that symbol filtering is disabled when -l is used but no library paths are available (either in -L options or in java.library.path). >> >> That said, I think having an extra option to explicitly turn on, or off, the symbol checking is a good idea as well. >> >> Jorn >> >> [1]: http://cr.openjdk.java.net/~jvernee/panama/webrevs/jlibpath/webrev.02/ >> >> Henry Jen schreef op 2019-01-22 17:40: >>> It is preferred to keep options compatible with cc when applicable. >>> >>> -L is only to provide the path for library at tooling time, that?s jextract. >>> ?infer-path is similar to -R, will record the path for searching at >>> runtime, that?s is, the path specified with -L will be added into >>> search path of library. >>> >>> As symbol check, it should be enabled with -l. -L is simply provide >>> extra path to search for the library, without -L, it will simply >>> search in java.library.path. >>> >>> Cheers, >>> Henry >>> >>> >>>> On Jan 22, 2019, at 4:31 AM, Maurizio Cimadamore wrote: >>>> >>>> >>>> On 22/01/2019 12:09, Jorn Vernee wrote: >>>>> This sounds good, I really like the idea of a separate option to enable the symbol filtering. But can you share what you think the role of java.library.path should be as well? >>>> >>>> I think using java.library.path as a default for the missing symbol check could be ok. But I don't think it would be ok to use it as a basis for infer-rpath. That is, I don't want static properties (e.g. valid at extraction time) to spill onto the runtime. If the user really wants to set some dynamic property, it has to use an explicit flag to do so (e.g. -L). >>>> >>>> Maurizio >>>> >>>>> >>>>> Jorn >>>>> >>>>> Maurizio Cimadamore schreef op 2019-01-22 12:58: >>>>>> Looking at this, I remember being confused about this too. >>>>>> >>>>>> Let me try to see if we can find a better stacking for the existing >>>>>> options - as Sundar said, we currently have: >>>>>> >>>>>> * -l >>>>>> >>>>>> This option is used to specify library _names_. >>>>>> >>>>>> The main goal of this option is to alter the contents of the >>>>>> @NativeHeader annotation (by adding the library name) but there are, >>>>>> as we shall see, other subtle side-effects. >>>>>> >>>>>> * -L + -l >>>>>> >>>>>> When both -L and -l are specified, the so called "missing symbols >>>>>> check" will kick in, that is, jextract will check that all symbols in >>>>>> the library are indeed defined in the header files being extracted. A >>>>>> subtle side-effect of that check, is that when -l and -L are specified >>>>>> together, and the missing symbol check is enabled, jextract will warn >>>>>> for symbols not found and _it will exclude them_ from the extracted >>>>>> classfile (w/o need for --include-symbols or --exclude-symbols). >>>>>> >>>>>> * -L + -l + -infer-rpath >>>>>> >>>>>> When -L and -l are used together, and the -infer-rpath option is >>>>>> given, a runtime library path will be inferred from the contents of >>>>>> -L, and will be stored in @NativeHeader, so that the binder can use >>>>>> it. >>>>>> >>>>>> >>>>>> I think the status quo is a bit confusing - because -L has multiple >>>>>> functions (it serves up the library paths to be used as inferred >>>>>> rpaths, and it also serves up the library paths to be used for the >>>>>> missing symbol check). I think a more consistent stacking could be >>>>>> something like this: >>>>>> >>>>>> -l --> used to specify library _names_; only side-effect is contents >>>>>> of @NativeHeader >>>>>> >>>>>> -L --> used to specify _custom_ library _paths_; no side-effects >>>>>> >>>>>> -exclude-missing -> must be used in conjunction with -l and -L ; >>>>>> enables the missing symbol check and auto-exclusion >>>>>> >>>>>> -infer-rpath -> must be used in conjunction with -l and -L ; enables >>>>>> rpath inference (rpath inferred with paths specified in -L) >>>>>> >>>>>> >>>>>> Thoughts? >>>>>> >>>>>> Maurizio >>>>>> >>>>>> >>>>>> On 22/01/2019 05:41, Sundararajan Athijegannathan wrote: >>>>>>> I don't think it is a bug - afaik it is as per design. The primary use of "-l" is to record the library in annotation of the generated jar - so that binder can auto-load the library (either from java.library.path configuration or -rpath value recorded in annotation). It is okay to record name of the shared object alone and leave the library path configuration to java.library.path setting. >>>>>>> >>>>>>> "-L" option is added feature to perform missing symbols checking. "-rpath" option is to add a path for library search - so that binder can locate the shared object in the specific directory. If no -rpath is specified, "-L" is used for runtime search as well. >>>>>>> >>>>>>> -Sundar >>>>>>> >>>>>>> On 22/01/19, 12:01 AM, Jorn Vernee wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> I've recently updated the instructions for using libraries on Windows. For python the jextract example I gave was: >>>>>>>> >>>>>>>> jextract -l python27 -o "python.jar" -t "org.python" C:\Python27\include\Python.h >>>>>>>> >>>>>>>> I'm lacking an `-L` option here (for specifying library directories) since the contents of PATH seems to be added to java.library.path by default, and this is presumably also how jextract is able to load the library. But, since I'm not using an `-L` option, SymbolFilter is not checking if the symbols are in the python27.dll [1] >>>>>>>> >>>>>>>> private void initSymChecker(List linkCheckPaths) { >>>>>>>> if (!libraryNames.isEmpty() && !linkCheckPaths.isEmpty()) { >>>>>>>> // ... init symChecker >>>>>>>> } else { >>>>>>>> symChecker = null; >>>>>>>> } >>>>>>>> } >>>>>>>> >>>>>>>> (linkCheckPaths comes from the -L option values) >>>>>>>> >>>>>>>> This behaviour is somewhat unexpected. At least a warning that missing an `-L` option will turn off symbol checking would be nice. >>>>>>>> >>>>>>>> We could also add the paths in `java.library.path` to the list of link check paths in jextract [2]. That would mean that the symbol checker would run for the example command. >>>>>>>> >>>>>>>> What do you think? >>>>>>>> >>>>>>>> Jorn >>>>>>>> >>>>>>>> [1] : http://hg.openjdk.java.net/panama/dev/file/eaca2d16b80b/src/jdk.jextract/share/classes/com/sun/tools/jextract/SymbolFilter.java#l89 >>>>>>>> [2] : http://cr.openjdk.java.net/~jvernee/panama/webrevs/jlibpath/webrev.01/ From sundararajan.athijegannathan at oracle.com Wed Jan 23 02:57:24 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Wed, 23 Jan 2019 08:27:24 +0530 Subject: [foreign] RFC: Jextract -l options ignored by SymbolFilter when -L is not specified In-Reply-To: <19941C48-DF18-414B-AC02-83F1C62F6712@oracle.com> References: <698464e8f7692e12a7062155c7e599d9@xs4all.nl> <5C46AD23.9000207@oracle.com> <4089850d-fec6-40cf-c036-e666169a84b3@oracle.com> <30080ab5a99a1a9abf34a89a6c70b8b8@xs4all.nl> <4de2ba3b-4971-95d3-cb3b-9a0d478656d9@oracle.com> <036DADBC-851D-494F-8FC0-094505BE5696@oracle.com> <2f86de9e0e566913c1130e89556c541e@xs4all.nl> <5C47CB05.6060701@oracle.com> <19941C48-DF18-414B-AC02-83F1C62F6712@oracle.com> Message-ID: <5C47D814.5090203@oracle.com> Yes, I got that part - that's why I wrote my reason as to why that is not a good idea. java.library.path's default value includes only the directories that contain pre-built platform JNI libs - unlike platform native default library path used by "ld". You won't find anything useful for the general C shared objects. Of course user can set it - which requires -J-D... option as I mentioned. -Sundar On 23/01/19, 8:19 AM, Henry Jen wrote: > What we are suggesting is that, -L is supplement to java.library.path. that -L is optional while -l is required for validating again the shared library. > > In fact, -L is pretty much like a shortcut to prepend more search path for java.library.path. > > Cheers, > Henry > >> On Jan 22, 2019, at 6:01 PM, Sundararajan Athijegannathan wrote: >> >> java.library.path contains only the JRE's own directories + system specific java dirs. This does not include directories like /usr/lib or /usr/local/lib (where you're likely to find shared objects of C libraries that are installed). In other words, the default value of java.library.path system property does not get you too far. >> >> If user has to pass java.library.path for jextract, s/he has to use >> >> jextract -J-Djava.library.path=... >> >> -J is needed to set pass JVM option to jdk/bin tools other than "java". That is not super friendly. S/he might as well use -L. So I'm not sure I agree with using "java.library.path" as fallback. >> >> -Sundar >> >> On 23/01/19, 12:50 AM, Jorn Vernee wrote: >>> This also seems the most natural to me, since it follows what the linker flags do. >>> >>> -L is to specify additional linker directories. We would consider java.library.path to be the "default"/"system" directories. >>> >>> This is also what I tried to do in the patch [1] (minor update). With the addition of emitting a warning that symbol filtering is disabled when -l is used but no library paths are available (either in -L options or in java.library.path). >>> >>> That said, I think having an extra option to explicitly turn on, or off, the symbol checking is a good idea as well. >>> >>> Jorn >>> >>> [1]: http://cr.openjdk.java.net/~jvernee/panama/webrevs/jlibpath/webrev.02/ >>> >>> Henry Jen schreef op 2019-01-22 17:40: >>>> It is preferred to keep options compatible with cc when applicable. >>>> >>>> -L is only to provide the path for library at tooling time, that?s jextract. >>>> ?infer-path is similar to -R, will record the path for searching at >>>> runtime, that?s is, the path specified with -L will be added into >>>> search path of library. >>>> >>>> As symbol check, it should be enabled with -l. -L is simply provide >>>> extra path to search for the library, without -L, it will simply >>>> search in java.library.path. >>>> >>>> Cheers, >>>> Henry >>>> >>>> >>>>> On Jan 22, 2019, at 4:31 AM, Maurizio Cimadamore wrote: >>>>> >>>>> >>>>> On 22/01/2019 12:09, Jorn Vernee wrote: >>>>>> This sounds good, I really like the idea of a separate option to enable the symbol filtering. But can you share what you think the role of java.library.path should be as well? >>>>> I think using java.library.path as a default for the missing symbol check could be ok. But I don't think it would be ok to use it as a basis for infer-rpath. That is, I don't want static properties (e.g. valid at extraction time) to spill onto the runtime. If the user really wants to set some dynamic property, it has to use an explicit flag to do so (e.g. -L). >>>>> >>>>> Maurizio >>>>> >>>>>> Jorn >>>>>> >>>>>> Maurizio Cimadamore schreef op 2019-01-22 12:58: >>>>>>> Looking at this, I remember being confused about this too. >>>>>>> >>>>>>> Let me try to see if we can find a better stacking for the existing >>>>>>> options - as Sundar said, we currently have: >>>>>>> >>>>>>> * -l >>>>>>> >>>>>>> This option is used to specify library _names_. >>>>>>> >>>>>>> The main goal of this option is to alter the contents of the >>>>>>> @NativeHeader annotation (by adding the library name) but there are, >>>>>>> as we shall see, other subtle side-effects. >>>>>>> >>>>>>> * -L + -l >>>>>>> >>>>>>> When both -L and -l are specified, the so called "missing symbols >>>>>>> check" will kick in, that is, jextract will check that all symbols in >>>>>>> the library are indeed defined in the header files being extracted. A >>>>>>> subtle side-effect of that check, is that when -l and -L are specified >>>>>>> together, and the missing symbol check is enabled, jextract will warn >>>>>>> for symbols not found and _it will exclude them_ from the extracted >>>>>>> classfile (w/o need for --include-symbols or --exclude-symbols). >>>>>>> >>>>>>> * -L + -l + -infer-rpath >>>>>>> >>>>>>> When -L and -l are used together, and the -infer-rpath option is >>>>>>> given, a runtime library path will be inferred from the contents of >>>>>>> -L, and will be stored in @NativeHeader, so that the binder can use >>>>>>> it. >>>>>>> >>>>>>> >>>>>>> I think the status quo is a bit confusing - because -L has multiple >>>>>>> functions (it serves up the library paths to be used as inferred >>>>>>> rpaths, and it also serves up the library paths to be used for the >>>>>>> missing symbol check). I think a more consistent stacking could be >>>>>>> something like this: >>>>>>> >>>>>>> -l --> used to specify library _names_; only side-effect is contents >>>>>>> of @NativeHeader >>>>>>> >>>>>>> -L --> used to specify _custom_ library _paths_; no side-effects >>>>>>> >>>>>>> -exclude-missing -> must be used in conjunction with -l and -L ; >>>>>>> enables the missing symbol check and auto-exclusion >>>>>>> >>>>>>> -infer-rpath -> must be used in conjunction with -l and -L ; enables >>>>>>> rpath inference (rpath inferred with paths specified in -L) >>>>>>> >>>>>>> >>>>>>> Thoughts? >>>>>>> >>>>>>> Maurizio >>>>>>> >>>>>>> >>>>>>> On 22/01/2019 05:41, Sundararajan Athijegannathan wrote: >>>>>>>> I don't think it is a bug - afaik it is as per design. The primary use of "-l" is to record the library in annotation of the generated jar - so that binder can auto-load the library (either from java.library.path configuration or -rpath value recorded in annotation). It is okay to record name of the shared object alone and leave the library path configuration to java.library.path setting. >>>>>>>> >>>>>>>> "-L" option is added feature to perform missing symbols checking. "-rpath" option is to add a path for library search - so that binder can locate the shared object in the specific directory. If no -rpath is specified, "-L" is used for runtime search as well. >>>>>>>> >>>>>>>> -Sundar >>>>>>>> >>>>>>>> On 22/01/19, 12:01 AM, Jorn Vernee wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> I've recently updated the instructions for using libraries on Windows. For python the jextract example I gave was: >>>>>>>>> >>>>>>>>> jextract -l python27 -o "python.jar" -t "org.python" C:\Python27\include\Python.h >>>>>>>>> >>>>>>>>> I'm lacking an `-L` option here (for specifying library directories) since the contents of PATH seems to be added to java.library.path by default, and this is presumably also how jextract is able to load the library. But, since I'm not using an `-L` option, SymbolFilter is not checking if the symbols are in the python27.dll [1] >>>>>>>>> >>>>>>>>> private void initSymChecker(List linkCheckPaths) { >>>>>>>>> if (!libraryNames.isEmpty()&& !linkCheckPaths.isEmpty()) { >>>>>>>>> // ... init symChecker >>>>>>>>> } else { >>>>>>>>> symChecker = null; >>>>>>>>> } >>>>>>>>> } >>>>>>>>> >>>>>>>>> (linkCheckPaths comes from the -L option values) >>>>>>>>> >>>>>>>>> This behaviour is somewhat unexpected. At least a warning that missing an `-L` option will turn off symbol checking would be nice. >>>>>>>>> >>>>>>>>> We could also add the paths in `java.library.path` to the list of link check paths in jextract [2]. That would mean that the symbol checker would run for the example command. >>>>>>>>> >>>>>>>>> What do you think? >>>>>>>>> >>>>>>>>> Jorn >>>>>>>>> >>>>>>>>> [1] : http://hg.openjdk.java.net/panama/dev/file/eaca2d16b80b/src/jdk.jextract/share/classes/com/sun/tools/jextract/SymbolFilter.java#l89 >>>>>>>>> [2] : http://cr.openjdk.java.net/~jvernee/panama/webrevs/jlibpath/webrev.01/ From sundararajan.athijegannathan at oracle.com Wed Jan 23 03:02:20 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Wed, 23 Jan 2019 08:32:20 +0530 Subject: [foreign] RFC: Jextract -l options ignored by SymbolFilter when -L is not specified In-Reply-To: <5C47D814.5090203@oracle.com> References: <698464e8f7692e12a7062155c7e599d9@xs4all.nl> <5C46AD23.9000207@oracle.com> <4089850d-fec6-40cf-c036-e666169a84b3@oracle.com> <30080ab5a99a1a9abf34a89a6c70b8b8@xs4all.nl> <4de2ba3b-4971-95d3-cb3b-9a0d478656d9@oracle.com> <036DADBC-851D-494F-8FC0-094505BE5696@oracle.com> <2f86de9e0e566913c1130e89556c541e@xs4all.nl> <5C47CB05.6060701@oracle.com> <19941C48-DF18-414B-AC02-83F1C62F6712@oracle.com> <5C47D814.5090203@oracle.com> Message-ID: <5C47D93C.7040008@oracle.com> I think it'd be better to use new jextract specific options: --library-path / -lp to specify library paths. Similar to --module-path/--class-path etc. If possible, we should detect and use platform specific library paths that "ld" would use. This way, default would be useful --missing-symbols=warn|error|exclude option. Tells what should be done on missing native symbols. -Sundar On 23/01/19, 8:27 AM, Sundararajan Athijegannathan wrote: > Yes, I got that part - that's why I wrote my reason as to why that is > not a good idea. > > java.library.path's default value includes only the directories that > contain pre-built platform JNI libs - unlike platform native default > library path used by "ld". You won't find anything useful for the > general C shared objects. Of course user can set it - which requires > -J-D... option as I mentioned. > > -Sundar > > On 23/01/19, 8:19 AM, Henry Jen wrote: >> What we are suggesting is that, -L is supplement to >> java.library.path. that -L is optional while -l is required for >> validating again the shared library. >> >> In fact, -L is pretty much like a shortcut to prepend more search >> path for java.library.path. >> >> Cheers, >> Henry >> >>> On Jan 22, 2019, at 6:01 PM, Sundararajan >>> Athijegannathan wrote: >>> >>> java.library.path contains only the JRE's own directories + system >>> specific java dirs. This does not include directories like /usr/lib >>> or /usr/local/lib (where you're likely to find shared objects of C >>> libraries that are installed). In other words, the default value of >>> java.library.path system property does not get you too far. >>> >>> If user has to pass java.library.path for jextract, s/he has to use >>> >>> jextract -J-Djava.library.path=... >>> >>> -J is needed to set pass JVM option to jdk/bin tools other than >>> "java". That is not super friendly. S/he might as well use -L. So >>> I'm not sure I agree with using "java.library.path" as fallback. >>> >>> -Sundar >>> >>> On 23/01/19, 12:50 AM, Jorn Vernee wrote: >>>> This also seems the most natural to me, since it follows what the >>>> linker flags do. >>>> >>>> -L is to specify additional linker directories. We would consider >>>> java.library.path to be the "default"/"system" directories. >>>> >>>> This is also what I tried to do in the patch [1] (minor update). >>>> With the addition of emitting a warning that symbol filtering is >>>> disabled when -l is used but no library paths are available (either >>>> in -L options or in java.library.path). >>>> >>>> That said, I think having an extra option to explicitly turn on, or >>>> off, the symbol checking is a good idea as well. >>>> >>>> Jorn >>>> >>>> [1]: >>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/jlibpath/webrev.02/ >>>> >>>> Henry Jen schreef op 2019-01-22 17:40: >>>>> It is preferred to keep options compatible with cc when applicable. >>>>> >>>>> -L is only to provide the path for library at tooling time, that?s >>>>> jextract. >>>>> ?infer-path is similar to -R, will record the path for searching at >>>>> runtime, that?s is, the path specified with -L will be added into >>>>> search path of library. >>>>> >>>>> As symbol check, it should be enabled with -l. -L is simply provide >>>>> extra path to search for the library, without -L, it will simply >>>>> search in java.library.path. >>>>> >>>>> Cheers, >>>>> Henry >>>>> >>>>> >>>>>> On Jan 22, 2019, at 4:31 AM, Maurizio >>>>>> Cimadamore wrote: >>>>>> >>>>>> >>>>>> On 22/01/2019 12:09, Jorn Vernee wrote: >>>>>>> This sounds good, I really like the idea of a separate option to >>>>>>> enable the symbol filtering. But can you share what you think >>>>>>> the role of java.library.path should be as well? >>>>>> I think using java.library.path as a default for the missing >>>>>> symbol check could be ok. But I don't think it would be ok to use >>>>>> it as a basis for infer-rpath. That is, I don't want static >>>>>> properties (e.g. valid at extraction time) to spill onto the >>>>>> runtime. If the user really wants to set some dynamic property, >>>>>> it has to use an explicit flag to do so (e.g. -L). >>>>>> >>>>>> Maurizio >>>>>> >>>>>>> Jorn >>>>>>> >>>>>>> Maurizio Cimadamore schreef op 2019-01-22 12:58: >>>>>>>> Looking at this, I remember being confused about this too. >>>>>>>> >>>>>>>> Let me try to see if we can find a better stacking for the >>>>>>>> existing >>>>>>>> options - as Sundar said, we currently have: >>>>>>>> >>>>>>>> * -l >>>>>>>> >>>>>>>> This option is used to specify library _names_. >>>>>>>> >>>>>>>> The main goal of this option is to alter the contents of the >>>>>>>> @NativeHeader annotation (by adding the library name) but there >>>>>>>> are, >>>>>>>> as we shall see, other subtle side-effects. >>>>>>>> >>>>>>>> * -L + -l >>>>>>>> >>>>>>>> When both -L and -l are specified, the so called "missing symbols >>>>>>>> check" will kick in, that is, jextract will check that all >>>>>>>> symbols in >>>>>>>> the library are indeed defined in the header files being >>>>>>>> extracted. A >>>>>>>> subtle side-effect of that check, is that when -l and -L are >>>>>>>> specified >>>>>>>> together, and the missing symbol check is enabled, jextract >>>>>>>> will warn >>>>>>>> for symbols not found and _it will exclude them_ from the >>>>>>>> extracted >>>>>>>> classfile (w/o need for --include-symbols or --exclude-symbols). >>>>>>>> >>>>>>>> * -L + -l + -infer-rpath >>>>>>>> >>>>>>>> When -L and -l are used together, and the -infer-rpath option is >>>>>>>> given, a runtime library path will be inferred from the >>>>>>>> contents of >>>>>>>> -L, and will be stored in @NativeHeader, so that the binder can >>>>>>>> use >>>>>>>> it. >>>>>>>> >>>>>>>> >>>>>>>> I think the status quo is a bit confusing - because -L has >>>>>>>> multiple >>>>>>>> functions (it serves up the library paths to be used as inferred >>>>>>>> rpaths, and it also serves up the library paths to be used for the >>>>>>>> missing symbol check). I think a more consistent stacking could be >>>>>>>> something like this: >>>>>>>> >>>>>>>> -l --> used to specify library _names_; only side-effect is >>>>>>>> contents >>>>>>>> of @NativeHeader >>>>>>>> >>>>>>>> -L --> used to specify _custom_ library _paths_; no side-effects >>>>>>>> >>>>>>>> -exclude-missing -> must be used in conjunction with -l and -L ; >>>>>>>> enables the missing symbol check and auto-exclusion >>>>>>>> >>>>>>>> -infer-rpath -> must be used in conjunction with -l and -L ; >>>>>>>> enables >>>>>>>> rpath inference (rpath inferred with paths specified in -L) >>>>>>>> >>>>>>>> >>>>>>>> Thoughts? >>>>>>>> >>>>>>>> Maurizio >>>>>>>> >>>>>>>> >>>>>>>> On 22/01/2019 05:41, Sundararajan Athijegannathan wrote: >>>>>>>>> I don't think it is a bug - afaik it is as per design. The >>>>>>>>> primary use of "-l" is to record the library in annotation of >>>>>>>>> the generated jar - so that binder can auto-load the library >>>>>>>>> (either from java.library.path configuration or -rpath value >>>>>>>>> recorded in annotation). It is okay to record name of the >>>>>>>>> shared object alone and leave the library path configuration >>>>>>>>> to java.library.path setting. >>>>>>>>> >>>>>>>>> "-L" option is added feature to perform missing symbols >>>>>>>>> checking. "-rpath" option is to add a path for library search >>>>>>>>> - so that binder can locate the shared object in the specific >>>>>>>>> directory. If no -rpath is specified, "-L" is used for runtime >>>>>>>>> search as well. >>>>>>>>> >>>>>>>>> -Sundar >>>>>>>>> >>>>>>>>> On 22/01/19, 12:01 AM, Jorn Vernee wrote: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> I've recently updated the instructions for using libraries on >>>>>>>>>> Windows. For python the jextract example I gave was: >>>>>>>>>> >>>>>>>>>> jextract -l python27 -o "python.jar" -t "org.python" >>>>>>>>>> C:\Python27\include\Python.h >>>>>>>>>> >>>>>>>>>> I'm lacking an `-L` option here (for specifying library >>>>>>>>>> directories) since the contents of PATH seems to be added to >>>>>>>>>> java.library.path by default, and this is presumably also how >>>>>>>>>> jextract is able to load the library. But, since I'm not >>>>>>>>>> using an `-L` option, SymbolFilter is not checking if the >>>>>>>>>> symbols are in the python27.dll [1] >>>>>>>>>> >>>>>>>>>> private void initSymChecker(List linkCheckPaths) { >>>>>>>>>> if (!libraryNames.isEmpty()&& >>>>>>>>>> !linkCheckPaths.isEmpty()) { >>>>>>>>>> // ... init symChecker >>>>>>>>>> } else { >>>>>>>>>> symChecker = null; >>>>>>>>>> } >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> (linkCheckPaths comes from the -L option values) >>>>>>>>>> >>>>>>>>>> This behaviour is somewhat unexpected. At least a warning >>>>>>>>>> that missing an `-L` option will turn off symbol checking >>>>>>>>>> would be nice. >>>>>>>>>> >>>>>>>>>> We could also add the paths in `java.library.path` to the >>>>>>>>>> list of link check paths in jextract [2]. That would mean >>>>>>>>>> that the symbol checker would run for the example command. >>>>>>>>>> >>>>>>>>>> What do you think? >>>>>>>>>> >>>>>>>>>> Jorn >>>>>>>>>> >>>>>>>>>> [1] : >>>>>>>>>> http://hg.openjdk.java.net/panama/dev/file/eaca2d16b80b/src/jdk.jextract/share/classes/com/sun/tools/jextract/SymbolFilter.java#l89 >>>>>>>>>> [2] : >>>>>>>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/jlibpath/webrev.01/ >>>>>>>>>> From vladimir.x.ivanov at oracle.com Wed Jan 23 03:12:04 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 22 Jan 2019 19:12:04 -0800 Subject: [vector api] Vector elements shift confusion In-Reply-To: References: Message-ID: <89f7aa3e-1967-add4-e24a-10226e92bd30@oracle.com> Good catch, Tomasz! I confirm that rotateER is completely broken :-) There's definitely disagreement between javadoc & implementation on shiftEL/ER variants, but I'm in favor of the javadoc here and would like to see the implementation aligned with it. While fromArray() and [shift|rotate]EL/ER seem to disagree (little-endian vs big-endian style), it aligns with the behavior of shifts on primitives where: "left" == "towards higher bits" == "towards higher indices" "right" == "towards lower bits" == "towards higher indices" And as you noted, implementation behaves in the opposite direction while rotateEL agrees with the spec [1]. Best regards, Vladimir Ivanov [1] $ jshell --add-modules=jdk.incubator.vector > import jdk.incubator.vector.* > var I256 = IntVector.species(jdk.incubator.vector.Vector.Shape.S_256_BIT); > var v1 = I256.fromArray(new int[] { 1, 2, 3, 4, 5, 6, 7, 8 }, 0) v1 ==> [1, 2, 3, 4, 5, 6, 7, 8] > v1.get(0) ==> 1 > v1.get(7) ==> 8 > v1.shiftEL(1) ==> [2, 3, 4, 5, 6, 7, 8, 0] > v1.shiftEL(1).get(0) ==> 2 > v1.shiftEL(1).get(7) ==> 0 > v1.shiftER(1) ==> [0, 1, 2, 3, 4, 5, 6, 7] > v1.shiftER(1).get(0) ==> 0 > v1.shiftER(1).get(7) ==> 7 > v1.rotateEL(1) ==> [8, 1, 2, 3, 4, 5, 6, 7] jshell> v1.rotateEL(1).get(0) ==> 8 jshell> v1.rotateEL(1).get(7) ==> 7 > v1.rotateER(1) | Exception java.lang.ArrayIndexOutOfBoundsException: Index -1 out of bounds for length 8 | at Int256Vector.rotateER (Int256Vector.java:924) | at Int256Vector.rotateER (Int256Vector.java:39) | at (#44:1) > I started this email having problems using rotateER on vectors. I have > not found any way to call this method without getting an exception :). > More investigation revealed that shift-ing vector elements does not > work in accordance with javadoc. I realised this email will be quite > long so I split it up and will describe just shift issues. If my > suspicions are correct I will follow up with rotations. > > First shiftEL operation describes itself "as if rotating left the lane > elements by i [...] zero value is placed into the result vector at > lane indexes less than i % this.length()." > > *I suspect that the rotation is described in the wrong direction*. > Lets try to confirm it but just looking at zeroed lane indices (using > get() which "Gets the lane element at lane index i"): > > @Test > public void shouldShiftElementsLeft() { > // Given > IntVector intVector = > IntVector.species(Vector.Shape.S_128_BIT).fromArray(new int[]{ > 1, 2, 3, 4 > }, 0); > > // When > intVector = intVector.shiftEL(2); > > // Then > assertThat(intVector.get(0)).isEqualTo(0); > assertThat(intVector.get(1)).isEqualTo(0); > } > > So after the shift of "2" lane indexes less than "2" should be zero. > This test fails as the rotation is done in the opposite (to me - > correct) direction: > > @Test > public void shouldShiftElementsLeft() { > // Given > IntVector intVector = > IntVector.species(Vector.Shape.S_128_BIT).fromArray(new int[]{ > 1, 2, 3, 4 > }, 0); > > // When > intVector = intVector.shiftEL(2); > > // Then > assertThat(intVector.toArray()).containsExactly(3, 4, 0, 0); > } > > Same case is with shiftER. Java doc says that "zero value is placed > into the result vector at lane indexes greater or equal to > this.length() - (i % this.length()).". But zeroed are indexes < i % > this.length(). > > Looks like implementations are correct but javadoc-s are swapped (I > hope I am not making fool of myself). > > Please advise, > Tomasz > From jbvernee at xs4all.nl Wed Jan 23 09:43:05 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Wed, 23 Jan 2019 10:43:05 +0100 Subject: [foreign] RFC: Jextract -l options ignored by SymbolFilter when -L is not specified In-Reply-To: <5C47D93C.7040008@oracle.com> References: <698464e8f7692e12a7062155c7e599d9@xs4all.nl> <5C46AD23.9000207@oracle.com> <4089850d-fec6-40cf-c036-e666169a84b3@oracle.com> <30080ab5a99a1a9abf34a89a6c70b8b8@xs4all.nl> <4de2ba3b-4971-95d3-cb3b-9a0d478656d9@oracle.com> <036DADBC-851D-494F-8FC0-094505BE5696@oracle.com> <2f86de9e0e566913c1130e89556c541e@xs4all.nl> <5C47CB05.6060701@oracle.com> <19941C48-DF18-414B-AC02-83F1C62F6712@oracle.com> <5C47D814.5090203@oracle.com> <5C47D93C.7040008@oracle.com> Message-ID: <0477c3fc285fc48c81d9a7f24561c118@xs4all.nl> The default value of the java.library.path seems to have changed over time, or maybe there's a difference between operating systems. Previously I understood that java.library.path defaulted to the value of the classpath. But now, if I open jshell and query java.library.path I get some paths that seem to be injected, which seem to be the paths of the pre-built platform JNI libs you mention. Appended to that there is the contents of PATH, which on Windows is used to look up DLLs. And finally appended to that is the `.` path. (FWIW, the same value is returned when I create a simple class which queries and returns the value) So "you won't find anything useful for the general C shared objects" is not true in my case, as we essentially get the system's default DLL search path with java.library.path (i.e. the contents of PATH). Like you said, setting the java.library.path requires -J, but having users do that should not be the goal imho. Rather, it acts as a default set of search paths, and if more need to be added, the user should use -L. > --library-path / -lp to specify library paths. Similar to > --module-path/--class-path etc. If possible, we should detect and use > platform specific library paths that "ld" would use. This way, default > would be useful Tbh, I don't really see how "--library-path / -lp" differs from "-L" that we have currently, except that the paths are passed in a different format. If the contents of java.library.path is not good enough on other platforms, maybe we could use an environment variable like JEXTRACT_LIBRARY_PATH which contains the default search paths. The advantage of using an environment variable is that it can be set once and then used across multiple jextract runs, where command line options have to be passed every time. Jorn Sundararajan Athijegannathan schreef op 2019-01-23 04:02: > I think it'd be better to use new jextract specific options: > > --library-path / -lp to specify library paths. Similar to > --module-path/--class-path etc. If possible, we should detect and use > platform specific library paths that "ld" would use. This way, default > would be useful > > --missing-symbols=warn|error|exclude option. Tells what should be done > on missing native symbols. > > -Sundar > > On 23/01/19, 8:27 AM, Sundararajan Athijegannathan wrote: >> Yes, I got that part - that's why I wrote my reason as to why that is >> not a good idea. >> >> java.library.path's default value includes only the directories that >> contain pre-built platform JNI libs - unlike platform native default >> library path used by "ld". You won't find anything useful for the >> general C shared objects. Of course user can set it - which requires >> -J-D... option as I mentioned. >> >> -Sundar >> >> On 23/01/19, 8:19 AM, Henry Jen wrote: >>> What we are suggesting is that, -L is supplement to >>> java.library.path. that -L is optional while -l is required for >>> validating again the shared library. >>> >>> In fact, -L is pretty much like a shortcut to prepend more search >>> path for java.library.path. >>> >>> Cheers, >>> Henry >>> >>>> On Jan 22, 2019, at 6:01 PM, Sundararajan >>>> Athijegannathan wrote: >>>> >>>> java.library.path contains only the JRE's own directories + system >>>> specific java dirs. This does not include directories like /usr/lib >>>> or /usr/local/lib (where you're likely to find shared objects of C >>>> libraries that are installed). In other words, the default value of >>>> java.library.path system property does not get you too far. >>>> >>>> If user has to pass java.library.path for jextract, s/he has to use >>>> >>>> jextract -J-Djava.library.path=... >>>> >>>> -J is needed to set pass JVM option to jdk/bin tools other than >>>> "java". That is not super friendly. S/he might as well use -L. So >>>> I'm not sure I agree with using "java.library.path" as fallback. >>>> >>>> -Sundar >>>> >>>> On 23/01/19, 12:50 AM, Jorn Vernee wrote: >>>>> This also seems the most natural to me, since it follows what the >>>>> linker flags do. >>>>> >>>>> -L is to specify additional linker directories. We would consider >>>>> java.library.path to be the "default"/"system" directories. >>>>> >>>>> This is also what I tried to do in the patch [1] (minor update). >>>>> With the addition of emitting a warning that symbol filtering is >>>>> disabled when -l is used but no library paths are available (either >>>>> in -L options or in java.library.path). >>>>> >>>>> That said, I think having an extra option to explicitly turn on, or >>>>> off, the symbol checking is a good idea as well. >>>>> >>>>> Jorn >>>>> >>>>> [1]: >>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/jlibpath/webrev.02/ >>>>> >>>>> Henry Jen schreef op 2019-01-22 17:40: >>>>>> It is preferred to keep options compatible with cc when >>>>>> applicable. >>>>>> >>>>>> -L is only to provide the path for library at tooling time, that?s >>>>>> jextract. >>>>>> ?infer-path is similar to -R, will record the path for searching >>>>>> at >>>>>> runtime, that?s is, the path specified with -L will be added into >>>>>> search path of library. >>>>>> >>>>>> As symbol check, it should be enabled with -l. -L is simply >>>>>> provide >>>>>> extra path to search for the library, without -L, it will simply >>>>>> search in java.library.path. >>>>>> >>>>>> Cheers, >>>>>> Henry >>>>>> >>>>>> >>>>>>> On Jan 22, 2019, at 4:31 AM, Maurizio >>>>>>> Cimadamore wrote: >>>>>>> >>>>>>> >>>>>>> On 22/01/2019 12:09, Jorn Vernee wrote: >>>>>>>> This sounds good, I really like the idea of a separate option to >>>>>>>> enable the symbol filtering. But can you share what you think >>>>>>>> the role of java.library.path should be as well? >>>>>>> I think using java.library.path as a default for the missing >>>>>>> symbol check could be ok. But I don't think it would be ok to use >>>>>>> it as a basis for infer-rpath. That is, I don't want static >>>>>>> properties (e.g. valid at extraction time) to spill onto the >>>>>>> runtime. If the user really wants to set some dynamic property, >>>>>>> it has to use an explicit flag to do so (e.g. -L). >>>>>>> >>>>>>> Maurizio >>>>>>> >>>>>>>> Jorn >>>>>>>> >>>>>>>> Maurizio Cimadamore schreef op 2019-01-22 12:58: >>>>>>>>> Looking at this, I remember being confused about this too. >>>>>>>>> >>>>>>>>> Let me try to see if we can find a better stacking for the >>>>>>>>> existing >>>>>>>>> options - as Sundar said, we currently have: >>>>>>>>> >>>>>>>>> * -l >>>>>>>>> >>>>>>>>> This option is used to specify library _names_. >>>>>>>>> >>>>>>>>> The main goal of this option is to alter the contents of the >>>>>>>>> @NativeHeader annotation (by adding the library name) but there >>>>>>>>> are, >>>>>>>>> as we shall see, other subtle side-effects. >>>>>>>>> >>>>>>>>> * -L + -l >>>>>>>>> >>>>>>>>> When both -L and -l are specified, the so called "missing >>>>>>>>> symbols >>>>>>>>> check" will kick in, that is, jextract will check that all >>>>>>>>> symbols in >>>>>>>>> the library are indeed defined in the header files being >>>>>>>>> extracted. A >>>>>>>>> subtle side-effect of that check, is that when -l and -L are >>>>>>>>> specified >>>>>>>>> together, and the missing symbol check is enabled, jextract >>>>>>>>> will warn >>>>>>>>> for symbols not found and _it will exclude them_ from the >>>>>>>>> extracted >>>>>>>>> classfile (w/o need for --include-symbols or >>>>>>>>> --exclude-symbols). >>>>>>>>> >>>>>>>>> * -L + -l + -infer-rpath >>>>>>>>> >>>>>>>>> When -L and -l are used together, and the -infer-rpath option >>>>>>>>> is >>>>>>>>> given, a runtime library path will be inferred from the >>>>>>>>> contents of >>>>>>>>> -L, and will be stored in @NativeHeader, so that the binder can >>>>>>>>> use >>>>>>>>> it. >>>>>>>>> >>>>>>>>> >>>>>>>>> I think the status quo is a bit confusing - because -L has >>>>>>>>> multiple >>>>>>>>> functions (it serves up the library paths to be used as >>>>>>>>> inferred >>>>>>>>> rpaths, and it also serves up the library paths to be used for >>>>>>>>> the >>>>>>>>> missing symbol check). I think a more consistent stacking could >>>>>>>>> be >>>>>>>>> something like this: >>>>>>>>> >>>>>>>>> -l --> used to specify library _names_; only side-effect is >>>>>>>>> contents >>>>>>>>> of @NativeHeader >>>>>>>>> >>>>>>>>> -L --> used to specify _custom_ library _paths_; no >>>>>>>>> side-effects >>>>>>>>> >>>>>>>>> -exclude-missing -> must be used in conjunction with -l and -L >>>>>>>>> ; >>>>>>>>> enables the missing symbol check and auto-exclusion >>>>>>>>> >>>>>>>>> -infer-rpath -> must be used in conjunction with -l and -L ; >>>>>>>>> enables >>>>>>>>> rpath inference (rpath inferred with paths specified in -L) >>>>>>>>> >>>>>>>>> >>>>>>>>> Thoughts? >>>>>>>>> >>>>>>>>> Maurizio >>>>>>>>> >>>>>>>>> >>>>>>>>> On 22/01/2019 05:41, Sundararajan Athijegannathan wrote: >>>>>>>>>> I don't think it is a bug - afaik it is as per design. The >>>>>>>>>> primary use of "-l" is to record the library in annotation of >>>>>>>>>> the generated jar - so that binder can auto-load the library >>>>>>>>>> (either from java.library.path configuration or -rpath value >>>>>>>>>> recorded in annotation). It is okay to record name of the >>>>>>>>>> shared object alone and leave the library path configuration >>>>>>>>>> to java.library.path setting. >>>>>>>>>> >>>>>>>>>> "-L" option is added feature to perform missing symbols >>>>>>>>>> checking. "-rpath" option is to add a path for library search >>>>>>>>>> - so that binder can locate the shared object in the specific >>>>>>>>>> directory. If no -rpath is specified, "-L" is used for runtime >>>>>>>>>> search as well. >>>>>>>>>> >>>>>>>>>> -Sundar >>>>>>>>>> >>>>>>>>>> On 22/01/19, 12:01 AM, Jorn Vernee wrote: >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> I've recently updated the instructions for using libraries on >>>>>>>>>>> Windows. For python the jextract example I gave was: >>>>>>>>>>> >>>>>>>>>>> jextract -l python27 -o "python.jar" -t "org.python" >>>>>>>>>>> C:\Python27\include\Python.h >>>>>>>>>>> >>>>>>>>>>> I'm lacking an `-L` option here (for specifying library >>>>>>>>>>> directories) since the contents of PATH seems to be added to >>>>>>>>>>> java.library.path by default, and this is presumably also how >>>>>>>>>>> jextract is able to load the library. But, since I'm not >>>>>>>>>>> using an `-L` option, SymbolFilter is not checking if the >>>>>>>>>>> symbols are in the python27.dll [1] >>>>>>>>>>> >>>>>>>>>>> private void initSymChecker(List linkCheckPaths) >>>>>>>>>>> { >>>>>>>>>>> if (!libraryNames.isEmpty()&& >>>>>>>>>>> !linkCheckPaths.isEmpty()) { >>>>>>>>>>> // ... init symChecker >>>>>>>>>>> } else { >>>>>>>>>>> symChecker = null; >>>>>>>>>>> } >>>>>>>>>>> } >>>>>>>>>>> >>>>>>>>>>> (linkCheckPaths comes from the -L option values) >>>>>>>>>>> >>>>>>>>>>> This behaviour is somewhat unexpected. At least a warning >>>>>>>>>>> that missing an `-L` option will turn off symbol checking >>>>>>>>>>> would be nice. >>>>>>>>>>> >>>>>>>>>>> We could also add the paths in `java.library.path` to the >>>>>>>>>>> list of link check paths in jextract [2]. That would mean >>>>>>>>>>> that the symbol checker would run for the example command. >>>>>>>>>>> >>>>>>>>>>> What do you think? >>>>>>>>>>> >>>>>>>>>>> Jorn >>>>>>>>>>> >>>>>>>>>>> [1] : >>>>>>>>>>> http://hg.openjdk.java.net/panama/dev/file/eaca2d16b80b/src/jdk.jextract/share/classes/com/sun/tools/jextract/SymbolFilter.java#l89 >>>>>>>>>>> [2] : >>>>>>>>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/jlibpath/webrev.01/ From sundararajan.athijegannathan at oracle.com Wed Jan 23 09:54:10 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Wed, 23 Jan 2019 15:24:10 +0530 Subject: [foreign] RFC: Jextract -l options ignored by SymbolFilter when -L is not specified In-Reply-To: <0477c3fc285fc48c81d9a7f24561c118@xs4all.nl> References: <698464e8f7692e12a7062155c7e599d9@xs4all.nl> <5C46AD23.9000207@oracle.com> <4089850d-fec6-40cf-c036-e666169a84b3@oracle.com> <30080ab5a99a1a9abf34a89a6c70b8b8@xs4all.nl> <4de2ba3b-4971-95d3-cb3b-9a0d478656d9@oracle.com> <036DADBC-851D-494F-8FC0-094505BE5696@oracle.com> <2f86de9e0e566913c1130e89556c541e@xs4all.nl> <5C47CB05.6060701@oracle.com> <19941C48-DF18-414B-AC02-83F1C62F6712@oracle.com> <5C47D814.5090203@oracle.com> <5C47D93C.7040008@oracle.com> <0477c3fc285fc48c81d9a7f24561c118@xs4all.nl> Message-ID: <5C4839C2.2000000@oracle.com> Inline responses below.. On 23/01/19, 3:13 PM, Jorn Vernee wrote: > The default value of the java.library.path seems to have changed over > time, or maybe there's a difference between operating systems. > Previously I understood that java.library.path defaulted to the value > of the classpath. But now, if I open jshell and query > java.library.path I get some paths that seem to be injected, which > seem to be the paths of the pre-built platform JNI libs you mention. > Appended to that there is the contents of PATH, which on Windows is > used to look up DLLs. And finally appended to that is the `.` path. > (FWIW, the same value is returned when I create a simple class which > queries and returns the value) > > So "you won't find anything useful for the general C shared objects" > is not true in my case, as we essentially get the system's default DLL > search path with java.library.path (i.e. the contents of PATH). > On Mac: jshell> System.getProperty("java.library.path") $1 ==> "/Users/SATHIJEG/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java:." As you can see, this does not have the usual places like /usr/lib etc. As always, Windows is different :) > Like you said, setting the java.library.path requires -J, but having > users do that should not be the goal imho. Rather, it acts as a > default set of search paths, and if more need to be added, the user > should use -L. > If the default is useless for C shared objects (which it is on Mac for example), user has to set the property and -J is the only way. >> --library-path / -lp to specify library paths. Similar to >> --module-path/--class-path etc. If possible, we should detect and use >> platform specific library paths that "ld" would use. This way, default >> would be useful > > Tbh, I don't really see how "--library-path / -lp" differs from "-L" > that we have currently, except that the paths are passed in a > different format. > Because -L seems to suggest that the option is same/similar to platform linker option. That is why we started. But now that we are deviating, a different name is better. Besides --library-path makes it more like --module-path, --class-path of the Java world. > If the contents of java.library.path is not good enough on other > platforms, maybe we could use an environment variable like > JEXTRACT_LIBRARY_PATH which contains the default search paths. The > advantage of using an environment variable is that it can be set once > and then used across multiple jextract runs, where command line > options have to be passed every time. Not sure of new env. var. If we want to use env. var, we might as well use LD_LIBRARY_PATH or PATH (depending on platform :) ) -Sundar > > Jorn > > Sundararajan Athijegannathan schreef op 2019-01-23 04:02: >> I think it'd be better to use new jextract specific options: >> >> --library-path / -lp to specify library paths. Similar to >> --module-path/--class-path etc. If possible, we should detect and use >> platform specific library paths that "ld" would use. This way, default >> would be useful >> >> --missing-symbols=warn|error|exclude option. Tells what should be done >> on missing native symbols. >> >> -Sundar >> >> On 23/01/19, 8:27 AM, Sundararajan Athijegannathan wrote: >>> Yes, I got that part - that's why I wrote my reason as to why that >>> is not a good idea. >>> >>> java.library.path's default value includes only the directories that >>> contain pre-built platform JNI libs - unlike platform native default >>> library path used by "ld". You won't find anything useful for the >>> general C shared objects. Of course user can set it - which requires >>> -J-D... option as I mentioned. >>> >>> -Sundar >>> >>> On 23/01/19, 8:19 AM, Henry Jen wrote: >>>> What we are suggesting is that, -L is supplement to >>>> java.library.path. that -L is optional while -l is required for >>>> validating again the shared library. >>>> >>>> In fact, -L is pretty much like a shortcut to prepend more search >>>> path for java.library.path. >>>> >>>> Cheers, >>>> Henry >>>> >>>>> On Jan 22, 2019, at 6:01 PM, Sundararajan >>>>> Athijegannathan wrote: >>>>> >>>>> java.library.path contains only the JRE's own directories + system >>>>> specific java dirs. This does not include directories like >>>>> /usr/lib or /usr/local/lib (where you're likely to find shared >>>>> objects of C libraries that are installed). In other words, the >>>>> default value of java.library.path system property does not get >>>>> you too far. >>>>> >>>>> If user has to pass java.library.path for jextract, s/he has to use >>>>> >>>>> jextract -J-Djava.library.path=... >>>>> >>>>> -J is needed to set pass JVM option to jdk/bin tools other than >>>>> "java". That is not super friendly. S/he might as well use -L. So >>>>> I'm not sure I agree with using "java.library.path" as fallback. >>>>> >>>>> -Sundar >>>>> >>>>> On 23/01/19, 12:50 AM, Jorn Vernee wrote: >>>>>> This also seems the most natural to me, since it follows what the >>>>>> linker flags do. >>>>>> >>>>>> -L is to specify additional linker directories. We would consider >>>>>> java.library.path to be the "default"/"system" directories. >>>>>> >>>>>> This is also what I tried to do in the patch [1] (minor update). >>>>>> With the addition of emitting a warning that symbol filtering is >>>>>> disabled when -l is used but no library paths are available >>>>>> (either in -L options or in java.library.path). >>>>>> >>>>>> That said, I think having an extra option to explicitly turn on, >>>>>> or off, the symbol checking is a good idea as well. >>>>>> >>>>>> Jorn >>>>>> >>>>>> [1]: >>>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/jlibpath/webrev.02/ >>>>>> >>>>>> >>>>>> Henry Jen schreef op 2019-01-22 17:40: >>>>>>> It is preferred to keep options compatible with cc when applicable. >>>>>>> >>>>>>> -L is only to provide the path for library at tooling time, >>>>>>> that?s jextract. >>>>>>> ?infer-path is similar to -R, will record the path for searching at >>>>>>> runtime, that?s is, the path specified with -L will be added into >>>>>>> search path of library. >>>>>>> >>>>>>> As symbol check, it should be enabled with -l. -L is simply provide >>>>>>> extra path to search for the library, without -L, it will simply >>>>>>> search in java.library.path. >>>>>>> >>>>>>> Cheers, >>>>>>> Henry >>>>>>> >>>>>>> >>>>>>>> On Jan 22, 2019, at 4:31 AM, Maurizio >>>>>>>> Cimadamore wrote: >>>>>>>> >>>>>>>> >>>>>>>> On 22/01/2019 12:09, Jorn Vernee wrote: >>>>>>>>> This sounds good, I really like the idea of a separate option >>>>>>>>> to enable the symbol filtering. But can you share what you >>>>>>>>> think the role of java.library.path should be as well? >>>>>>>> I think using java.library.path as a default for the missing >>>>>>>> symbol check could be ok. But I don't think it would be ok to >>>>>>>> use it as a basis for infer-rpath. That is, I don't want static >>>>>>>> properties (e.g. valid at extraction time) to spill onto the >>>>>>>> runtime. If the user really wants to set some dynamic property, >>>>>>>> it has to use an explicit flag to do so (e.g. -L). >>>>>>>> >>>>>>>> Maurizio >>>>>>>> >>>>>>>>> Jorn >>>>>>>>> >>>>>>>>> Maurizio Cimadamore schreef op 2019-01-22 12:58: >>>>>>>>>> Looking at this, I remember being confused about this too. >>>>>>>>>> >>>>>>>>>> Let me try to see if we can find a better stacking for the >>>>>>>>>> existing >>>>>>>>>> options - as Sundar said, we currently have: >>>>>>>>>> >>>>>>>>>> * -l >>>>>>>>>> >>>>>>>>>> This option is used to specify library _names_. >>>>>>>>>> >>>>>>>>>> The main goal of this option is to alter the contents of the >>>>>>>>>> @NativeHeader annotation (by adding the library name) but >>>>>>>>>> there are, >>>>>>>>>> as we shall see, other subtle side-effects. >>>>>>>>>> >>>>>>>>>> * -L + -l >>>>>>>>>> >>>>>>>>>> When both -L and -l are specified, the so called "missing >>>>>>>>>> symbols >>>>>>>>>> check" will kick in, that is, jextract will check that all >>>>>>>>>> symbols in >>>>>>>>>> the library are indeed defined in the header files being >>>>>>>>>> extracted. A >>>>>>>>>> subtle side-effect of that check, is that when -l and -L are >>>>>>>>>> specified >>>>>>>>>> together, and the missing symbol check is enabled, jextract >>>>>>>>>> will warn >>>>>>>>>> for symbols not found and _it will exclude them_ from the >>>>>>>>>> extracted >>>>>>>>>> classfile (w/o need for --include-symbols or --exclude-symbols). >>>>>>>>>> >>>>>>>>>> * -L + -l + -infer-rpath >>>>>>>>>> >>>>>>>>>> When -L and -l are used together, and the -infer-rpath option is >>>>>>>>>> given, a runtime library path will be inferred from the >>>>>>>>>> contents of >>>>>>>>>> -L, and will be stored in @NativeHeader, so that the binder >>>>>>>>>> can use >>>>>>>>>> it. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> I think the status quo is a bit confusing - because -L has >>>>>>>>>> multiple >>>>>>>>>> functions (it serves up the library paths to be used as inferred >>>>>>>>>> rpaths, and it also serves up the library paths to be used >>>>>>>>>> for the >>>>>>>>>> missing symbol check). I think a more consistent stacking >>>>>>>>>> could be >>>>>>>>>> something like this: >>>>>>>>>> >>>>>>>>>> -l --> used to specify library _names_; only side-effect is >>>>>>>>>> contents >>>>>>>>>> of @NativeHeader >>>>>>>>>> >>>>>>>>>> -L --> used to specify _custom_ library _paths_; no >>>>>>>>>> side-effects >>>>>>>>>> >>>>>>>>>> -exclude-missing -> must be used in conjunction with -l and >>>>>>>>>> -L ; >>>>>>>>>> enables the missing symbol check and auto-exclusion >>>>>>>>>> >>>>>>>>>> -infer-rpath -> must be used in conjunction with -l and -L ; >>>>>>>>>> enables >>>>>>>>>> rpath inference (rpath inferred with paths specified in -L) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thoughts? >>>>>>>>>> >>>>>>>>>> Maurizio >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 22/01/2019 05:41, Sundararajan Athijegannathan wrote: >>>>>>>>>>> I don't think it is a bug - afaik it is as per design. The >>>>>>>>>>> primary use of "-l" is to record the library in annotation >>>>>>>>>>> of the generated jar - so that binder can auto-load the >>>>>>>>>>> library (either from java.library.path configuration or >>>>>>>>>>> -rpath value recorded in annotation). It is okay to record >>>>>>>>>>> name of the shared object alone and leave the library path >>>>>>>>>>> configuration to java.library.path setting. >>>>>>>>>>> >>>>>>>>>>> "-L" option is added feature to perform missing symbols >>>>>>>>>>> checking. "-rpath" option is to add a path for library >>>>>>>>>>> search - so that binder can locate the shared object in the >>>>>>>>>>> specific directory. If no -rpath is specified, "-L" is used >>>>>>>>>>> for runtime search as well. >>>>>>>>>>> >>>>>>>>>>> -Sundar >>>>>>>>>>> >>>>>>>>>>> On 22/01/19, 12:01 AM, Jorn Vernee wrote: >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> I've recently updated the instructions for using libraries >>>>>>>>>>>> on Windows. For python the jextract example I gave was: >>>>>>>>>>>> >>>>>>>>>>>> jextract -l python27 -o "python.jar" -t "org.python" >>>>>>>>>>>> C:\Python27\include\Python.h >>>>>>>>>>>> >>>>>>>>>>>> I'm lacking an `-L` option here (for specifying library >>>>>>>>>>>> directories) since the contents of PATH seems to be added >>>>>>>>>>>> to java.library.path by default, and this is presumably >>>>>>>>>>>> also how jextract is able to load the library. But, since >>>>>>>>>>>> I'm not using an `-L` option, SymbolFilter is not checking >>>>>>>>>>>> if the symbols are in the python27.dll [1] >>>>>>>>>>>> >>>>>>>>>>>> private void initSymChecker(List >>>>>>>>>>>> linkCheckPaths) { >>>>>>>>>>>> if (!libraryNames.isEmpty()&& >>>>>>>>>>>> !linkCheckPaths.isEmpty()) { >>>>>>>>>>>> // ... init symChecker >>>>>>>>>>>> } else { >>>>>>>>>>>> symChecker = null; >>>>>>>>>>>> } >>>>>>>>>>>> } >>>>>>>>>>>> >>>>>>>>>>>> (linkCheckPaths comes from the -L option values) >>>>>>>>>>>> >>>>>>>>>>>> This behaviour is somewhat unexpected. At least a warning >>>>>>>>>>>> that missing an `-L` option will turn off symbol checking >>>>>>>>>>>> would be nice. >>>>>>>>>>>> >>>>>>>>>>>> We could also add the paths in `java.library.path` to the >>>>>>>>>>>> list of link check paths in jextract [2]. That would mean >>>>>>>>>>>> that the symbol checker would run for the example command. >>>>>>>>>>>> >>>>>>>>>>>> What do you think? >>>>>>>>>>>> >>>>>>>>>>>> Jorn >>>>>>>>>>>> >>>>>>>>>>>> [1] : >>>>>>>>>>>> http://hg.openjdk.java.net/panama/dev/file/eaca2d16b80b/src/jdk.jextract/share/classes/com/sun/tools/jextract/SymbolFilter.java#l89 >>>>>>>>>>>> [2] : >>>>>>>>>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/jlibpath/webrev.01/ >>>>>>>>>>>> From sundararajan.athijegannathan at oracle.com Wed Jan 23 09:59:18 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Wed, 23 Jan 2019 15:29:18 +0530 Subject: [foreign] RFC: Jextract -l options ignored by SymbolFilter when -L is not specified In-Reply-To: <5C4839C2.2000000@oracle.com> References: <698464e8f7692e12a7062155c7e599d9@xs4all.nl> <5C46AD23.9000207@oracle.com> <4089850d-fec6-40cf-c036-e666169a84b3@oracle.com> <30080ab5a99a1a9abf34a89a6c70b8b8@xs4all.nl> <4de2ba3b-4971-95d3-cb3b-9a0d478656d9@oracle.com> <036DADBC-851D-494F-8FC0-094505BE5696@oracle.com> <2f86de9e0e566913c1130e89556c541e@xs4all.nl> <5C47CB05.6060701@oracle.com> <19941C48-DF18-414B-AC02-83F1C62F6712@oracle.com> <5C47D814.5090203@oracle.com> <5C47D93C.7040008@oracle.com> <0477c3fc285fc48c81d9a7f24561c118@xs4all.nl> <5C4839C2.2000000@oracle.com> Message-ID: <5C483AF6.6050208@oracle.com> On 23/01/19, 3:24 PM, Sundararajan Athijegannathan wrote: > > Inline responses below.. > > On 23/01/19, 3:13 PM, Jorn Vernee wrote: >> The default value of the java.library.path seems to have changed over >> time, or maybe there's a difference between operating systems. >> Previously I understood that java.library.path defaulted to the value >> of the classpath. But now, if I open jshell and query >> java.library.path I get some paths that seem to be injected, which >> seem to be the paths of the pre-built platform JNI libs you mention. >> Appended to that there is the contents of PATH, which on Windows is >> used to look up DLLs. And finally appended to that is the `.` path. >> (FWIW, the same value is returned when I create a simple class which >> queries and returns the value) >> >> So "you won't find anything useful for the general C shared objects" >> is not true in my case, as we essentially get the system's default >> DLL search path with java.library.path (i.e. the contents of PATH). >> > On Mac: > > jshell> System.getProperty("java.library.path") > $1 ==> > "/Users/SATHIJEG/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java:." > > As you can see, this does not have the usual places like /usr/lib etc. > As always, Windows is different :) >> Like you said, setting the java.library.path requires -J, but having >> users do that should not be the goal imho. Rather, it acts as a >> default set of search paths, and if more need to be added, the user >> should use -L. >> > If the default is useless for C shared objects (which it is on Mac for > example), user has to set the property and -J is the only way. I meant that (on Mac) user cannot make use of the default. Either s/he set the property or use explicit library path option. Setting System property involves -J-.. => might as well always use explicit command line option to pass library path. > >>> --library-path / -lp to specify library paths. Similar to >>> --module-path/--class-path etc. If possible, we should detect and use >>> platform specific library paths that "ld" would use. This way, default >>> would be useful >> >> Tbh, I don't really see how "--library-path / -lp" differs from "-L" >> that we have currently, except that the paths are passed in a >> different format. >> > Because -L seems to suggest that the option is same/similar to > platform linker option. That is why we started. But now that we are > deviating, a different name is better. Besides --library-path makes it > more like --module-path, --class-path of the Java world. > >> If the contents of java.library.path is not good enough on other >> platforms, maybe we could use an environment variable like >> JEXTRACT_LIBRARY_PATH which contains the default search paths. The >> advantage of using an environment variable is that it can be set once >> and then used across multiple jextract runs, where command line >> options have to be passed every time. > Not sure of new env. var. If we want to use env. var, we might as well > use LD_LIBRARY_PATH or PATH (depending on platform :) ) > > -Sundar >> >> Jorn >> >> Sundararajan Athijegannathan schreef op 2019-01-23 04:02: >>> I think it'd be better to use new jextract specific options: >>> >>> --library-path / -lp to specify library paths. Similar to >>> --module-path/--class-path etc. If possible, we should detect and use >>> platform specific library paths that "ld" would use. This way, default >>> would be useful >>> >>> --missing-symbols=warn|error|exclude option. Tells what should be done >>> on missing native symbols. >>> >>> -Sundar >>> >>> On 23/01/19, 8:27 AM, Sundararajan Athijegannathan wrote: >>>> Yes, I got that part - that's why I wrote my reason as to why that >>>> is not a good idea. >>>> >>>> java.library.path's default value includes only the directories >>>> that contain pre-built platform JNI libs - unlike platform native >>>> default library path used by "ld". You won't find anything useful >>>> for the general C shared objects. Of course user can set it - which >>>> requires -J-D... option as I mentioned. >>>> >>>> -Sundar >>>> >>>> On 23/01/19, 8:19 AM, Henry Jen wrote: >>>>> What we are suggesting is that, -L is supplement to >>>>> java.library.path. that -L is optional while -l is required for >>>>> validating again the shared library. >>>>> >>>>> In fact, -L is pretty much like a shortcut to prepend more search >>>>> path for java.library.path. >>>>> >>>>> Cheers, >>>>> Henry >>>>> >>>>>> On Jan 22, 2019, at 6:01 PM, Sundararajan >>>>>> Athijegannathan wrote: >>>>>> >>>>>> java.library.path contains only the JRE's own directories + >>>>>> system specific java dirs. This does not include directories like >>>>>> /usr/lib or /usr/local/lib (where you're likely to find shared >>>>>> objects of C libraries that are installed). In other words, the >>>>>> default value of java.library.path system property does not get >>>>>> you too far. >>>>>> >>>>>> If user has to pass java.library.path for jextract, s/he has to use >>>>>> >>>>>> jextract -J-Djava.library.path=... >>>>>> >>>>>> -J is needed to set pass JVM option to jdk/bin tools other than >>>>>> "java". That is not super friendly. S/he might as well use -L. So >>>>>> I'm not sure I agree with using "java.library.path" as fallback. >>>>>> >>>>>> -Sundar >>>>>> >>>>>> On 23/01/19, 12:50 AM, Jorn Vernee wrote: >>>>>>> This also seems the most natural to me, since it follows what >>>>>>> the linker flags do. >>>>>>> >>>>>>> -L is to specify additional linker directories. We would >>>>>>> consider java.library.path to be the "default"/"system" >>>>>>> directories. >>>>>>> >>>>>>> This is also what I tried to do in the patch [1] (minor update). >>>>>>> With the addition of emitting a warning that symbol filtering is >>>>>>> disabled when -l is used but no library paths are available >>>>>>> (either in -L options or in java.library.path). >>>>>>> >>>>>>> That said, I think having an extra option to explicitly turn on, >>>>>>> or off, the symbol checking is a good idea as well. >>>>>>> >>>>>>> Jorn >>>>>>> >>>>>>> [1]: >>>>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/jlibpath/webrev.02/ >>>>>>> >>>>>>> >>>>>>> Henry Jen schreef op 2019-01-22 17:40: >>>>>>>> It is preferred to keep options compatible with cc when >>>>>>>> applicable. >>>>>>>> >>>>>>>> -L is only to provide the path for library at tooling time, >>>>>>>> that?s jextract. >>>>>>>> ?infer-path is similar to -R, will record the path for >>>>>>>> searching at >>>>>>>> runtime, that?s is, the path specified with -L will be added into >>>>>>>> search path of library. >>>>>>>> >>>>>>>> As symbol check, it should be enabled with -l. -L is simply >>>>>>>> provide >>>>>>>> extra path to search for the library, without -L, it will simply >>>>>>>> search in java.library.path. >>>>>>>> >>>>>>>> Cheers, >>>>>>>> Henry >>>>>>>> >>>>>>>> >>>>>>>>> On Jan 22, 2019, at 4:31 AM, Maurizio >>>>>>>>> Cimadamore wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> On 22/01/2019 12:09, Jorn Vernee wrote: >>>>>>>>>> This sounds good, I really like the idea of a separate option >>>>>>>>>> to enable the symbol filtering. But can you share what you >>>>>>>>>> think the role of java.library.path should be as well? >>>>>>>>> I think using java.library.path as a default for the missing >>>>>>>>> symbol check could be ok. But I don't think it would be ok to >>>>>>>>> use it as a basis for infer-rpath. That is, I don't want >>>>>>>>> static properties (e.g. valid at extraction time) to spill >>>>>>>>> onto the runtime. If the user really wants to set some dynamic >>>>>>>>> property, it has to use an explicit flag to do so (e.g. -L). >>>>>>>>> >>>>>>>>> Maurizio >>>>>>>>> >>>>>>>>>> Jorn >>>>>>>>>> >>>>>>>>>> Maurizio Cimadamore schreef op 2019-01-22 12:58: >>>>>>>>>>> Looking at this, I remember being confused about this too. >>>>>>>>>>> >>>>>>>>>>> Let me try to see if we can find a better stacking for the >>>>>>>>>>> existing >>>>>>>>>>> options - as Sundar said, we currently have: >>>>>>>>>>> >>>>>>>>>>> * -l >>>>>>>>>>> >>>>>>>>>>> This option is used to specify library _names_. >>>>>>>>>>> >>>>>>>>>>> The main goal of this option is to alter the contents of the >>>>>>>>>>> @NativeHeader annotation (by adding the library name) but >>>>>>>>>>> there are, >>>>>>>>>>> as we shall see, other subtle side-effects. >>>>>>>>>>> >>>>>>>>>>> * -L + -l >>>>>>>>>>> >>>>>>>>>>> When both -L and -l are specified, the so called "missing >>>>>>>>>>> symbols >>>>>>>>>>> check" will kick in, that is, jextract will check that all >>>>>>>>>>> symbols in >>>>>>>>>>> the library are indeed defined in the header files being >>>>>>>>>>> extracted. A >>>>>>>>>>> subtle side-effect of that check, is that when -l and -L are >>>>>>>>>>> specified >>>>>>>>>>> together, and the missing symbol check is enabled, jextract >>>>>>>>>>> will warn >>>>>>>>>>> for symbols not found and _it will exclude them_ from the >>>>>>>>>>> extracted >>>>>>>>>>> classfile (w/o need for --include-symbols or >>>>>>>>>>> --exclude-symbols). >>>>>>>>>>> >>>>>>>>>>> * -L + -l + -infer-rpath >>>>>>>>>>> >>>>>>>>>>> When -L and -l are used together, and the -infer-rpath >>>>>>>>>>> option is >>>>>>>>>>> given, a runtime library path will be inferred from the >>>>>>>>>>> contents of >>>>>>>>>>> -L, and will be stored in @NativeHeader, so that the binder >>>>>>>>>>> can use >>>>>>>>>>> it. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I think the status quo is a bit confusing - because -L has >>>>>>>>>>> multiple >>>>>>>>>>> functions (it serves up the library paths to be used as >>>>>>>>>>> inferred >>>>>>>>>>> rpaths, and it also serves up the library paths to be used >>>>>>>>>>> for the >>>>>>>>>>> missing symbol check). I think a more consistent stacking >>>>>>>>>>> could be >>>>>>>>>>> something like this: >>>>>>>>>>> >>>>>>>>>>> -l --> used to specify library _names_; only side-effect is >>>>>>>>>>> contents >>>>>>>>>>> of @NativeHeader >>>>>>>>>>> >>>>>>>>>>> -L --> used to specify _custom_ library _paths_; no >>>>>>>>>>> side-effects >>>>>>>>>>> >>>>>>>>>>> -exclude-missing -> must be used in conjunction with -l and >>>>>>>>>>> -L ; >>>>>>>>>>> enables the missing symbol check and auto-exclusion >>>>>>>>>>> >>>>>>>>>>> -infer-rpath -> must be used in conjunction with -l and -L >>>>>>>>>>> ; enables >>>>>>>>>>> rpath inference (rpath inferred with paths specified in -L) >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thoughts? >>>>>>>>>>> >>>>>>>>>>> Maurizio >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 22/01/2019 05:41, Sundararajan Athijegannathan wrote: >>>>>>>>>>>> I don't think it is a bug - afaik it is as per design. The >>>>>>>>>>>> primary use of "-l" is to record the library in annotation >>>>>>>>>>>> of the generated jar - so that binder can auto-load the >>>>>>>>>>>> library (either from java.library.path configuration or >>>>>>>>>>>> -rpath value recorded in annotation). It is okay to record >>>>>>>>>>>> name of the shared object alone and leave the library path >>>>>>>>>>>> configuration to java.library.path setting. >>>>>>>>>>>> >>>>>>>>>>>> "-L" option is added feature to perform missing symbols >>>>>>>>>>>> checking. "-rpath" option is to add a path for library >>>>>>>>>>>> search - so that binder can locate the shared object in the >>>>>>>>>>>> specific directory. If no -rpath is specified, "-L" is used >>>>>>>>>>>> for runtime search as well. >>>>>>>>>>>> >>>>>>>>>>>> -Sundar >>>>>>>>>>>> >>>>>>>>>>>> On 22/01/19, 12:01 AM, Jorn Vernee wrote: >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> I've recently updated the instructions for using libraries >>>>>>>>>>>>> on Windows. For python the jextract example I gave was: >>>>>>>>>>>>> >>>>>>>>>>>>> jextract -l python27 -o "python.jar" -t "org.python" >>>>>>>>>>>>> C:\Python27\include\Python.h >>>>>>>>>>>>> >>>>>>>>>>>>> I'm lacking an `-L` option here (for specifying library >>>>>>>>>>>>> directories) since the contents of PATH seems to be added >>>>>>>>>>>>> to java.library.path by default, and this is presumably >>>>>>>>>>>>> also how jextract is able to load the library. But, since >>>>>>>>>>>>> I'm not using an `-L` option, SymbolFilter is not checking >>>>>>>>>>>>> if the symbols are in the python27.dll [1] >>>>>>>>>>>>> >>>>>>>>>>>>> private void initSymChecker(List >>>>>>>>>>>>> linkCheckPaths) { >>>>>>>>>>>>> if (!libraryNames.isEmpty()&& >>>>>>>>>>>>> !linkCheckPaths.isEmpty()) { >>>>>>>>>>>>> // ... init symChecker >>>>>>>>>>>>> } else { >>>>>>>>>>>>> symChecker = null; >>>>>>>>>>>>> } >>>>>>>>>>>>> } >>>>>>>>>>>>> >>>>>>>>>>>>> (linkCheckPaths comes from the -L option values) >>>>>>>>>>>>> >>>>>>>>>>>>> This behaviour is somewhat unexpected. At least a warning >>>>>>>>>>>>> that missing an `-L` option will turn off symbol checking >>>>>>>>>>>>> would be nice. >>>>>>>>>>>>> >>>>>>>>>>>>> We could also add the paths in `java.library.path` to the >>>>>>>>>>>>> list of link check paths in jextract [2]. That would mean >>>>>>>>>>>>> that the symbol checker would run for the example command. >>>>>>>>>>>>> >>>>>>>>>>>>> What do you think? >>>>>>>>>>>>> >>>>>>>>>>>>> Jorn >>>>>>>>>>>>> >>>>>>>>>>>>> [1] : >>>>>>>>>>>>> http://hg.openjdk.java.net/panama/dev/file/eaca2d16b80b/src/jdk.jextract/share/classes/com/sun/tools/jextract/SymbolFilter.java#l89 >>>>>>>>>>>>> [2] : >>>>>>>>>>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/jlibpath/webrev.01/ >>>>>>>>>>>>> From jbvernee at xs4all.nl Wed Jan 23 10:05:07 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Wed, 23 Jan 2019 11:05:07 +0100 Subject: [foreign] RFC: Jextract -l options ignored by SymbolFilter when -L is not specified In-Reply-To: <5C4839C2.2000000@oracle.com> References: <698464e8f7692e12a7062155c7e599d9@xs4all.nl> <5C46AD23.9000207@oracle.com> <4089850d-fec6-40cf-c036-e666169a84b3@oracle.com> <30080ab5a99a1a9abf34a89a6c70b8b8@xs4all.nl> <4de2ba3b-4971-95d3-cb3b-9a0d478656d9@oracle.com> <036DADBC-851D-494F-8FC0-094505BE5696@oracle.com> <2f86de9e0e566913c1130e89556c541e@xs4all.nl> <5C47CB05.6060701@oracle.com> <19941C48-DF18-414B-AC02-83F1C62F6712@oracle.com> <5C47D814.5090203@oracle.com> <5C47D93C.7040008@oracle.com> <0477c3fc285fc48c81d9a7f24561c118@xs4all.nl> <5C4839C2.2000000@oracle.com> Message-ID: <16a22bcf7c55a3ac4dbd517aec6a555a@xs4all.nl> >> Tbh, I don't really see how "--library-path / -lp" differs from "-L" >> that we have currently, except that the paths are passed in a >> different format. >> > Because -L seems to suggest that the option is same/similar to > platform linker option. That is why we started. But now that we are > deviating, a different name is better. Besides --library-path makes it > more like --module-path, --class-path of the Java world. Ok, I see. That makes sense, especially when combined with the `--missing-symbols=warn|error|exclude` option you mentioned as well. >> If the contents of java.library.path is not good enough on other >> platforms, maybe we could use an environment variable like >> JEXTRACT_LIBRARY_PATH which contains the default search paths. The >> advantage of using an environment variable is that it can be set once >> and then used across multiple jextract runs, where command line >> options have to be passed every time. > Not sure of new env. var. If we want to use env. var, we might as well > use LD_LIBRARY_PATH or PATH (depending on platform :) ) Using LD_LIBRARY_PATH on Mac/Linux and PATH on Windows also seems like a good option to me. Having _some_ tweak-able default search paths would be nice. Jorn Sundararajan Athijegannathan schreef op 2019-01-23 10:54: > Inline responses below.. > > On 23/01/19, 3:13 PM, Jorn Vernee wrote: >> The default value of the java.library.path seems to have changed over >> time, or maybe there's a difference between operating systems. >> Previously I understood that java.library.path defaulted to the value >> of the classpath. But now, if I open jshell and query >> java.library.path I get some paths that seem to be injected, which >> seem to be the paths of the pre-built platform JNI libs you mention. >> Appended to that there is the contents of PATH, which on Windows is >> used to look up DLLs. And finally appended to that is the `.` path. >> (FWIW, the same value is returned when I create a simple class which >> queries and returns the value) >> >> So "you won't find anything useful for the general C shared objects" >> is not true in my case, as we essentially get the system's default DLL >> search path with java.library.path (i.e. the contents of PATH). >> > On Mac: > > jshell> System.getProperty("java.library.path") > $1 ==> > "/Users/SATHIJEG/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java:." > > As you can see, this does not have the usual places like /usr/lib etc. > As always, Windows is different :) >> Like you said, setting the java.library.path requires -J, but having >> users do that should not be the goal imho. Rather, it acts as a >> default set of search paths, and if more need to be added, the user >> should use -L. >> > If the default is useless for C shared objects (which it is on Mac for > example), user has to set the property and -J is the only way. > >>> --library-path / -lp to specify library paths. Similar to >>> --module-path/--class-path etc. If possible, we should detect and use >>> platform specific library paths that "ld" would use. This way, >>> default >>> would be useful >> >> Tbh, I don't really see how "--library-path / -lp" differs from "-L" >> that we have currently, except that the paths are passed in a >> different format. >> > Because -L seems to suggest that the option is same/similar to > platform linker option. That is why we started. But now that we are > deviating, a different name is better. Besides --library-path makes it > more like --module-path, --class-path of the Java world. > >> If the contents of java.library.path is not good enough on other >> platforms, maybe we could use an environment variable like >> JEXTRACT_LIBRARY_PATH which contains the default search paths. The >> advantage of using an environment variable is that it can be set once >> and then used across multiple jextract runs, where command line >> options have to be passed every time. > Not sure of new env. var. If we want to use env. var, we might as well > use LD_LIBRARY_PATH or PATH (depending on platform :) ) > > -Sundar >> >> Jorn >> >> Sundararajan Athijegannathan schreef op 2019-01-23 04:02: >>> I think it'd be better to use new jextract specific options: >>> >>> --library-path / -lp to specify library paths. Similar to >>> --module-path/--class-path etc. If possible, we should detect and use >>> platform specific library paths that "ld" would use. This way, >>> default >>> would be useful >>> >>> --missing-symbols=warn|error|exclude option. Tells what should be >>> done >>> on missing native symbols. >>> >>> -Sundar >>> >>> On 23/01/19, 8:27 AM, Sundararajan Athijegannathan wrote: >>>> Yes, I got that part - that's why I wrote my reason as to why that >>>> is not a good idea. >>>> >>>> java.library.path's default value includes only the directories that >>>> contain pre-built platform JNI libs - unlike platform native default >>>> library path used by "ld". You won't find anything useful for the >>>> general C shared objects. Of course user can set it - which requires >>>> -J-D... option as I mentioned. >>>> >>>> -Sundar >>>> >>>> On 23/01/19, 8:19 AM, Henry Jen wrote: >>>>> What we are suggesting is that, -L is supplement to >>>>> java.library.path. that -L is optional while -l is required for >>>>> validating again the shared library. >>>>> >>>>> In fact, -L is pretty much like a shortcut to prepend more search >>>>> path for java.library.path. >>>>> >>>>> Cheers, >>>>> Henry >>>>> >>>>>> On Jan 22, 2019, at 6:01 PM, Sundararajan >>>>>> Athijegannathan wrote: >>>>>> >>>>>> java.library.path contains only the JRE's own directories + system >>>>>> specific java dirs. This does not include directories like >>>>>> /usr/lib or /usr/local/lib (where you're likely to find shared >>>>>> objects of C libraries that are installed). In other words, the >>>>>> default value of java.library.path system property does not get >>>>>> you too far. >>>>>> >>>>>> If user has to pass java.library.path for jextract, s/he has to >>>>>> use >>>>>> >>>>>> jextract -J-Djava.library.path=... >>>>>> >>>>>> -J is needed to set pass JVM option to jdk/bin tools other than >>>>>> "java". That is not super friendly. S/he might as well use -L. So >>>>>> I'm not sure I agree with using "java.library.path" as fallback. >>>>>> >>>>>> -Sundar >>>>>> >>>>>> On 23/01/19, 12:50 AM, Jorn Vernee wrote: >>>>>>> This also seems the most natural to me, since it follows what the >>>>>>> linker flags do. >>>>>>> >>>>>>> -L is to specify additional linker directories. We would consider >>>>>>> java.library.path to be the "default"/"system" directories. >>>>>>> >>>>>>> This is also what I tried to do in the patch [1] (minor update). >>>>>>> With the addition of emitting a warning that symbol filtering is >>>>>>> disabled when -l is used but no library paths are available >>>>>>> (either in -L options or in java.library.path). >>>>>>> >>>>>>> That said, I think having an extra option to explicitly turn on, >>>>>>> or off, the symbol checking is a good idea as well. >>>>>>> >>>>>>> Jorn >>>>>>> >>>>>>> [1]: >>>>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/jlibpath/webrev.02/ >>>>>>> Henry Jen schreef op 2019-01-22 17:40: >>>>>>>> It is preferred to keep options compatible with cc when >>>>>>>> applicable. >>>>>>>> >>>>>>>> -L is only to provide the path for library at tooling time, >>>>>>>> that?s jextract. >>>>>>>> ?infer-path is similar to -R, will record the path for searching >>>>>>>> at >>>>>>>> runtime, that?s is, the path specified with -L will be added >>>>>>>> into >>>>>>>> search path of library. >>>>>>>> >>>>>>>> As symbol check, it should be enabled with -l. -L is simply >>>>>>>> provide >>>>>>>> extra path to search for the library, without -L, it will simply >>>>>>>> search in java.library.path. >>>>>>>> >>>>>>>> Cheers, >>>>>>>> Henry >>>>>>>> >>>>>>>> >>>>>>>>> On Jan 22, 2019, at 4:31 AM, Maurizio >>>>>>>>> Cimadamore wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> On 22/01/2019 12:09, Jorn Vernee wrote: >>>>>>>>>> This sounds good, I really like the idea of a separate option >>>>>>>>>> to enable the symbol filtering. But can you share what you >>>>>>>>>> think the role of java.library.path should be as well? >>>>>>>>> I think using java.library.path as a default for the missing >>>>>>>>> symbol check could be ok. But I don't think it would be ok to >>>>>>>>> use it as a basis for infer-rpath. That is, I don't want static >>>>>>>>> properties (e.g. valid at extraction time) to spill onto the >>>>>>>>> runtime. If the user really wants to set some dynamic property, >>>>>>>>> it has to use an explicit flag to do so (e.g. -L). >>>>>>>>> >>>>>>>>> Maurizio >>>>>>>>> >>>>>>>>>> Jorn >>>>>>>>>> >>>>>>>>>> Maurizio Cimadamore schreef op 2019-01-22 12:58: >>>>>>>>>>> Looking at this, I remember being confused about this too. >>>>>>>>>>> >>>>>>>>>>> Let me try to see if we can find a better stacking for the >>>>>>>>>>> existing >>>>>>>>>>> options - as Sundar said, we currently have: >>>>>>>>>>> >>>>>>>>>>> * -l >>>>>>>>>>> >>>>>>>>>>> This option is used to specify library _names_. >>>>>>>>>>> >>>>>>>>>>> The main goal of this option is to alter the contents of the >>>>>>>>>>> @NativeHeader annotation (by adding the library name) but >>>>>>>>>>> there are, >>>>>>>>>>> as we shall see, other subtle side-effects. >>>>>>>>>>> >>>>>>>>>>> * -L + -l >>>>>>>>>>> >>>>>>>>>>> When both -L and -l are specified, the so called "missing >>>>>>>>>>> symbols >>>>>>>>>>> check" will kick in, that is, jextract will check that all >>>>>>>>>>> symbols in >>>>>>>>>>> the library are indeed defined in the header files being >>>>>>>>>>> extracted. A >>>>>>>>>>> subtle side-effect of that check, is that when -l and -L are >>>>>>>>>>> specified >>>>>>>>>>> together, and the missing symbol check is enabled, jextract >>>>>>>>>>> will warn >>>>>>>>>>> for symbols not found and _it will exclude them_ from the >>>>>>>>>>> extracted >>>>>>>>>>> classfile (w/o need for --include-symbols or >>>>>>>>>>> --exclude-symbols). >>>>>>>>>>> >>>>>>>>>>> * -L + -l + -infer-rpath >>>>>>>>>>> >>>>>>>>>>> When -L and -l are used together, and the -infer-rpath option >>>>>>>>>>> is >>>>>>>>>>> given, a runtime library path will be inferred from the >>>>>>>>>>> contents of >>>>>>>>>>> -L, and will be stored in @NativeHeader, so that the binder >>>>>>>>>>> can use >>>>>>>>>>> it. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I think the status quo is a bit confusing - because -L has >>>>>>>>>>> multiple >>>>>>>>>>> functions (it serves up the library paths to be used as >>>>>>>>>>> inferred >>>>>>>>>>> rpaths, and it also serves up the library paths to be used >>>>>>>>>>> for the >>>>>>>>>>> missing symbol check). I think a more consistent stacking >>>>>>>>>>> could be >>>>>>>>>>> something like this: >>>>>>>>>>> >>>>>>>>>>> -l --> used to specify library _names_; only side-effect is >>>>>>>>>>> contents >>>>>>>>>>> of @NativeHeader >>>>>>>>>>> >>>>>>>>>>> -L --> used to specify _custom_ library _paths_; no >>>>>>>>>>> side-effects >>>>>>>>>>> >>>>>>>>>>> -exclude-missing -> must be used in conjunction with -l and >>>>>>>>>>> -L ; >>>>>>>>>>> enables the missing symbol check and auto-exclusion >>>>>>>>>>> >>>>>>>>>>> -infer-rpath -> must be used in conjunction with -l and -L ; >>>>>>>>>>> enables >>>>>>>>>>> rpath inference (rpath inferred with paths specified in -L) >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thoughts? >>>>>>>>>>> >>>>>>>>>>> Maurizio >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 22/01/2019 05:41, Sundararajan Athijegannathan wrote: >>>>>>>>>>>> I don't think it is a bug - afaik it is as per design. The >>>>>>>>>>>> primary use of "-l" is to record the library in annotation >>>>>>>>>>>> of the generated jar - so that binder can auto-load the >>>>>>>>>>>> library (either from java.library.path configuration or >>>>>>>>>>>> -rpath value recorded in annotation). It is okay to record >>>>>>>>>>>> name of the shared object alone and leave the library path >>>>>>>>>>>> configuration to java.library.path setting. >>>>>>>>>>>> >>>>>>>>>>>> "-L" option is added feature to perform missing symbols >>>>>>>>>>>> checking. "-rpath" option is to add a path for library >>>>>>>>>>>> search - so that binder can locate the shared object in the >>>>>>>>>>>> specific directory. If no -rpath is specified, "-L" is used >>>>>>>>>>>> for runtime search as well. >>>>>>>>>>>> >>>>>>>>>>>> -Sundar >>>>>>>>>>>> >>>>>>>>>>>> On 22/01/19, 12:01 AM, Jorn Vernee wrote: >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> I've recently updated the instructions for using libraries >>>>>>>>>>>>> on Windows. For python the jextract example I gave was: >>>>>>>>>>>>> >>>>>>>>>>>>> jextract -l python27 -o "python.jar" -t "org.python" >>>>>>>>>>>>> C:\Python27\include\Python.h >>>>>>>>>>>>> >>>>>>>>>>>>> I'm lacking an `-L` option here (for specifying library >>>>>>>>>>>>> directories) since the contents of PATH seems to be added >>>>>>>>>>>>> to java.library.path by default, and this is presumably >>>>>>>>>>>>> also how jextract is able to load the library. But, since >>>>>>>>>>>>> I'm not using an `-L` option, SymbolFilter is not checking >>>>>>>>>>>>> if the symbols are in the python27.dll [1] >>>>>>>>>>>>> >>>>>>>>>>>>> private void initSymChecker(List >>>>>>>>>>>>> linkCheckPaths) { >>>>>>>>>>>>> if (!libraryNames.isEmpty()&& >>>>>>>>>>>>> !linkCheckPaths.isEmpty()) { >>>>>>>>>>>>> // ... init symChecker >>>>>>>>>>>>> } else { >>>>>>>>>>>>> symChecker = null; >>>>>>>>>>>>> } >>>>>>>>>>>>> } >>>>>>>>>>>>> >>>>>>>>>>>>> (linkCheckPaths comes from the -L option values) >>>>>>>>>>>>> >>>>>>>>>>>>> This behaviour is somewhat unexpected. At least a warning >>>>>>>>>>>>> that missing an `-L` option will turn off symbol checking >>>>>>>>>>>>> would be nice. >>>>>>>>>>>>> >>>>>>>>>>>>> We could also add the paths in `java.library.path` to the >>>>>>>>>>>>> list of link check paths in jextract [2]. That would mean >>>>>>>>>>>>> that the symbol checker would run for the example command. >>>>>>>>>>>>> >>>>>>>>>>>>> What do you think? >>>>>>>>>>>>> >>>>>>>>>>>>> Jorn >>>>>>>>>>>>> >>>>>>>>>>>>> [1] : >>>>>>>>>>>>> http://hg.openjdk.java.net/panama/dev/file/eaca2d16b80b/src/jdk.jextract/share/classes/com/sun/tools/jextract/SymbolFilter.java#l89 >>>>>>>>>>>>> [2] : >>>>>>>>>>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/jlibpath/webrev.01/ From maurizio.cimadamore at oracle.com Wed Jan 23 10:37:46 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 23 Jan 2019 10:37:46 +0000 Subject: [foreign] RFC: Jextract -l options ignored by SymbolFilter when -L is not specified In-Reply-To: <0477c3fc285fc48c81d9a7f24561c118@xs4all.nl> References: <698464e8f7692e12a7062155c7e599d9@xs4all.nl> <5C46AD23.9000207@oracle.com> <4089850d-fec6-40cf-c036-e666169a84b3@oracle.com> <30080ab5a99a1a9abf34a89a6c70b8b8@xs4all.nl> <4de2ba3b-4971-95d3-cb3b-9a0d478656d9@oracle.com> <036DADBC-851D-494F-8FC0-094505BE5696@oracle.com> <2f86de9e0e566913c1130e89556c541e@xs4all.nl> <5C47CB05.6060701@oracle.com> <19941C48-DF18-414B-AC02-83F1C62F6712@oracle.com> <5C47D814.5090203@oracle.com> <5C47D93C.7040008@oracle.com> <0477c3fc285fc48c81d9a7f24561c118@xs4all.nl> Message-ID: <8c7079b5-4fd1-eaba-4923-445a55b34a0e@oracle.com> For what is worth (not arguing in favor or against this or that proposal), on Ubuntu I have observed that Oracle JDK has a different java.library.path than on Ubuntu's OpenJDK (which you install via apt). So java.library.path is not very stable, even on the same platform/OS :-) Maurizio On 23/01/2019 09:43, Jorn Vernee wrote: > So "you won't find anything useful for the general C shared objects" > is not true in my case, as we essentially get the system's default DLL > search path with java.library.path (i.e. the contents of PATH). From maurizio.cimadamore at oracle.com Wed Jan 23 10:41:12 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 23 Jan 2019 10:41:12 +0000 Subject: [foreign] RFC: Jextract -l options ignored by SymbolFilter when -L is not specified In-Reply-To: <16a22bcf7c55a3ac4dbd517aec6a555a@xs4all.nl> References: <698464e8f7692e12a7062155c7e599d9@xs4all.nl> <5C46AD23.9000207@oracle.com> <4089850d-fec6-40cf-c036-e666169a84b3@oracle.com> <30080ab5a99a1a9abf34a89a6c70b8b8@xs4all.nl> <4de2ba3b-4971-95d3-cb3b-9a0d478656d9@oracle.com> <036DADBC-851D-494F-8FC0-094505BE5696@oracle.com> <2f86de9e0e566913c1130e89556c541e@xs4all.nl> <5C47CB05.6060701@oracle.com> <19941C48-DF18-414B-AC02-83F1C62F6712@oracle.com> <5C47D814.5090203@oracle.com> <5C47D93C.7040008@oracle.com> <0477c3fc285fc48c81d9a7f24561c118@xs4all.nl> <5C4839C2.2000000@oracle.com> <16a22bcf7c55a3ac4dbd517aec6a555a@xs4all.nl> Message-ID: <95cdc7fc-925d-93f4-d5f6-2eff3df548ff@oracle.com> On 23/01/2019 10:05, Jorn Vernee wrote: >> Because -L seems to suggest that the option is same/similar to >> platform linker option. That is why we started. But now that we are >> deviating, a different name is better. Besides --library-path makes it >> more like --module-path, --class-path of the Java world. > > Ok, I see. That makes sense, especially when combined with the > `--missing-symbols=warn|error|exclude` option you mentioned as well. I'm not sure I understand this point - what is the deviation we're talking about? Maurizio From maurizio.cimadamore at oracle.com Wed Jan 23 10:46:46 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 23 Jan 2019 10:46:46 +0000 Subject: scopes writeup In-Reply-To: <0e23525d-aefb-8484-8f46-d2e4897ee93d@gmail.com> References: <6fbff2eb-b100-0c03-7663-c732a3904302@oracle.com> <0e23525d-aefb-8484-8f46-d2e4897ee93d@gmail.com> Message-ID: <064ce943-e73e-015c-aa3f-1b6784baee36@oracle.com> On 23/01/2019 01:03, Samuel Audet wrote: > Hi, Maurizio, > > I see there is an "interface Resource", but it doesn't allow us to > provide custom allocators and deallocators. How could we, for example, > manage all resources including those that are not pointers, structs, > etc with a single Scope? Do we need to provide a new implementation of > Scope for each type of Resource? In the current proposal, the only kind of resources available are coming directly from Scope, so Scope knows how to clean them up. Note also that the name 'Scope' is too general, and will probably be changed to something more specific like MemoryScope (to align with what's done in other projects - see Loom's FiberScope). In the future (provided we have concrete use cases for it) we can evolve the Scope API to cater for custom deallocation strategies - not sure if the custom deallocation goes in Resource - probably should be passed as a lambda when you call Scope::allocateXYZ. But in any case, as mentioned in the email, this proposal starts (very) simple, we can add to it as use cases pop up. Maurizio > > Samuel > > On 1/23/19 1:27 AM, Maurizio Cimadamore wrote: >> Hi, >> at the end of last year I tried coming up with a proposal [1] which >> aimed at finding the basic concepts behind Panama scopes - I've been >> iterating (with the help of some internal feedback) on that proposal, >> and put it back together in a different form which, all things >> considered, it's not too far off from where we are now. >> >> https://cr.openjdk.java.net/~mcimadamore/panama/scopes.html >> >> That is, we still have a Scope abstraction - memory regions are still >> hidden under Scopes, and not part of the public API (in an attempt to >> limit the number of concepts exposed by the API - at least for now). >> Pointer is an immutable cursor into a memory region. Resources are >> just things with a scope (turns out, a native library is another such >> thing, since a native library needs a library-wide scope for >> allocating its own globals, etc.). >> >> On top of what we have now, this writeup pushes forward an improved >> ownership model, as well as some discussions concerning >> thread-confinement of scopes. >> >> I think this tweaked API proposal has the potential of fixing the >> current lifecycle issues, w/o making the API too complex to use. >> Additional abstraction (such as MemoryRegion) can be added back to >> the public API on a by need basis (e.g. provided relevant use cases). >> >> An experimental patch implementing the proposed approach can be found >> here: >> >> http://cr.openjdk.java.net/~mcimadamore/panama/scope-ownership_v2/ >> >> Comments welcome! >> >> Maurizio >> >> [1] - >> https://mail.openjdk.java.net/pipermail/panama-dev/2018-December/003605.html >> >> > From jbvernee at xs4all.nl Wed Jan 23 10:49:51 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Wed, 23 Jan 2019 11:49:51 +0100 Subject: [foreign] RFC: Jextract -l options ignored by SymbolFilter when -L is not specified In-Reply-To: <95cdc7fc-925d-93f4-d5f6-2eff3df548ff@oracle.com> References: <698464e8f7692e12a7062155c7e599d9@xs4all.nl> <5C46AD23.9000207@oracle.com> <4089850d-fec6-40cf-c036-e666169a84b3@oracle.com> <30080ab5a99a1a9abf34a89a6c70b8b8@xs4all.nl> <4de2ba3b-4971-95d3-cb3b-9a0d478656d9@oracle.com> <036DADBC-851D-494F-8FC0-094505BE5696@oracle.com> <2f86de9e0e566913c1130e89556c541e@xs4all.nl> <5C47CB05.6060701@oracle.com> <19941C48-DF18-414B-AC02-83F1C62F6712@oracle.com> <5C47D814.5090203@oracle.com> <5C47D93C.7040008@oracle.com> <0477c3fc285fc48c81d9a7f24561c118@xs4all.nl> <5C4839C2.2000000@oracle.com> <16a22bcf7c55a3ac4dbd517aec6a555a@xs4all.nl> <95cdc7fc-925d-93f4-d5f6-2eff3df548ff@oracle.com> Message-ID: That jextract's -L also has the added functionality of turning on symbol checking. Jorn Maurizio Cimadamore schreef op 2019-01-23 11:41: > On 23/01/2019 10:05, Jorn Vernee wrote: > >>> Because -L seems to suggest that the option is same/similar to >>> platform linker option. That is why we started. But now that we >>> are >>> deviating, a different name is better. Besides --library-path >>> makes it >>> more like --module-path, --class-path of the Java world. >> >> Ok, I see. That makes sense, especially when combined with the >> `--missing-symbols=warn|error|exclude` option you mentioned as well. > > I'm not sure I understand this point - what is the deviation we're > talking about? > > Maurizio From jbvernee at xs4all.nl Wed Jan 23 10:56:43 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Wed, 23 Jan 2019 11:56:43 +0100 Subject: scopes writeup In-Reply-To: <6fbff2eb-b100-0c03-7663-c732a3904302@oracle.com> References: <6fbff2eb-b100-0c03-7663-c732a3904302@oracle.com> Message-ID: Hi Maurizio, Per the proposal, a Pointer from a parent scope can only be assigned to a Pointer> from a child scope (but not from an ancestor scope). How does this work for native function calls in general? - What, if any, are the restrictions on the Scopes of Pointers passed to native library functions? - You've said before that a Pointer returned from a library call will get the library's scope, is that sufficient for the case where a library returns a Pointer that we passed into it from Java? Does this imply you can only pass pointers to native calls if they have been allocated from the library's scope (or an ancestor)? Jorn Maurizio Cimadamore schreef op 2019-01-22 17:27: > Hi, > at the end of last year I tried coming up with a proposal [1] which > aimed at finding the basic concepts behind Panama scopes - I've been > iterating (with the help of some internal feedback) on that proposal, > and put it back together in a different form which, all things > considered, it's not too far off from where we are now. > > https://cr.openjdk.java.net/~mcimadamore/panama/scopes.html > > That is, we still have a Scope abstraction - memory regions are still > hidden under Scopes, and not part of the public API (in an attempt to > limit the number of concepts exposed by the API - at least for now). > Pointer is an immutable cursor into a memory region. Resources are > just things with a scope (turns out, a native library is another such > thing, since a native library needs a library-wide scope for > allocating its own globals, etc.). > > On top of what we have now, this writeup pushes forward an improved > ownership model, as well as some discussions concerning > thread-confinement of scopes. > > I think this tweaked API proposal has the potential of fixing the > current lifecycle issues, w/o making the API too complex to use. > Additional abstraction (such as MemoryRegion) can be added back to the > public API on a by need basis (e.g. provided relevant use cases). > > An experimental patch implementing the proposed approach can be found > here: > > http://cr.openjdk.java.net/~mcimadamore/panama/scope-ownership_v2/ > > Comments welcome! > > Maurizio > > [1] - > https://mail.openjdk.java.net/pipermail/panama-dev/2018-December/003605.html From maurizio.cimadamore at oracle.com Wed Jan 23 11:10:35 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 23 Jan 2019 11:10:35 +0000 Subject: [foreign] RFC: Jextract -l options ignored by SymbolFilter when -L is not specified In-Reply-To: References: <698464e8f7692e12a7062155c7e599d9@xs4all.nl> <5C46AD23.9000207@oracle.com> <4089850d-fec6-40cf-c036-e666169a84b3@oracle.com> <30080ab5a99a1a9abf34a89a6c70b8b8@xs4all.nl> <4de2ba3b-4971-95d3-cb3b-9a0d478656d9@oracle.com> <036DADBC-851D-494F-8FC0-094505BE5696@oracle.com> <2f86de9e0e566913c1130e89556c541e@xs4all.nl> <5C47CB05.6060701@oracle.com> <19941C48-DF18-414B-AC02-83F1C62F6712@oracle.com> <5C47D814.5090203@oracle.com> <5C47D93C.7040008@oracle.com> <0477c3fc285fc48c81d9a7f24561c118@xs4all.nl> <5C4839C2.2000000@oracle.com> <16a22bcf7c55a3ac4dbd517aec6a555a@xs4all.nl> <95cdc7fc-925d-93f4-d5f6-2eff3df548ff@oracle.com> Message-ID: On 23/01/2019 10:49, Jorn Vernee wrote: > That jextract's -L also has the added functionality of turning on > symbol checking. So... if we follow the 'should be like the C toolchain' metaphore, this doesn't make a lot of sense. With clang or gcc, I can specify loads of paths (or none) in -L and that's fine. The linker trigger is "-l", not "-L". In other words, I think the 'divergence' we are talking about here, is pretty much our own creation. What happens when you say, e.g. gcc -l GL the linker will try its best to locate the GL library - if it cannot find it, error. How do you solve the error? With -L gcc -l GL -L foo/bar So, there is a possible universe in which jextract "-l" triggers symbol checks (which we can define as the equivalent of the linker job at extraction time). If jextract cannot find a shared library, it will fail badly. Then the user will have to specify -L to tell jextract where to find the desired library. This seems pretty much on par with what we have on the C toolchain side. The problem right now is that, as you say, the absence of -L turns off the symbol checking. That is, jextract won't even try (and won't give any error). This is the bit that is not compatible with the C toolchain, IMHO, and is something that can be rectified. Then there's an orthogonal question as to what should the default -L search path be if none is specified - and I can see different answers there (none, current dir, java.library.path) - but that is an orthogonal issue. Maurizio > > Jorn > > Maurizio Cimadamore schreef op 2019-01-23 11:41: >> On 23/01/2019 10:05, Jorn Vernee wrote: >> >>>> Because -L seems to suggest that the option is same/similar to >>>> platform linker option. That is why we started. But now that we >>>> are >>>> deviating, a different name is better. Besides --library-path >>>> makes it >>>> more like --module-path, --class-path of the Java world. >>> >>> Ok, I see. That makes sense, especially when combined with the >>> `--missing-symbols=warn|error|exclude` option you mentioned as well. >> >> I'm not sure I understand this point - what is the deviation we're >> talking about? >> >> Maurizio From maurizio.cimadamore at oracle.com Wed Jan 23 11:15:23 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 23 Jan 2019 11:15:23 +0000 Subject: access modes for pointers and memory regions In-Reply-To: <8E6C9030-8E00-45BF-AA8F-294DBDFBB212@oracle.com> References: <60B00FE8-1B85-4C97-9ECC-CB4DD106F90C@oracle.com> <796b0698-c080-a62b-c922-ce054e56c09e@oracle.com> <8E6C9030-8E00-45BF-AA8F-294DBDFBB212@oracle.com> Message-ID: On 22/01/2019 22:45, John Rose wrote: > My money is on views, > and the question is whether P+A+MR is too many different > views or not. Can we get away with two or even one? > If so, how does it all factor out? I agree that is the fundamental question. My line of thinking (which prompted my reply) was that if Pointer was a 'real' view (with access and all), then MemoryRegion could be our hidden bit of implementation which represented _The Memory_ (rather than a view of). Both API and implementation-wise, I think MemoryRegion is sitting in the middle of very important abstractions: Pointer and Arrays on one side, and Scope on the other side. It is easy to imagine a world (at least in terms of the impl) where there's not even a MemoryRegion thingie underneath, and all the relevant info is collapsed one way or another. For instance, the base/limit info can be part of the pointer (as these are constants), etc. I like views - and I like (immutable) cursors. But a view that is a cursor into another view (which is what Pointer + MemoryRegion seems to point to), seems too hard to grasp from an user perspective. Which is why I got to the point where MemoryRegion has an intrinsic state (R/W/RW) which is likely dictated by the Scope that is used to allocate it, and Pointer is a cursor view over that. As for your other question, whether Arrays and Pointers is one view too many, I tried some experiments early this year. In terms of code the unification is possible - but I did not like where it all ended up. An experimental patch is available here: http://cr.openjdk.java.net/~mcimadamore/panama/array_pointer_lump/ The main issue with this approach is that you have a carrier like Pointer which can either be a true pointer (e.g. an address) or a sequence of contiguous objects (e.g. an array) and the only way to query for it is to look for the pointer's size presence. I think this is a very sneaky lumping move. Also, the semantics of struct field copy for arrays and pointer is completely different (arrays are copied in bulk), so again collapsing both under the same abstraction feels like a dirty move. Maurizio From maurizio.cimadamore at oracle.com Wed Jan 23 11:27:08 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 23 Jan 2019 11:27:08 +0000 Subject: scopes writeup In-Reply-To: References: <6fbff2eb-b100-0c03-7663-c732a3904302@oracle.com> Message-ID: <3e4546e7-f768-6550-3a31-b38dae835d7b@oracle.com> On 23/01/2019 10:56, Jorn Vernee wrote: > Hi Maurizio, > > Per the proposal, a Pointer from a parent scope can only be > assigned to a Pointer> from a child scope (but not > from an ancestor scope). How does this work for native function calls > in general? Accommodating native function calls is the primary reason for having an unchecked scope which is used internally by the binder. When calling a function it is typical to copy all incoming arguments into a temporary area - and if you do that, with scope ownership you quickly realize that there's no safe way to do that (you'd have to find a scope which is the 'child' of all scopes associated to all the arguments). But this is, I believe, a very special case, and also an internal one, so using some kind of 'trust-me' scope works well in this case. > > - What, if any, are the restrictions on the Scopes of Pointers passed > to native library functions? I think the proposed patch doesn't add any new restriction on this side (because of unchecked scopes). > - You've said before that a Pointer returned from a library call will > get the library's scope, is that sufficient for the case where a > library returns a Pointer that we passed into it from Java? Does this > imply you can only pass pointers to native calls if they have been > allocated from the library's scope (or an ancestor)? I think this is a valid concern, but ultimately, I think it's pointless to try and enforce structure on pointers that are coming out of a native call. There's no way the Java code can know what's coming out of the library and what its life-cycle would be like. In other words, the safety goals for the Scope API, is that if you allocate memory, play with pointers etc., read/write you can never shoot yourself in the foot - that is, the API should be safe - and should prevent common mistakes, such as leaving dandling pointers and such. Calling a native method is a completely different problem - and we can't expect there to have (much) more guarantees than those we get with JNI. That is, calling a native method has an inherent safety risk for the code involved in your application - which means you have to know what you are doing, and what the library is doing. Extreme example: the library could return a pointer that is already dandling (e.g. the library allocate memory with malloc, saves the pointer somewhere, frees the memory, and then returns the pointer). There's no possible scope abstraction that can save you from that. So, when it comes to native calls, I lean towards flexibility - you should be able to pass any pointer to the native library, and you should make minimal assumption about the lifecycle of the pointers that come out of that library (the most basic assumption is that these pointers will be alive for the duration of the library). Maurizio > > Jorn > > Maurizio Cimadamore schreef op 2019-01-22 17:27: >> Hi, >> at the end of last year I tried coming up with a proposal [1] which >> aimed at finding the basic concepts behind Panama scopes - I've been >> iterating (with the help of some internal feedback) on that proposal, >> and put it back together in a different form which, all things >> considered, it's not too far off from where we are now. >> >> https://cr.openjdk.java.net/~mcimadamore/panama/scopes.html >> >> That is, we still have a Scope abstraction - memory regions are still >> hidden under Scopes, and not part of the public API (in an attempt to >> limit the number of concepts exposed by the API - at least for now). >> Pointer is an immutable cursor into a memory region. Resources are >> just things with a scope (turns out, a native library is another such >> thing, since a native library needs a library-wide scope for >> allocating its own globals, etc.). >> >> On top of what we have now, this writeup pushes forward an improved >> ownership model, as well as some discussions concerning >> thread-confinement of scopes. >> >> I think this tweaked API proposal has the potential of fixing the >> current lifecycle issues, w/o making the API too complex to use. >> Additional abstraction (such as MemoryRegion) can be added back to the >> public API on a by need basis (e.g. provided relevant use cases). >> >> An experimental patch implementing the proposed approach can be found >> here: >> >> http://cr.openjdk.java.net/~mcimadamore/panama/scope-ownership_v2/ >> >> Comments welcome! >> >> Maurizio >> >> [1] - >> https://mail.openjdk.java.net/pipermail/panama-dev/2018-December/003605.html >> From jbvernee at xs4all.nl Wed Jan 23 11:33:05 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Wed, 23 Jan 2019 12:33:05 +0100 Subject: [foreign] RFR: Update (Windows) build docs Message-ID: <60002e51650d6bd5712f915ce4ab9a20@xs4all.nl> Hi, I've taken a second pass over the build documentation. I have now: - Added Windows instructions for Tensorflow. - Simplified some of the Windows commands by taking advantage of --infer-rpath - Updated the OpenGL example to take advantage of static forwarders. - Miscellaneous fixes & simplifications here and there. Please review the changes. Webrev: http://cr.openjdk.java.net/~jvernee/panama/webrevs/updateddocs/webrev.00/ Thanks, Jorn From maurizio.cimadamore at oracle.com Wed Jan 23 11:47:21 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 23 Jan 2019 11:47:21 +0000 Subject: [foreign] RFR: Update (Windows) build docs In-Reply-To: <60002e51650d6bd5712f915ce4ab9a20@xs4all.nl> References: <60002e51650d6bd5712f915ce4ab9a20@xs4all.nl> Message-ID: Looks good. The only weird bit is that there seems to be a spurious line (after the table of contents): "% Using Panama "foreign" JDK" here: http://cr.openjdk.java.net/~jvernee/panama/webrevs/updateddocs/webrev.00/raw_files/new/doc/panama_foreign.html Which is missing in the current version: http://hg.openjdk.java.net/panama/dev/raw-file/foreign/doc/panama_foreign.html In fact, if you look at the HTML diff, you will find this extra paragraph: +

% Using Panama "foreign" JDK

It seems like you have added? some extra newline on top of the "%" directive in the md file - maybe that's the culprit. Or maybe it's just to do with the pandoc version being used? Maurizio On 23/01/2019 11:33, Jorn Vernee wrote: > Hi, > > I've taken a second pass over the build documentation. I have now: > > - Added Windows instructions for Tensorflow. > - Simplified some of the Windows commands by taking advantage of > --infer-rpath > - Updated the OpenGL example to take advantage of static forwarders. > - Miscellaneous fixes & simplifications here and there. > > Please review the changes. > > Webrev: > http://cr.openjdk.java.net/~jvernee/panama/webrevs/updateddocs/webrev.00/ > > Thanks, > Jorn From jbvernee at xs4all.nl Wed Jan 23 11:55:02 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Wed, 23 Jan 2019 12:55:02 +0100 Subject: [foreign] RFR: Update (Windows) build docs In-Reply-To: References: <60002e51650d6bd5712f915ce4ab9a20@xs4all.nl> Message-ID: Good catch. I missed this when reviewing the webrev locally. It must have been added when copy-pasting the markdown from the online editor I was using to the local .md file. Updated webrev: http://cr.openjdk.java.net/~jvernee/panama/webrevs/updateddocs/webrev.01/ Jorn Maurizio Cimadamore schreef op 2019-01-23 12:47: > Looks good. > > The only weird bit is that there seems to be a spurious line (after > the table of contents): > > "% Using Panama "foreign" JDK" > > here: > http://cr.openjdk.java.net/~jvernee/panama/webrevs/updateddocs/webrev.00/raw_files/new/doc/panama_foreign.html > > Which is missing in the current version: > http://hg.openjdk.java.net/panama/dev/raw-file/foreign/doc/panama_foreign.html > > In fact, if you look at the HTML diff, you will find this extra > paragraph: > > +

% Using Panama "foreign" JDK

> > It seems like you have added? some extra newline on top of the "%" > directive in the md file - maybe that's the culprit. Or maybe it's > just to do with the pandoc version being used? > > Maurizio > > On 23/01/2019 11:33, Jorn Vernee wrote: >> Hi, >> >> I've taken a second pass over the build documentation. I have now: >> >> - Added Windows instructions for Tensorflow. >> - Simplified some of the Windows commands by taking advantage of >> --infer-rpath >> - Updated the OpenGL example to take advantage of static forwarders. >> - Miscellaneous fixes & simplifications here and there. >> >> Please review the changes. >> >> Webrev: >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/updateddocs/webrev.00/ >> >> Thanks, >> Jorn From maurizio.cimadamore at oracle.com Wed Jan 23 12:11:11 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 23 Jan 2019 12:11:11 +0000 Subject: [foreign] RFR: Update (Windows) build docs In-Reply-To: References: <60002e51650d6bd5712f915ce4ab9a20@xs4all.nl> Message-ID: Good thanks! Maurizio On 23/01/2019 11:55, Jorn Vernee wrote: > Good catch. I missed this when reviewing the webrev locally. It must > have been added when copy-pasting the markdown from the online editor > I was using to the local .md file. > > Updated webrev: > http://cr.openjdk.java.net/~jvernee/panama/webrevs/updateddocs/webrev.01/ > > Jorn > > Maurizio Cimadamore schreef op 2019-01-23 12:47: >> Looks good. >> >> The only weird bit is that there seems to be a spurious line (after >> the table of contents): >> >> "% Using Panama "foreign" JDK" >> >> here: >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/updateddocs/webrev.00/raw_files/new/doc/panama_foreign.html >> >> >> Which is missing in the current version: >> http://hg.openjdk.java.net/panama/dev/raw-file/foreign/doc/panama_foreign.html >> >> >> In fact, if you look at the HTML diff, you will find this extra >> paragraph: >> >> +

% Using Panama "foreign" JDK

>> >> It seems like you have added? some extra newline on top of the "%" >> directive in the md file - maybe that's the culprit. Or maybe it's >> just to do with the pandoc version being used? >> >> Maurizio >> >> On 23/01/2019 11:33, Jorn Vernee wrote: >>> Hi, >>> >>> I've taken a second pass over the build documentation. I have now: >>> >>> - Added Windows instructions for Tensorflow. >>> - Simplified some of the Windows commands by taking advantage of >>> --infer-rpath >>> - Updated the OpenGL example to take advantage of static forwarders. >>> - Miscellaneous fixes & simplifications here and there. >>> >>> Please review the changes. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/updateddocs/webrev.00/ >>> >>> Thanks, >>> Jorn From jbvernee at xs4all.nl Wed Jan 23 12:12:23 2019 From: jbvernee at xs4all.nl (jbvernee at xs4all.nl) Date: Wed, 23 Jan 2019 12:12:23 +0000 Subject: hg: panama/dev: Summary: Update (Windows) build docs Message-ID: <201901231212.x0NCCOKQ021715@aojmv0008.oracle.com> Changeset: c40a5a6dc24d Author: jvernee Date: 2019-01-23 12:56 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/c40a5a6dc24d Summary: Update (Windows) build docs Reviewed-by: mcimadamore ! doc/panama_foreign.html ! doc/panama_foreign.md From jbvernee at xs4all.nl Wed Jan 23 12:12:53 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Wed, 23 Jan 2019 13:12:53 +0100 Subject: [foreign] RFR: Update (Windows) build docs In-Reply-To: References: <60002e51650d6bd5712f915ce4ab9a20@xs4all.nl> Message-ID: Thanks for the review! Pushed. Jorn Maurizio Cimadamore schreef op 2019-01-23 13:11: > Good thanks! > > Maurizio > > On 23/01/2019 11:55, Jorn Vernee wrote: >> Good catch. I missed this when reviewing the webrev locally. It must >> have been added when copy-pasting the markdown from the online editor >> I was using to the local .md file. >> >> Updated webrev: >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/updateddocs/webrev.01/ >> >> Jorn >> >> Maurizio Cimadamore schreef op 2019-01-23 12:47: >>> Looks good. >>> >>> The only weird bit is that there seems to be a spurious line (after >>> the table of contents): >>> >>> "% Using Panama "foreign" JDK" >>> >>> here: >>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/updateddocs/webrev.00/raw_files/new/doc/panama_foreign.html >>> Which is missing in the current version: >>> http://hg.openjdk.java.net/panama/dev/raw-file/foreign/doc/panama_foreign.html >>> In fact, if you look at the HTML diff, you will find this extra >>> paragraph: >>> >>> +

% Using Panama "foreign" JDK

>>> >>> It seems like you have added? some extra newline on top of the "%" >>> directive in the md file - maybe that's the culprit. Or maybe it's >>> just to do with the pandoc version being used? >>> >>> Maurizio >>> >>> On 23/01/2019 11:33, Jorn Vernee wrote: >>>> Hi, >>>> >>>> I've taken a second pass over the build documentation. I have now: >>>> >>>> - Added Windows instructions for Tensorflow. >>>> - Simplified some of the Windows commands by taking advantage of >>>> --infer-rpath >>>> - Updated the OpenGL example to take advantage of static forwarders. >>>> - Miscellaneous fixes & simplifications here and there. >>>> >>>> Please review the changes. >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/updateddocs/webrev.00/ >>>> >>>> Thanks, >>>> Jorn From maurizio.cimadamore at oracle.com Wed Jan 23 12:15:14 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 23 Jan 2019 12:15:14 +0000 Subject: hg: panama/dev: Automatic merge with foreign Message-ID: <201901231215.x0NCFEED023152@aojmv0008.oracle.com> Changeset: 3ade27987264 Author: mcimadamore Date: 2019-01-23 13:15 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/3ade27987264 Automatic merge with foreign From maurizio.cimadamore at oracle.com Wed Jan 23 13:07:36 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 23 Jan 2019 13:07:36 +0000 Subject: scopes writeup In-Reply-To: <6fbff2eb-b100-0c03-7663-c732a3904302@oracle.com> References: <6fbff2eb-b100-0c03-7663-c732a3904302@oracle.com> Message-ID: <0c647bf7-f622-7903-6295-ffca7472c74e@oracle.com> Slightly updated webrev: http://cr.openjdk.java.net/~mcimadamore/panama/scope-ownership_v3/ I realized that the library scope was incorrect - in the sense that it was a 'static' scope shared across all instances of a given library class. To rectify this required some major surgery to HeaderImplGenerator. The second thing this patch does is it removes the (odd) requirement for @NativeHeader interface to extend from Scope.Resource (just to be able to get a scope out of them). Libraries are not resources, they might *use* resources, but that's a different matter. To expose library scopes to client I opted for a simple helper method on the Libraries class: Libraries::libraryScope(Object o) That is, you pass the library instance to the static method and it gives you the scope object associated with that library back. I think this is a decent trade-off. Maurizio On 22/01/2019 16:27, Maurizio Cimadamore wrote: > Hi, > at the end of last year I tried coming up with a proposal [1] which > aimed at finding the basic concepts behind Panama scopes - I've been > iterating (with the help of some internal feedback) on that proposal, > and put it back together in a different form which, all things > considered, it's not too far off from where we are now. > > https://cr.openjdk.java.net/~mcimadamore/panama/scopes.html > > That is, we still have a Scope abstraction - memory regions are still > hidden under Scopes, and not part of the public API (in an attempt to > limit the number of concepts exposed by the API - at least for now). > Pointer is an immutable cursor into a memory region. Resources are > just things with a scope (turns out, a native library is another such > thing, since a native library needs a library-wide scope for > allocating its own globals, etc.). > > On top of what we have now, this writeup pushes forward an improved > ownership model, as well as some discussions concerning > thread-confinement of scopes. > > I think this tweaked API proposal has the potential of fixing the > current lifecycle issues, w/o making the API too complex to use. > Additional abstraction (such as MemoryRegion) can be added back to the > public API on a by need basis (e.g. provided relevant use cases). > > An experimental patch implementing the proposed approach can be found > here: > > http://cr.openjdk.java.net/~mcimadamore/panama/scope-ownership_v2/ > > Comments welcome! > > Maurizio > > [1] - > https://mail.openjdk.java.net/pipermail/panama-dev/2018-December/003605.html > From sundararajan.athijegannathan at oracle.com Wed Jan 23 13:11:02 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Wed, 23 Jan 2019 18:41:02 +0530 Subject: [foreign] RFC: Jextract -l options ignored by SymbolFilter when -L is not specified In-Reply-To: References: <698464e8f7692e12a7062155c7e599d9@xs4all.nl> <5C46AD23.9000207@oracle.com> <4089850d-fec6-40cf-c036-e666169a84b3@oracle.com> <30080ab5a99a1a9abf34a89a6c70b8b8@xs4all.nl> <4de2ba3b-4971-95d3-cb3b-9a0d478656d9@oracle.com> <036DADBC-851D-494F-8FC0-094505BE5696@oracle.com> <2f86de9e0e566913c1130e89556c541e@xs4all.nl> <5C47CB05.6060701@oracle.com> <19941C48-DF18-414B-AC02-83F1C62F6712@oracle.com> <5C47D814.5090203@oracle.com> <5C47D93C.7040008@oracle.com> <0477c3fc285fc48c81d9a7f24561c118@xs4all.nl> <5C4839C2.2000000@oracle.com> <16a22bcf7c55a3ac4dbd517aec6a555a@xs4all.nl> <95cdc7fc-925d-93f4-d5f6-2eff3df548ff@oracle.com> Message-ID: <5C4867E6.5090900@oracle.com> * Not finding a library in default library search path is fatal for "ld". For jextract, it is just that additional functionality would be missed. We just record library name and leave the configuration to binder. So -L is not mandatory for jextract. Symbol missing is left for the binder to find & report. If we allow even late resolution in binder (per native method instead of per interface), it'd be like JNI. I'm not sure we should fix it by jextract throwing error when not finding a library. * When -L is specified, jextract warns missing symbols and does not generate corresponding function in interface. "ld" would throw error for missing symbols and won't link. * Finally jextract uses -L value for runtime search path as well if --infer-path is turned on. Yes, all divergences are our creation. Initially started with similarities but over the time kept adding features. Hence I thought we should perhaps use a different name and be uniform with --module-path, --class-path. That said, I'm not too religious about it :) Summary: * Should it be okay to skip jextract time ("tool time") checks completely and just record library name in annotation? (consistent with Java's late binding). * We need a way to specify library path for search (a) for jextract time checks. (b) to help binder for runtime search of the libraries * Should there be a default for library path? If so, how to infer it? java.library.path or other System property, env. var. (old/new) ? - java.library.path is not stable across platforms and may not be useful. Should we use the same default as used by the platform "ld"? * We need a way to tell what to do on missing symbols? (none/warn/error/warn-and-skip-method) -Sundar On 23/01/19, 4:40 PM, Maurizio Cimadamore wrote: > > On 23/01/2019 10:49, Jorn Vernee wrote: >> That jextract's -L also has the added functionality of turning on >> symbol checking. > > So... if we follow the 'should be like the C toolchain' metaphore, > this doesn't make a lot of sense. > > With clang or gcc, I can specify loads of paths (or none) in -L and > that's fine. The linker trigger is "-l", not "-L". > > In other words, I think the 'divergence' we are talking about here, is > pretty much our own creation. > > What happens when you say, e.g. > > gcc -l GL > > the linker will try its best to locate the GL library - if it cannot > find it, error. > > How do you solve the error? With -L > > gcc -l GL -L foo/bar > > So, there is a possible universe in which jextract "-l" triggers > symbol checks (which we can define as the equivalent of the linker job > at extraction time). If jextract cannot find a shared library, it will > fail badly. Then the user will have to specify -L to tell jextract > where to find the desired library. > > This seems pretty much on par with what we have on the C toolchain side. > > The problem right now is that, as you say, the absence of -L turns off > the symbol checking. That is, jextract won't even try (and won't give > any error). This is the bit that is not compatible with the C > toolchain, IMHO, and is something that can be rectified. > > Then there's an orthogonal question as to what should the default -L > search path be if none is specified - and I can see different answers > there (none, current dir, java.library.path) - but that is an > orthogonal issue. > > Maurizio > >> >> Jorn >> >> Maurizio Cimadamore schreef op 2019-01-23 11:41: >>> On 23/01/2019 10:05, Jorn Vernee wrote: >>> >>>>> Because -L seems to suggest that the option is same/similar to >>>>> platform linker option. That is why we started. But now that we >>>>> are >>>>> deviating, a different name is better. Besides --library-path >>>>> makes it >>>>> more like --module-path, --class-path of the Java world. >>>> >>>> Ok, I see. That makes sense, especially when combined with the >>>> `--missing-symbols=warn|error|exclude` option you mentioned as well. >>> >>> I'm not sure I understand this point - what is the deviation we're >>> talking about? >>> >>> Maurizio From maurizio.cimadamore at oracle.com Wed Jan 23 14:19:32 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 23 Jan 2019 14:19:32 +0000 Subject: [foreign] RFC: Jextract -l options ignored by SymbolFilter when -L is not specified In-Reply-To: <5C4867E6.5090900@oracle.com> References: <698464e8f7692e12a7062155c7e599d9@xs4all.nl> <5C46AD23.9000207@oracle.com> <4089850d-fec6-40cf-c036-e666169a84b3@oracle.com> <30080ab5a99a1a9abf34a89a6c70b8b8@xs4all.nl> <4de2ba3b-4971-95d3-cb3b-9a0d478656d9@oracle.com> <036DADBC-851D-494F-8FC0-094505BE5696@oracle.com> <2f86de9e0e566913c1130e89556c541e@xs4all.nl> <5C47CB05.6060701@oracle.com> <19941C48-DF18-414B-AC02-83F1C62F6712@oracle.com> <5C47D814.5090203@oracle.com> <5C47D93C.7040008@oracle.com> <0477c3fc285fc48c81d9a7f24561c118@xs4all.nl> <5C4839C2.2000000@oracle.com> <16a22bcf7c55a3ac4dbd517aec6a555a@xs4all.nl> <95cdc7fc-925d-93f4-d5f6-2eff3df548ff@oracle.com> <5C4867E6.5090900@oracle.com> Message-ID: <4c869549-a72c-1723-815a-a8fa9084c70e@oracle.com> One extra element to your good summary - it is true that with ld, if a library is not found, its a fatal error. But, if we decided to go ahead and implement the library-centric approach we've been talking about, I think jextract should always be able to find the library at extraction time, to be able to compute the root symbols accordingly. I think this means that the answer to the first bullet in your summary should be no (at least with the library-centric approach it doesn't make sense not to have a library available at extraction time, I think). Maurizio On 23/01/2019 13:11, Sundararajan Athijegannathan wrote: > * Not finding a library in default library search path is fatal for > "ld". For jextract, it is just that additional functionality would be > missed. We just record library name and leave the configuration to > binder. So -L is not mandatory for jextract. Symbol missing is left > for the binder to find & report. If we allow even late resolution in > binder (per native method instead of per interface), it'd be like JNI. > I'm not sure we should fix it by jextract throwing error when not > finding a library. > > * When -L is specified, jextract warns missing symbols and does not > generate corresponding function in interface. "ld" would throw error > for missing symbols and won't link. > > * Finally jextract uses -L value for runtime search path as well if > --infer-path is turned on. > > Yes, all divergences are our creation. Initially started with > similarities but over the time kept adding features. Hence I thought > we should perhaps use a different name and be uniform with > --module-path, --class-path. That said, I'm not too religious about it :) > > Summary: > > * Should it be okay to skip jextract time ("tool time") checks > completely and just record library name in annotation? (consistent > with Java's late binding). > > * We need a way to specify library path for search (a) for jextract > time checks. (b) to help binder for runtime search of the libraries > > * Should there be a default for library path? If so, how to infer it?? > java.library.path or other System property, env. var. (old/new) ? > - java.library.path is not stable across platforms and may not be > useful. Should we use the same default as used by the platform "ld"? > > * We need a way to tell what to do on missing symbols? > (none/warn/error/warn-and-skip-method) > > -Sundar > > On 23/01/19, 4:40 PM, Maurizio Cimadamore wrote: >> >> On 23/01/2019 10:49, Jorn Vernee wrote: >>> That jextract's -L also has the added functionality of turning on >>> symbol checking. >> >> So... if we follow the 'should be like the C toolchain' metaphore, >> this doesn't make a lot of sense. >> >> With clang or gcc, I can specify loads of paths (or none) in -L and >> that's fine. The linker trigger is "-l", not "-L". >> >> In other words, I think the 'divergence' we are talking about here, >> is pretty much our own creation. >> >> What happens when you say, e.g. >> >> gcc -l GL >> >> the linker will try its best to locate the GL library - if it cannot >> find it, error. >> >> How do you solve the error? With -L >> >> gcc -l GL -L foo/bar >> >> So, there is a possible universe in which jextract "-l" triggers >> symbol checks (which we can define as the equivalent of the linker >> job at extraction time). If jextract cannot find a shared library, it >> will fail badly. Then the user will have to specify -L to tell >> jextract where to find the desired library. >> >> This seems pretty much on par with what we have on the C toolchain side. >> >> The problem right now is that, as you say, the absence of -L turns >> off the symbol checking. That is, jextract won't even try (and won't >> give any error). This is the bit that is not compatible with the C >> toolchain, IMHO, and is something that can be rectified. >> >> Then there's an orthogonal question as to what should the default -L >> search path be if none is specified - and I can see different answers >> there (none, current dir, java.library.path) - but that is an >> orthogonal issue. >> >> Maurizio >> >>> >>> Jorn >>> >>> Maurizio Cimadamore schreef op 2019-01-23 11:41: >>>> On 23/01/2019 10:05, Jorn Vernee wrote: >>>> >>>>>> Because -L seems to suggest that the option is same/similar to >>>>>> platform linker option. That is why we started. But now that we >>>>>> are >>>>>> deviating, a different name is better. Besides --library-path >>>>>> makes it >>>>>> more like --module-path, --class-path of the Java world. >>>>> >>>>> Ok, I see. That makes sense, especially when combined with the >>>>> `--missing-symbols=warn|error|exclude` option you mentioned as well. >>>> >>>> I'm not sure I understand this point - what is the deviation we're >>>> talking about? >>>> >>>> Maurizio From maurizio.cimadamore at oracle.com Wed Jan 23 14:23:06 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 23 Jan 2019 14:23:06 +0000 Subject: [foreign] RFC: Jextract -l options ignored by SymbolFilter when -L is not specified In-Reply-To: <5C4867E6.5090900@oracle.com> References: <698464e8f7692e12a7062155c7e599d9@xs4all.nl> <5C46AD23.9000207@oracle.com> <4089850d-fec6-40cf-c036-e666169a84b3@oracle.com> <30080ab5a99a1a9abf34a89a6c70b8b8@xs4all.nl> <4de2ba3b-4971-95d3-cb3b-9a0d478656d9@oracle.com> <036DADBC-851D-494F-8FC0-094505BE5696@oracle.com> <2f86de9e0e566913c1130e89556c541e@xs4all.nl> <5C47CB05.6060701@oracle.com> <19941C48-DF18-414B-AC02-83F1C62F6712@oracle.com> <5C47D814.5090203@oracle.com> <5C47D93C.7040008@oracle.com> <0477c3fc285fc48c81d9a7f24561c118@xs4all.nl> <5C4839C2.2000000@oracle.com> <16a22bcf7c55a3ac4dbd517aec6a555a@xs4all.nl> <95cdc7fc-925d-93f4-d5f6-2eff3df548ff@oracle.com> <5C4867E6.5090900@oracle.com> Message-ID: On 23/01/2019 13:11, Sundararajan Athijegannathan wrote: > * We need a way to tell what to do on missing symbols? > (none/warn/error/warn-and-skip-method) Suggestion to unlog the jam: why don't we start here? The option you proposed "--missing-symbols=warn|error|exclude" is a good one and we should add it to control the missing symbol check logic. That will at least make it clear that, when that option is enabled, jextract will have to be able to find the required library (meaning -L and -l, currently). I think this would be a net improvement over the status quo? Maurizio From sundararajan.athijegannathan at oracle.com Wed Jan 23 14:28:30 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Wed, 23 Jan 2019 19:58:30 +0530 Subject: [foreign] RFC: Jextract -l options ignored by SymbolFilter when -L is not specified In-Reply-To: References: <698464e8f7692e12a7062155c7e599d9@xs4all.nl> <5C46AD23.9000207@oracle.com> <4089850d-fec6-40cf-c036-e666169a84b3@oracle.com> <30080ab5a99a1a9abf34a89a6c70b8b8@xs4all.nl> <4de2ba3b-4971-95d3-cb3b-9a0d478656d9@oracle.com> <036DADBC-851D-494F-8FC0-094505BE5696@oracle.com> <2f86de9e0e566913c1130e89556c541e@xs4all.nl> <5C47CB05.6060701@oracle.com> <19941C48-DF18-414B-AC02-83F1C62F6712@oracle.com> <5C47D814.5090203@oracle.com> <5C47D93C.7040008@oracle.com> <0477c3fc285fc48c81d9a7f24561c118@xs4all.nl> <5C4839C2.2000000@oracle.com> <16a22bcf7c55a3ac4dbd517aec6a555a@xs4all.nl> <95cdc7fc-925d-93f4-d5f6-2eff3df548ff@oracle.com> <5C4867E6.5090900@oracle.com> Message-ID: <5C487A0E.2050509@oracle.com> Yes, sounds good. -Sundar On 23/01/19, 7:53 PM, Maurizio Cimadamore wrote: > > On 23/01/2019 13:11, Sundararajan Athijegannathan wrote: >> * We need a way to tell what to do on missing symbols? >> (none/warn/error/warn-and-skip-method) > > Suggestion to unlog the jam: why don't we start here? > > The option you proposed "--missing-symbols=warn|error|exclude" is a > good one and we should add it to control the missing symbol check logic. > > That will at least make it clear that, when that option is enabled, > jextract will have to be able to find the required library (meaning -L > and -l, currently). > > I think this would be a net improvement over the status quo? > > Maurizio > From henry.jen at oracle.com Wed Jan 23 16:14:54 2019 From: henry.jen at oracle.com (Henry Jen) Date: Wed, 23 Jan 2019 08:14:54 -0800 Subject: [foreign] RFC: Jextract -l options ignored by SymbolFilter when -L is not specified In-Reply-To: <4c869549-a72c-1723-815a-a8fa9084c70e@oracle.com> References: <698464e8f7692e12a7062155c7e599d9@xs4all.nl> <5C46AD23.9000207@oracle.com> <4089850d-fec6-40cf-c036-e666169a84b3@oracle.com> <30080ab5a99a1a9abf34a89a6c70b8b8@xs4all.nl> <4de2ba3b-4971-95d3-cb3b-9a0d478656d9@oracle.com> <036DADBC-851D-494F-8FC0-094505BE5696@oracle.com> <2f86de9e0e566913c1130e89556c541e@xs4all.nl> <5C47CB05.6060701@oracle.com> <19941C48-DF18-414B-AC02-83F1C62F6712@oracle.com> <5C47D814.5090203@oracle.com> <5C47D93C.7040008@oracle.com> <0477c3fc285fc48c81d9a7f24561c118@xs4all.nl> <5C4839C2.2000000@oracle.com> <16a22bcf7c55a3ac4dbd517aec6a555a@xs4all.nl> <95cdc7fc-925d-93f4-d5f6-2eff3df548ff@oracle.com> <5C4867E6.5090900@oracle.com> <4c869549-a72c-1723-815a-a8fa9084c70e@oracle.com> Message-ID: > On Jan 23, 2019, at 6:19 AM, Maurizio Cimadamore wrote: > > One extra element to your good summary - it is true that with ld, if a library is not found, its a fatal error. > > But, if we decided to go ahead and implement the library-centric approach we've been talking about, I think jextract should always be able to find the library at extraction time, to be able to compute the root symbols accordingly. > > I think this means that the answer to the first bullet in your summary should be no (at least with the library-centric approach it doesn't make sense not to have a library available at extraction time, I think). > This is what I am expecting. However, we can decide whether to tolerate missing library with a warning and run as if the -l not specified except to record the library name and generating the static forwarding class. Cheers, Henry > Maurizio > > On 23/01/2019 13:11, Sundararajan Athijegannathan wrote: >> * Not finding a library in default library search path is fatal for "ld". For jextract, it is just that additional functionality would be missed. We just record library name and leave the configuration to binder. So -L is not mandatory for jextract. Symbol missing is left for the binder to find & report. If we allow even late resolution in binder (per native method instead of per interface), it'd be like JNI. I'm not sure we should fix it by jextract throwing error when not finding a library. >> >> * When -L is specified, jextract warns missing symbols and does not generate corresponding function in interface. "ld" would throw error for missing symbols and won't link. >> >> * Finally jextract uses -L value for runtime search path as well if --infer-path is turned on. >> >> Yes, all divergences are our creation. Initially started with similarities but over the time kept adding features. Hence I thought we should perhaps use a different name and be uniform with --module-path, --class-path. That said, I'm not too religious about it :) >> >> Summary: >> >> * Should it be okay to skip jextract time ("tool time") checks completely and just record library name in annotation? (consistent with Java's late binding). >> >> * We need a way to specify library path for search (a) for jextract time checks. (b) to help binder for runtime search of the libraries >> >> * Should there be a default for library path? If so, how to infer it? java.library.path or other System property, env. var. (old/new) ? >> - java.library.path is not stable across platforms and may not be useful. Should we use the same default as used by the platform "ld"? >> >> * We need a way to tell what to do on missing symbols? (none/warn/error/warn-and-skip-method) >> >> -Sundar >> >> On 23/01/19, 4:40 PM, Maurizio Cimadamore wrote: >>> >>> On 23/01/2019 10:49, Jorn Vernee wrote: >>>> That jextract's -L also has the added functionality of turning on symbol checking. >>> >>> So... if we follow the 'should be like the C toolchain' metaphore, this doesn't make a lot of sense. >>> >>> With clang or gcc, I can specify loads of paths (or none) in -L and that's fine. The linker trigger is "-l", not "-L". >>> >>> In other words, I think the 'divergence' we are talking about here, is pretty much our own creation. >>> >>> What happens when you say, e.g. >>> >>> gcc -l GL >>> >>> the linker will try its best to locate the GL library - if it cannot find it, error. >>> >>> How do you solve the error? With -L >>> >>> gcc -l GL -L foo/bar >>> >>> So, there is a possible universe in which jextract "-l" triggers symbol checks (which we can define as the equivalent of the linker job at extraction time). If jextract cannot find a shared library, it will fail badly. Then the user will have to specify -L to tell jextract where to find the desired library. >>> >>> This seems pretty much on par with what we have on the C toolchain side. >>> >>> The problem right now is that, as you say, the absence of -L turns off the symbol checking. That is, jextract won't even try (and won't give any error). This is the bit that is not compatible with the C toolchain, IMHO, and is something that can be rectified. >>> >>> Then there's an orthogonal question as to what should the default -L search path be if none is specified - and I can see different answers there (none, current dir, java.library.path) - but that is an orthogonal issue. >>> >>> Maurizio >>> >>>> >>>> Jorn >>>> >>>> Maurizio Cimadamore schreef op 2019-01-23 11:41: >>>>> On 23/01/2019 10:05, Jorn Vernee wrote: >>>>> >>>>>>> Because -L seems to suggest that the option is same/similar to >>>>>>> platform linker option. That is why we started. But now that we >>>>>>> are >>>>>>> deviating, a different name is better. Besides --library-path >>>>>>> makes it >>>>>>> more like --module-path, --class-path of the Java world. >>>>>> >>>>>> Ok, I see. That makes sense, especially when combined with the >>>>>> `--missing-symbols=warn|error|exclude` option you mentioned as well. >>>>> >>>>> I'm not sure I understand this point - what is the deviation we're >>>>> talking about? >>>>> >>>>> Maurizio From tomasz.kowalczewski at gmail.com Wed Jan 23 18:14:52 2019 From: tomasz.kowalczewski at gmail.com (Tomasz Kowalczewski) Date: Wed, 23 Jan 2019 19:14:52 +0100 Subject: [vector api] Vector elements shift confusion In-Reply-To: <89f7aa3e-1967-add4-e24a-10226e92bd30@oracle.com> References: <89f7aa3e-1967-add4-e24a-10226e92bd30@oracle.com> Message-ID: Thanks Vladimir for taking time to look into it. If it makes sense I can try to fix these (should be simple - on Java level) and post a webrev (I have signed the OCA). -- Tomek On Wed, Jan 23, 2019 at 4:12 AM Vladimir Ivanov wrote: > > Good catch, Tomasz! I confirm that rotateER is completely broken :-) > > There's definitely disagreement between javadoc & implementation on > shiftEL/ER variants, but I'm in favor of the javadoc here and would like > to see the implementation aligned with it. > > While fromArray() and [shift|rotate]EL/ER seem to disagree > (little-endian vs big-endian style), it aligns with the behavior of > shifts on primitives where: > "left" == "towards higher bits" == "towards higher indices" > "right" == "towards lower bits" == "towards higher indices" > > And as you noted, implementation behaves in the opposite direction while > rotateEL agrees with the spec [1]. > > Best regards, > Vladimir Ivanov > > [1] $ jshell --add-modules=jdk.incubator.vector > > > import jdk.incubator.vector.* > > > var I256 = > IntVector.species(jdk.incubator.vector.Vector.Shape.S_256_BIT); > > > var v1 = I256.fromArray(new int[] { 1, 2, 3, 4, 5, 6, 7, 8 }, 0) > v1 ==> [1, 2, 3, 4, 5, 6, 7, 8] > > > v1.get(0) > ==> 1 > > > v1.get(7) > ==> 8 > > > v1.shiftEL(1) > ==> [2, 3, 4, 5, 6, 7, 8, 0] > > > v1.shiftEL(1).get(0) > ==> 2 > > > v1.shiftEL(1).get(7) > ==> 0 > > > v1.shiftER(1) > ==> [0, 1, 2, 3, 4, 5, 6, 7] > > > v1.shiftER(1).get(0) > ==> 0 > > > v1.shiftER(1).get(7) > ==> 7 > > > v1.rotateEL(1) > ==> [8, 1, 2, 3, 4, 5, 6, 7] > > jshell> v1.rotateEL(1).get(0) > ==> 8 > > jshell> v1.rotateEL(1).get(7) > ==> 7 > > > v1.rotateER(1) > | Exception java.lang.ArrayIndexOutOfBoundsException: Index -1 out of > bounds for length 8 > | at Int256Vector.rotateER (Int256Vector.java:924) > | at Int256Vector.rotateER (Int256Vector.java:39) > | at (#44:1) > > > > I started this email having problems using rotateER on vectors. I have > > not found any way to call this method without getting an exception :). > > More investigation revealed that shift-ing vector elements does not > > work in accordance with javadoc. I realised this email will be quite > > long so I split it up and will describe just shift issues. If my > > suspicions are correct I will follow up with rotations. > > > > First shiftEL operation describes itself "as if rotating left the lane > > elements by i [...] zero value is placed into the result vector at > > lane indexes less than i % this.length()." > > > > *I suspect that the rotation is described in the wrong direction*. > > Lets try to confirm it but just looking at zeroed lane indices (using > > get() which "Gets the lane element at lane index i"): > > > > @Test > > public void shouldShiftElementsLeft() { > > // Given > > IntVector intVector = > > IntVector.species(Vector.Shape.S_128_BIT).fromArray(new int[]{ > > 1, 2, 3, 4 > > }, 0); > > > > // When > > intVector = intVector.shiftEL(2); > > > > // Then > > assertThat(intVector.get(0)).isEqualTo(0); > > assertThat(intVector.get(1)).isEqualTo(0); > > } > > > > So after the shift of "2" lane indexes less than "2" should be zero. > > This test fails as the rotation is done in the opposite (to me - > > correct) direction: > > > > @Test > > public void shouldShiftElementsLeft() { > > // Given > > IntVector intVector = > > IntVector.species(Vector.Shape.S_128_BIT).fromArray(new int[]{ > > 1, 2, 3, 4 > > }, 0); > > > > // When > > intVector = intVector.shiftEL(2); > > > > // Then > > assertThat(intVector.toArray()).containsExactly(3, 4, 0, 0); > > } > > > > Same case is with shiftER. Java doc says that "zero value is placed > > into the result vector at lane indexes greater or equal to > > this.length() - (i % this.length()).". But zeroed are indexes < i % > > this.length(). > > > > Looks like implementations are correct but javadoc-s are swapped (I > > hope I am not making fool of myself). > > > > Please advise, > > Tomasz > > From vladimir.x.ivanov at oracle.com Wed Jan 23 18:24:07 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Wed, 23 Jan 2019 10:24:07 -0800 Subject: [vector api] Vector elements shift confusion In-Reply-To: References: <89f7aa3e-1967-add4-e24a-10226e92bd30@oracle.com> Message-ID: > Thanks Vladimir for taking time to look into it. If it makes sense I > can try to fix these (should be simple - on Java level) and post a > webrev (I have signed the OCA). Sure, contributions are more than welcome! At some point, rotateER/L and shiftER/L should be intrinsified, but unfortunately they haven't got enough attention yet. Sandhya, what are the plans on that front? Does anybody working on it? Best regards, Vladimir Ivanov > On Wed, Jan 23, 2019 at 4:12 AM Vladimir Ivanov > wrote: >> >> Good catch, Tomasz! I confirm that rotateER is completely broken :-) >> >> There's definitely disagreement between javadoc & implementation on >> shiftEL/ER variants, but I'm in favor of the javadoc here and would like >> to see the implementation aligned with it. >> >> While fromArray() and [shift|rotate]EL/ER seem to disagree >> (little-endian vs big-endian style), it aligns with the behavior of >> shifts on primitives where: >> "left" == "towards higher bits" == "towards higher indices" >> "right" == "towards lower bits" == "towards higher indices" >> >> And as you noted, implementation behaves in the opposite direction while >> rotateEL agrees with the spec [1]. >> >> Best regards, >> Vladimir Ivanov >> >> [1] $ jshell --add-modules=jdk.incubator.vector >> >> > import jdk.incubator.vector.* >> >> > var I256 = >> IntVector.species(jdk.incubator.vector.Vector.Shape.S_256_BIT); >> >> > var v1 = I256.fromArray(new int[] { 1, 2, 3, 4, 5, 6, 7, 8 }, 0) >> v1 ==> [1, 2, 3, 4, 5, 6, 7, 8] >> >> > v1.get(0) >> ==> 1 >> >> > v1.get(7) >> ==> 8 >> >> > v1.shiftEL(1) >> ==> [2, 3, 4, 5, 6, 7, 8, 0] >> >> > v1.shiftEL(1).get(0) >> ==> 2 >> >> > v1.shiftEL(1).get(7) >> ==> 0 >> >> > v1.shiftER(1) >> ==> [0, 1, 2, 3, 4, 5, 6, 7] >> >> > v1.shiftER(1).get(0) >> ==> 0 >> >> > v1.shiftER(1).get(7) >> ==> 7 >> >> > v1.rotateEL(1) >> ==> [8, 1, 2, 3, 4, 5, 6, 7] >> >> jshell> v1.rotateEL(1).get(0) >> ==> 8 >> >> jshell> v1.rotateEL(1).get(7) >> ==> 7 >> >> > v1.rotateER(1) >> | Exception java.lang.ArrayIndexOutOfBoundsException: Index -1 out of >> bounds for length 8 >> | at Int256Vector.rotateER (Int256Vector.java:924) >> | at Int256Vector.rotateER (Int256Vector.java:39) >> | at (#44:1) >> >> >>> I started this email having problems using rotateER on vectors. I have >>> not found any way to call this method without getting an exception :). >>> More investigation revealed that shift-ing vector elements does not >>> work in accordance with javadoc. I realised this email will be quite >>> long so I split it up and will describe just shift issues. If my >>> suspicions are correct I will follow up with rotations. >>> >>> First shiftEL operation describes itself "as if rotating left the lane >>> elements by i [...] zero value is placed into the result vector at >>> lane indexes less than i % this.length()." >>> >>> *I suspect that the rotation is described in the wrong direction*. >>> Lets try to confirm it but just looking at zeroed lane indices (using >>> get() which "Gets the lane element at lane index i"): >>> >>> @Test >>> public void shouldShiftElementsLeft() { >>> // Given >>> IntVector intVector = >>> IntVector.species(Vector.Shape.S_128_BIT).fromArray(new int[]{ >>> 1, 2, 3, 4 >>> }, 0); >>> >>> // When >>> intVector = intVector.shiftEL(2); >>> >>> // Then >>> assertThat(intVector.get(0)).isEqualTo(0); >>> assertThat(intVector.get(1)).isEqualTo(0); >>> } >>> >>> So after the shift of "2" lane indexes less than "2" should be zero. >>> This test fails as the rotation is done in the opposite (to me - >>> correct) direction: >>> >>> @Test >>> public void shouldShiftElementsLeft() { >>> // Given >>> IntVector intVector = >>> IntVector.species(Vector.Shape.S_128_BIT).fromArray(new int[]{ >>> 1, 2, 3, 4 >>> }, 0); >>> >>> // When >>> intVector = intVector.shiftEL(2); >>> >>> // Then >>> assertThat(intVector.toArray()).containsExactly(3, 4, 0, 0); >>> } >>> >>> Same case is with shiftER. Java doc says that "zero value is placed >>> into the result vector at lane indexes greater or equal to >>> this.length() - (i % this.length()).". But zeroed are indexes < i % >>> this.length(). >>> >>> Looks like implementations are correct but javadoc-s are swapped (I >>> hope I am not making fool of myself). >>> >>> Please advise, >>> Tomasz >>> From sandhya.viswanathan at intel.com Wed Jan 23 18:29:04 2019 From: sandhya.viswanathan at intel.com (Viswanathan, Sandhya) Date: Wed, 23 Jan 2019 18:29:04 +0000 Subject: [vector api] Vector elements shift confusion In-Reply-To: References: <89f7aa3e-1967-add4-e24a-10226e92bd30@oracle.com> Message-ID: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A52BB6@FMSMSX126.amr.corp.intel.com> Hi Vladimir, >> At some point, rotateER/L and shiftER/L should be intrinsified, but unfortunately they haven't got enough attention yet. Sandhya, what are the plans on that front? Does anybody working on it? Yes we plan to take it up in February (Jatin cced here) after the min/max/minAll/maxAll fix to match the new specification. Best Regards, Sandhya -----Original Message----- From: Vladimir Ivanov [mailto:vladimir.x.ivanov at oracle.com] Sent: Wednesday, January 23, 2019 10:24 AM To: Tomasz Kowalczewski ; Viswanathan, Sandhya Cc: panama-dev at openjdk.java.net Subject: Re: [vector api] Vector elements shift confusion > Thanks Vladimir for taking time to look into it. If it makes sense I > can try to fix these (should be simple - on Java level) and post a > webrev (I have signed the OCA). Sure, contributions are more than welcome! At some point, rotateER/L and shiftER/L should be intrinsified, but unfortunately they haven't got enough attention yet. Sandhya, what are the plans on that front? Does anybody working on it? Best regards, Vladimir Ivanov > On Wed, Jan 23, 2019 at 4:12 AM Vladimir Ivanov > wrote: >> >> Good catch, Tomasz! I confirm that rotateER is completely broken :-) >> >> There's definitely disagreement between javadoc & implementation on >> shiftEL/ER variants, but I'm in favor of the javadoc here and would >> like to see the implementation aligned with it. >> >> While fromArray() and [shift|rotate]EL/ER seem to disagree >> (little-endian vs big-endian style), it aligns with the behavior of >> shifts on primitives where: >> "left" == "towards higher bits" == "towards higher indices" >> "right" == "towards lower bits" == "towards higher indices" >> >> And as you noted, implementation behaves in the opposite direction >> while rotateEL agrees with the spec [1]. >> >> Best regards, >> Vladimir Ivanov >> >> [1] $ jshell --add-modules=jdk.incubator.vector >> >> > import jdk.incubator.vector.* >> >> > var I256 = >> IntVector.species(jdk.incubator.vector.Vector.Shape.S_256_BIT); >> >> > var v1 = I256.fromArray(new int[] { 1, 2, 3, 4, 5, 6, 7, 8 }, 0) >> v1 ==> [1, 2, 3, 4, 5, 6, 7, 8] >> >> > v1.get(0) >> ==> 1 >> >> > v1.get(7) >> ==> 8 >> >> > v1.shiftEL(1) >> ==> [2, 3, 4, 5, 6, 7, 8, 0] >> >> > v1.shiftEL(1).get(0) >> ==> 2 >> >> > v1.shiftEL(1).get(7) >> ==> 0 >> >> > v1.shiftER(1) >> ==> [0, 1, 2, 3, 4, 5, 6, 7] >> >> > v1.shiftER(1).get(0) >> ==> 0 >> >> > v1.shiftER(1).get(7) >> ==> 7 >> >> > v1.rotateEL(1) >> ==> [8, 1, 2, 3, 4, 5, 6, 7] >> >> jshell> v1.rotateEL(1).get(0) >> ==> 8 >> >> jshell> v1.rotateEL(1).get(7) >> ==> 7 >> >> > v1.rotateER(1) >> | Exception java.lang.ArrayIndexOutOfBoundsException: Index -1 out >> | of >> bounds for length 8 >> | at Int256Vector.rotateER (Int256Vector.java:924) >> | at Int256Vector.rotateER (Int256Vector.java:39) >> | at (#44:1) >> >> >>> I started this email having problems using rotateER on vectors. I >>> have not found any way to call this method without getting an exception :). >>> More investigation revealed that shift-ing vector elements does not >>> work in accordance with javadoc. I realised this email will be quite >>> long so I split it up and will describe just shift issues. If my >>> suspicions are correct I will follow up with rotations. >>> >>> First shiftEL operation describes itself "as if rotating left the >>> lane elements by i [...] zero value is placed into the result vector >>> at lane indexes less than i % this.length()." >>> >>> *I suspect that the rotation is described in the wrong direction*. >>> Lets try to confirm it but just looking at zeroed lane indices >>> (using >>> get() which "Gets the lane element at lane index i"): >>> >>> @Test >>> public void shouldShiftElementsLeft() { >>> // Given >>> IntVector intVector = >>> IntVector.species(Vector.Shape.S_128_BIT).fromArray(new int[]{ >>> 1, 2, 3, 4 >>> }, 0); >>> >>> // When >>> intVector = intVector.shiftEL(2); >>> >>> // Then >>> assertThat(intVector.get(0)).isEqualTo(0); >>> assertThat(intVector.get(1)).isEqualTo(0); >>> } >>> >>> So after the shift of "2" lane indexes less than "2" should be zero. >>> This test fails as the rotation is done in the opposite (to me - >>> correct) direction: >>> >>> @Test >>> public void shouldShiftElementsLeft() { >>> // Given >>> IntVector intVector = >>> IntVector.species(Vector.Shape.S_128_BIT).fromArray(new int[]{ >>> 1, 2, 3, 4 >>> }, 0); >>> >>> // When >>> intVector = intVector.shiftEL(2); >>> >>> // Then >>> assertThat(intVector.toArray()).containsExactly(3, 4, 0, 0); } >>> >>> Same case is with shiftER. Java doc says that "zero value is placed >>> into the result vector at lane indexes greater or equal to >>> this.length() - (i % this.length()).". But zeroed are indexes < i % >>> this.length(). >>> >>> Looks like implementations are correct but javadoc-s are swapped (I >>> hope I am not making fool of myself). >>> >>> Please advise, >>> Tomasz >>> From vladimir.x.ivanov at oracle.com Thu Jan 24 02:47:32 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Wed, 23 Jan 2019 18:47:32 -0800 Subject: [Vector] RFR: reshape, resize, rebracket, cast In-Reply-To: References: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A5012D@FMSMSX126.amr.corp.intel.com> Message-ID: <367c043f-d66f-17df-c4cb-910a19a5f30f@oracle.com> Kishor, > I have new webrev which modifies the tests to use the new methods - http://cr.openjdk.java.net/~kkharbas/vector-api/webrev-reshape_etc.01 > Incremental webrev - http://cr.openjdk.java.net/~kkharbas/vector-api/webrev-reshape_etc.01_to_base It looks good to me and aligned with what was discussed previously. Please, also update users: $ egrep -r -e resize -e reshape -e rebracket -I -l test/jdk/jdk/incubator/vector test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/AbstractVectorBenchmark.java test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/PopulationCount.java test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Rearrange.java test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/SumOfUnsignedBytes.java test/jdk/jdk/incubator/vector/VectorHash.java test/jdk/jdk/incubator/vector/VectorReshapeTests.java test/jdk/jdk/incubator/vector/CovarOverrideTest.java > New methods: > class Vector { > > /* E x S -> E' x S' */ > /* was reshape() and rebracket() before */ > Vector reinterpret(Species s) > > /* E x S -> E x S' */ > /* was resize() before */ > Vector reshape(Species s) > > /* E x S -> E' x S' */ > Vector cast(Species s) > } As a next step, I'd like to question the presence of `reshape`. It seemed reasonable to keep it at the time discussion happened, but based on my recent experience with the API, `reinterpret`/`cast` look more than enough unless truncation/padding is forbidden on `reinterpret`/`cast` and exposed through a dedicated API point (`reshape`). There's one nice thing about `reshape`: it preserves element type, so there's no need to cast result back to specialized vector type (e.g., Vector => IntVector). But `reinterpret`/`cast` suffer from that and it would be nice to provide a workaround/solution. For example, how about introducing Vector.toIntVector() et al which serve as assertions about vector shapes backed by runtime checks? IntVector v = ... ; // Int128Vector LongSpecies s = ... ; // Long256Species LongVector vl1 = (LongVector)v.cast(s); LongVector = v.cast(s).toLongVector(); Should work rather well for fluent-style code. Best regards, Vladimir Ivanov >> -----Original Message----- >> From: Viswanathan, Sandhya >> Sent: Wednesday, January 16, 2019 12:33 PM >> To: Kharbas, Kishor ; panama- >> dev at openjdk.java.net; Brian Goetz ; John Rose >> ; Vladimir Ivanov >> >> Cc: joe.darcy at oracle.com >> Subject: RE: [Vector] RFR: reshape, resize, rebracket, cast >> >> >> Adding Brian, John, Joe and Vladimir to the thread. Please do give your >> feedback on the patch from Kishor >> (http://cr.openjdk.java.net/~kkharbas/vector-api/webrev-reshape_etc) >> implementing the reinterpret API as suggested by Brian and Vladimir in >> thread https://mail.openjdk.java.net/pipermail/panama-dev/2018- >> December/003365.html. >> >> Best Regards, >> Sandhya >> >> >> -----Original Message----- >> From: panama-dev [mailto:panama-dev-bounces at openjdk.java.net] On >> Behalf Of Kharbas, Kishor >> Sent: Wednesday, January 16, 2019 12:20 AM >> To: panama-dev at openjdk.java.net >> Subject: [Vector] RFR: reshape, resize, rebracket, cast >> >> Hi, >> I have a patch which refactors the above methods as discussed in this thread >> - https://mail.openjdk.java.net/pipermail/panama-dev/2018- >> December/003365.html. >> >> Please review the changes at - http://cr.openjdk.java.net/~kkharbas/vector- >> api/webrev-reshape_etc >> Generation of intrinsics and correct vector boxing elimination have been >> verified. In the next patch I will add changes to jtreg tests. >> >> I wanted to bring forth one small issue(possibly) which programmers might >> face with this change - >> >> We provide specialized types like IntVector, FloatVector, etc for users to >> define their vectors and they would be able to write code like this with >> previous methods, >> FloatVector float256 = SPECIES_FLOAT256.cast(int256); Here FloatSpecies >> would always return FloatVector. >> >> However with this change, since cast() is defined on a vector and takes >> species of a generic type, it cannot return a specialized Vector like IntVector >> or FloatVector. User has to explicitly cast the return vector from Vector >> to specialized Vector or use a generic vector of corresponding element type. >> For example, >> FloatVector float256 = (FloatVector)int256.cast(SPECIES_FLOAT256); >> or >> Vector float256 = int256.cast(SPECIES_FLOAT256); >> >> I am not sure if this is even a problem, but I thought its worth mentioning. >> >> Thanks >> Kishor From jbvernee at xs4all.nl Thu Jan 24 13:04:44 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Thu, 24 Jan 2019 14:04:44 +0100 Subject: [foreign] RFR 8217664: jextract should check for error code returns from Type::getOffsetOf and Type::size Message-ID: <8feff9ee0e27c76976ad315d0b43cb02@xs4all.nl> Hi, jextract uses libclang functions to query the size of a type and offset of a field. The returned values are used for instance in layout calculations. But, these functions can also return error codes and there is currently no check for them. They are happily being used in the layout calculations, and this can cause some strange errors, or in some cases incorrect layouts are generated. To give an example: struct Foo { int x; void *list_of_data[]; }; The expected layout of this struct (which uses an incomplete/flexible array field) should be `[i32x32[0u64]]` but jextract instead generates `[i32[0u64]x32]`. This particular case is due to a bug in libclang where requesting the offset of a field in a type with a flexible array member always returns an incomplete type error, even though getting the offset of each fields should be possible. This error code (-2) is then used in the layout calculations and yields the incorrect result. So I've added checking to the returned value of Type::size and Type::getOffsetOf and throwing an exception in case an error code is returned. I'm also explicitly checking for the flexible array case and throwing a UOE in that case. This also means I had to disable the flexible array jextract test [1]. The test was passing, but at least on Windows the wrong layout was being generated. Since we can not reliably compute padding for types with flexible arrays using the current method, I think it's better to throw an exception for now until we can find a workaround for this bug in libclang, or the bug gets fixed. Using a type with a flexible/incomplete array member in the binder should still be fine of course, so I've added a separate test for that. Please review the following. Bug: https://bugs.openjdk.java.net/browse/JDK-8217664 Webrev: http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217664/webrev.00/ Thanks, Jorn [1] : http://hg.openjdk.java.net/panama/dev/file/c40a5a6dc24d/test/jdk/com/sun/tools/jextract/testStruct/struct.h#l85 From maurizio.cimadamore at oracle.com Thu Jan 24 13:07:25 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 24 Jan 2019 13:07:25 +0000 Subject: [foreign] RFR 8217727: drop AVX compiler directives from panama/foreign tests Message-ID: <12d70460-6a4b-757c-5c39-e5fec507ffa0@oracle.com> Hi, the use of some AVX specific settings in some Panama/foreign tests can cause spurious failures when tests are executed on platforms w/o AVX support. This patch removes AVX-specific dependencies from tests. http://cr.openjdk.java.net/~mcimadamore/panama/8217727/ Cheers Maurizio From jbvernee at xs4all.nl Thu Jan 24 13:24:40 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Thu, 24 Jan 2019 14:24:40 +0100 Subject: [foreign] RFR 8217727: drop AVX compiler directives from panama/foreign tests In-Reply-To: <12d70460-6a4b-757c-5c39-e5fec507ffa0@oracle.com> References: <12d70460-6a4b-757c-5c39-e5fec507ffa0@oracle.com> Message-ID: <9ecdf0cc3f9c876725ff3232aacfe43b@xs4all.nl> Hi Maurizio, This patch is throwing an error for me when running the tests: /bin/find: ?else?: No such file or directory It seems that there is a spurious `\` in /make/test/JtregNativeJdk.gmk: diff -r 3b9dc4b788e2 make/test/JtregNativeJdk.gmk --- a/make/test/JtregNativeJdk.gmk Thu Jan 24 14:10:29 2019 +0100 +++ b/make/test/JtregNativeJdk.gmk Thu Jan 24 14:24:10 2019 +0100 @@ -47,7 +47,7 @@ # Only build java.foreign tests on x64 for now ifneq ($(filter x86_64, $(OPENJDK_TARGET_CPU)),) BUILD_JDK_JTREG_NATIVE_SRC += \ - $(TOPDIR)/test/jdk/java/foreign \ + $(TOPDIR)/test/jdk/java/foreign else $(info Not building java.foreign tests) endif Otherwise this looks good. Cheers, Jorn Maurizio Cimadamore schreef op 2019-01-24 14:07: > Hi, > the use of some AVX specific settings in some Panama/foreign tests can > cause spurious failures when tests are executed on platforms w/o AVX > support. > > This patch removes AVX-specific dependencies from tests. > > http://cr.openjdk.java.net/~mcimadamore/panama/8217727/ > > Cheers > Maurizio From maurizio.cimadamore at oracle.com Thu Jan 24 13:33:42 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 24 Jan 2019 13:33:42 +0000 Subject: [foreign] RFR 8217727: drop AVX compiler directives from panama/foreign tests In-Reply-To: <9ecdf0cc3f9c876725ff3232aacfe43b@xs4all.nl> References: <12d70460-6a4b-757c-5c39-e5fec507ffa0@oracle.com> <9ecdf0cc3f9c876725ff3232aacfe43b@xs4all.nl> Message-ID: Well spotted - I ran tests on all platforms, but probably depends on how the shell is setup. Another attempt: http://cr.openjdk.java.net/~mcimadamore/panama/8217727_v2/ Maurizio On 24/01/2019 13:24, Jorn Vernee wrote: > Hi Maurizio, > > This patch is throwing an error for me when running the tests: > > ??? /bin/find: ?else?: No such file or directory > > It seems that there is a spurious `\` in /make/test/JtregNativeJdk.gmk: > > diff -r 3b9dc4b788e2 make/test/JtregNativeJdk.gmk > --- a/make/test/JtregNativeJdk.gmk????? Thu Jan 24 14:10:29 2019 +0100 > +++ b/make/test/JtregNativeJdk.gmk????? Thu Jan 24 14:24:10 2019 +0100 > @@ -47,7 +47,7 @@ > ?# Only build java.foreign tests on x64 for now > ?ifneq ($(filter x86_64, $(OPENJDK_TARGET_CPU)),) > ???? BUILD_JDK_JTREG_NATIVE_SRC += \ > -?????? $(TOPDIR)/test/jdk/java/foreign \ > +?????? $(TOPDIR)/test/jdk/java/foreign > ?else > ?$(info Not building java.foreign tests) > ?endif > > > Otherwise this looks good. > > Cheers, > Jorn > > Maurizio Cimadamore schreef op 2019-01-24 14:07: >> Hi, >> the use of some AVX specific settings in some Panama/foreign tests can >> cause spurious failures when tests are executed on platforms w/o AVX >> support. >> >> This patch removes AVX-specific dependencies from tests. >> >> http://cr.openjdk.java.net/~mcimadamore/panama/8217727/ >> >> Cheers >> Maurizio From jbvernee at xs4all.nl Thu Jan 24 13:50:53 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Thu, 24 Jan 2019 14:50:53 +0100 Subject: [foreign] RFR 8217727: drop AVX compiler directives from panama/foreign tests In-Reply-To: References: <12d70460-6a4b-757c-5c39-e5fec507ffa0@oracle.com> <9ecdf0cc3f9c876725ff3232aacfe43b@xs4all.nl> Message-ID: Maurizio Cimadamore schreef op 2019-01-24 14:33: > Well spotted - I ran tests on all platforms, but probably depends on > how the shell is setup. FWIW, I manually terminated the test run after spotting the error. I think on a supported platform the tests would still pass, but on an unsupported platform this would cause the jdk_foreign tests to be built any ways and then fail. > Another attempt: > > http://cr.openjdk.java.net/~mcimadamore/panama/8217727_v2/ Looks good - All tests passing on my machine as well. Jorn > Maurizio > > On 24/01/2019 13:24, Jorn Vernee wrote: >> Hi Maurizio, >> >> This patch is throwing an error for me when running the tests: >> >> ??? /bin/find: ?else?: No such file or directory >> >> It seems that there is a spurious `\` in >> /make/test/JtregNativeJdk.gmk: >> >> diff -r 3b9dc4b788e2 make/test/JtregNativeJdk.gmk >> --- a/make/test/JtregNativeJdk.gmk????? Thu Jan 24 14:10:29 2019 +0100 >> +++ b/make/test/JtregNativeJdk.gmk????? Thu Jan 24 14:24:10 2019 +0100 >> @@ -47,7 +47,7 @@ >> ?# Only build java.foreign tests on x64 for now >> ?ifneq ($(filter x86_64, $(OPENJDK_TARGET_CPU)),) >> ???? BUILD_JDK_JTREG_NATIVE_SRC += \ >> -?????? $(TOPDIR)/test/jdk/java/foreign \ >> +?????? $(TOPDIR)/test/jdk/java/foreign >> ?else >> ?$(info Not building java.foreign tests) >> ?endif >> >> >> Otherwise this looks good. >> >> Cheers, >> Jorn >> >> Maurizio Cimadamore schreef op 2019-01-24 14:07: >>> Hi, >>> the use of some AVX specific settings in some Panama/foreign tests >>> can >>> cause spurious failures when tests are executed on platforms w/o AVX >>> support. >>> >>> This patch removes AVX-specific dependencies from tests. >>> >>> http://cr.openjdk.java.net/~mcimadamore/panama/8217727/ >>> >>> Cheers >>> Maurizio From sundararajan.athijegannathan at oracle.com Thu Jan 24 14:37:55 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Thu, 24 Jan 2019 20:07:55 +0530 Subject: [foreign] RFR 8217727: drop AVX compiler directives from panama/foreign tests In-Reply-To: References: <12d70460-6a4b-757c-5c39-e5fec507ffa0@oracle.com> <9ecdf0cc3f9c876725ff3232aacfe43b@xs4all.nl> Message-ID: <5C49CDC3.5000705@oracle.com> Looks good -Sundar On 24/01/19, 7:03 PM, Maurizio Cimadamore wrote: > Well spotted - I ran tests on all platforms, but probably depends on > how the shell is setup. > > Another attempt: > > http://cr.openjdk.java.net/~mcimadamore/panama/8217727_v2/ > > Maurizio > > On 24/01/2019 13:24, Jorn Vernee wrote: >> Hi Maurizio, >> >> This patch is throwing an error for me when running the tests: >> >> /bin/find: ?else?: No such file or directory >> >> It seems that there is a spurious `\` in /make/test/JtregNativeJdk.gmk: >> >> diff -r 3b9dc4b788e2 make/test/JtregNativeJdk.gmk >> --- a/make/test/JtregNativeJdk.gmk Thu Jan 24 14:10:29 2019 +0100 >> +++ b/make/test/JtregNativeJdk.gmk Thu Jan 24 14:24:10 2019 +0100 >> @@ -47,7 +47,7 @@ >> # Only build java.foreign tests on x64 for now >> ifneq ($(filter x86_64, $(OPENJDK_TARGET_CPU)),) >> BUILD_JDK_JTREG_NATIVE_SRC += \ >> - $(TOPDIR)/test/jdk/java/foreign \ >> + $(TOPDIR)/test/jdk/java/foreign >> else >> $(info Not building java.foreign tests) >> endif >> >> >> Otherwise this looks good. >> >> Cheers, >> Jorn >> >> Maurizio Cimadamore schreef op 2019-01-24 14:07: >>> Hi, >>> the use of some AVX specific settings in some Panama/foreign tests can >>> cause spurious failures when tests are executed on platforms w/o AVX >>> support. >>> >>> This patch removes AVX-specific dependencies from tests. >>> >>> http://cr.openjdk.java.net/~mcimadamore/panama/8217727/ >>> >>> Cheers >>> Maurizio From maurizio.cimadamore at oracle.com Thu Jan 24 14:41:19 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Thu, 24 Jan 2019 14:41:19 +0000 Subject: hg: panama/dev: 8217727: drop AVX compiler directives from panama/foreign tests Message-ID: <201901241441.x0OEfJGI021552@aojmv0008.oracle.com> Changeset: f6698564775d Author: mcimadamore Date: 2019-01-24 14:41 +0000 URL: http://hg.openjdk.java.net/panama/dev/rev/f6698564775d 8217727: drop AVX compiler directives from panama/foreign tests Reviewed-by: sundar ! make/test/JtregNativeJdk.gmk ! test/jdk/java/foreign/GlobalVariable.java ! test/jdk/java/foreign/libGlobalVariable.c From maurizio.cimadamore at oracle.com Thu Jan 24 14:44:25 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 24 Jan 2019 14:44:25 +0000 Subject: [foreign] RFR 8217664: jextract should check for error code returns from Type::getOffsetOf and Type::size In-Reply-To: <8feff9ee0e27c76976ad315d0b43cb02@xs4all.nl> References: <8feff9ee0e27c76976ad315d0b43cb02@xs4all.nl> Message-ID: I'll leave the detailed review to Sundar - I just wanted to say that I noticed this issues as well in the past (e.g. Clang API returning weird things for incomplete arrays). On the binder test - maybe it shouldn't be in the jextract folder? Also, while you test that allocation works, the test is basically checking if two method calls return null as expected - which is a tad weird? Shouldn't we also check that we can actually get an instance of that array and that the size should be set to zero? (e.g. binder should not crash when dereferencing one of these odd zero-length arrays). Maurizio On 24/01/2019 13:04, Jorn Vernee wrote: > Hi, > > jextract uses libclang functions to query the size of a type and > offset of a field. The returned values are used for instance in layout > calculations. But, these functions can also return error codes and > there is currently no check for them. They are happily being used in > the layout calculations, and this can cause some strange errors, or in > some cases incorrect layouts are generated. To give an example: > > ??? struct Foo { > ??????? int x; > ??????? void *list_of_data[]; > ??? }; > > The expected layout of this struct (which uses an incomplete/flexible > array field) should be `[i32x32[0u64]]` but jextract instead generates > `[i32[0u64]x32]`. This particular case is due to a bug in libclang > where requesting the offset of a field in a type with a flexible array > member always returns an incomplete type error, even though getting > the offset of each fields should be possible. This error code (-2) is > then used in the layout calculations and yields the incorrect result. > > So I've added checking to the returned value of Type::size and > Type::getOffsetOf and throwing an exception in case an error code is > returned. I'm also explicitly checking for the flexible array case and > throwing a UOE in that case. This also means I had to disable the > flexible array jextract test [1]. The test was passing, but at least > on Windows the wrong layout was being generated. Since we can not > reliably compute padding for types with flexible arrays using the > current method, I think it's better to throw an exception for now > until we can find a workaround for this bug in libclang, or the bug > gets fixed. > > Using a type with a flexible/incomplete array member in the binder > should still be fine of course, so I've added a separate test for that. > > Please review the following. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8217664 > Webrev: > http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217664/webrev.00/ > > Thanks, > Jorn > > [1] : > http://hg.openjdk.java.net/panama/dev/file/c40a5a6dc24d/test/jdk/com/sun/tools/jextract/testStruct/struct.h#l85 From maurizio.cimadamore at oracle.com Thu Jan 24 14:46:37 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Thu, 24 Jan 2019 14:46:37 +0000 Subject: hg: panama/dev: Automatic merge with foreign Message-ID: <201901241446.x0OEkbow023269@aojmv0008.oracle.com> Changeset: 769235489c24 Author: mcimadamore Date: 2019-01-24 15:46 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/769235489c24 Automatic merge with foreign From jbvernee at xs4all.nl Thu Jan 24 15:55:07 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Thu, 24 Jan 2019 16:55:07 +0100 Subject: [foreign] RFR 8217664: jextract should check for error code returns from Type::getOffsetOf and Type::size In-Reply-To: References: <8feff9ee0e27c76976ad315d0b43cb02@xs4all.nl> Message-ID: <99a186494da6b93f999ab1210316a954@xs4all.nl> Maurizio Cimadamore schreef op 2019-01-24 15:44: > I'll leave the detailed review to Sundar - I just wanted to say that I > noticed this issues as well in the past (e.g. Clang API returning > weird things for incomplete arrays). > > On the binder test - maybe it shouldn't be in the jextract folder? > Also, while you test that allocation works, the test is basically > checking if two method calls return null as expected - which is a tad > weird? Shouldn't we also check that we can actually get an instance of > that array and that the size should be set to zero? (e.g. binder > should not crash when dereferencing one of these odd zero-length > arrays). Yes, good points - I just copied the functions being used by the jextract test. I've moved the test files to test/jdk/java/foreign. I fleshed out the test by setting/getting the fields of the struct, for which I also had to do a small tweak to BoundedMemoryRegion.copyTo to just return immediately if the length argument is 0. Updated webrev: http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217664/webrev.01/ Jorn > Maurizio > > On 24/01/2019 13:04, Jorn Vernee wrote: >> Hi, >> >> jextract uses libclang functions to query the size of a type and >> offset of a field. The returned values are used for instance in layout >> calculations. But, these functions can also return error codes and >> there is currently no check for them. They are happily being used in >> the layout calculations, and this can cause some strange errors, or in >> some cases incorrect layouts are generated. To give an example: >> >> ??? struct Foo { >> ??????? int x; >> ??????? void *list_of_data[]; >> ??? }; >> >> The expected layout of this struct (which uses an incomplete/flexible >> array field) should be `[i32x32[0u64]]` but jextract instead generates >> `[i32[0u64]x32]`. This particular case is due to a bug in libclang >> where requesting the offset of a field in a type with a flexible array >> member always returns an incomplete type error, even though getting >> the offset of each fields should be possible. This error code (-2) is >> then used in the layout calculations and yields the incorrect result. >> >> So I've added checking to the returned value of Type::size and >> Type::getOffsetOf and throwing an exception in case an error code is >> returned. I'm also explicitly checking for the flexible array case and >> throwing a UOE in that case. This also means I had to disable the >> flexible array jextract test [1]. The test was passing, but at least >> on Windows the wrong layout was being generated. Since we can not >> reliably compute padding for types with flexible arrays using the >> current method, I think it's better to throw an exception for now >> until we can find a workaround for this bug in libclang, or the bug >> gets fixed. >> >> Using a type with a flexible/incomplete array member in the binder >> should still be fine of course, so I've added a separate test for >> that. >> >> Please review the following. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8217664 >> Webrev: >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217664/webrev.00/ >> >> Thanks, >> Jorn >> >> [1] : >> http://hg.openjdk.java.net/panama/dev/file/c40a5a6dc24d/test/jdk/com/sun/tools/jextract/testStruct/struct.h#l85 From maurizio.cimadamore at oracle.com Thu Jan 24 15:59:49 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 24 Jan 2019 15:59:49 +0000 Subject: [foreign] RFR 8217664: jextract should check for error code returns from Type::getOffsetOf and Type::size In-Reply-To: <99a186494da6b93f999ab1210316a954@xs4all.nl> References: <8feff9ee0e27c76976ad315d0b43cb02@xs4all.nl> <99a186494da6b93f999ab1210316a954@xs4all.nl> Message-ID: Binder changes and test look good! Maurizio On 24/01/2019 15:55, Jorn Vernee wrote: > Maurizio Cimadamore schreef op 2019-01-24 15:44: >> I'll leave the detailed review to Sundar - I just wanted to say that I >> noticed this issues as well in the past (e.g. Clang API returning >> weird things for incomplete arrays). >> >> On the binder test - maybe it shouldn't be in the jextract folder? >> Also, while you test that allocation works, the test is basically >> checking if two method calls return null as expected - which is a tad >> weird? Shouldn't we also check that we can actually get an instance of >> that array and that the size should be set to zero? (e.g. binder >> should not crash when dereferencing one of these odd zero-length >> arrays). > > Yes, good points - I just copied the functions being used by the > jextract test. I've moved the test files to test/jdk/java/foreign. I > fleshed out the test by setting/getting the fields of the struct, for > which I also had to do a small tweak to BoundedMemoryRegion.copyTo to > just return immediately if the length argument is 0. > > Updated webrev: > http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217664/webrev.01/ > > Jorn > >> Maurizio >> >> On 24/01/2019 13:04, Jorn Vernee wrote: >>> Hi, >>> >>> jextract uses libclang functions to query the size of a type and >>> offset of a field. The returned values are used for instance in >>> layout calculations. But, these functions can also return error >>> codes and there is currently no check for them. They are happily >>> being used in the layout calculations, and this can cause some >>> strange errors, or in some cases incorrect layouts are generated. To >>> give an example: >>> >>> ??? struct Foo { >>> ??????? int x; >>> ??????? void *list_of_data[]; >>> ??? }; >>> >>> The expected layout of this struct (which uses an >>> incomplete/flexible array field) should be `[i32x32[0u64]]` but >>> jextract instead generates `[i32[0u64]x32]`. This particular case is >>> due to a bug in libclang where requesting the offset of a field in a >>> type with a flexible array member always returns an incomplete type >>> error, even though getting the offset of each fields should be >>> possible. This error code (-2) is then used in the layout >>> calculations and yields the incorrect result. >>> >>> So I've added checking to the returned value of Type::size and >>> Type::getOffsetOf and throwing an exception in case an error code is >>> returned. I'm also explicitly checking for the flexible array case >>> and throwing a UOE in that case. This also means I had to disable >>> the flexible array jextract test [1]. The test was passing, but at >>> least on Windows the wrong layout was being generated. Since we can >>> not reliably compute padding for types with flexible arrays using >>> the current method, I think it's better to throw an exception for >>> now until we can find a workaround for this bug in libclang, or the >>> bug gets fixed. >>> >>> Using a type with a flexible/incomplete array member in the binder >>> should still be fine of course, so I've added a separate test for that. >>> >>> Please review the following. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8217664 >>> Webrev: >>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217664/webrev.00/ >>> >>> Thanks, >>> Jorn >>> >>> [1] : >>> http://hg.openjdk.java.net/panama/dev/file/c40a5a6dc24d/test/jdk/com/sun/tools/jextract/testStruct/struct.h#l85 From sundararajan.athijegannathan at oracle.com Thu Jan 24 17:15:22 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Thu, 24 Jan 2019 22:45:22 +0530 Subject: [foreign] RFR 8217738: Refactor LayoutUtils code for handling structs and unions Message-ID: <5C49F2AA.9000104@oracle.com> Please review. Bug: https://bugs.openjdk.java.net/browse/JDK-8217738 Webrev: https://cr.openjdk.java.net/~sundar/8217738/webrev.01/ Thanks, -Sundar From maurizio.cimadamore at oracle.com Thu Jan 24 17:57:14 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 24 Jan 2019 17:57:14 +0000 Subject: [foreign] RFR 8217738: Refactor LayoutUtils code for handling structs and unions In-Reply-To: <5C49F2AA.9000104@oracle.com> References: <5C49F2AA.9000104@oracle.com> Message-ID: <64f96719-b359-8141-a830-23c273300954@oracle.com> Hi, I think it is a very good piece of code, which tries to put some sense into the monolithic blabbering I wrote few months ago :-) As discussed privately, I think there might be a further opportunity for improvement in the struct layout computer - that is, if we could fold the size/offset field together (which could be tricky), then we could have an helper method to add padding like this: void addPaddingIfNeeded(long expectedOffset) { ??? addFieldLayout(Padding.of(expectedOffset - offset)); ??? offfset = expectedOffset; } And then you can call this in the two places where padding is required (this relies on the fact that, at the end of struct processing, the offset we end up with should be the same as the size computed so far). But this can be addressed as a separate improvement. Maurizio On 24/01/2019 17:15, Sundararajan Athijegannathan wrote: > Please review. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8217738 > Webrev: https://cr.openjdk.java.net/~sundar/8217738/webrev.01/ > > Thanks, > -Sundar > From sundararajan.athijegannathan at oracle.com Thu Jan 24 18:14:50 2019 From: sundararajan.athijegannathan at oracle.com (sundararajan.athijegannathan at oracle.com) Date: Thu, 24 Jan 2019 18:14:50 +0000 Subject: hg: panama/dev: 8217738: Refactor LayoutUtils code for handling structs and unions Message-ID: <201901241814.x0OIEoqf018259@aojmv0008.oracle.com> Changeset: 7aadec261d59 Author: sundar Date: 2019-01-24 23:45 +0530 URL: http://hg.openjdk.java.net/panama/dev/rev/7aadec261d59 8217738: Refactor LayoutUtils code for handling structs and unions Reviewed-by: mcimadamore ! src/jdk.jextract/share/classes/com/sun/tools/jextract/AsmCodeFactory.java ! src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/LayoutUtils.java + src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/RecordLayoutComputer.java + src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/StructLayoutComputer.java ! src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/StructTree.java ! src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/TreePrinter.java + src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/UnionLayoutComputer.java From maurizio.cimadamore at oracle.com Thu Jan 24 18:20:12 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Thu, 24 Jan 2019 18:20:12 +0000 Subject: hg: panama/dev: Automatic merge with foreign Message-ID: <201901241820.x0OIKCRm020059@aojmv0008.oracle.com> Changeset: 0324aaf73279 Author: mcimadamore Date: 2019-01-24 19:20 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/0324aaf73279 Automatic merge with foreign From jbvernee at xs4all.nl Thu Jan 24 19:40:34 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Thu, 24 Jan 2019 20:40:34 +0100 Subject: [foreign] RFR 8217664: jextract should check for error code returns from Type::getOffsetOf and Type::size In-Reply-To: <8feff9ee0e27c76976ad315d0b43cb02@xs4all.nl> References: <8feff9ee0e27c76976ad315d0b43cb02@xs4all.nl> Message-ID: <263fe7d19c5f371316cb851b2e79f20b@xs4all.nl> Updated webrev for recent push: http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217664/webrev.03/ Jorn Jorn Vernee schreef op 2019-01-24 14:04: > Hi, > > jextract uses libclang functions to query the size of a type and > offset of a field. The returned values are used for instance in layout > calculations. But, these functions can also return error codes and > there is currently no check for them. They are happily being used in > the layout calculations, and this can cause some strange errors, or in > some cases incorrect layouts are generated. To give an example: > > struct Foo { > int x; > void *list_of_data[]; > }; > > The expected layout of this struct (which uses an incomplete/flexible > array field) should be `[i32x32[0u64]]` but jextract instead generates > `[i32[0u64]x32]`. This particular case is due to a bug in libclang > where requesting the offset of a field in a type with a flexible array > member always returns an incomplete type error, even though getting > the offset of each fields should be possible. This error code (-2) is > then used in the layout calculations and yields the incorrect result. > > So I've added checking to the returned value of Type::size and > Type::getOffsetOf and throwing an exception in case an error code is > returned. I'm also explicitly checking for the flexible array case and > throwing a UOE in that case. This also means I had to disable the > flexible array jextract test [1]. The test was passing, but at least > on Windows the wrong layout was being generated. Since we can not > reliably compute padding for types with flexible arrays using the > current method, I think it's better to throw an exception for now > until we can find a workaround for this bug in libclang, or the bug > gets fixed. > > Using a type with a flexible/incomplete array member in the binder > should still be fine of course, so I've added a separate test for > that. > > Please review the following. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8217664 > Webrev: > http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217664/webrev.00/ > > Thanks, > Jorn > > [1] : > http://hg.openjdk.java.net/panama/dev/file/c40a5a6dc24d/test/jdk/com/sun/tools/jextract/testStruct/struct.h#l85 From sundararajan.athijegannathan at oracle.com Fri Jan 25 02:21:04 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Fri, 25 Jan 2019 07:51:04 +0530 Subject: [foreign] RFR 8217664: jextract should check for error code returns from Type::getOffsetOf and Type::size In-Reply-To: <263fe7d19c5f371316cb851b2e79f20b@xs4all.nl> References: <8feff9ee0e27c76976ad315d0b43cb02@xs4all.nl> <263fe7d19c5f371316cb851b2e79f20b@xs4all.nl> Message-ID: <5C4A7290.4030005@oracle.com> Unused code should be removed and not just commented (struct.h for eg). We've history in the repo and we can always go back. PS. I'll build/test on Mac and respond. -Sundar On 25/01/19, 1:10 AM, Jorn Vernee wrote: > Updated webrev for recent push: > http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217664/webrev.03/ > > Jorn > > Jorn Vernee schreef op 2019-01-24 14:04: >> Hi, >> >> jextract uses libclang functions to query the size of a type and >> offset of a field. The returned values are used for instance in layout >> calculations. But, these functions can also return error codes and >> there is currently no check for them. They are happily being used in >> the layout calculations, and this can cause some strange errors, or in >> some cases incorrect layouts are generated. To give an example: >> >> struct Foo { >> int x; >> void *list_of_data[]; >> }; >> >> The expected layout of this struct (which uses an incomplete/flexible >> array field) should be `[i32x32[0u64]]` but jextract instead generates >> `[i32[0u64]x32]`. This particular case is due to a bug in libclang >> where requesting the offset of a field in a type with a flexible array >> member always returns an incomplete type error, even though getting >> the offset of each fields should be possible. This error code (-2) is >> then used in the layout calculations and yields the incorrect result. >> >> So I've added checking to the returned value of Type::size and >> Type::getOffsetOf and throwing an exception in case an error code is >> returned. I'm also explicitly checking for the flexible array case and >> throwing a UOE in that case. This also means I had to disable the >> flexible array jextract test [1]. The test was passing, but at least >> on Windows the wrong layout was being generated. Since we can not >> reliably compute padding for types with flexible arrays using the >> current method, I think it's better to throw an exception for now >> until we can find a workaround for this bug in libclang, or the bug >> gets fixed. >> >> Using a type with a flexible/incomplete array member in the binder >> should still be fine of course, so I've added a separate test for >> that. >> >> Please review the following. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8217664 >> Webrev: >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217664/webrev.00/ >> >> Thanks, >> Jorn >> >> [1] : >> http://hg.openjdk.java.net/panama/dev/file/c40a5a6dc24d/test/jdk/com/sun/tools/jextract/testStruct/struct.h#l85 >> From sundararajan.athijegannathan at oracle.com Fri Jan 25 08:57:26 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Fri, 25 Jan 2019 14:27:26 +0530 Subject: [foreign] RFR 8217664: jextract should check for error code returns from Type::getOffsetOf and Type::size In-Reply-To: <5C4A7290.4030005@oracle.com> References: <8feff9ee0e27c76976ad315d0b43cb02@xs4all.nl> <263fe7d19c5f371316cb851b2e79f20b@xs4all.nl> <5C4A7290.4030005@oracle.com> Message-ID: <5C4ACF76.6060902@oracle.com> jextract/clang changes fine. Tests pass on Mac. But as I mentioned, please remove unused code in tests rather than commenting it. -Sundar On 25/01/19, 7:51 AM, Sundararajan Athijegannathan wrote: > Unused code should be removed and not just commented (struct.h for > eg). We've history in the repo and we can always go back. > > PS. I'll build/test on Mac and respond. > > -Sundar > > On 25/01/19, 1:10 AM, Jorn Vernee wrote: >> Updated webrev for recent push: >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217664/webrev.03/ >> >> Jorn >> >> Jorn Vernee schreef op 2019-01-24 14:04: >>> Hi, >>> >>> jextract uses libclang functions to query the size of a type and >>> offset of a field. The returned values are used for instance in layout >>> calculations. But, these functions can also return error codes and >>> there is currently no check for them. They are happily being used in >>> the layout calculations, and this can cause some strange errors, or in >>> some cases incorrect layouts are generated. To give an example: >>> >>> struct Foo { >>> int x; >>> void *list_of_data[]; >>> }; >>> >>> The expected layout of this struct (which uses an incomplete/flexible >>> array field) should be `[i32x32[0u64]]` but jextract instead generates >>> `[i32[0u64]x32]`. This particular case is due to a bug in libclang >>> where requesting the offset of a field in a type with a flexible array >>> member always returns an incomplete type error, even though getting >>> the offset of each fields should be possible. This error code (-2) is >>> then used in the layout calculations and yields the incorrect result. >>> >>> So I've added checking to the returned value of Type::size and >>> Type::getOffsetOf and throwing an exception in case an error code is >>> returned. I'm also explicitly checking for the flexible array case and >>> throwing a UOE in that case. This also means I had to disable the >>> flexible array jextract test [1]. The test was passing, but at least >>> on Windows the wrong layout was being generated. Since we can not >>> reliably compute padding for types with flexible arrays using the >>> current method, I think it's better to throw an exception for now >>> until we can find a workaround for this bug in libclang, or the bug >>> gets fixed. >>> >>> Using a type with a flexible/incomplete array member in the binder >>> should still be fine of course, so I've added a separate test for >>> that. >>> >>> Please review the following. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8217664 >>> Webrev: >>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217664/webrev.00/ >>> >>> Thanks, >>> Jorn >>> >>> [1] : >>> http://hg.openjdk.java.net/panama/dev/file/c40a5a6dc24d/test/jdk/com/sun/tools/jextract/testStruct/struct.h#l85 >>> From jbvernee at xs4all.nl Fri Jan 25 11:29:31 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Fri, 25 Jan 2019 12:29:31 +0100 Subject: [foreign] RFR 8217664: jextract should check for error code returns from Type::getOffsetOf and Type::size In-Reply-To: <5C4ACF76.6060902@oracle.com> References: <8feff9ee0e27c76976ad315d0b43cb02@xs4all.nl> <263fe7d19c5f371316cb851b2e79f20b@xs4all.nl> <5C4A7290.4030005@oracle.com> <5C4ACF76.6060902@oracle.com> Message-ID: <0d37841e64ddd768b775032505ad2a15@xs4all.nl> Hi Sundar, I have removed the unused code. I've also added a test for the various flexible/incomplete array cases that should be caught (jextract/incompleteArrays/IncompleteArrayTest.java), can you take a look? Updated webrev: http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217664/webrev.04/ Thanks, Jorn Sundararajan Athijegannathan schreef op 2019-01-25 09:57: > jextract/clang changes fine. Tests pass on Mac. But as I mentioned, > please remove unused code in tests rather than commenting it. > > -Sundar > > On 25/01/19, 7:51 AM, Sundararajan Athijegannathan wrote: >> Unused code should be removed and not just commented (struct.h for >> eg). We've history in the repo and we can always go back. >> >> PS. I'll build/test on Mac and respond. >> >> -Sundar >> >> On 25/01/19, 1:10 AM, Jorn Vernee wrote: >>> Updated webrev for recent push: >>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217664/webrev.03/ >>> >>> Jorn >>> >>> Jorn Vernee schreef op 2019-01-24 14:04: >>>> Hi, >>>> >>>> jextract uses libclang functions to query the size of a type and >>>> offset of a field. The returned values are used for instance in >>>> layout >>>> calculations. But, these functions can also return error codes and >>>> there is currently no check for them. They are happily being used in >>>> the layout calculations, and this can cause some strange errors, or >>>> in >>>> some cases incorrect layouts are generated. To give an example: >>>> >>>> struct Foo { >>>> int x; >>>> void *list_of_data[]; >>>> }; >>>> >>>> The expected layout of this struct (which uses an >>>> incomplete/flexible >>>> array field) should be `[i32x32[0u64]]` but jextract instead >>>> generates >>>> `[i32[0u64]x32]`. This particular case is due to a bug in libclang >>>> where requesting the offset of a field in a type with a flexible >>>> array >>>> member always returns an incomplete type error, even though getting >>>> the offset of each fields should be possible. This error code (-2) >>>> is >>>> then used in the layout calculations and yields the incorrect >>>> result. >>>> >>>> So I've added checking to the returned value of Type::size and >>>> Type::getOffsetOf and throwing an exception in case an error code is >>>> returned. I'm also explicitly checking for the flexible array case >>>> and >>>> throwing a UOE in that case. This also means I had to disable the >>>> flexible array jextract test [1]. The test was passing, but at least >>>> on Windows the wrong layout was being generated. Since we can not >>>> reliably compute padding for types with flexible arrays using the >>>> current method, I think it's better to throw an exception for now >>>> until we can find a workaround for this bug in libclang, or the bug >>>> gets fixed. >>>> >>>> Using a type with a flexible/incomplete array member in the binder >>>> should still be fine of course, so I've added a separate test for >>>> that. >>>> >>>> Please review the following. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8217664 >>>> Webrev: >>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217664/webrev.00/ >>>> >>>> Thanks, >>>> Jorn >>>> >>>> [1] : >>>> http://hg.openjdk.java.net/panama/dev/file/c40a5a6dc24d/test/jdk/com/sun/tools/jextract/testStruct/struct.h#l85 From sundararajan.athijegannathan at oracle.com Fri Jan 25 11:42:22 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Fri, 25 Jan 2019 17:12:22 +0530 Subject: [foreign] RFR 8217664: jextract should check for error code returns from Type::getOffsetOf and Type::size In-Reply-To: <0d37841e64ddd768b775032505ad2a15@xs4all.nl> References: <8feff9ee0e27c76976ad315d0b43cb02@xs4all.nl> <263fe7d19c5f371316cb851b2e79f20b@xs4all.nl> <5C4A7290.4030005@oracle.com> <5C4ACF76.6060902@oracle.com> <0d37841e64ddd768b775032505ad2a15@xs4all.nl> Message-ID: <5C4AF61E.5080507@oracle.com> Looks good PS. I did not re-run the tests with this latest patch though -Sundar On 25/01/19, 4:59 PM, Jorn Vernee wrote: > Hi Sundar, > > I have removed the unused code. I've also added a test for the various > flexible/incomplete array cases that should be caught > (jextract/incompleteArrays/IncompleteArrayTest.java), can you take a > look? > > Updated webrev: > http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217664/webrev.04/ > > Thanks, > Jorn > > Sundararajan Athijegannathan schreef op 2019-01-25 09:57: >> jextract/clang changes fine. Tests pass on Mac. But as I mentioned, >> please remove unused code in tests rather than commenting it. >> >> -Sundar >> >> On 25/01/19, 7:51 AM, Sundararajan Athijegannathan wrote: >>> Unused code should be removed and not just commented (struct.h for >>> eg). We've history in the repo and we can always go back. >>> >>> PS. I'll build/test on Mac and respond. >>> >>> -Sundar >>> >>> On 25/01/19, 1:10 AM, Jorn Vernee wrote: >>>> Updated webrev for recent push: >>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217664/webrev.03/ >>>> >>>> Jorn >>>> >>>> Jorn Vernee schreef op 2019-01-24 14:04: >>>>> Hi, >>>>> >>>>> jextract uses libclang functions to query the size of a type and >>>>> offset of a field. The returned values are used for instance in >>>>> layout >>>>> calculations. But, these functions can also return error codes and >>>>> there is currently no check for them. They are happily being used in >>>>> the layout calculations, and this can cause some strange errors, >>>>> or in >>>>> some cases incorrect layouts are generated. To give an example: >>>>> >>>>> struct Foo { >>>>> int x; >>>>> void *list_of_data[]; >>>>> }; >>>>> >>>>> The expected layout of this struct (which uses an incomplete/flexible >>>>> array field) should be `[i32x32[0u64]]` but jextract instead >>>>> generates >>>>> `[i32[0u64]x32]`. This particular case is due to a bug in libclang >>>>> where requesting the offset of a field in a type with a flexible >>>>> array >>>>> member always returns an incomplete type error, even though getting >>>>> the offset of each fields should be possible. This error code (-2) is >>>>> then used in the layout calculations and yields the incorrect result. >>>>> >>>>> So I've added checking to the returned value of Type::size and >>>>> Type::getOffsetOf and throwing an exception in case an error code is >>>>> returned. I'm also explicitly checking for the flexible array case >>>>> and >>>>> throwing a UOE in that case. This also means I had to disable the >>>>> flexible array jextract test [1]. The test was passing, but at least >>>>> on Windows the wrong layout was being generated. Since we can not >>>>> reliably compute padding for types with flexible arrays using the >>>>> current method, I think it's better to throw an exception for now >>>>> until we can find a workaround for this bug in libclang, or the bug >>>>> gets fixed. >>>>> >>>>> Using a type with a flexible/incomplete array member in the binder >>>>> should still be fine of course, so I've added a separate test for >>>>> that. >>>>> >>>>> Please review the following. >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8217664 >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217664/webrev.00/ >>>>> >>>>> Thanks, >>>>> Jorn >>>>> >>>>> [1] : >>>>> http://hg.openjdk.java.net/panama/dev/file/c40a5a6dc24d/test/jdk/com/sun/tools/jextract/testStruct/struct.h#l85 >>>>> From jbvernee at xs4all.nl Fri Jan 25 12:00:20 2019 From: jbvernee at xs4all.nl (jbvernee at xs4all.nl) Date: Fri, 25 Jan 2019 12:00:20 +0000 Subject: hg: panama/dev: 8217664: jextract should check for error code returns from Type::getOffsetOf and Type::size Message-ID: <201901251200.x0PC0KR6023610@aojmv0008.oracle.com> Changeset: 907c209c9401 Author: jvernee Date: 2019-01-25 12:59 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/907c209c9401 8217664: jextract should check for error code returns from Type::getOffsetOf and Type::size Reviewed-by: sundar ! src/java.base/share/classes/jdk/internal/foreign/memory/BoundedMemoryRegion.java ! src/jdk.internal.clang/share/classes/jdk/internal/clang/Type.java + src/jdk.internal.clang/share/classes/jdk/internal/clang/TypeLayoutError.java ! src/jdk.internal.clang/share/native/libjclang/jdk_internal_clang.cpp ! src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/RecordLayoutComputer.java + test/jdk/com/sun/tools/jextract/incompleteArrays/IncompleteArrayTest.java + test/jdk/com/sun/tools/jextract/incompleteArrays/incompleteArray1.h + test/jdk/com/sun/tools/jextract/incompleteArrays/incompleteArray2.h + test/jdk/com/sun/tools/jextract/incompleteArrays/incompleteArray3.h ! test/jdk/com/sun/tools/jextract/jclang-ffi/src/jdk/internal/clang/Type.java + test/jdk/com/sun/tools/jextract/jclang-ffi/src/jdk/internal/clang/TypeLayoutError.java ! test/jdk/com/sun/tools/jextract/testStruct/StructTest.java ! test/jdk/com/sun/tools/jextract/testStruct/libStruct.c ! test/jdk/com/sun/tools/jextract/testStruct/struct.h + test/jdk/java/foreign/IncompleteArrayTest.java + test/jdk/java/foreign/libIncompleteArray.c From maurizio.cimadamore at oracle.com Fri Jan 25 12:06:01 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Fri, 25 Jan 2019 12:06:01 +0000 Subject: hg: panama/dev: Automatic merge with foreign Message-ID: <201901251206.x0PC61d3025614@aojmv0008.oracle.com> Changeset: 50e86d0dde56 Author: mcimadamore Date: 2019-01-25 13:05 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/50e86d0dde56 Automatic merge with foreign From jbvernee at xs4all.nl Fri Jan 25 12:26:18 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Fri, 25 Jan 2019 13:26:18 +0100 Subject: [foreign] RFR 8217781: Filter child cursors when doing recursive offset lookup Message-ID: <481fd4e444e00a8cda38d1e7c18f294e@xs4all.nl> Hi, Based on JDK-8217664 [1], I have found the following bug with this layout: ``` struct Foo { struct { union { int x; } Bar; }; }; ``` To find the offset of the anonymous struct in Foo, we have to find the offset of it's first field. But, the current code is looking for the first child _cursor_ of the anonymous struct, which is the union declaration, and not the field 'Bar'. This again tries to look up the first field of 'Bar' which is 'x', so we end up looking up the field 'x' in Foo, which throws an error. The fix is simple, we just have to filter for anonymous struct/union and FieldDecl child cursors when doing a recursive offset lookup. I've also added an exception message to the recursive lookup, which helped debug this. Please review the following. Bug: https://bugs.openjdk.java.net/browse/JDK-8217781 Webrev: http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217781/webrev.00/ Thanks, Jorn [1] : https://bugs.openjdk.java.net/browse/JDK-8217664 From jbvernee at xs4all.nl Fri Jan 25 13:52:16 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Fri, 25 Jan 2019 14:52:16 +0100 Subject: [foreign] RFR 8217784: Ignore anonymous bitfields so they are handled as padding Message-ID: <951b99d4c7d7e7667b6a192952f87e26@xs4all.nl> Hi, From the bug description: Jextract fails on the following example: ``` struct Foo { unsigned int x: 1; int :7; unsigned int y: 8; int :16; int z; }; ``` The current code tries to look up the second field, but because it has no name this throws an illegal field name error. --- I've fixed this by ignoring anonymous bitfields, since the StructLayoutComputer inserts padding automatically before the next field. Please review the following. Bug: https://bugs.openjdk.java.net/browse/JDK-8217784 Webrev: http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217784/webrev.00/ Thanks, Jorn From maurizio.cimadamore at oracle.com Fri Jan 25 17:49:59 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Fri, 25 Jan 2019 17:49:59 +0000 Subject: Lazy memory initialization for Unsafe (calloc vs. malloc) In-Reply-To: <8c81dd4a-7b25-f6c4-8dc4-cc3eec06b336@kuppe.org> References: <310950e3-c22f-32ef-d1ce-546ec0af2f7c@oracle.com> <8c81dd4a-7b25-f6c4-8dc4-cc3eec06b336@kuppe.org> Message-ID: <79eb930a-db83-493b-2437-7e8e02379133@oracle.com> On 25/01/2019 17:29, mail.openjdk.java.net at lemmster.de wrote: > On 16.01.19 04:07, Maurizio Cimadamore wrote: >> * while adding a new method to Unsafe seems like the most natural path, >> please be aware that we have committed to reduce the footprint of that >> API over time; your proposal goes in the opposite direction. That said, >> we could still have some internal functionality for performing calloc >> that is only used internally by the Panama offheap API >> >> * I'm not an expert in the area, but I'm wondering whether the lazy >> zeroing scheme is truly cross-platform as opposed to being just a Linux >> optimization; of course adding a new functionality that adds benefit on >> a single platform has a different cost compared to adding something that >> provides advantages across the board > > Hi Maurizio, > > I'd argue that the functionality of allocating initialized/zeroed memory > (calloc as opposed to malloc) offers advantages that are orthogonal to > the optimistic memory allocation strategy provided by the OS. That > said, I checked with a colleague who is intimately familiar with Windows > memory management. The Win32 API comes with VirtualAlloc* [1] which > supports a similar optimistic allocation strategy to what I described > for Linux. > I haven't tried implementing a prototype on Windows yet, but let me know > if it makes sense at this point. Please also advise how to move the API > out of Unsafe into Panama. Hi Markus, I think that, as an experiment you could try adding that functionality as a native method in the Panama internal ScopeImpl class, and have Scope use that for allocating memory, rather than Unsafe. Regarding platforms, it would be nice to know what is the performance cost for going with this option (which I agree would be desirable from an usability perspective), possibly on all the three supported platforms (Mac x64, Linux x64 and Windows x64). Cheers Maurizio > > I'd have to do more research to find out what's possible on macOS. > > Thanks > Markus > > [1] > https://msdn.microsoft.com/en-us/library/windows/desktop/aa366890(v=vs.85).aspx From maurizio.cimadamore at oracle.com Fri Jan 25 18:25:35 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Fri, 25 Jan 2019 18:25:35 +0000 Subject: [foreign] RFR 8217784: Ignore anonymous bitfields so they are handled as padding In-Reply-To: <951b99d4c7d7e7667b6a192952f87e26@xs4all.nl> References: <951b99d4c7d7e7667b6a192952f87e26@xs4all.nl> Message-ID: <01cc6681-1c2e-0e4f-e23d-af7b7713ff55@oracle.com> Looks good - but what happens with unions? I'm worried we'd get a similar failure there... Maurizio On 25/01/2019 13:52, Jorn Vernee wrote: > Hi, > > From the bug description: > > Jextract fails on the following example: > > ``` > struct Foo { > ??? unsigned int x: 1; > ??? int :7; > ??? unsigned int y: 8; > ??? int :16; > ??? int z; > }; > ``` > > The current code tries to look up the second field, but because it has > no name this throws an illegal field name error. > > --- > > I've fixed this by ignoring anonymous bitfields, since the > StructLayoutComputer inserts padding automatically before the next field. > > Please review the following. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8217784 > Webrev: > http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217784/webrev.00/ > > Thanks, > Jorn From maurizio.cimadamore at oracle.com Fri Jan 25 18:41:35 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Fri, 25 Jan 2019 18:41:35 +0000 Subject: [foreign] RFR 8217781: Filter child cursors when doing recursive offset lookup In-Reply-To: <496d5b3d-5bb9-bc04-1ebf-2b3784f344ed@oracle.com> References: <481fd4e444e00a8cda38d1e7c18f294e@xs4all.nl> <496d5b3d-5bb9-bc04-1ebf-2b3784f344ed@oracle.com> Message-ID: On 25/01/2019 18:40, Maurizio Cimadamore wrote: > But then, as I was writing this, I realized that RecordLayoutComputer > seems to have several helper methods which take a Cursor as input, > when in reality the only input we ever pass is the record cursor > itself (which is already stashed in a field) - so maybe we should get > rid of these extra Cursor params and use the field? Click send too fast :-) Of course these methods should be made non-static to access the field Maurizio From maurizio.cimadamore at oracle.com Fri Jan 25 18:40:14 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Fri, 25 Jan 2019 18:40:14 +0000 Subject: [foreign] RFR 8217781: Filter child cursors when doing recursive offset lookup In-Reply-To: <481fd4e444e00a8cda38d1e7c18f294e@xs4all.nl> References: <481fd4e444e00a8cda38d1e7c18f294e@xs4all.nl> Message-ID: <496d5b3d-5bb9-bc04-1ebf-2b3784f344ed@oracle.com> Looks good, Some minor suggestions - these kind of issues arise because the code should never call cursor.children(). I suggest getting rid of isFlattenable, and have an accessor which does this: Stream children(Cursor c) { ?? return c.children() ?????????????????? .filter(RecordLayoutComputer::isFlattenable) } But then, as I was writing this, I realized that RecordLayoutComputer seems to have several helper methods which take a Cursor as input, when in reality the only input we ever pass is the record cursor itself (which is already stashed in a field) - so maybe we should get rid of these extra Cursor params and use the field? Maurizio On 25/01/2019 12:26, Jorn Vernee wrote: > Hi, > > Based on JDK-8217664 [1], I have found the following bug with this > layout: > > ``` > struct Foo { > ??? struct { > ??????? union { > ??????????? int x; > ??????? } Bar; > ??? }; > }; > ``` > > To find the offset of the anonymous struct in Foo, we have to find the > offset of it's first field. But, the current code is looking for the > first child _cursor_ of the anonymous struct, which is the union > declaration, and not the field 'Bar'. This again tries to look up the > first field of 'Bar' which is 'x', so we end up looking up the field > 'x' in Foo, which throws an error. > > The fix is simple, we just have to filter for anonymous struct/union > and FieldDecl child cursors when doing a recursive offset lookup. I've > also added an exception message to the recursive lookup, which helped > debug this. > > Please review the following. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8217781 > Webrev: > http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217781/webrev.00/ > > Thanks, > Jorn > > [1] : https://bugs.openjdk.java.net/browse/JDK-8217664 From jbvernee at xs4all.nl Fri Jan 25 19:18:19 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Fri, 25 Jan 2019 20:18:19 +0100 Subject: [foreign] RFR 8217784: Ignore anonymous bitfields so they are handled as padding In-Reply-To: <01cc6681-1c2e-0e4f-e23d-af7b7713ff55@oracle.com> References: <951b99d4c7d7e7667b6a192952f87e26@xs4all.nl> <01cc6681-1c2e-0e4f-e23d-af7b7713ff55@oracle.com> Message-ID: <35c8f6bcec5732a2f5e54256c441bc53@xs4all.nl> Good catch. I kinda glossed over unions since I thought there would be no point in having anonymous bitfields in a union, since there is no need for padding - but on second thought, these could be used to manually widen the type. Any ways, the compiler seems to allow anonymous bitfields in unions, so it's good to support that as well. Updated webrev: http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217784/webrev.01/ Jorn Maurizio Cimadamore schreef op 2019-01-25 19:25: > Looks good - but what happens with unions? I'm worried we'd get a > similar failure there... > > Maurizio > > On 25/01/2019 13:52, Jorn Vernee wrote: >> Hi, >> >> From the bug description: >> >> Jextract fails on the following example: >> >> ``` >> struct Foo { >> ??? unsigned int x: 1; >> ??? int :7; >> ??? unsigned int y: 8; >> ??? int :16; >> ??? int z; >> }; >> ``` >> >> The current code tries to look up the second field, but because it has >> no name this throws an illegal field name error. >> >> --- >> >> I've fixed this by ignoring anonymous bitfields, since the >> StructLayoutComputer inserts padding automatically before the next >> field. >> >> Please review the following. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8217784 >> Webrev: >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217784/webrev.00/ >> >> Thanks, >> Jorn From jbvernee at xs4all.nl Fri Jan 25 19:45:02 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Fri, 25 Jan 2019 20:45:02 +0100 Subject: [foreign] RFR 8217781: Filter child cursors when doing recursive offset lookup In-Reply-To: <496d5b3d-5bb9-bc04-1ebf-2b3784f344ed@oracle.com> References: <481fd4e444e00a8cda38d1e7c18f294e@xs4all.nl> <496d5b3d-5bb9-bc04-1ebf-2b3784f344ed@oracle.com> Message-ID: <5a7368f419256b80bf7867d73c300de8@xs4all.nl> > But then, as I was writing this, I realized that RecordLayoutComputer > seems to have several helper methods which take a Cursor as input, > when in reality the only input we ever pass is the record cursor > itself (which is already stashed in a field) - so maybe we should get > rid of these extra Cursor params and use the field? This is not quite possible. `hasIncompleteArray` also calls these methods with the child cursors of anonymous nested structs and unions. So we can't just make the helper methods non-static and use the field. It's quite unfortunate the way the bug works. We have to check for incomplete array fields before trying to process _any_ of the fields, which means doing a recursive lookup into the anonymous structs and unions inside the type. But there is a simplification possible by replacing isFlattenable with the method you suggest (which I've named `flattenableChildren` to show the difference with `children()`) Updated webrev: http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217781/webrev.01/ Cheer, Jorn Maurizio Cimadamore schreef op 2019-01-25 19:40: > Looks good, > Some minor suggestions - these kind of issues arise because the code > should never call cursor.children(). > > I suggest getting rid of isFlattenable, and have an accessor which does > this: > > Stream children(Cursor c) { > ?? return c.children() > ?????????????????? .filter(RecordLayoutComputer::isFlattenable) > } > > But then, as I was writing this, I realized that RecordLayoutComputer > seems to have several helper methods which take a Cursor as input, > when in reality the only input we ever pass is the record cursor > itself (which is already stashed in a field) - so maybe we should get > rid of these extra Cursor params and use the field? > > Maurizio > > On 25/01/2019 12:26, Jorn Vernee wrote: >> Hi, >> >> Based on JDK-8217664 [1], I have found the following bug with this >> layout: >> >> ``` >> struct Foo { >> ??? struct { >> ??????? union { >> ??????????? int x; >> ??????? } Bar; >> ??? }; >> }; >> ``` >> >> To find the offset of the anonymous struct in Foo, we have to find the >> offset of it's first field. But, the current code is looking for the >> first child _cursor_ of the anonymous struct, which is the union >> declaration, and not the field 'Bar'. This again tries to look up the >> first field of 'Bar' which is 'x', so we end up looking up the field >> 'x' in Foo, which throws an error. >> >> The fix is simple, we just have to filter for anonymous struct/union >> and FieldDecl child cursors when doing a recursive offset lookup. I've >> also added an exception message to the recursive lookup, which helped >> debug this. >> >> Please review the following. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8217781 >> Webrev: >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217781/webrev.00/ >> >> Thanks, >> Jorn >> >> [1] : https://bugs.openjdk.java.net/browse/JDK-8217664 From brian.goetz at oracle.com Fri Jan 25 21:00:14 2019 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 25 Jan 2019 16:00:14 -0500 Subject: [vector] Vector API -- alignment with value types Message-ID: Coming back to this after a long break... As I mentioned in the past, I think the high-order bit of where we want to rotate the API is to ensure that when we have value types, the key abstractions -- Species and Vector -- can be values, because then we'll get most of the optimizations we want from the general properties of values, rather than ad-hoc tricks surrounding the vector types. Part I ------ Here's an idea for simplifying Species, which is: let's drive Species down to be a simple class that is really just a constant holder for (element type, shape), and move all the behavior to static methods on XxxVector.? (At that point, Species can just be an enum, or a group of enums with a common interface parent, if we like.)? I think we can greatly reduce the importance of Species in the API, making XxxVector the star player. (The cost here is it becomes harder to write code that is agnostic to _both_ element type and size -- but I am not convinced this is an important use case?) Here are the methods on Species currently: ?- Simple state methods: elementType, elementSize, shape, length, bitSize ?- Generic factories: zero(), fromByteArray(), fromByteBuffer(), maskXxx(), shuffleXxx() ?- Transforms: reshape(), rebracket(), resize(), cast() ?- Specialized factories: broadcast(), single(), random(), scalars(), fromArray() My somewhat radical suggestion is: let's get all of these, except the first line, off of Species, and onto XxxVector, with versions that take an explicit species argument, and versions that take no species argument (defaulting to the preferred species for that shape.) One positive result of this is that code that just wants to multiply int vectors can remain _entirely ignorant of species_; you just use the defaults: ??? IntVector.fromArray(...).add(...).intoArray(...) Only if users want to have finer control over the vector width do they need to use species at all. A slightly negative result is that one loses the ability to write code that is agnostic across both element type and width. Another slightly negative result is that some of these methods will have to dispatch on the species argument, which means we will need strong constant propagation so that these dispatches fold away.? However, I think those fold away in exactly the same cases that the virtual methods on species do today, so I don't think this is necessarily a change in reality. At this point, we're ready for species to become values, which will only help our constant propagation story. Part II ------- To make the Vector types value-ready, we have to flatten out the inheritance hierarchy.? This is an easy enough game; we make the concrete vector types into values, and the abstract vector types into interfaces.? So we have: ??? public interface Vector { .. } ??? public interface IntVector <: Vector { .. } ??? private value class Int64Vector <: IntVector { .. } This is easy enough, but I'm sure if we just did this, the carefully crafted optimizations we've done for vectors would fail (at first) when the abstract vector types become interfaces. We'll also need to ensure that type sharpening / nullity analysis is up to the task, so we get scalarization.? But this seems the straightest path to get from the API we have to one that use value types. Another path, slightly more circuitous, is to collapse the XxxNnVector types to a single XxxVector value type, which looks something like: ??? IntVector { ??????? IntSpecies species; ??????? T vector; ??? } where there are SuperLong types for 64, 128, 256, and 512.? This gets us away from using arrays, which is eventually where we'll want to go, but it's a longer road. Thoughts on these directions? From maurizio.cimadamore at oracle.com Fri Jan 25 21:24:53 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Fri, 25 Jan 2019 21:24:53 +0000 Subject: [foreign] RFR 8217784: Ignore anonymous bitfields so they are handled as padding In-Reply-To: <35c8f6bcec5732a2f5e54256c441bc53@xs4all.nl> References: <951b99d4c7d7e7667b6a192952f87e26@xs4all.nl> <01cc6681-1c2e-0e4f-e23d-af7b7713ff55@oracle.com> <35c8f6bcec5732a2f5e54256c441bc53@xs4all.nl> Message-ID: The patch is functionally good, I just wonder if it's too magic - e.g. wouldn't be better if the record computer processed the anonymous bitfield (so that the correct padding can be added) instead of relying on the right thing to happen (which will be fragile if e.g. the impl changes a bit) ? Maurizio On 25/01/2019 19:18, Jorn Vernee wrote: > Good catch. I kinda glossed over unions since I thought there would be > no point in having anonymous bitfields in a union, since there is no > need for padding - but on second thought, these could be used to > manually widen the type. Any ways, the compiler seems to allow > anonymous bitfields in unions, so it's good to support that as well. > > Updated webrev: > http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217784/webrev.01/ > > Jorn > > Maurizio Cimadamore schreef op 2019-01-25 19:25: >> Looks good - but what happens with unions? I'm worried we'd get a >> similar failure there... >> >> Maurizio >> >> On 25/01/2019 13:52, Jorn Vernee wrote: >>> Hi, >>> >>> From the bug description: >>> >>> Jextract fails on the following example: >>> >>> ``` >>> struct Foo { >>> ??? unsigned int x: 1; >>> ??? int :7; >>> ??? unsigned int y: 8; >>> ??? int :16; >>> ??? int z; >>> }; >>> ``` >>> >>> The current code tries to look up the second field, but because it >>> has no name this throws an illegal field name error. >>> >>> --- >>> >>> I've fixed this by ignoring anonymous bitfields, since the >>> StructLayoutComputer inserts padding automatically before the next >>> field. >>> >>> Please review the following. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8217784 >>> Webrev: >>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217784/webrev.00/ >>> >>> Thanks, >>> Jorn From maurizio.cimadamore at oracle.com Fri Jan 25 21:26:19 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Fri, 25 Jan 2019 21:26:19 +0000 Subject: [foreign] RFR 8217784: Ignore anonymous bitfields so they are handled as padding In-Reply-To: References: <951b99d4c7d7e7667b6a192952f87e26@xs4all.nl> <01cc6681-1c2e-0e4f-e23d-af7b7713ff55@oracle.com> <35c8f6bcec5732a2f5e54256c441bc53@xs4all.nl> Message-ID: <2ab9e217-529c-cc33-efea-14a09fadcb8e@oracle.com> Looks good by me - I suggest you also wait for Sundar's opinion, since he's fresh with this code Maurizio On 25/01/2019 21:24, Maurizio Cimadamore wrote: > The patch is functionally good, I just wonder if it's too magic - e.g. > wouldn't be better if the record computer processed the anonymous > bitfield (so that the correct padding can be added) instead of relying > on the right thing to happen (which will be fragile if e.g. the impl > changes a bit) ? > > Maurizio > > On 25/01/2019 19:18, Jorn Vernee wrote: >> Good catch. I kinda glossed over unions since I thought there would >> be no point in having anonymous bitfields in a union, since there is >> no need for padding - but on second thought, these could be used to >> manually widen the type. Any ways, the compiler seems to allow >> anonymous bitfields in unions, so it's good to support that as well. >> >> Updated webrev: >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217784/webrev.01/ >> >> Jorn >> >> Maurizio Cimadamore schreef op 2019-01-25 19:25: >>> Looks good - but what happens with unions? I'm worried we'd get a >>> similar failure there... >>> >>> Maurizio >>> >>> On 25/01/2019 13:52, Jorn Vernee wrote: >>>> Hi, >>>> >>>> From the bug description: >>>> >>>> Jextract fails on the following example: >>>> >>>> ``` >>>> struct Foo { >>>> ??? unsigned int x: 1; >>>> ??? int :7; >>>> ??? unsigned int y: 8; >>>> ??? int :16; >>>> ??? int z; >>>> }; >>>> ``` >>>> >>>> The current code tries to look up the second field, but because it >>>> has no name this throws an illegal field name error. >>>> >>>> --- >>>> >>>> I've fixed this by ignoring anonymous bitfields, since the >>>> StructLayoutComputer inserts padding automatically before the next >>>> field. >>>> >>>> Please review the following. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8217784 >>>> Webrev: >>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217784/webrev.00/ >>>> >>>> Thanks, >>>> Jorn From jbvernee at xs4all.nl Fri Jan 25 22:43:49 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Fri, 25 Jan 2019 23:43:49 +0100 Subject: [foreign] RFR 8217784: Ignore anonymous bitfields so they are handled as padding In-Reply-To: References: <951b99d4c7d7e7667b6a192952f87e26@xs4all.nl> <01cc6681-1c2e-0e4f-e23d-af7b7713ff55@oracle.com> <35c8f6bcec5732a2f5e54256c441bc53@xs4all.nl> Message-ID: <7b527b6f582b211e2dc61c92fcff6af7@xs4all.nl> I think it's okay. The anonymous bitfield is still handled in the sense that libClang will use it to compute the offset for the next named field. We just derive the padding from that offset instead of trying to compute it ourselves from the anonymous bitfield. Handling the anonymous bitfield manually would at first glance also seem to make the code a bit more complex, since we are breaking the assumption that we can get the offset of any field we are processing. Ignoring the anonymous bitfield just kind of works out nicely. The change in UnionLayoutComputer looks kind of magic. There might be other reasons why the computed size of a union is smaller than the size reported by libClang. An anonymous bitfield is the only reason I could think of, so that's why I wrote that specific comment there. The change in implementation is because the previous implementation was incorrect AFACT; if the computed size is too small, we need a padding member with the expected size of the union, since all fields of a union overlap. Jorn Maurizio Cimadamore schreef op 2019-01-25 22:24: > The patch is functionally good, I just wonder if it's too magic - e.g. > wouldn't be better if the record computer processed the anonymous > bitfield (so that the correct padding can be added) instead of relying > on the right thing to happen (which will be fragile if e.g. the impl > changes a bit) ? > > Maurizio > > On 25/01/2019 19:18, Jorn Vernee wrote: >> Good catch. I kinda glossed over unions since I thought there would be >> no point in having anonymous bitfields in a union, since there is no >> need for padding - but on second thought, these could be used to >> manually widen the type. Any ways, the compiler seems to allow >> anonymous bitfields in unions, so it's good to support that as well. >> >> Updated webrev: >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217784/webrev.01/ >> >> Jorn >> >> Maurizio Cimadamore schreef op 2019-01-25 19:25: >>> Looks good - but what happens with unions? I'm worried we'd get a >>> similar failure there... >>> >>> Maurizio >>> >>> On 25/01/2019 13:52, Jorn Vernee wrote: >>>> Hi, >>>> >>>> From the bug description: >>>> >>>> Jextract fails on the following example: >>>> >>>> ``` >>>> struct Foo { >>>> ??? unsigned int x: 1; >>>> ??? int :7; >>>> ??? unsigned int y: 8; >>>> ??? int :16; >>>> ??? int z; >>>> }; >>>> ``` >>>> >>>> The current code tries to look up the second field, but because it >>>> has no name this throws an illegal field name error. >>>> >>>> --- >>>> >>>> I've fixed this by ignoring anonymous bitfields, since the >>>> StructLayoutComputer inserts padding automatically before the next >>>> field. >>>> >>>> Please review the following. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8217784 >>>> Webrev: >>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217784/webrev.00/ >>>> >>>> Thanks, >>>> Jorn From vladimir.x.ivanov at oracle.com Fri Jan 25 23:44:09 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 25 Jan 2019 15:44:09 -0800 Subject: [Vector] RFR: reshape, resize, rebracket, cast In-Reply-To: <367c043f-d66f-17df-c4cb-910a19a5f30f@oracle.com> References: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A5012D@FMSMSX126.amr.corp.intel.com> <367c043f-d66f-17df-c4cb-910a19a5f30f@oracle.com> Message-ID: <6d935f65-36b7-2824-9bf6-74a2b75e9a60@oracle.com> >> I have new webrev which modifies the tests to use the new methods - >> http://cr.openjdk.java.net/~kkharbas/vector-api/webrev-reshape_etc.01 >> Incremental webrev - >> http://cr.openjdk.java.net/~kkharbas/vector-api/webrev-reshape_etc.01_to_base One comment about the patch: src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Vector.java: + public Vector defaultReinterpret(Species s) { + public abstract Class vectorType(); Those operations shouldn't be part of public API. Best regards, Vladimir Ivanov > It looks good to me and aligned with what was discussed previously. > > Please, also update users: > > $ egrep -r -e resize -e reshape -e rebracket -I -l > test/jdk/jdk/incubator/vector > test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/AbstractVectorBenchmark.java > > test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/PopulationCount.java > > test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Rearrange.java > > test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/SumOfUnsignedBytes.java > > test/jdk/jdk/incubator/vector/VectorHash.java > test/jdk/jdk/incubator/vector/VectorReshapeTests.java > test/jdk/jdk/incubator/vector/CovarOverrideTest.java > >> New methods: >> ???? class Vector { >> >> ???????? /* E x S? ->? E' x S' */ >> ???????? /* was reshape() and rebracket() before */ >> ???????? Vector reinterpret(Species s) >> >> ???????? /* E x S? ->? E x S' */ >> ???????? /* was resize() before */ >> ???????? Vector reshape(Species s) >> ???????? /* E x S? ->? E' x S' */ >> ???????? Vector cast(Species s) >> ???? } > > As a next step, I'd like to question the presence of `reshape`. > > It seemed reasonable to keep it at the time discussion happened, but > based on my recent experience with the API, `reinterpret`/`cast` look > more than enough unless truncation/padding is forbidden on > `reinterpret`/`cast` and exposed through a dedicated API point (`reshape`). > > There's one nice thing about `reshape`: it preserves element type, so > there's no need to cast result back to specialized vector type (e.g., > Vector => IntVector). But `reinterpret`/`cast` suffer from that > and it would be nice to provide a workaround/solution. > > For example, how about introducing Vector.toIntVector() et al which > serve as assertions about vector shapes backed by runtime checks? > > ? IntVector?? v = ... ; // Int128Vector > ? LongSpecies s = ... ; // Long256Species > ? LongVector vl1 = (LongVector)v.cast(s); > ? LongVector = v.cast(s).toLongVector(); > > Should work rather well for fluent-style code. > > Best regards, > Vladimir Ivanov > >>> -----Original Message----- >>> From: Viswanathan, Sandhya >>> Sent: Wednesday, January 16, 2019 12:33 PM >>> To: Kharbas, Kishor ; panama- >>> dev at openjdk.java.net; Brian Goetz ; John Rose >>> ; Vladimir Ivanov >>> >>> Cc: joe.darcy at oracle.com >>> Subject: RE: [Vector] RFR: reshape, resize, rebracket, cast >>> >>> >>> Adding Brian, John, Joe and Vladimir to the thread. Please do give your >>> feedback on the patch from Kishor >>> (http://cr.openjdk.java.net/~kkharbas/vector-api/webrev-reshape_etc) >>> implementing the reinterpret API as suggested by Brian and Vladimir in >>> thread https://mail.openjdk.java.net/pipermail/panama-dev/2018- >>> December/003365.html. >>> >>> Best Regards, >>> Sandhya >>> >>> >>> -----Original Message----- >>> From: panama-dev [mailto:panama-dev-bounces at openjdk.java.net] On >>> Behalf Of Kharbas, Kishor >>> Sent: Wednesday, January 16, 2019 12:20 AM >>> To: panama-dev at openjdk.java.net >>> Subject: [Vector] RFR: reshape, resize, rebracket, cast >>> >>> Hi, >>> I have a patch which refactors the above methods as discussed in this >>> thread >>> - https://mail.openjdk.java.net/pipermail/panama-dev/2018- >>> December/003365.html. >>> >>> Please review the changes at - >>> http://cr.openjdk.java.net/~kkharbas/vector- >>> api/webrev-reshape_etc >>> Generation of intrinsics and correct vector boxing elimination have been >>> verified. In the next patch I will add changes to jtreg tests. >>> >>> I wanted to bring forth one small issue(possibly) which programmers >>> might >>> face with this change - >>> >>> We provide specialized types like IntVector, FloatVector, etc for >>> users to >>> define their vectors and they would be able to write code like this with >>> previous methods, >>> ???? FloatVector float256 = SPECIES_FLOAT256.cast(int256); Here >>> FloatSpecies >>> would always return FloatVector. >>> >>> However with this change, since cast() is defined on a vector and takes >>> species of a generic type, it cannot return a specialized Vector like >>> IntVector >>> or FloatVector. User has to explicitly cast the return vector from >>> Vector >>> to specialized Vector or use a generic vector of corresponding >>> element type. >>> For example, >>> ???? FloatVector float256 = (FloatVector)int256.cast(SPECIES_FLOAT256); >>> ?? or >>> ???? Vector float256 = int256.cast(SPECIES_FLOAT256); >>> >>> I am not sure if this is even a problem, but I thought its worth >>> mentioning. >>> >>> Thanks >>> Kishor From vladimir.x.ivanov at oracle.com Sat Jan 26 00:14:42 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 25 Jan 2019 16:14:42 -0800 Subject: [vector] Vector API -- alignment with value types In-Reply-To: References: Message-ID: > As I mentioned in the past, I think the high-order bit of where we want > to rotate the API is to ensure that when we have value types, the key > abstractions -- Species and Vector -- can be values, because then we'll > get most of the optimizations we want from the general properties of > values, rather than ad-hoc tricks surrounding the vector types. > > Part I > ------ > > Here's an idea for simplifying Species, which is: let's drive Species > down to be a simple class that is really just a constant holder for > (element type, shape), and move all the behavior to static methods on > XxxVector.? (At that point, Species can just be an enum, or a group of > enums with a common interface parent, if we like.)? I think we can > greatly reduce the importance of Species in the API, making XxxVector > the star player. > > (The cost here is it becomes harder to write code that is agnostic to > _both_ element type and size -- but I am not convinced this is an > important use case?) > > Here are the methods on Species currently: > > ?- Simple state methods: elementType, elementSize, shape, length, bitSize > ?- Generic factories: zero(), fromByteArray(), fromByteBuffer(), > maskXxx(), shuffleXxx() > ?- Transforms: reshape(), rebracket(), resize(), cast() > ?- Specialized factories: broadcast(), single(), random(), scalars(), > fromArray() > > My somewhat radical suggestion is: let's get all of these, except the > first line, off of Species, and onto XxxVector, with versions that take > an explicit species argument, and versions that take no species argument > (defaulting to the preferred species for that shape.) > > One positive result of this is that code that just wants to multiply int > vectors can remain _entirely ignorant of species_; you just use the > defaults: > > ??? IntVector.fromArray(...).add(...).intoArray(...) > > Only if users want to have finer control over the vector width do they > need to use species at all. > A slightly negative result is that one loses the ability to write code > that is agnostic across both element type and width. Another downside: for factories it means 2 overloads per operation - default and parameterized by Species. Leaving parameterized variants on Species and introducing default ones (as static methods) looks more attractive to me. > Another slightly negative result is that some of these methods will have > to dispatch on the species argument, which means we will need strong > constant propagation so that these dispatches fold away.? However, I > think those fold away in exactly the same cases that the virtual methods > on species do today, so I don't think this is necessarily a change in > reality. Agree. It can even stay the same virtual call in the implementation: IntVector.fromArray(Species s, ...) { return s.fromArray(...); } > At this point, we're ready for species to become values, which will only > help our constant propagation story. > Part II > ------- > > To make the Vector types value-ready, we have to flatten out the > inheritance hierarchy.? This is an easy enough game; we make the > concrete vector types into values, and the abstract vector types into > interfaces.? So we have: > > ??? public interface Vector { .. } > ??? public interface IntVector <: Vector { .. } > > ??? private value class Int64Vector <: IntVector { .. } > > This is easy enough, but I'm sure if we just did this, the carefully > crafted optimizations we've done for vectors would fail (at first) when > the abstract vector types become interfaces. We'll also need to ensure > that type sharpening / nullity analysis is up to the task, so we get > scalarization.? But this seems the straightest path to get from the API > we have to one that use value types. > > Another path, slightly more circuitous, is to collapse the XxxNnVector > types to a single XxxVector value type, which looks something like: > > ??? IntVector { > ??????? IntSpecies species; > ??????? T vector; > ??? } > > where there are SuperLong types for 64, 128, 256, and 512.? This gets us > away from using arrays, which is eventually where we'll want to go, but > it's a longer road. As a first step, I'm in favor of the former proposal. Reducing the number of specializations is attractive, but additional performance work is needed to ensure JIT-compiler is capable of eliminating the boxes. So, I'd prefer to see it as a separate experiment. Right now, the hierarchy is the following: public abstract class Vector { .. } public abstract class IntVector extends Vector { .. } /*package-private*/ class Int64Vector implements IntVector { .. } The only reason why Vector & IntVector are abstract classes is simplification of custom box elimination logic. My understanding is it shouldn't be too much work to extend it. The plan was to get a bare minimum to keep the implementation viable while waiting for value types. Best regards, Vladimir Ivanov From Yang.Zhang at arm.com Mon Jan 28 02:44:45 2019 From: Yang.Zhang at arm.com (Yang Zhang (Arm Technology China)) Date: Mon, 28 Jan 2019 02:44:45 +0000 Subject: [vector api] RFR: Implement Vector API andAll/orAll/xorAll for AArch64 NEON Message-ID: Hi I have a patch which implements Vector API andAll/orAll/xorAll for AArch64 NEON. Could you please help to review it? http://cr.openjdk.java.net/~yzhang/vectorapi.andall/webrev.00/ Regards, Yang From sundararajan.athijegannathan at oracle.com Mon Jan 28 09:41:44 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Mon, 28 Jan 2019 15:11:44 +0530 Subject: [foreign] RFR 8217784: Ignore anonymous bitfields so they are handled as padding In-Reply-To: <2ab9e217-529c-cc33-efea-14a09fadcb8e@oracle.com> References: <951b99d4c7d7e7667b6a192952f87e26@xs4all.nl> <01cc6681-1c2e-0e4f-e23d-af7b7713ff55@oracle.com> <35c8f6bcec5732a2f5e54256c441bc53@xs4all.nl> <2ab9e217-529c-cc33-efea-14a09fadcb8e@oracle.com> Message-ID: <5C4ECE58.5080900@oracle.com> Looks good. Mac build/test fine. Minor: why is the specific imports in bitfields.java are replaced with import *. Your IDE? -Sundar On 26/01/19, 2:56 AM, Maurizio Cimadamore wrote: > Looks good by me - I suggest you also wait for Sundar's opinion, since > he's fresh with this code > > Maurizio > > On 25/01/2019 21:24, Maurizio Cimadamore wrote: >> The patch is functionally good, I just wonder if it's too magic - >> e.g. wouldn't be better if the record computer processed the >> anonymous bitfield (so that the correct padding can be added) instead >> of relying on the right thing to happen (which will be fragile if >> e.g. the impl changes a bit) ? >> >> Maurizio >> >> On 25/01/2019 19:18, Jorn Vernee wrote: >>> Good catch. I kinda glossed over unions since I thought there would >>> be no point in having anonymous bitfields in a union, since there is >>> no need for padding - but on second thought, these could be used to >>> manually widen the type. Any ways, the compiler seems to allow >>> anonymous bitfields in unions, so it's good to support that as well. >>> >>> Updated webrev: >>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217784/webrev.01/ >>> >>> Jorn >>> >>> Maurizio Cimadamore schreef op 2019-01-25 19:25: >>>> Looks good - but what happens with unions? I'm worried we'd get a >>>> similar failure there... >>>> >>>> Maurizio >>>> >>>> On 25/01/2019 13:52, Jorn Vernee wrote: >>>>> Hi, >>>>> >>>>> From the bug description: >>>>> >>>>> Jextract fails on the following example: >>>>> >>>>> ``` >>>>> struct Foo { >>>>> unsigned int x: 1; >>>>> int :7; >>>>> unsigned int y: 8; >>>>> int :16; >>>>> int z; >>>>> }; >>>>> ``` >>>>> >>>>> The current code tries to look up the second field, but because it >>>>> has no name this throws an illegal field name error. >>>>> >>>>> --- >>>>> >>>>> I've fixed this by ignoring anonymous bitfields, since the >>>>> StructLayoutComputer inserts padding automatically before the next >>>>> field. >>>>> >>>>> Please review the following. >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8217784 >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217784/webrev.00/ >>>>> >>>>> Thanks, >>>>> Jorn From aph at redhat.com Mon Jan 28 10:13:02 2019 From: aph at redhat.com (Andrew Haley) Date: Mon, 28 Jan 2019 10:13:02 +0000 Subject: [vector api] RFR: Implement Vector API andAll/orAll/xorAll for AArch64 NEON In-Reply-To: References: Message-ID: <838a6bca-8f67-eca5-f223-c1a5f377541a@redhat.com> On 1/28/19 2:44 AM, Yang Zhang (Arm Technology China) wrote: > I have a patch which implements Vector API andAll/orAll/xorAll for AArch64 NEON. Could you please help to review it? > http://cr.openjdk.java.net/~yzhang/vectorapi.andall/webrev.00/ OK, but: You're doing a great deal of repetitive stuff in here. This makes the patch hard to maintain and review. Instead, I suggest that you have a look at aarch64_ad.m4 to see how it's possible to generate all of these patterns from a script: 487 lines of that script generates 1970 lines of patterns. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From maurizio.cimadamore at oracle.com Mon Jan 28 11:02:26 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 28 Jan 2019 11:02:26 +0000 Subject: [foreign] RFR 8217784: Ignore anonymous bitfields so they are handled as padding In-Reply-To: <5C4ECE58.5080900@oracle.com> References: <951b99d4c7d7e7667b6a192952f87e26@xs4all.nl> <01cc6681-1c2e-0e4f-e23d-af7b7713ff55@oracle.com> <35c8f6bcec5732a2f5e54256c441bc53@xs4all.nl> <2ab9e217-529c-cc33-efea-14a09fadcb8e@oracle.com> <5C4ECE58.5080900@oracle.com> Message-ID: <96615ed1-f39d-0e4e-d110-ad1e5599a427@oracle.com> On 28/01/2019 09:41, Sundararajan Athijegannathan wrote: > Looks good. Mac build/test fine. > > Minor: why is the specific imports in bitfields.java are replaced with > import *. Your IDE? That happens with me too with IntelliJ - you need to tweak import setting and tell the IDE to use star import when you have a ridiculously high number of inner classes (e.g. 9999). :-) Maurizio > > -Sundar > > On 26/01/19, 2:56 AM, Maurizio Cimadamore wrote: >> Looks good by me - I suggest you also wait for Sundar's opinion, >> since he's fresh with this code >> >> Maurizio >> >> On 25/01/2019 21:24, Maurizio Cimadamore wrote: >>> The patch is functionally good, I just wonder if it's too magic - >>> e.g. wouldn't be better if the record computer processed the >>> anonymous bitfield (so that the correct padding can be added) >>> instead of relying on the right thing to happen (which will be >>> fragile if e.g. the impl changes a bit) ? >>> >>> Maurizio >>> >>> On 25/01/2019 19:18, Jorn Vernee wrote: >>>> Good catch. I kinda glossed over unions since I thought there would >>>> be no point in having anonymous bitfields in a union, since there >>>> is no need for padding - but on second thought, these could be used >>>> to manually widen the type. Any ways, the compiler seems to allow >>>> anonymous bitfields in unions, so it's good to support that as well. >>>> >>>> Updated webrev: >>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217784/webrev.01/ >>>> >>>> Jorn >>>> >>>> Maurizio Cimadamore schreef op 2019-01-25 19:25: >>>>> Looks good - but what happens with unions? I'm worried we'd get a >>>>> similar failure there... >>>>> >>>>> Maurizio >>>>> >>>>> On 25/01/2019 13:52, Jorn Vernee wrote: >>>>>> Hi, >>>>>> >>>>>> From the bug description: >>>>>> >>>>>> Jextract fails on the following example: >>>>>> >>>>>> ``` >>>>>> struct Foo { >>>>>> ??? unsigned int x: 1; >>>>>> ??? int :7; >>>>>> ??? unsigned int y: 8; >>>>>> ??? int :16; >>>>>> ??? int z; >>>>>> }; >>>>>> ``` >>>>>> >>>>>> The current code tries to look up the second field, but because >>>>>> it has no name this throws an illegal field name error. >>>>>> >>>>>> --- >>>>>> >>>>>> I've fixed this by ignoring anonymous bitfields, since the >>>>>> StructLayoutComputer inserts padding automatically before the >>>>>> next field. >>>>>> >>>>>> Please review the following. >>>>>> >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8217784 >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217784/webrev.00/ >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Jorn From jbvernee at xs4all.nl Mon Jan 28 11:52:05 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Mon, 28 Jan 2019 12:52:05 +0100 Subject: [foreign] RFR 8217784: Ignore anonymous bitfields so they are handled as padding In-Reply-To: <96615ed1-f39d-0e4e-d110-ad1e5599a427@oracle.com> References: <951b99d4c7d7e7667b6a192952f87e26@xs4all.nl> <01cc6681-1c2e-0e4f-e23d-af7b7713ff55@oracle.com> <35c8f6bcec5732a2f5e54256c441bc53@xs4all.nl> <2ab9e217-529c-cc33-efea-14a09fadcb8e@oracle.com> <5C4ECE58.5080900@oracle.com> <96615ed1-f39d-0e4e-d110-ad1e5599a427@oracle.com> Message-ID: Yes, it's my IDE. I'm also using IntelliJ, it does that automatically when you have 5 or more imports from the same package. I will fix it before pushing. Jorn Maurizio Cimadamore schreef op 2019-01-28 12:02: > On 28/01/2019 09:41, Sundararajan Athijegannathan wrote: >> Looks good. Mac build/test fine. >> >> Minor: why is the specific imports in bitfields.java are replaced with >> import *. Your IDE? > > That happens with me too with IntelliJ - you need to tweak import > setting and tell the IDE to use star import when you have a > ridiculously high number of inner classes (e.g. 9999). :-) > > Maurizio > >> >> -Sundar >> >> On 26/01/19, 2:56 AM, Maurizio Cimadamore wrote: >>> Looks good by me - I suggest you also wait for Sundar's opinion, >>> since he's fresh with this code >>> >>> Maurizio >>> >>> On 25/01/2019 21:24, Maurizio Cimadamore wrote: >>>> The patch is functionally good, I just wonder if it's too magic - >>>> e.g. wouldn't be better if the record computer processed the >>>> anonymous bitfield (so that the correct padding can be added) >>>> instead of relying on the right thing to happen (which will be >>>> fragile if e.g. the impl changes a bit) ? >>>> >>>> Maurizio >>>> >>>> On 25/01/2019 19:18, Jorn Vernee wrote: >>>>> Good catch. I kinda glossed over unions since I thought there would >>>>> be no point in having anonymous bitfields in a union, since there >>>>> is no need for padding - but on second thought, these could be used >>>>> to manually widen the type. Any ways, the compiler seems to allow >>>>> anonymous bitfields in unions, so it's good to support that as >>>>> well. >>>>> >>>>> Updated webrev: >>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217784/webrev.01/ >>>>> >>>>> Jorn >>>>> >>>>> Maurizio Cimadamore schreef op 2019-01-25 19:25: >>>>>> Looks good - but what happens with unions? I'm worried we'd get a >>>>>> similar failure there... >>>>>> >>>>>> Maurizio >>>>>> >>>>>> On 25/01/2019 13:52, Jorn Vernee wrote: >>>>>>> Hi, >>>>>>> >>>>>>> From the bug description: >>>>>>> >>>>>>> Jextract fails on the following example: >>>>>>> >>>>>>> ``` >>>>>>> struct Foo { >>>>>>> ??? unsigned int x: 1; >>>>>>> ??? int :7; >>>>>>> ??? unsigned int y: 8; >>>>>>> ??? int :16; >>>>>>> ??? int z; >>>>>>> }; >>>>>>> ``` >>>>>>> >>>>>>> The current code tries to look up the second field, but because >>>>>>> it has no name this throws an illegal field name error. >>>>>>> >>>>>>> --- >>>>>>> >>>>>>> I've fixed this by ignoring anonymous bitfields, since the >>>>>>> StructLayoutComputer inserts padding automatically before the >>>>>>> next field. >>>>>>> >>>>>>> Please review the following. >>>>>>> >>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8217784 >>>>>>> Webrev: >>>>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217784/webrev.00/ >>>>>>> Thanks, >>>>>>> Jorn From maurizio.cimadamore at oracle.com Mon Jan 28 12:00:28 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Mon, 28 Jan 2019 12:00:28 +0000 Subject: hg: panama/dev: Automatic merge with foreign Message-ID: <201901281200.x0SC0SDw007541@aojmv0008.oracle.com> Changeset: 0a3b7f8bbaf9 Author: mcimadamore Date: 2019-01-28 13:00 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/0a3b7f8bbaf9 Automatic merge with foreign From jbvernee at xs4all.nl Mon Jan 28 11:58:37 2019 From: jbvernee at xs4all.nl (jbvernee at xs4all.nl) Date: Mon, 28 Jan 2019 11:58:37 +0000 Subject: hg: panama/dev: 8217784: jextract fails on anonymous bitfields used as manual padding Message-ID: <201901281158.x0SBwb9P006354@aojmv0008.oracle.com> Changeset: c1cba2114615 Author: jvernee Date: 2019-01-28 12:58 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/c1cba2114615 8217784: jextract fails on anonymous bitfields used as manual padding Summary: Ignore anonymous bitfields so they are handled as padding Reviewed-by: mcimadamore, sundar ! src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/RecordLayoutComputer.java ! src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/UnionLayoutComputer.java ! test/jdk/com/sun/tools/jextract/bitfields.h ! test/jdk/com/sun/tools/jextract/compare/bitfields.java ! test/jdk/com/sun/tools/jextract/compare/windows/bitfields.java From jbvernee at xs4all.nl Mon Jan 28 13:06:21 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Mon, 28 Jan 2019 14:06:21 +0100 Subject: [foreign] RFR: Fix UndefinedLayoutException exception message Message-ID: <53d45f14d8e7d7277fec7f6aa89320cb@xs4all.nl> Hi, JDK-8217245 changed the syntax for Unresolved layouts from `$(Name)` to `${Name}`. This also means Unresolved now has a dedicated `layoutExpression` field instead of relying on the name annotation. LayoutResolver.UndefinedLayoutException is still using the name annotation in the exception message, which is now incorrect. This small patch fixes that, and changes it to use Unresolved::layoutExpression(). Please review. Webrev: http://cr.openjdk.java.net/~jvernee/panama/webrevs/fixmessage/webrev.00/ Cheers, Jorn From sundararajan.athijegannathan at oracle.com Mon Jan 28 13:12:25 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Mon, 28 Jan 2019 18:42:25 +0530 Subject: [foreign] RFR: Fix UndefinedLayoutException exception message In-Reply-To: <53d45f14d8e7d7277fec7f6aa89320cb@xs4all.nl> References: <53d45f14d8e7d7277fec7f6aa89320cb@xs4all.nl> Message-ID: <5C4EFFB9.2020509@oracle.com> Is it possible to add a test with invalid (unresolvable) name reference in a layout descriptor and check the exception message? -Sundar On 28/01/19, 6:36 PM, Jorn Vernee wrote: > Hi, > > JDK-8217245 changed the syntax for Unresolved layouts from `$(Name)` > to `${Name}`. This also means Unresolved now has a dedicated > `layoutExpression` field instead of relying on the name annotation. > > LayoutResolver.UndefinedLayoutException is still using the name > annotation in the exception message, which is now incorrect. This > small patch fixes that, and changes it to use > Unresolved::layoutExpression(). > > Please review. > > Webrev: > http://cr.openjdk.java.net/~jvernee/panama/webrevs/fixmessage/webrev.00/ > > Cheers, > Jorn From jbvernee at xs4all.nl Mon Jan 28 13:16:41 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Mon, 28 Jan 2019 14:16:41 +0100 Subject: [foreign] RFR: Fix UndefinedLayoutException exception message In-Reply-To: <5C4EFFB9.2020509@oracle.com> References: <53d45f14d8e7d7277fec7f6aa89320cb@xs4all.nl> <5C4EFFB9.2020509@oracle.com> Message-ID: Yes, I will do that. (wasn't sure if it was useful enough). Jorn Sundararajan Athijegannathan schreef op 2019-01-28 14:12: > Is it possible to add a test with invalid (unresolvable) name > reference in a layout descriptor and check the exception message? > > -Sundar > > On 28/01/19, 6:36 PM, Jorn Vernee wrote: >> Hi, >> >> JDK-8217245 changed the syntax for Unresolved layouts from `$(Name)` >> to `${Name}`. This also means Unresolved now has a dedicated >> `layoutExpression` field instead of relying on the name annotation. >> >> LayoutResolver.UndefinedLayoutException is still using the name >> annotation in the exception message, which is now incorrect. This >> small patch fixes that, and changes it to use >> Unresolved::layoutExpression(). >> >> Please review. >> >> Webrev: >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/fixmessage/webrev.00/ >> >> Cheers, >> Jorn From jbvernee at xs4all.nl Mon Jan 28 14:52:33 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Mon, 28 Jan 2019 15:52:33 +0100 Subject: [foreign] RFR: Fix UndefinedLayoutException exception message In-Reply-To: References: <53d45f14d8e7d7277fec7f6aa89320cb@xs4all.nl> <5C4EFFB9.2020509@oracle.com> Message-ID: Ok, it took a while to find a good test case. I was seeing the exception due to a cross-header layout reference, so that's also the test case I added. It seems that that is currently the only realistic use-case where this exception is being thrown. I believe not being able to resolve cross-header layout references is a known issue as well? Updated webrev: http://cr.openjdk.java.net/~jvernee/panama/webrevs/fixmessage/webrev.01/ Jorn Jorn Vernee schreef op 2019-01-28 14:16: > Yes, I will do that. (wasn't sure if it was useful enough). > > Jorn > > Sundararajan Athijegannathan schreef op 2019-01-28 14:12: >> Is it possible to add a test with invalid (unresolvable) name >> reference in a layout descriptor and check the exception message? >> >> -Sundar >> >> On 28/01/19, 6:36 PM, Jorn Vernee wrote: >>> Hi, >>> >>> JDK-8217245 changed the syntax for Unresolved layouts from `$(Name)` >>> to `${Name}`. This also means Unresolved now has a dedicated >>> `layoutExpression` field instead of relying on the name annotation. >>> >>> LayoutResolver.UndefinedLayoutException is still using the name >>> annotation in the exception message, which is now incorrect. This >>> small patch fixes that, and changes it to use >>> Unresolved::layoutExpression(). >>> >>> Please review. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/fixmessage/webrev.00/ >>> >>> Cheers, >>> Jorn From brian.goetz at oracle.com Mon Jan 28 22:41:39 2019 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 28 Jan 2019 17:41:39 -0500 Subject: [vector] Vector API -- alignment with value types In-Reply-To: References: Message-ID: <39773340-9352-8608-c9a6-aa8be7d468cf@oracle.com> On 1/25/2019 7:14 PM, Vladimir Ivanov wrote: >> Part I >> ------ >> >> Here's an idea for simplifying Species, which is: let's drive Species >> down to be a simple class that is really just a constant holder for >> (element type, shape), and move all the behavior to static methods on >> XxxVector.? (At that point, Species can just be an enum, or a group >> of enums with a common interface parent, if we like.) > > Leaving parameterized variants on Species and introducing default ones > (as static methods) looks more attractive to me. To be clear -- the end result of this is that there is almost nothing left in Species _at all_.? I think the Species abstraction, from an API perspective -- is over-engineered; it doesn't serve the _user_ very much, and you (currently) can't understand this API before you understand species. If Species could take a back seat, being a simple value-wrapper for (type, lanes) -- I think that would be a worthwhile result. There would probably be some short-term regression in inlining/intrinsification, but I suspect that would be a short-term speed bump? From brian.goetz at oracle.com Mon Jan 28 23:38:25 2019 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 28 Jan 2019 18:38:25 -0500 Subject: [Vector] RFR: reshape, resize, rebracket, cast In-Reply-To: References: Message-ID: The issue you raise is a minor frustration, but overall I think the direction of movement -- from species to vector -- is the right one.? We can work on ironing out these small issues later. On 1/16/2019 3:20 AM, Kharbas, Kishor wrote: > > Hi, > > I have a patch which refactors the above methods as discussed in this > thread - > https://mail.openjdk.java.net/pipermail/panama-dev/2018-December/003365.html. > > Please review the changes at - > http://cr.openjdk.java.net/~kkharbas/vector-api/webrev-reshape_etc > > Generation of intrinsics and correct vector boxing elimination have > been verified. In the next patch I will add changes to jtreg tests. > > I wanted to bring forth one small issue(possibly) which programmers > might face with this change ? > > We provide specialized types like IntVector, FloatVector, etc for > users to define their vectors and they would be able to write code > like this with previous methods, > > /FloatVector float256 = SPECIES_FLOAT256.cast(int256);/ > > Here FloatSpecies would always return FloatVector. > > However with this change, since cast() is defined on a vector and > takes species of a generic type, it cannot return a specialized Vector > like IntVector or FloatVector. User has to explicitly cast the return > vector from Vector to specialized Vector or use a generic vector of > corresponding element type. For example, > > /? ??FloatVector float256 = (FloatVector)int256.cast(SPECIES_FLOAT256);/ > > /? or/ > > /? ??Vector float256 = int256.cast(SPECIES_FLOAT256);/ > > I am not sure if this is even a problem, but I thought its worth > mentioning. > > Thanks > > Kishor > From Yang.Zhang at arm.com Tue Jan 29 09:33:48 2019 From: Yang.Zhang at arm.com (Yang Zhang (Arm Technology China)) Date: Tue, 29 Jan 2019 09:33:48 +0000 Subject: [vector api] RFR: Implement Vector API andAll/orAll/xorAll for AArch64 NEON In-Reply-To: <838a6bca-8f67-eca5-f223-c1a5f377541a@redhat.com> References: <838a6bca-8f67-eca5-f223-c1a5f377541a@redhat.com> Message-ID: Hi Andrew Thanks very much for your suggestion. aarch64_ad.m4 will make the code easy to maintain and review. I have updated the patch. Could you please help to review it again? http://cr.openjdk.java.net/~yzhang/vectorapi.andall/webrev.01/ Regards Yang -----Original Message----- From: Andrew Haley Sent: Monday, January 28, 2019 6:13 PM To: Yang Zhang (Arm Technology China) ; 'panama-dev at openjdk.java.net' Subject: Re: [vector api] RFR: Implement Vector API andAll/orAll/xorAll for AArch64 NEON On 1/28/19 2:44 AM, Yang Zhang (Arm Technology China) wrote: > I have a patch which implements Vector API andAll/orAll/xorAll for AArch64 NEON. Could you please help to review it? > http://cr.openjdk.java.net/~yzhang/vectorapi.andall/webrev.00/ OK, but: You're doing a great deal of repetitive stuff in here. This makes the patch hard to maintain and review. Instead, I suggest that you have a look at aarch64_ad.m4 to see how it's possible to generate all of these patterns from a script: 487 lines of that script generates 1970 lines of patterns. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From maurizio.cimadamore at oracle.com Tue Jan 29 14:38:09 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 29 Jan 2019 14:38:09 +0000 Subject: [foreign] RFR 8217989: Consolidate BoundedPointer implementation Message-ID: <2b353835-52ea-f6f7-e449-77f71f64bfc3@oracle.com> Hi, in the light of recent discussions re. access modes, I decided to go ahead and cleanup the split between BoundedPointer and BoundedMemoryRegion, to avoid duplication of functionalities. This patch does two things: * moves Scope, AccessMode on BoundedPointer (where they belong, IMHO) * reduce BoundedMemoryRegion to a simple helper class used to check pointer bounds (and renames it to MemoryBoundInfo) I also took care of minor cleanups, such as adding better encapsulation for pointer/array implementation fields. Webrev: http://cr.openjdk.java.net/~mcimadamore/panama/8217989/ Maurizio From maurizio.cimadamore at oracle.com Tue Jan 29 14:59:46 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 29 Jan 2019 14:59:46 +0000 Subject: [foreign] RFR: Fix UndefinedLayoutException exception message In-Reply-To: References: <53d45f14d8e7d7277fec7f6aa89320cb@xs4all.nl> <5C4EFFB9.2020509@oracle.com> Message-ID: Resolving references across interfaces is not expected to work reliably. This is the primary reason which sends us towards an extraction model that is library-centric, rather than header-centric, so that all references in a library should be self-contained. As for your test, it seems like you can get the same exception simply by having a single struct interface whose layout has some dandling unresolved layout (e.g. an unresolved layout whose layout expression is not defined anywhere)? Maurizio On 28/01/2019 14:52, Jorn Vernee wrote: > Ok, it took a while to find a good test case. I was seeing the > exception due to a cross-header layout reference, so that's also the > test case I added. It seems that that is currently the only realistic > use-case where this exception is being thrown. I believe not being > able to resolve cross-header layout references is a known issue as well? > > Updated webrev: > http://cr.openjdk.java.net/~jvernee/panama/webrevs/fixmessage/webrev.01/ > > Jorn > > Jorn Vernee schreef op 2019-01-28 14:16: >> Yes, I will do that. (wasn't sure if it was useful enough). >> >> Jorn >> >> Sundararajan Athijegannathan schreef op 2019-01-28 14:12: >>> Is it possible to add a test with invalid (unresolvable) name >>> reference in a layout descriptor and check the exception message? >>> >>> -Sundar >>> >>> On 28/01/19, 6:36 PM, Jorn Vernee wrote: >>>> Hi, >>>> >>>> JDK-8217245 changed the syntax for Unresolved layouts from >>>> `$(Name)` to `${Name}`. This also means Unresolved now has a >>>> dedicated `layoutExpression` field instead of relying on the name >>>> annotation. >>>> >>>> LayoutResolver.UndefinedLayoutException is still using the name >>>> annotation in the exception message, which is now incorrect. This >>>> small patch fixes that, and changes it to use >>>> Unresolved::layoutExpression(). >>>> >>>> Please review. >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/fixmessage/webrev.00/ >>>> >>>> >>>> Cheers, >>>> Jorn From jbvernee at xs4all.nl Tue Jan 29 15:19:46 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Tue, 29 Jan 2019 16:19:46 +0100 Subject: [foreign] RFR: Fix UndefinedLayoutException exception message In-Reply-To: References: <53d45f14d8e7d7277fec7f6aa89320cb@xs4all.nl> <5C4EFFB9.2020509@oracle.com> Message-ID: Maurizio Cimadamore schreef op 2019-01-29 15:59: > Resolving references across interfaces is not expected to work > reliably. This is the primary reason which sends us towards an > extraction model that is library-centric, rather than header-centric, > so that all references in a library should be self-contained. I've been using a mitigating patch that also scans methods of a Struct implementation in LayoutResolver when scanning a struct. Under the assumption that a type's layout is defined on that type itself. > As for your test, it seems like you can get the same exception simply > by having a single struct interface whose layout has some dandling > unresolved layout (e.g. an unresolved layout whose layout expression > is not defined anywhere)? The tricky part is triggering the resolution of the layout through the public API. Trying to allocate a struct will not resolve the struct's layout. The only place I found that does this is either the test case I've used, where the dangling layout is in a @NativeFunction descriptor, or in StructImplGenerator's constructor, which is called when de referencing a pointer to a struct. Of course, I could also just use `LayoutResolver.get(Object.class).resolve(Layout.of("${Undefined}"));`. Would that be good enough? Jorn > Maurizio > > On 28/01/2019 14:52, Jorn Vernee wrote: >> Ok, it took a while to find a good test case. I was seeing the >> exception due to a cross-header layout reference, so that's also the >> test case I added. It seems that that is currently the only realistic >> use-case where this exception is being thrown. I believe not being >> able to resolve cross-header layout references is a known issue as >> well? >> >> Updated webrev: >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/fixmessage/webrev.01/ >> >> Jorn >> >> Jorn Vernee schreef op 2019-01-28 14:16: >>> Yes, I will do that. (wasn't sure if it was useful enough). >>> >>> Jorn >>> >>> Sundararajan Athijegannathan schreef op 2019-01-28 14:12: >>>> Is it possible to add a test with invalid (unresolvable) name >>>> reference in a layout descriptor and check the exception message? >>>> >>>> -Sundar >>>> >>>> On 28/01/19, 6:36 PM, Jorn Vernee wrote: >>>>> Hi, >>>>> >>>>> JDK-8217245 changed the syntax for Unresolved layouts from >>>>> `$(Name)` to `${Name}`. This also means Unresolved now has a >>>>> dedicated `layoutExpression` field instead of relying on the name >>>>> annotation. >>>>> >>>>> LayoutResolver.UndefinedLayoutException is still using the name >>>>> annotation in the exception message, which is now incorrect. This >>>>> small patch fixes that, and changes it to use >>>>> Unresolved::layoutExpression(). >>>>> >>>>> Please review. >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/fixmessage/webrev.00/ >>>>> Cheers, >>>>> Jorn From maurizio.cimadamore at oracle.com Tue Jan 29 15:46:47 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 29 Jan 2019 15:46:47 +0000 Subject: [foreign] RFR: Fix UndefinedLayoutException exception message In-Reply-To: References: <53d45f14d8e7d7277fec7f6aa89320cb@xs4all.nl> <5C4EFFB9.2020509@oracle.com> Message-ID: <2e50c58e-7655-4198-4a07-8c95025136c8@oracle.com> On 29/01/2019 15:19, Jorn Vernee wrote: > Maurizio Cimadamore schreef op 2019-01-29 15:59: >> Resolving references across interfaces is not expected to work >> reliably. This is the primary reason which sends us towards an >> extraction model that is library-centric, rather than header-centric, >> so that all references in a library should be self-contained. > > I've been using a mitigating patch that also scans methods of a Struct > implementation in LayoutResolver when scanning a struct. Under the > assumption that a type's layout is defined on that type itself. > >> As for your test, it seems like you can get the same exception simply >> by having a single struct interface whose layout has some dandling >> unresolved layout (e.g. an unresolved layout whose layout expression >> is not defined anywhere)? > > The tricky part is triggering the resolution of the layout through the > public API. Trying to allocate a struct will not resolve the struct's > layout. The only place I found that does this is either the test case > I've used, where the dangling layout is in a @NativeFunction > descriptor, or in StructImplGenerator's constructor, which is called > when de referencing a pointer to a struct. So, I guess what I'm saying is: @NativeStruct("[ ${foo}(foo) ]") interface BadStruct extends Struct { ??? @NativeGetter("foo") ??? Foo foo(); ??? interface Foo { } } Shouldn't this fail when trying to generate the struct implementation? (e.g. Scope.allocateStruct(BadStruct.class)) Maurizio > > Of course, I could also just use > `LayoutResolver.get(Object.class).resolve(Layout.of("${Undefined}"));`. > Would that be good enough? > > Jorn > >> Maurizio >> >> On 28/01/2019 14:52, Jorn Vernee wrote: >>> Ok, it took a while to find a good test case. I was seeing the >>> exception due to a cross-header layout reference, so that's also the >>> test case I added. It seems that that is currently the only >>> realistic use-case where this exception is being thrown. I believe >>> not being able to resolve cross-header layout references is a known >>> issue as well? >>> >>> Updated webrev: >>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/fixmessage/webrev.01/ >>> >>> >>> Jorn >>> >>> Jorn Vernee schreef op 2019-01-28 14:16: >>>> Yes, I will do that. (wasn't sure if it was useful enough). >>>> >>>> Jorn >>>> >>>> Sundararajan Athijegannathan schreef op 2019-01-28 14:12: >>>>> Is it possible to add a test with invalid (unresolvable) name >>>>> reference in a layout descriptor and check the exception message? >>>>> >>>>> -Sundar >>>>> >>>>> On 28/01/19, 6:36 PM, Jorn Vernee wrote: >>>>>> Hi, >>>>>> >>>>>> JDK-8217245 changed the syntax for Unresolved layouts from >>>>>> `$(Name)` to `${Name}`. This also means Unresolved now has a >>>>>> dedicated `layoutExpression` field instead of relying on the name >>>>>> annotation. >>>>>> >>>>>> LayoutResolver.UndefinedLayoutException is still using the name >>>>>> annotation in the exception message, which is now incorrect. This >>>>>> small patch fixes that, and changes it to use >>>>>> Unresolved::layoutExpression(). >>>>>> >>>>>> Please review. >>>>>> >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/fixmessage/webrev.00/ >>>>>> Cheers, >>>>>> Jorn From jbvernee at xs4all.nl Tue Jan 29 15:57:51 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Tue, 29 Jan 2019 16:57:51 +0100 Subject: [foreign] RFR: Fix UndefinedLayoutException exception message In-Reply-To: <2e50c58e-7655-4198-4a07-8c95025136c8@oracle.com> References: <53d45f14d8e7d7277fec7f6aa89320cb@xs4all.nl> <5C4EFFB9.2020509@oracle.com> <2e50c58e-7655-4198-4a07-8c95025136c8@oracle.com> Message-ID: Maurizio Cimadamore schreef op 2019-01-29 16:46: > On 29/01/2019 15:19, Jorn Vernee wrote: >> Maurizio Cimadamore schreef op 2019-01-29 15:59: >>> Resolving references across interfaces is not expected to work >>> reliably. This is the primary reason which sends us towards an >>> extraction model that is library-centric, rather than header-centric, >>> so that all references in a library should be self-contained. >> >> I've been using a mitigating patch that also scans methods of a Struct >> implementation in LayoutResolver when scanning a struct. Under the >> assumption that a type's layout is defined on that type itself. >> >>> As for your test, it seems like you can get the same exception simply >>> by having a single struct interface whose layout has some dandling >>> unresolved layout (e.g. an unresolved layout whose layout expression >>> is not defined anywhere)? >> >> The tricky part is triggering the resolution of the layout through the >> public API. Trying to allocate a struct will not resolve the struct's >> layout. The only place I found that does this is either the test case >> I've used, where the dangling layout is in a @NativeFunction >> descriptor, or in StructImplGenerator's constructor, which is called >> when de referencing a pointer to a struct. > > So, I guess what I'm saying is: > > @NativeStruct("[ ${foo}(foo) ]") > interface BadStruct extends Struct { > ??? @NativeGetter("foo") > ??? Foo foo(); > > ??? interface Foo { } > } > > Shouldn't this fail when trying to generate the struct implementation? > (e.g. Scope.allocateStruct(BadStruct.class)) No, that will fail with "UnsupportedOperationException: bitsSize on Unresolved" Scope.allocateStruct(BadStruct.class) will call allocateInternal first which throws that UOE, only after that would the implementation be generated when calling get() on the returned pointer; ``` private BoundedPointer allocateInternal(LayoutType type, long count, int align) { ... long size = Util.alignUp(type.bytesSize(), align); // !!! calling bitSize() on Unresolved throws UOE ... } @Override public > T allocateStruct(Class clazz) { // FIXME: is this alignment needed? return allocate(LayoutType.ofStruct(clazz), 8).get(); // call to 'get()' triggers struct generation, and resolution of struct's layout. } ``` Jorn > Maurizio > >> >> Of course, I could also just use >> `LayoutResolver.get(Object.class).resolve(Layout.of("${Undefined}"));`. >> Would that be good enough? >> >> Jorn >> >>> Maurizio >>> >>> On 28/01/2019 14:52, Jorn Vernee wrote: >>>> Ok, it took a while to find a good test case. I was seeing the >>>> exception due to a cross-header layout reference, so that's also the >>>> test case I added. It seems that that is currently the only >>>> realistic use-case where this exception is being thrown. I believe >>>> not being able to resolve cross-header layout references is a known >>>> issue as well? >>>> >>>> Updated webrev: >>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/fixmessage/webrev.01/ >>>> Jorn >>>> >>>> Jorn Vernee schreef op 2019-01-28 14:16: >>>>> Yes, I will do that. (wasn't sure if it was useful enough). >>>>> >>>>> Jorn >>>>> >>>>> Sundararajan Athijegannathan schreef op 2019-01-28 14:12: >>>>>> Is it possible to add a test with invalid (unresolvable) name >>>>>> reference in a layout descriptor and check the exception message? >>>>>> >>>>>> -Sundar >>>>>> >>>>>> On 28/01/19, 6:36 PM, Jorn Vernee wrote: >>>>>>> Hi, >>>>>>> >>>>>>> JDK-8217245 changed the syntax for Unresolved layouts from >>>>>>> `$(Name)` to `${Name}`. This also means Unresolved now has a >>>>>>> dedicated `layoutExpression` field instead of relying on the name >>>>>>> annotation. >>>>>>> >>>>>>> LayoutResolver.UndefinedLayoutException is still using the name >>>>>>> annotation in the exception message, which is now incorrect. This >>>>>>> small patch fixes that, and changes it to use >>>>>>> Unresolved::layoutExpression(). >>>>>>> >>>>>>> Please review. >>>>>>> >>>>>>> Webrev: >>>>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/fixmessage/webrev.00/ >>>>>>> Cheers, >>>>>>> Jorn From maurizio.cimadamore at oracle.com Tue Jan 29 16:09:22 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 29 Jan 2019 16:09:22 +0000 Subject: [foreign] RFR: Fix UndefinedLayoutException exception message In-Reply-To: References: <53d45f14d8e7d7277fec7f6aa89320cb@xs4all.nl> <5C4EFFB9.2020509@oracle.com> <2e50c58e-7655-4198-4a07-8c95025136c8@oracle.com> Message-ID: On 29/01/2019 15:57, Jorn Vernee wrote: > Maurizio Cimadamore schreef op 2019-01-29 16:46: >> On 29/01/2019 15:19, Jorn Vernee wrote: >>> Maurizio Cimadamore schreef op 2019-01-29 15:59: >>>> Resolving references across interfaces is not expected to work >>>> reliably. This is the primary reason which sends us towards an >>>> extraction model that is library-centric, rather than header-centric, >>>> so that all references in a library should be self-contained. >>> >>> I've been using a mitigating patch that also scans methods of a >>> Struct implementation in LayoutResolver when scanning a struct. >>> Under the assumption that a type's layout is defined on that type >>> itself. >>> >>>> As for your test, it seems like you can get the same exception simply >>>> by having a single struct interface whose layout has some dandling >>>> unresolved layout (e.g. an unresolved layout whose layout expression >>>> is not defined anywhere)? >>> >>> The tricky part is triggering the resolution of the layout through >>> the public API. Trying to allocate a struct will not resolve the >>> struct's layout. The only place I found that does this is either the >>> test case I've used, where the dangling layout is in a >>> @NativeFunction descriptor, or in StructImplGenerator's constructor, >>> which is called when de referencing a pointer to a struct. >> >> So, I guess what I'm saying is: >> >> @NativeStruct("[ ${foo}(foo) ]") >> interface BadStruct extends Struct { >> ??? @NativeGetter("foo") >> ??? Foo foo(); >> >> ??? interface Foo { } >> } >> >> Shouldn't this fail when trying to generate the struct implementation? >> (e.g. Scope.allocateStruct(BadStruct.class)) > > No, that will fail with "UnsupportedOperationException: bitsSize on > Unresolved" Then try with scope.allocate(LayoutType.ofStruct(BadStruct.class)).get().foo() Maurizio > > Scope.allocateStruct(BadStruct.class) will call allocateInternal first > which throws that UOE, only after that would the implementation be > generated when calling get() on the returned pointer; > > ``` > private BoundedPointer allocateInternal(LayoutType type, > long count, int align) { > ??? ... > ??? long size = Util.alignUp(type.bytesSize(), align); // !!! calling > bitSize() on Unresolved throws UOE > ??? ... > } > > @Override > public > T allocateStruct(Class clazz) { > ??? // FIXME: is this alignment needed? > ??? return allocate(LayoutType.ofStruct(clazz), 8).get(); // call to > 'get()' triggers struct generation, and resolution of struct's layout. > } > ``` > > Jorn > >> Maurizio >> >>> >>> Of course, I could also just use >>> `LayoutResolver.get(Object.class).resolve(Layout.of("${Undefined}"));`. >>> Would that be good enough? >>> >>> Jorn >>> >>>> Maurizio >>>> >>>> On 28/01/2019 14:52, Jorn Vernee wrote: >>>>> Ok, it took a while to find a good test case. I was seeing the >>>>> exception due to a cross-header layout reference, so that's also >>>>> the test case I added. It seems that that is currently the only >>>>> realistic use-case where this exception is being thrown. I believe >>>>> not being able to resolve cross-header layout references is a >>>>> known issue as well? >>>>> >>>>> Updated webrev: >>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/fixmessage/webrev.01/ >>>>> Jorn >>>>> >>>>> Jorn Vernee schreef op 2019-01-28 14:16: >>>>>> Yes, I will do that. (wasn't sure if it was useful enough). >>>>>> >>>>>> Jorn >>>>>> >>>>>> Sundararajan Athijegannathan schreef op 2019-01-28 14:12: >>>>>>> Is it possible to add a test with invalid (unresolvable) name >>>>>>> reference in a layout descriptor and check the exception message? >>>>>>> >>>>>>> -Sundar >>>>>>> >>>>>>> On 28/01/19, 6:36 PM, Jorn Vernee wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> JDK-8217245 changed the syntax for Unresolved layouts from >>>>>>>> `$(Name)` to `${Name}`. This also means Unresolved now has a >>>>>>>> dedicated `layoutExpression` field instead of relying on the >>>>>>>> name annotation. >>>>>>>> >>>>>>>> LayoutResolver.UndefinedLayoutException is still using the name >>>>>>>> annotation in the exception message, which is now incorrect. >>>>>>>> This small patch fixes that, and changes it to use >>>>>>>> Unresolved::layoutExpression(). >>>>>>>> >>>>>>>> Please review. >>>>>>>> >>>>>>>> Webrev: >>>>>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/fixmessage/webrev.00/ >>>>>>>> Cheers, >>>>>>>> Jorn From jbvernee at xs4all.nl Tue Jan 29 17:41:39 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Tue, 29 Jan 2019 18:41:39 +0100 Subject: [foreign] RFR: Fix UndefinedLayoutException exception message In-Reply-To: References: <53d45f14d8e7d7277fec7f6aa89320cb@xs4all.nl> <5C4EFFB9.2020509@oracle.com> <2e50c58e-7655-4198-4a07-8c95025136c8@oracle.com> Message-ID: <1fe1d64ca9fa8f4f65851ecaaf1288fc@xs4all.nl> Maurizio Cimadamore schreef op 2019-01-29 17:09: > On 29/01/2019 15:57, Jorn Vernee wrote: >> Maurizio Cimadamore schreef op 2019-01-29 16:46: >>> On 29/01/2019 15:19, Jorn Vernee wrote: >>>> Maurizio Cimadamore schreef op 2019-01-29 15:59: >>>>> Resolving references across interfaces is not expected to work >>>>> reliably. This is the primary reason which sends us towards an >>>>> extraction model that is library-centric, rather than >>>>> header-centric, >>>>> so that all references in a library should be self-contained. >>>> >>>> I've been using a mitigating patch that also scans methods of a >>>> Struct implementation in LayoutResolver when scanning a struct. >>>> Under the assumption that a type's layout is defined on that type >>>> itself. >>>> >>>>> As for your test, it seems like you can get the same exception >>>>> simply >>>>> by having a single struct interface whose layout has some dandling >>>>> unresolved layout (e.g. an unresolved layout whose layout >>>>> expression >>>>> is not defined anywhere)? >>>> >>>> The tricky part is triggering the resolution of the layout through >>>> the public API. Trying to allocate a struct will not resolve the >>>> struct's layout. The only place I found that does this is either the >>>> test case I've used, where the dangling layout is in a >>>> @NativeFunction descriptor, or in StructImplGenerator's constructor, >>>> which is called when de referencing a pointer to a struct. >>> >>> So, I guess what I'm saying is: >>> >>> @NativeStruct("[ ${foo}(foo) ]") >>> interface BadStruct extends Struct { >>> ??? @NativeGetter("foo") >>> ??? Foo foo(); >>> >>> ??? interface Foo { } >>> } >>> >>> Shouldn't this fail when trying to generate the struct >>> implementation? >>> (e.g. Scope.allocateStruct(BadStruct.class)) >> >> No, that will fail with "UnsupportedOperationException: bitsSize on >> Unresolved" > > Then try with > > scope.allocate(LayoutType.ofStruct(BadStruct.class)).get().foo() Same there, that also calls allocateInternal. Another way to trigger the exception could be something like; ``` try (var scope = Scope.newNativeScope()) { Pointer ptr = scope.allocate(NativeTypes.VOID); ptr.cast(LayoutType.ofStruct(BasStruct.class)).get(); // trigger resolution } ``` But that didn't seem like a realistic enough test. So, I went with the case in which I was seeing the exception; which is a cross-header layout reference. Jorn > Maurizio > >> >> Scope.allocateStruct(BadStruct.class) will call allocateInternal first >> which throws that UOE, only after that would the implementation be >> generated when calling get() on the returned pointer; >> >> ``` >> private BoundedPointer allocateInternal(LayoutType type, >> long count, int align) { >> ??? ... >> ??? long size = Util.alignUp(type.bytesSize(), align); // !!! calling >> bitSize() on Unresolved throws UOE >> ??? ... >> } >> >> @Override >> public > T allocateStruct(Class clazz) { >> ??? // FIXME: is this alignment needed? >> ??? return allocate(LayoutType.ofStruct(clazz), 8).get(); // call to >> 'get()' triggers struct generation, and resolution of struct's layout. >> } >> ``` >> >> Jorn >> >>> Maurizio >>> >>>> >>>> Of course, I could also just use >>>> `LayoutResolver.get(Object.class).resolve(Layout.of("${Undefined}"));`. >>>> Would that be good enough? >>>> >>>> Jorn >>>> >>>>> Maurizio >>>>> >>>>> On 28/01/2019 14:52, Jorn Vernee wrote: >>>>>> Ok, it took a while to find a good test case. I was seeing the >>>>>> exception due to a cross-header layout reference, so that's also >>>>>> the test case I added. It seems that that is currently the only >>>>>> realistic use-case where this exception is being thrown. I believe >>>>>> not being able to resolve cross-header layout references is a >>>>>> known issue as well? >>>>>> >>>>>> Updated webrev: >>>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/fixmessage/webrev.01/ >>>>>> Jorn >>>>>> >>>>>> Jorn Vernee schreef op 2019-01-28 14:16: >>>>>>> Yes, I will do that. (wasn't sure if it was useful enough). >>>>>>> >>>>>>> Jorn >>>>>>> >>>>>>> Sundararajan Athijegannathan schreef op 2019-01-28 14:12: >>>>>>>> Is it possible to add a test with invalid (unresolvable) name >>>>>>>> reference in a layout descriptor and check the exception >>>>>>>> message? >>>>>>>> >>>>>>>> -Sundar >>>>>>>> >>>>>>>> On 28/01/19, 6:36 PM, Jorn Vernee wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> JDK-8217245 changed the syntax for Unresolved layouts from >>>>>>>>> `$(Name)` to `${Name}`. This also means Unresolved now has a >>>>>>>>> dedicated `layoutExpression` field instead of relying on the >>>>>>>>> name annotation. >>>>>>>>> >>>>>>>>> LayoutResolver.UndefinedLayoutException is still using the name >>>>>>>>> annotation in the exception message, which is now incorrect. >>>>>>>>> This small patch fixes that, and changes it to use >>>>>>>>> Unresolved::layoutExpression(). >>>>>>>>> >>>>>>>>> Please review. >>>>>>>>> >>>>>>>>> Webrev: >>>>>>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/fixmessage/webrev.00/ >>>>>>>>> Cheers, >>>>>>>>> Jorn From maurizio.cimadamore at oracle.com Tue Jan 29 17:51:23 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 29 Jan 2019 17:51:23 +0000 Subject: [foreign] RFR: Fix UndefinedLayoutException exception message In-Reply-To: <1fe1d64ca9fa8f4f65851ecaaf1288fc@xs4all.nl> References: <53d45f14d8e7d7277fec7f6aa89320cb@xs4all.nl> <5C4EFFB9.2020509@oracle.com> <2e50c58e-7655-4198-4a07-8c95025136c8@oracle.com> <1fe1d64ca9fa8f4f65851ecaaf1288fc@xs4all.nl> Message-ID: On 29/01/2019 17:41, Jorn Vernee wrote: > But that didn't seem like a realistic enough test. So, I went with the > case in which I was seeing the exception; which is a cross-header > layout reference. I see. I guess the enclosing header for MyDep can probably be removed, right? Maurizio From maurizio.cimadamore at oracle.com Tue Jan 29 17:56:13 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 29 Jan 2019 17:56:13 +0000 Subject: [foreign] RFR: Fix UndefinedLayoutException exception message In-Reply-To: References: <53d45f14d8e7d7277fec7f6aa89320cb@xs4all.nl> <5C4EFFB9.2020509@oracle.com> <2e50c58e-7655-4198-4a07-8c95025136c8@oracle.com> Message-ID: > Then try with > > scope.allocate(LayoutType.ofStruct(BadStruct.class)).get().foo() > > Ok, the above also doesn't work - but this should? scope.allocate(LayoutType.ofStruct(BadStruct.class).pointer()).get().get().foo() This time we should be able to bypass Scope.allocateInternal... The main issue about having a cross header ref is that the test is gonna be sensitive to whatever we do in that area - e.g. that failure is, in a way, temporary; it would be preferrable to have a test that really checked something Bad (TM). Maurizio From jbvernee at xs4all.nl Tue Jan 29 19:18:20 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Tue, 29 Jan 2019 20:18:20 +0100 Subject: [foreign] RFR: Fix UndefinedLayoutException exception message In-Reply-To: References: <53d45f14d8e7d7277fec7f6aa89320cb@xs4all.nl> <5C4EFFB9.2020509@oracle.com> <2e50c58e-7655-4198-4a07-8c95025136c8@oracle.com> Message-ID: <45fbc51ee7afa7ad49ecde599a1ee0e6@xs4all.nl> Maurizio Cimadamore schreef op 2019-01-29 18:56: >> Then try with >> >> scope.allocate(LayoutType.ofStruct(BadStruct.class)).get().foo() >> >> > Ok, the above also doesn't work - but this should? > > scope.allocate(LayoutType.ofStruct(BadStruct.class).pointer()).get().get().foo() This also works, but tbh it doesn't seem much different than `scope.allocate(VOID).cast(LayoutType.ofStruct(BadStruct.class)).get()`, since we're dereferencing an invalid pointer in both cases. > > This time we should be able to bypass Scope.allocateInternal... > > The main issue about having a cross header ref is that the test is > gonna be sensitive to whatever we do in that area - e.g. that failure > is, in a way, temporary; it would be preferrable to have a test that > really checked something Bad (TM). FWIW, someone could write their own descriptor for an @NativeFunction and accidentally put in a reference to something that just doesn't exist at all, e.g.: ``` @NativeHeader interface Lib { @NativeFunction("()${Garbage}") int func(); } ``` That seems like a relatively simple but still realistic test case. Updated webrev: http://cr.openjdk.java.net/~jvernee/panama/webrevs/fixmessage/webrev.02/ Jorn > Maurizio From richard at openkappa.co.uk Tue Jan 29 19:37:59 2019 From: richard at openkappa.co.uk (Richard Startin) Date: Tue, 29 Jan 2019 19:37:59 +0000 Subject: Allocation caused by vector rebracketing twice In-Reply-To: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A51F11@FMSMSX126.amr.corp.intel.com> References: , <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A51F11@FMSMSX126.amr.corp.intel.com> Message-ID: Hi Sandhya, Thanks for looking into this. You and Vladimir were correct about the cause. I tried again and this now works well. Thanks, Richard ________________________________ From: Viswanathan, Sandhya Sent: 21 January 2019 19:29:16 To: Richard Startin; panama-dev at openjdk.java.net Subject: RE: Allocation caused by vector rebracketing twice Hi Richard, Thanks a lot for your feedback. I analyzed this further and it looks like the high allocation rate is happening because the Long addAll reduction intrinsic is not in place for AVX < 3 and so boxing is happening to call the Java implementation. I will send out a patch shortly which should fix this. Best Regards, Sandhya -----Original Message----- From: panama-dev [mailto:panama-dev-bounces at openjdk.java.net] On Behalf Of Richard Startin Sent: Saturday, January 19, 2019 11:29 AM To: panama-dev at openjdk.java.net Subject: Allocation caused by vector rebracketing twice It's great to see that there is now a shiftR method because this was the missing link to make vector bit counts possible. With great excitement, I tried this at 54348:a8516a4be714 but it doesn't work very well yet. @Benchmark public int vectorBitCount() { int bitCount = 0; var lookupPos = YMM_BYTE.fromArray(LOOKUP_POS, 0); var lookupNeg = YMM_BYTE.fromArray(LOOKUP_NEG, 0); var lowMask = YMM_BYTE.broadcast((byte)0x0F); for (int i = 0; i < data.length; i+= 4) { var bytes = (ByteVector)YMM_LONG.fromArray(data, i).rebracket(YMM_BYTE); bitCount += (int)((LongVector)lookupPos.rearrange(bytes.and(lowMask).toShuffle()) .add(lookupNeg.rearrange(bytes.shiftR(4).and(lowMask).toShuffle())) .rebracket(YMM_LONG)).addAll(); } return bitCount; } JMH -prof gc shows high allocation rates: Iteration 1: 0.008 ops/us ?gc.alloc.rate: 1112.997 MB/sec ?gc.alloc.rate.norm: 219264.051 B/op ?gc.churn.G1_Eden_Space: 1042.296 MB/sec ?gc.churn.G1_Eden_Space.norm: 205335.753 B/op ?gc.churn.G1_Old_Gen: 0.001 MB/sec ?gc.churn.G1_Old_Gen.norm: 0.258 B/op ?gc.count: 6.000 counts ?gc.time: 6.000 ms Stripping the code down until negligible allocation rates are observed, the smallest reproducer is where the vector is rebracketed and then the reverse rebracket is performed: @Benchmark public int vectorBitCount() { int bitCount = 0; for (int i = 0; i < data.length; i+= 4) { bitCount += (int)((LongVector)YMM_LONG.fromArray(data, i).rebracket(YMM_BYTE).rebracket(YMM_LONG)).addAll(); } return bitCount; } -prof gc: Iteration 1: 0.310 ops/us ?gc.alloc.rate: 3223.539 MB/sec ?gc.alloc.rate.norm: 16384.001 B/op ?gc.churn.G1_Eden_Space: 3084.887 MB/sec ?gc.churn.G1_Eden_Space.norm: 15679.287 B/op ?gc.churn.G1_Old_Gen: 0.005 MB/sec ?gc.churn.G1_Old_Gen.norm: 0.026 B/op ?gc.count: 8.000 counts ?gc.time: 9.000 ms Without reversing the rebracket I see negligible allocation @Benchmark public int vectorBitCount() { int bitCount = 0; for (int i = 0; i < data.length; i+= 4) { bitCount += (int)((ByteVector)YMM_LONG.fromArray(data, i).rebracket(YMM_BYTE)).addAll(); } return bitCount; } Iteration 1: 1.456 ops/us ?gc.alloc.rate: ? 10?? MB/sec ?gc.alloc.rate.norm: ? 10?? B/op ?gc.count: ? 0 counts Thanks, Richard From john.r.rose at oracle.com Tue Jan 29 20:03:13 2019 From: john.r.rose at oracle.com (John Rose) Date: Tue, 29 Jan 2019 12:03:13 -0800 Subject: scopes writeup In-Reply-To: <3e4546e7-f768-6550-3a31-b38dae835d7b@oracle.com> References: <6fbff2eb-b100-0c03-7663-c732a3904302@oracle.com> <3e4546e7-f768-6550-3a31-b38dae835d7b@oracle.com> Message-ID: <6E570AB5-88D3-43E6-B58C-E2297CD6D5CA@oracle.com> On Jan 23, 2019, at 3:27 AM, Maurizio Cimadamore wrote: > > Extreme example: the library could return a pointer that is already dandling (e.g. the library allocate memory with malloc, saves the pointer somewhere, frees the memory, and then returns the pointer). There's no possible scope abstraction that can save you from that. > > So, when it comes to native calls, I lean towards flexibility - you should be able to pass any pointer to the native library, and you should make minimal assumption about the lifecycle of the pointers that come out of that library (the most basic assumption is that these pointers will be alive for the duration of the library). Here's a simple example: strchr(char*)char* returns an internal pointer into its argument. This is documented on the man page, but there's nothing in the types or the header file that tells jextract this fact. The scope of the returned pointer can be copied from the scope of the input pointer. How can that be done? Some sort of manual intervention is required. This manual intervention is what I like to call "civilization": Somebody has to take the raw API from the header file only, and tweak it to better conform to the documented behavior of the API. In the end, the civilization process is intended to produce a Java API which is safe and secure: Easy to use safely, and impossible to subvert into a security exploit. Because C is an unsafe language and Java is safe, there is always going to be this mismatch between what comes out of jextract and what is safe for use by arbitrary users. This means either the extracted C API will be restricted to platform programmers, or the extracted C API is wrapped in a civilizing layer. Assigning accurate (safe and secure) scopes to pointers is one of the hard tasks of civilization. Probably there are some C APIs that just can't be made safe and secure for Java, but we are betting that many useful APIs will be easily civilized. Currently we are focusing on accurately extracting all possible raw APIs, and providing efficient access to them. This is challenging enough for now. So you won't see much help for civilizing yet. I expect that folks will start to experiment with civilizing layers when the raw extraction mechanisms stabilize. ? John From john.r.rose at oracle.com Tue Jan 29 20:09:07 2019 From: john.r.rose at oracle.com (John Rose) Date: Tue, 29 Jan 2019 12:09:07 -0800 Subject: [foreign] RFR: Fix UndefinedLayoutException exception message In-Reply-To: References: <53d45f14d8e7d7277fec7f6aa89320cb@xs4all.nl> <5C4EFFB9.2020509@oracle.com> Message-ID: On Jan 29, 2019, at 6:59 AM, Maurizio Cimadamore wrote: > > Resolving references across interfaces is not expected to work reliably. This is the primary reason which sends us towards an extraction model that is library-centric, rather than header-centric, so that all references in a library should be self-contained. The important thing is that when a header file is loaded into the binder, the binder knows which library it belongs to, so that information global to the library, and non-local to the header, can be accessed by the binder. This is pushing us towards a single-session mode of jextract, where everything is extracted in one go, and so it's obvious which headers go together. I think we also need to consider how to compose separate jextract sessions into a single library, and even (in some cases) compose libraries together, inheriting common definitions from one library into several others. The libc library is obviously a candidate for this, but as long as it is the only shared uber-library, we can special-case it, or even just copy definitions from it into all other sessions. Eventually I want to apply jextract to the HotSpot header files, which have a very complex structure. This may require us to work harder on composable jextract sessions. ? John From brian.goetz at oracle.com Tue Jan 29 20:48:25 2019 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 29 Jan 2019 15:48:25 -0500 Subject: [vector] failing tests Message-ID: I have done a clean checkout of the vectorIntrinsics branch, and I'm finding that the test for jdk.incubator.vector are not all passing: ============================== Test summary ============================== ?? TEST????????????????????????????????????????????? TOTAL? PASS FAIL ERROR >> jtreg:test/jdk/jdk/incubator/vector 68??? 59???? 9???? 0 << ============================== TEST FAILURE The failures seem to have been introduced before changeset:?? 54346:7d834373b72d branch:????? vectorIntrinsics parent:????? 54189:90ddf1dc631d parent:????? 54339:4a59f7042325 user:??????? vlivanov date:??????? Thu Jan 17 09:37:09 2019 -0800 summary:???? Manual merge with default but after changeset:?? 54149:d3b456d84b4e branch:????? vectorIntrinsics parent:????? 54054:890e996dfd1a parent:????? 54147:bccff579c2ff user:??????? mcimadamore date:??????? Wed Jan 09 22:09:11 2019 +0100 summary:???? Automatic merge with default The intervening commits look like they were all related to AMD support, one of these is the culprit: changeset:?? 54189:90ddf1dc631d branch:????? vectorIntrinsics parent:????? 54178:25099fb76afb user:??????? srukmannagar date:??????? Wed Jan 16 12:37:45 2019 -0800 summary:???? Contant Shift support for Byte and Short datatypes in VectorAPI changeset:?? 54178:25099fb76afb branch:????? vectorIntrinsics parent:????? 54176:9c63e91265fb user:??????? yzhang date:??????? Wed Jan 16 11:49:33 2019 +0800 summary:???? Fix byte vector test failures about bad ad file and unsupported vector size for AArch64 changeset:?? 54176:9c63e91265fb branch:????? vectorIntrinsics parent:????? 54149:d3b456d84b4e user:??????? yzhang date:??????? Tue Jan 15 13:46:39 2019 +0800 summary:???? Update max/min/maxAll/minAll Vector Api implementations and test cases From maurizio.cimadamore at oracle.com Tue Jan 29 21:03:33 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 29 Jan 2019 21:03:33 +0000 Subject: [foreign] RFR: Fix UndefinedLayoutException exception message In-Reply-To: <45fbc51ee7afa7ad49ecde599a1ee0e6@xs4all.nl> References: <53d45f14d8e7d7277fec7f6aa89320cb@xs4all.nl> <5C4EFFB9.2020509@oracle.com> <2e50c58e-7655-4198-4a07-8c95025136c8@oracle.com> <45fbc51ee7afa7ad49ecde599a1ee0e6@xs4all.nl> Message-ID: <6ff2647c-0911-137f-ef94-3ec703ee8a06@oracle.com> This revision looks good to me - I like the simple test. Maurizio On 29/01/2019 19:18, Jorn Vernee wrote: > Maurizio Cimadamore schreef op 2019-01-29 18:56: >>> Then try with >>> >>> scope.allocate(LayoutType.ofStruct(BadStruct.class)).get().foo() >>> >>> >> Ok, the above also doesn't work - but this should? >> >> scope.allocate(LayoutType.ofStruct(BadStruct.class).pointer()).get().get().foo() >> > > This also works, but tbh it doesn't seem much different than > `scope.allocate(VOID).cast(LayoutType.ofStruct(BadStruct.class)).get()`, > since we're dereferencing an invalid pointer in both cases. > >> >> This time we should be able to bypass Scope.allocateInternal... >> >> The main issue about having a cross header ref is that the test is >> gonna be sensitive to whatever we do in that area - e.g. that failure >> is, in a way, temporary; it would be preferrable to have a test that >> really checked something Bad (TM). > > FWIW, someone could write their own descriptor for an @NativeFunction > and accidentally put in a reference to something that just doesn't > exist at all, e.g.: > > ``` > @NativeHeader > interface Lib { > ??? @NativeFunction("()${Garbage}") > ??? int func(); > } > ``` > > That seems like a relatively simple but still realistic test case. > > Updated webrev: > http://cr.openjdk.java.net/~jvernee/panama/webrevs/fixmessage/webrev.02/ > > Jorn > >> Maurizio From jbvernee at xs4all.nl Tue Jan 29 21:09:08 2019 From: jbvernee at xs4all.nl (jbvernee at xs4all.nl) Date: Tue, 29 Jan 2019 21:09:08 +0000 Subject: hg: panama/dev: Summary: Fix UndefinedLayoutException exception message Message-ID: <201901292109.x0TL99x9002446@aojmv0008.oracle.com> Changeset: d82636575392 Author: jvernee Date: 2019-01-29 22:08 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/d82636575392 Summary: Fix UndefinedLayoutException exception message Reviewed-by: mcimadamore ! src/java.base/share/classes/jdk/internal/foreign/LayoutResolver.java + test/jdk/java/foreign/UndefinedLayoutTest.java From maurizio.cimadamore at oracle.com Tue Jan 29 21:15:12 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Tue, 29 Jan 2019 21:15:12 +0000 Subject: hg: panama/dev: Automatic merge with foreign Message-ID: <201901292115.x0TLFDqt005196@aojmv0008.oracle.com> Changeset: 27aef8743470 Author: mcimadamore Date: 2019-01-29 22:15 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/27aef8743470 Automatic merge with foreign From richard at openkappa.co.uk Tue Jan 29 21:28:23 2019 From: richard at openkappa.co.uk (Richard Startin) Date: Tue, 29 Jan 2019 21:28:23 +0000 Subject: [vector] AVX2 ByteVector.shiftR performance and semantics Message-ID: I noticed that in the current implementation ByteVector.shiftR is much slower than the equivalent logic using IntVector for AVX2. I don't fully understand the compiled code in the ByteVector case but most of the time is spent inside VectorIntrinsics::broadcastInt (see attachment): @Benchmark public int shiftRByte() { return ((ByteVector)YMM_LONG.fromArray(data, 0) .rebracket(YMM_BYTE)).shiftR(4).and((byte)0x0F) .get(0); } @Benchmark public int shiftRInt() { return ((ByteVector)((IntVector)YMM_LONG.fromArray(data, 0) .rebracket(YMM_INT)).shiftR(4).and(0x0F0F0F0F) .rebracket(YMM_BYTE)) .get(0); } Benchmark (size) Mode Cnt Score Error Units PopCount.shiftRByte 1024 thrpt 5 29.310 ? 0.680 ops/us PopCount.shiftRInt 1024 thrpt 5 257.261 ? 17.210 ops/us It's not clear whether shiftR is a signed or unsigned shift. If it is an unsigned shift, I think it can be implemented efficiently in terms of ints, which already works very well. Currently, ByteVector.shiftR seems to preserve sign, at least in the interpreter. Printing println(((ByteVector)YMM_LONG.fromArray(data, 0).rebracket(YMM_BYTE))); println(((ByteVector)YMM_LONG.fromArray(data, 0).rebracket(YMM_BYTE)).shiftR(4)); println(((ByteVector)((IntVector)YMM_LONG.fromArray(data, 0).rebracket(YMM_INT)).shiftR(4).and(0x0F0F0F0F) .rebracket(YMM_BYTE))); printed this: [9, 15, -41, 21, 85, 116, -18, 41, 30, -8, -40, -96, 119, -75, 119, -85, -48, -28, -55, 84, 118, 7, -46, 15, -103, -37, 48, -52, 14, -32, -105, -84] [0, 0, -3, 1, 5, 7, -2, 2, 1, -1, -3, -6, 7, -5, 7, -6, -3, -2, -4, 5, 7, 0, -3, 0, -7, -3, 3, -4, 0, -2, -7, -6] [0, 0, 13, 1, 5, 7, 14, 2, 1, 15, 13, 10, 7, 11, 7, 10, 13, 14, 12, 5, 7, 0, 13, 0, 9, 13, 3, 12, 0, 14, 9, 10] On the other hand, I saw that the shr instruction was used in the JIT compiled code. Thanks, Richard -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: perfasm.txt URL: From maurizio.cimadamore at oracle.com Tue Jan 29 21:38:53 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 29 Jan 2019 21:38:53 +0000 Subject: [foreign] RFR 8217781: Filter child cursors when doing recursive offset lookup In-Reply-To: <5a7368f419256b80bf7867d73c300de8@xs4all.nl> References: <481fd4e444e00a8cda38d1e7c18f294e@xs4all.nl> <496d5b3d-5bb9-bc04-1ebf-2b3784f344ed@oracle.com> <5a7368f419256b80bf7867d73c300de8@xs4all.nl> Message-ID: Looks good - Sundar please also take a look Cheers Maurizio On 25/01/2019 19:45, Jorn Vernee wrote: >> But then, as I was writing this, I realized that RecordLayoutComputer >> seems to have several helper methods which take a Cursor as input, >> when in reality the only input we ever pass is the record cursor >> itself (which is already stashed in a field) - so maybe we should get >> rid of these extra Cursor params and use the field? > > This is not quite possible. `hasIncompleteArray` also calls these > methods with the child cursors of anonymous nested structs and unions. > So we can't just make the helper methods non-static and use the field. > > It's quite unfortunate the way the bug works. We have to check for > incomplete array fields before trying to process _any_ of the fields, > which means doing a recursive lookup into the anonymous structs and > unions inside the type. > > But there is a simplification possible by replacing isFlattenable with > the method you suggest (which I've named `flattenableChildren` to show > the difference with `children()`) > > Updated webrev: > http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217781/webrev.01/ > > Cheer, > Jorn > > Maurizio Cimadamore schreef op 2019-01-25 19:40: >> Looks good, >> Some minor suggestions - these kind of issues arise because the code >> should never call cursor.children(). >> >> I suggest getting rid of isFlattenable, and have an accessor which >> does this: >> >> Stream children(Cursor c) { >> ?? return c.children() >> ?????????????????? .filter(RecordLayoutComputer::isFlattenable) >> } >> >> But then, as I was writing this, I realized that RecordLayoutComputer >> seems to have several helper methods which take a Cursor as input, >> when in reality the only input we ever pass is the record cursor >> itself (which is already stashed in a field) - so maybe we should get >> rid of these extra Cursor params and use the field? >> >> Maurizio >> >> On 25/01/2019 12:26, Jorn Vernee wrote: >>> Hi, >>> >>> Based on JDK-8217664 [1], I have found the following bug with this >>> layout: >>> >>> ``` >>> struct Foo { >>> ??? struct { >>> ??????? union { >>> ??????????? int x; >>> ??????? } Bar; >>> ??? }; >>> }; >>> ``` >>> >>> To find the offset of the anonymous struct in Foo, we have to find >>> the offset of it's first field. But, the current code is looking for >>> the first child _cursor_ of the anonymous struct, which is the union >>> declaration, and not the field 'Bar'. This again tries to look up >>> the first field of 'Bar' which is 'x', so we end up looking up the >>> field 'x' in Foo, which throws an error. >>> >>> The fix is simple, we just have to filter for anonymous struct/union >>> and FieldDecl child cursors when doing a recursive offset lookup. >>> I've also added an exception message to the recursive lookup, which >>> helped debug this. >>> >>> Please review the following. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8217781 >>> Webrev: >>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217781/webrev.00/ >>> >>> Thanks, >>> Jorn >>> >>> [1] : https://bugs.openjdk.java.net/browse/JDK-8217664 From brian.goetz at oracle.com Tue Jan 29 21:41:51 2019 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 29 Jan 2019 16:41:51 -0500 Subject: [vector] Small patch to rationalize species/shape factories Message-ID: <9d468ac7-3bae-4863-697c-5036972d1886@oracle.com> Please find attached a small patch to rationalize species/shape factory methods.? It compiles and tests cleanly against the last checkin for which the tests pass.? This is the patch for only the ungenerated sources (for ease of review.) It moves the species factories to Species.of() / Species.ofPreferred(), and the Shape factory to Shape.forBitSize(). -------------- next part -------------- # HG changeset patch # Parent d3b456d84b4ed3b6f4a15a09f9e26433b0d91bd5 diff -r d3b456d84b4e -r 8ab9aad380d0 src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Vector.java --- a/src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Vector.java Wed Jan 09 22:09:11 2019 +0100 +++ b/src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Vector.java Tue Jan 29 16:15:44 2019 -0500 @@ -74,9 +74,9 @@ * factory called a {@link Species}. A Species has an * element type and shape and creates Vector values of the same element type * and shape. - * A species can be {@link #species(Class, Shape)} obtained} given an element + * A species can be {@link Species#of(Class, Shape)} obtained} given an element * type and shape, or a preferred species can be - * {@link #preferredSpecies obtained} given just an element type where the most + * {@link Species#ofPreferred(Class)} obtained} given just an element type where the most * optimal shape is selected for the current platform. It is recommended that * Species instances be held in {@code static final} fields for optimal creation * and usage of Vector values by the runtime compiler. @@ -908,6 +908,32 @@ int length(Species s) { return bitSize() / s.elementSize(); } + + /** + * Finds appropriate shape depending on bitsize. + * + * @param bitSize the size in bits + * @return the shape corresponding to bitsize + * @see #bitSize + */ + public static Shape forBitSize(int bitSize) { + switch (bitSize) { + case 64: + return Shape.S_64_BIT; + case 128: + return Shape.S_128_BIT; + case 256: + return Shape.S_256_BIT; + case 512: + return Shape.S_512_BIT; + default: + if ((bitSize > 0) && (bitSize <= 2048) && (bitSize % 128 == 0)) { + return Shape.S_Max_BIT; + } else { + throw new IllegalArgumentException("Bad vector bit size: " + bitSize); + } + } + } } @@ -962,6 +988,65 @@ // Factory /** + * Finds a species for an element type and shape. + * + * @param c the element type + * @param s the shape + * @param the boxed element type + * @return a species for an element type and shape + * @throws IllegalArgumentException if no such species exists for the + * element type and/or shape + */ + @SuppressWarnings("unchecked") + public static Vector.Species of(Class c, Shape s) { + if (c == float.class) { + return (Vector.Species) FloatVector.species(s); + } + else if (c == double.class) { + return (Vector.Species) DoubleVector.species(s); + } + else if (c == byte.class) { + return (Vector.Species) ByteVector.species(s); + } + else if (c == short.class) { + return (Vector.Species) ShortVector.species(s); + } + else if (c == int.class) { + return (Vector.Species) IntVector.species(s); + } + else if (c == long.class) { + return (Vector.Species) LongVector.species(s); + } + else { + throw new IllegalArgumentException("Bad vector element type: " + c.getName()); + } + } + + /** + * Finds a preferred species for an element type. + *

+ * A preferred species is a species chosen by the platform that has a + * shape of maximal bit size. A preferred species for different element + * types will have the same shape, and therefore vectors created from + * such species will be shape compatible. + * + * @param c the element type + * @param the boxed element type + * @return a preferred species for an element type + * @throws IllegalArgumentException if no such species exists for the + * element type + */ + @SuppressWarnings("unchecked") + public static Vector.Species ofPreferred(Class c) { + Unsafe u = Unsafe.getUnsafe(); + + int vectorLength = u.getMaxVectorSize(c); + int vectorBitSize = bitSizeForVectorLength(c, vectorLength); + Shape s = Shape.forBitSize(vectorBitSize); + return Species.of(c, s); + } + + /** * Returns a vector where all lane elements are set to the default * primitive value. * @@ -1726,30 +1811,6 @@ } /** - * Finds a preferred species for an element type. - *

- * A preferred species is a species chosen by the platform that has a - * shape of maximal bit size. A preferred species for different element - * types will have the same shape, and therefore vectors created from - * such species will be shape compatible. - * - * @param c the element type - * @param the boxed element type - * @return a preferred species for an element type - * @throws IllegalArgumentException if no such species exists for the - * element type - */ - @SuppressWarnings("unchecked") - public static Vector.Species preferredSpecies(Class c) { - Unsafe u = Unsafe.getUnsafe(); - - int vectorLength = u.getMaxVectorSize(c); - int vectorBitSize = bitSizeForVectorLength(c, vectorLength); - Shape s = shapeForVectorBitSize(vectorBitSize); - return species(c, s); - } - - /** * Find bit size based on element type and number of elements. * * @param c the element type @@ -1779,65 +1840,4 @@ throw new IllegalArgumentException("Bad vector type: " + c.getName()); } } - - /** - * Finds appropriate shape depending on bitsize. - * - * @param bitSize the size in bits - * @return the shape corresponding to bitsize - * @see #bitSize - */ - public static Shape shapeForVectorBitSize(int bitSize) { - switch (bitSize) { - case 64: - return Shape.S_64_BIT; - case 128: - return Shape.S_128_BIT; - case 256: - return Shape.S_256_BIT; - case 512: - return Shape.S_512_BIT; - default: - if ((bitSize > 0) && (bitSize <= 2048) && (bitSize % 128 == 0)) { - return Shape.S_Max_BIT; - } else { - throw new IllegalArgumentException("Bad vector bit size: " + bitSize); - } - } - } - - /** - * Finds a species for an element type and shape. - * - * @param c the element type - * @param s the shape - * @param the boxed element type - * @return a species for an element type and shape - * @throws IllegalArgumentException if no such species exists for the - * element type and/or shape - */ - @SuppressWarnings("unchecked") - public static Vector.Species species(Class c, Shape s) { - if (c == float.class) { - return (Vector.Species) FloatVector.species(s); - } - else if (c == double.class) { - return (Vector.Species) DoubleVector.species(s); - } - else if (c == byte.class) { - return (Vector.Species) ByteVector.species(s); - } - else if (c == short.class) { - return (Vector.Species) ShortVector.species(s); - } - else if (c == int.class) { - return (Vector.Species) IntVector.species(s); - } - else if (c == long.class) { - return (Vector.Species) LongVector.species(s); - } - else { - throw new IllegalArgumentException("Bad vector element type: " + c.getName()); - } - } } diff -r d3b456d84b4e -r 8ab9aad380d0 src/jdk.incubator.vector/share/classes/jdk/incubator/vector/X-Vector.java.template --- a/src/jdk.incubator.vector/share/classes/jdk/incubator/vector/X-Vector.java.template Wed Jan 09 22:09:11 2019 +0100 +++ b/src/jdk.incubator.vector/share/classes/jdk/incubator/vector/X-Vector.java.template Tue Jan 29 16:15:44 2019 -0500 @@ -2240,7 +2240,7 @@ */ @SuppressWarnings("unchecked") public static $Type$Species preferredSpecies() { - return ($Type$Species) Vector.preferredSpecies($type$.class); + return ($Type$Species) Species.ofPreferred($type$.class); } /** diff -r d3b456d84b4e -r 8ab9aad380d0 src/jdk.incubator.vector/share/classes/jdk/incubator/vector/X-VectorBits.java.template --- a/src/jdk.incubator.vector/share/classes/jdk/incubator/vector/X-VectorBits.java.template Wed Jan 09 22:09:11 2019 +0100 +++ b/src/jdk.incubator.vector/share/classes/jdk/incubator/vector/X-VectorBits.java.template Tue Jan 29 16:15:44 2019 -0500 @@ -52,8 +52,8 @@ private static final IntVector.IntSpecies INDEX_SPEC; static { int bitSize = Vector.bitSizeForVectorLength(int.class, LENGTH); - Vector.Shape shape = Vector.shapeForVectorBitSize(bitSize); - INDEX_SPEC = (IntVector.IntSpecies) Vector.species(int.class, shape); + Vector.Shape shape = Shape.forBitSize(bitSize); + INDEX_SPEC = (IntVector.IntSpecies) Species.of(int.class, shape); } #end[!longOrDouble64] diff -r d3b456d84b4e -r 8ab9aad380d0 test/jdk/jdk/incubator/vector/PreferredSpeciesTest.java --- a/test/jdk/jdk/incubator/vector/PreferredSpeciesTest.java Wed Jan 09 22:09:11 2019 +0100 +++ b/test/jdk/jdk/incubator/vector/PreferredSpeciesTest.java Tue Jan 29 16:15:44 2019 -0500 @@ -51,7 +51,7 @@ @Test(dataProvider = "classesProvider") void testVectorLength(Class c) { Vector.Species species = - Vector.preferredSpecies(c); + Vector.Species.ofPreferred(c); Assert.assertEquals(species.length(), U.getMaxVectorSize(c)); } From sandhya.viswanathan at intel.com Tue Jan 29 21:50:21 2019 From: sandhya.viswanathan at intel.com (Viswanathan, Sandhya) Date: Tue, 29 Jan 2019 21:50:21 +0000 Subject: [vector] failing tests In-Reply-To: References: Message-ID: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A55476@FMSMSX126.amr.corp.intel.com> Hi Brian, Yes the tests are failing since the following changeset that changed the definition of Vector min/max/MinAll/MaxAll to match Math.min/Math/max: changeset: 54176:9c63e91265fb branch: vectorIntrinsics parent: 54149:d3b456d84b4e user: yzhang date: Tue Jan 15 13:46:39 2019 +0800 summary: Update max/min/maxAll/minAll Vector Api implementations and test cases We are in the process of updating the x86 implementation to match the new Java specification. Best Regards, Sandhya -----Original Message----- From: panama-dev [mailto:panama-dev-bounces at openjdk.java.net] On Behalf Of Brian Goetz Sent: Tuesday, January 29, 2019 12:48 PM To: panama-dev at openjdk.java.net Subject: [vector] failing tests I have done a clean checkout of the vectorIntrinsics branch, and I'm finding that the test for jdk.incubator.vector are not all passing: ============================== Test summary ============================== ?? TEST????????????????????????????????????????????? TOTAL? PASS FAIL ERROR >> jtreg:test/jdk/jdk/incubator/vector 68??? 59???? 9???? 0 << ============================== TEST FAILURE The failures seem to have been introduced before changeset:?? 54346:7d834373b72d branch:????? vectorIntrinsics parent:????? 54189:90ddf1dc631d parent:????? 54339:4a59f7042325 user:??????? vlivanov date:??????? Thu Jan 17 09:37:09 2019 -0800 summary:???? Manual merge with default but after changeset:?? 54149:d3b456d84b4e branch:????? vectorIntrinsics parent:????? 54054:890e996dfd1a parent:????? 54147:bccff579c2ff user:??????? mcimadamore date:??????? Wed Jan 09 22:09:11 2019 +0100 summary:???? Automatic merge with default The intervening commits look like they were all related to AMD support, one of these is the culprit: changeset:?? 54189:90ddf1dc631d branch:????? vectorIntrinsics parent:????? 54178:25099fb76afb user:??????? srukmannagar date:??????? Wed Jan 16 12:37:45 2019 -0800 summary:???? Contant Shift support for Byte and Short datatypes in VectorAPI changeset:?? 54178:25099fb76afb branch:????? vectorIntrinsics parent:????? 54176:9c63e91265fb user:??????? yzhang date:??????? Wed Jan 16 11:49:33 2019 +0800 summary:???? Fix byte vector test failures about bad ad file and unsupported vector size for AArch64 changeset:?? 54176:9c63e91265fb branch:????? vectorIntrinsics parent:????? 54149:d3b456d84b4e user:??????? yzhang date:??????? Tue Jan 15 13:46:39 2019 +0800 summary:???? Update max/min/maxAll/minAll Vector Api implementations and test cases From richard at openkappa.co.uk Tue Jan 29 22:44:47 2019 From: richard at openkappa.co.uk (Richard Startin) Date: Tue, 29 Jan 2019 22:44:47 +0000 Subject: [vector] Partial permutations Message-ID: Quite a lot of vector code uses shuffle instructions, not with the intent of rearranging a vector, but of doing a parallel lookup as fast as possible, even if it means wasting some of the result. For instance, ignoring one of vpshufb's parallel permutations it can be used to look up 16 values by nibble-valued keys in a single instruction. This technique is used in existing implementations of base 64 encoding, and various compression algorithms. As an example of how this would look in the vector API, below is a routine to count the bits in a bitmap which actually works even if it isn't very fast yet. The ByteVector.rearrange looks up the precomputed bit counts of the 16 nibbles in each vector, it doesn't seem to have been implemented yet, but if the rearrange method must do a full 32 byte permutation (requiring several instructions) I suspect it might be doing too much work for the routine below to be viable (compared to a loop of Long.bitCount calls). It would be nice to be able to hint somehow that only half of the result need be permuted, allowing the permutation to "short circuit" - potentially using faster code. private static byte[] NIBBLE_COUNTS = new byte[] { 0, 1, 1, 2, 1, 2, 2, 3, 1, 2, 2, 3, 2, 3, 3, 4, 0, 1, 1, 2, 1, 2, 2, 3, 1, 2, 2, 3, 2, 3, 3, 4 }; private long[] data; @Benchmark public int vectorBitCount() { int bitCount = 0; int block = 256; for (int i = 0; i < data.length; i += block) { var lo = YMM_INT.zero(); var hi = YMM_INT.zero(); var counts = YMM_BYTE.fromArray(NIBBLE_COUNTS, 0); for (int j = 0; j < block; j += 4) { var v1 = (IntVector)YMM_LONG.fromArray(data, i + j).rebracket(YMM_INT); lo = lo.add(counts.rearrange(v1.and(0x0F0F0F0F).rebracket(YMM_BYTE).toShuffle()).rebracket(YMM_INT)); hi = hi.add(counts.rearrange(v1.shiftR(4).and(0x0F0F0F0F).rebracket(YMM_BYTE).toShuffle()).rebracket(YMM_INT)); } bitCount += unsignedSum(lo); bitCount += unsignedSum(hi); } return bitCount; } private int unsignedSum(IntVector bv) { // convert to LongVector because Vector.get is slow var lv = (LongVector) bv.rebracket(YMM_LONG); return sumBytes(lv.get(0)) + sumBytes(lv.get(1)) + sumBytes(lv.get(2)) + sumBytes(lv.get(3)); } private int sumBytes(long w) { return ((int)w & 0xFF) + (((int)(w >>> 8)) & 0xFF) + (((int)(w >>> 16)) & 0xFF) + (((int)(w >>> 24)) & 0xFF) + (((int)(w >>> 32)) & 0xFF) + (((int)(w >>> 40)) & 0xFF) + (((int)(w >>> 48)) & 0xFF) + (((int)(w >>> 56)) & 0xFF); } From jbvernee at xs4all.nl Wed Jan 30 00:05:03 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Wed, 30 Jan 2019 01:05:03 +0100 Subject: hg: panama/dev: Summary: Fix UndefinedLayoutException exception message In-Reply-To: <201901292109.x0TL99x9002446@aojmv0008.oracle.com> References: <201901292109.x0TL99x9002446@aojmv0008.oracle.com> Message-ID: Ok. New test is failing intermittently because it's loading a native library that's also being loaded by another test, and this is causing a 'Native Library already loaded in another classloader' error. AFAIK it's a matter of when the GC decides to collect the ClassLoader of the previous test (and unload the library). The shared library use was already the case for 2 other tests (Hello.java and LoadLibraryTest.java), but adding a third test to this group apparently causes problems :/ I didn't run into this before while testing locally. Using /othervm for the test seems to fix the issue: http://cr.openjdk.java.net/~jvernee/panama/webrevs/conclib/webrev.00/ Sorry about that, Jorn jbvernee at xs4all.nl schreef op 2019-01-29 22:09: > Changeset: d82636575392 > Author: jvernee > Date: 2019-01-29 22:08 +0100 > URL: http://hg.openjdk.java.net/panama/dev/rev/d82636575392 > > Summary: Fix UndefinedLayoutException exception message > Reviewed-by: mcimadamore > > ! src/java.base/share/classes/jdk/internal/foreign/LayoutResolver.java > + test/jdk/java/foreign/UndefinedLayoutTest.java From sandhya.viswanathan at intel.com Wed Jan 30 01:30:30 2019 From: sandhya.viswanathan at intel.com (Viswanathan, Sandhya) Date: Wed, 30 Jan 2019 01:30:30 +0000 Subject: [Vector] RFR: reshape, resize, rebracket, cast In-Reply-To: References: Message-ID: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A555A8@FMSMSX126.amr.corp.intel.com> Hi All, Please find the updated webrev for reinterpret, reshape and cast at: http://cr.openjdk.java.net/~sviswanathan/vectorIntrinsics/ReinterpretReshapeCast/webrev.02/ This update implements the two review inputs from Vladimir: 1) Update users: test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/AbstractVectorBenchmark.java test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/PopulationCount.java test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/SumOfUnsignedBytes.java test/jdk/jdk/incubator/vector/VectorHash.java test/jdk/jdk/incubator/vector/VectorReshapeTests.java test/jdk/jdk/incubator/vector/CovarOverrideTest.java 2) src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Vector.java: + public Vector defaultReinterpret(Species s) { + public abstract Class vectorType(); Those operations shouldn't be part of public API. Please let me know if this looks ok to push. Best Regards, Sandhya -----Original Message----- From: panama-dev [mailto:panama-dev-bounces at openjdk.java.net] On Behalf Of Brian Goetz Sent: Monday, January 28, 2019 3:38 PM To: Kharbas, Kishor ; panama-dev at openjdk.java.net Subject: Re: [Vector] RFR: reshape, resize, rebracket, cast The issue you raise is a minor frustration, but overall I think the direction of movement -- from species to vector -- is the right one.? We can work on ironing out these small issues later. On 1/16/2019 3:20 AM, Kharbas, Kishor wrote: > > Hi, > > I have a patch which refactors the above methods as discussed in this > thread - > https://mail.openjdk.java.net/pipermail/panama-dev/2018-December/003365.html. > > Please review the changes at - > http://cr.openjdk.java.net/~kkharbas/vector-api/webrev-reshape_etc > > Generation of intrinsics and correct vector boxing elimination have > been verified. In the next patch I will add changes to jtreg tests. > > I wanted to bring forth one small issue(possibly) which programmers > might face with this change - > > We provide specialized types like IntVector, FloatVector, etc for > users to define their vectors and they would be able to write code > like this with previous methods, > > /FloatVector float256 = SPECIES_FLOAT256.cast(int256);/ > > Here FloatSpecies would always return FloatVector. > > However with this change, since cast() is defined on a vector and > takes species of a generic type, it cannot return a specialized Vector > like IntVector or FloatVector. User has to explicitly cast the return > vector from Vector to specialized Vector or use a generic vector of > corresponding element type. For example, > > /? ??FloatVector float256 = (FloatVector)int256.cast(SPECIES_FLOAT256);/ > > /? or/ > > /? ??Vector float256 = int256.cast(SPECIES_FLOAT256);/ > > I am not sure if this is even a problem, but I thought its worth > mentioning. > > Thanks > > Kishor > From john.r.rose at oracle.com Wed Jan 30 01:34:53 2019 From: john.r.rose at oracle.com (John Rose) Date: Tue, 29 Jan 2019 17:34:53 -0800 Subject: [vector] Partial permutations In-Reply-To: References: Message-ID: <9970C4BC-FDC8-49A4-93FE-1B9F72282DFC@oracle.com> On Jan 29, 2019, at 2:44 PM, Richard Startin wrote: > > lo = lo.add(counts.rearrange(v1.and(0x0F0F0F0F).rebracket(YMM_BYTE).toShuffle()).rebracket(YMM_INT)); Breaking it down: var v1 = ?; // u4[8] var nibblePerByte = v1.and(0x0F0F0F0F).rebracket(YMM_BYTE); // u1[32] var bitCountPerByte = counts.rearrange(nibblePerByte.toShuffle()); // u1[32] var swarLanes = bitCountPerByte.rebracket(YMM_INT); // u4[8] lo = lo.add(expandedCounts); // danger: carry-out across bytes? The second line zeroes out half of the bit positions and breaks out the 8 int lanes into 32 byte lanes. The third line replaces each byte (whose value is in 0..15) with another byte that gives the bit-count (value in 0..4). The fourth line undoes the action of the second line. Now we have four lanes each of which is an integer containing 4 "SIMD across a register" sub-lanes. It seems like the top half of your NIBBLE_COUNTS table could be any old garbage, not a copy of the bottom half. (Did I get it right? This kind of code is tricky!) The final line adds 8x4 lanes and their sub-lanes. I think you have a bug in your block size, since if all the input bits are set, then each nibble count will always be 4, and so each sub-lane count will accumulate that value, 256/4 times, which will cause a carry to propagate across your SWAR sub-lanes. The idiom counts.rearrange(nibblePerByte.toShuffle()) would read better, in this algorithm, as nibblePerByte.selectFrom(counts). It's really akin to a gather operation in this use case, and the similarity could be made more clear by using a gather-like API (which we are still working on). To respond to your question: If I follow you, you note that since the top half of the NIBBLE_COUNTS array is never used, then any instructions which would do the work of loading from that half of the array are useless. To me, this calls the question of how to do strength-reduction optimizations on shuffles. I don't think we want explicit API points for simplified shuffles, any more than we want library routines or operators for division when both operands are positive, even though there are optimizations relevant in both cases. The best way to get improved code for simplified inputs is to work hard to detect those inputs and adjust the instruction selection (or IR generation) accordingly. In this case, the shuffle vector is the logical "and" of 0x0F and some random input, which means that, ideally the optimizer should be able to prove that the selected lanes of "counts" are in the range 0..15. That's probably enough to simplify a complex shuffle idiom in the back end. Failing lanewise type analysis (but why should we fail that?), a pattern match in the C2 backend might also detect that the shuffle was immediately derived from a masking operation. In either case, the optimization might be confounded by the SWAR hacks going on. Perhaps the right answer is to rebracket to YMM_BYTE before the logical "and" (of a byte 0x0F). This is sadly delicate, but can be made more robust by an improvement to the C2 type system to track individual bit positions, or adoption of Graal, which already has that feature. The overall point here is that some of our good code will come from a strength reduction of expensive operations, gated by type and pattern analysis. The design problem is to put enough primitives into the user API but not too many, and enough optimization in the backend. ? John From sundararajan.athijegannathan at oracle.com Wed Jan 30 02:06:02 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Wed, 30 Jan 2019 07:36:02 +0530 Subject: [foreign] RFR: Fix UndefinedLayoutException exception message In-Reply-To: <6ff2647c-0911-137f-ef94-3ec703ee8a06@oracle.com> References: <53d45f14d8e7d7277fec7f6aa89320cb@xs4all.nl> <5C4EFFB9.2020509@oracle.com> <2e50c58e-7655-4198-4a07-8c95025136c8@oracle.com> <45fbc51ee7afa7ad49ecde599a1ee0e6@xs4all.nl> <6ff2647c-0911-137f-ef94-3ec703ee8a06@oracle.com> Message-ID: <5C51068A.4090503@oracle.com> +1 -Sundar On 30/01/19, 2:33 AM, Maurizio Cimadamore wrote: > This revision looks good to me - I like the simple test. > > Maurizio > > On 29/01/2019 19:18, Jorn Vernee wrote: >> Maurizio Cimadamore schreef op 2019-01-29 18:56: >>>> Then try with >>>> >>>> scope.allocate(LayoutType.ofStruct(BadStruct.class)).get().foo() >>>> >>>> >>> Ok, the above also doesn't work - but this should? >>> >>> scope.allocate(LayoutType.ofStruct(BadStruct.class).pointer()).get().get().foo() >>> >> >> This also works, but tbh it doesn't seem much different than >> `scope.allocate(VOID).cast(LayoutType.ofStruct(BadStruct.class)).get()`, >> since we're dereferencing an invalid pointer in both cases. >> >>> >>> This time we should be able to bypass Scope.allocateInternal... >>> >>> The main issue about having a cross header ref is that the test is >>> gonna be sensitive to whatever we do in that area - e.g. that failure >>> is, in a way, temporary; it would be preferrable to have a test that >>> really checked something Bad (TM). >> >> FWIW, someone could write their own descriptor for an @NativeFunction >> and accidentally put in a reference to something that just doesn't >> exist at all, e.g.: >> >> ``` >> @NativeHeader >> interface Lib { >> @NativeFunction("()${Garbage}") >> int func(); >> } >> ``` >> >> That seems like a relatively simple but still realistic test case. >> >> Updated webrev: >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/fixmessage/webrev.02/ >> >> Jorn >> >>> Maurizio From sundararajan.athijegannathan at oracle.com Wed Jan 30 02:08:56 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Wed, 30 Jan 2019 07:38:56 +0530 Subject: hg: panama/dev: Summary: Fix UndefinedLayoutException exception message In-Reply-To: References: <201901292109.x0TL99x9002446@aojmv0008.oracle.com> Message-ID: <5C510738.30908@oracle.com> Hmm.. That needs to be investigated. But it is better to avoid test failure. -othervm is fine (for now) -Sundar On 30/01/19, 5:35 AM, Jorn Vernee wrote: > Ok. New test is failing intermittently because it's loading a native > library that's also being loaded by another test, and this is causing > a 'Native Library already loaded in another classloader' error. AFAIK > it's a matter of when the GC decides to collect the ClassLoader of the > previous test (and unload the library). The shared library use was > already the case for 2 other tests (Hello.java and > LoadLibraryTest.java), but adding a third test to this group > apparently causes problems :/ I didn't run into this before while > testing locally. > > Using /othervm for the test seems to fix the issue: > http://cr.openjdk.java.net/~jvernee/panama/webrevs/conclib/webrev.00/ > > Sorry about that, > Jorn > > jbvernee at xs4all.nl schreef op 2019-01-29 22:09: >> Changeset: d82636575392 >> Author: jvernee >> Date: 2019-01-29 22:08 +0100 >> URL: http://hg.openjdk.java.net/panama/dev/rev/d82636575392 >> >> Summary: Fix UndefinedLayoutException exception message >> Reviewed-by: mcimadamore >> >> ! src/java.base/share/classes/jdk/internal/foreign/LayoutResolver.java >> + test/jdk/java/foreign/UndefinedLayoutTest.java From samuel.audet at gmail.com Wed Jan 30 02:21:01 2019 From: samuel.audet at gmail.com (Samuel Audet) Date: Wed, 30 Jan 2019 11:21:01 +0900 Subject: scopes writeup In-Reply-To: <6E570AB5-88D3-43E6-B58C-E2297CD6D5CA@oracle.com> References: <6fbff2eb-b100-0c03-7663-c732a3904302@oracle.com> <3e4546e7-f768-6550-3a31-b38dae835d7b@oracle.com> <6E570AB5-88D3-43E6-B58C-E2297CD6D5CA@oracle.com> Message-ID: Hi, I would like to hear more about what John mentions below about being "challenging enough" and leaving work for the future. I would go one step further. The Substrate VM team already has something working as part of their C interface, and they did not need to parse anything to get it working. It does not try to do as much as Panama, but they did what is in my opinion most important: A simpler more efficient variant of JNI. If we could get this bit of Panama stabilized first, instead of trying to parse everything, that would be great I think. What do you think? Or if that is not possible, how is Panama different from Substrate VM such that the same approach would not work? I am starting the get the impression that Panama is basically reimplementing portions of LLVM in Java as part of SystemABI. How much effort would it take to, for example, implement the required subset of the Java ABI in LLVM? Doing it that way, we would not need to test the C/C++ ABI, at least, making it possible instead to reuse tests from the JDK itself, or am I talking nonsense here? Looks like that might be challenging as well: http://openjdk.java.net/jeps/8189173. In any case, it would be great if we could have a discussion about these things instead of having OpenJDK dictate everything! Samuel On 1/30/19 5:03 AM, John Rose wrote: > Currently we are focusing on accurately extracting all possible > raw APIs, and providing efficient access to them. This is > challenging enough for now. So you won't see much help > for civilizing yet. I expect that folks will start to experiment > with civilizing layers when the raw extraction mechanisms > stabilize. From sundararajan.athijegannathan at oracle.com Wed Jan 30 03:58:29 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Wed, 30 Jan 2019 09:28:29 +0530 Subject: [foreign] RFR 8217781: Filter child cursors when doing recursive offset lookup In-Reply-To: References: <481fd4e444e00a8cda38d1e7c18f294e@xs4all.nl> <496d5b3d-5bb9-bc04-1ebf-2b3784f344ed@oracle.com> <5a7368f419256b80bf7867d73c300de8@xs4all.nl> Message-ID: <5C5120E5.900@oracle.com> +1. Good fix! subtle bug. Minor: Cursor.toString is a very long line. Perhaps it can folded into 2 lines. -Sundar On 30/01/19, 3:08 AM, Maurizio Cimadamore wrote: > Looks good - Sundar please also take a look > > Cheers > Maurizio > > On 25/01/2019 19:45, Jorn Vernee wrote: >>> But then, as I was writing this, I realized that RecordLayoutComputer >>> seems to have several helper methods which take a Cursor as input, >>> when in reality the only input we ever pass is the record cursor >>> itself (which is already stashed in a field) - so maybe we should get >>> rid of these extra Cursor params and use the field? >> >> This is not quite possible. `hasIncompleteArray` also calls these >> methods with the child cursors of anonymous nested structs and >> unions. So we can't just make the helper methods non-static and use >> the field. >> >> It's quite unfortunate the way the bug works. We have to check for >> incomplete array fields before trying to process _any_ of the fields, >> which means doing a recursive lookup into the anonymous structs and >> unions inside the type. >> >> But there is a simplification possible by replacing isFlattenable >> with the method you suggest (which I've named `flattenableChildren` >> to show the difference with `children()`) >> >> Updated webrev: >> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217781/webrev.01/ >> >> Cheer, >> Jorn >> >> Maurizio Cimadamore schreef op 2019-01-25 19:40: >>> Looks good, >>> Some minor suggestions - these kind of issues arise because the code >>> should never call cursor.children(). >>> >>> I suggest getting rid of isFlattenable, and have an accessor which >>> does this: >>> >>> Stream children(Cursor c) { >>> return c.children() >>> .filter(RecordLayoutComputer::isFlattenable) >>> } >>> >>> But then, as I was writing this, I realized that RecordLayoutComputer >>> seems to have several helper methods which take a Cursor as input, >>> when in reality the only input we ever pass is the record cursor >>> itself (which is already stashed in a field) - so maybe we should get >>> rid of these extra Cursor params and use the field? >>> >>> Maurizio >>> >>> On 25/01/2019 12:26, Jorn Vernee wrote: >>>> Hi, >>>> >>>> Based on JDK-8217664 [1], I have found the following bug with this >>>> layout: >>>> >>>> ``` >>>> struct Foo { >>>> struct { >>>> union { >>>> int x; >>>> } Bar; >>>> }; >>>> }; >>>> ``` >>>> >>>> To find the offset of the anonymous struct in Foo, we have to find >>>> the offset of it's first field. But, the current code is looking >>>> for the first child _cursor_ of the anonymous struct, which is the >>>> union declaration, and not the field 'Bar'. This again tries to >>>> look up the first field of 'Bar' which is 'x', so we end up looking >>>> up the field 'x' in Foo, which throws an error. >>>> >>>> The fix is simple, we just have to filter for anonymous >>>> struct/union and FieldDecl child cursors when doing a recursive >>>> offset lookup. I've also added an exception message to the >>>> recursive lookup, which helped debug this. >>>> >>>> Please review the following. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8217781 >>>> Webrev: >>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217781/webrev.00/ >>>> >>>> Thanks, >>>> Jorn >>>> >>>> [1] : https://bugs.openjdk.java.net/browse/JDK-8217664 From maurizio.cimadamore at oracle.com Wed Jan 30 10:59:01 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 30 Jan 2019 10:59:01 +0000 Subject: scopes writeup In-Reply-To: References: <6fbff2eb-b100-0c03-7663-c732a3904302@oracle.com> <3e4546e7-f768-6550-3a31-b38dae835d7b@oracle.com> <6E570AB5-88D3-43E6-B58C-E2297CD6D5CA@oracle.com> Message-ID: <20528665-d4be-e67d-83bb-2060344ce9e0@oracle.com> Hi Samuel, the point you raise has nothing to do with the scope discussion. Please kindly consider moving it to a separate thread. Cheers Maurizio On 30/01/2019 02:21, Samuel Audet wrote: > Hi, > > I would like to hear more about what John mentions below about being > "challenging enough" and leaving work for the future. I would go one > step further. The Substrate VM team already has something working as > part of their C interface, and they did not need to parse anything to > get it working. It does not try to do as much as Panama, but they did > what is in my opinion most important: A simpler more efficient variant > of JNI. If we could get this bit of Panama stabilized first, instead > of trying to parse everything, that would be great I think. What do > you think? Or if that is not possible, how is Panama different from > Substrate VM such that the same approach would not work? > > I am starting the get the impression that Panama is basically > reimplementing portions of LLVM in Java as part of SystemABI. How much > effort would it take to, for example, implement the required subset of > the Java ABI in LLVM? Doing it that way, we would not need to test the > C/C++ ABI, at least, making it possible instead to reuse tests from > the JDK itself, or am I talking nonsense here? Looks like that might > be challenging as well: http://openjdk.java.net/jeps/8189173. In any > case, it would be great if we could have a discussion about these > things instead of having OpenJDK dictate everything! > > Samuel > > On 1/30/19 5:03 AM, John Rose wrote: >> Currently we are focusing on accurately extracting all possible >> raw APIs, and providing efficient access to them.? This is >> challenging enough for now.? So you won't see much help >> for civilizing yet.? I expect that folks will start to experiment >> with civilizing layers when the raw extraction mechanisms >> stabilize. From jbvernee at xs4all.nl Wed Jan 30 11:01:50 2019 From: jbvernee at xs4all.nl (jbvernee at xs4all.nl) Date: Wed, 30 Jan 2019 11:01:50 +0000 Subject: hg: panama/dev: Summary: Fix intermittent test failure due to multiple tests loading the same library Message-ID: <201901301101.x0UB1oOm002687@aojmv0008.oracle.com> Changeset: 589aa22eaf53 Author: jvernee Date: 2019-01-30 12:00 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/589aa22eaf53 Summary: Fix intermittent test failure due to multiple tests loading the same library Reviewed-by: sundar ! test/jdk/java/foreign/UndefinedLayoutTest.java From maurizio.cimadamore at oracle.com Wed Jan 30 11:05:18 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 30 Jan 2019 11:05:18 +0000 Subject: hg: panama/dev: Automatic merge with foreign Message-ID: <201901301105.x0UB5I9j003917@aojmv0008.oracle.com> Changeset: 60634fb79a0f Author: mcimadamore Date: 2019-01-30 12:05 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/60634fb79a0f Automatic merge with foreign From jbvernee at xs4all.nl Wed Jan 30 11:28:34 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Wed, 30 Jan 2019 12:28:34 +0100 Subject: [foreign] RFR 8217781: Filter child cursors when doing recursive offset lookup In-Reply-To: <5C5120E5.900@oracle.com> References: <481fd4e444e00a8cda38d1e7c18f294e@xs4all.nl> <496d5b3d-5bb9-bc04-1ebf-2b3784f344ed@oracle.com> <5a7368f419256b80bf7867d73c300de8@xs4all.nl> <5C5120E5.900@oracle.com> Message-ID: <6a7a395d456608e08f14498711226543@xs4all.nl> Thanks, Updated webrev: http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217781/webrev.02/ I also split the exception message I added in RecordLayoutComputer since that was also pretty long. Jorn Sundararajan Athijegannathan schreef op 2019-01-30 04:58: > +1. > > Good fix! subtle bug. > > Minor: Cursor.toString is a very long line. Perhaps it can folded into > 2 lines. > > -Sundar > > On 30/01/19, 3:08 AM, Maurizio Cimadamore wrote: >> Looks good - Sundar please also take a look >> >> Cheers >> Maurizio >> >> On 25/01/2019 19:45, Jorn Vernee wrote: >>>> But then, as I was writing this, I realized that >>>> RecordLayoutComputer >>>> seems to have several helper methods which take a Cursor as input, >>>> when in reality the only input we ever pass is the record cursor >>>> itself (which is already stashed in a field) - so maybe we should >>>> get >>>> rid of these extra Cursor params and use the field? >>> >>> This is not quite possible. `hasIncompleteArray` also calls these >>> methods with the child cursors of anonymous nested structs and >>> unions. So we can't just make the helper methods non-static and use >>> the field. >>> >>> It's quite unfortunate the way the bug works. We have to check for >>> incomplete array fields before trying to process _any_ of the fields, >>> which means doing a recursive lookup into the anonymous structs and >>> unions inside the type. >>> >>> But there is a simplification possible by replacing isFlattenable >>> with the method you suggest (which I've named `flattenableChildren` >>> to show the difference with `children()`) >>> >>> Updated webrev: >>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217781/webrev.01/ >>> >>> Cheer, >>> Jorn >>> >>> Maurizio Cimadamore schreef op 2019-01-25 19:40: >>>> Looks good, >>>> Some minor suggestions - these kind of issues arise because the code >>>> should never call cursor.children(). >>>> >>>> I suggest getting rid of isFlattenable, and have an accessor which >>>> does this: >>>> >>>> Stream children(Cursor c) { >>>> return c.children() >>>> .filter(RecordLayoutComputer::isFlattenable) >>>> } >>>> >>>> But then, as I was writing this, I realized that >>>> RecordLayoutComputer >>>> seems to have several helper methods which take a Cursor as input, >>>> when in reality the only input we ever pass is the record cursor >>>> itself (which is already stashed in a field) - so maybe we should >>>> get >>>> rid of these extra Cursor params and use the field? >>>> >>>> Maurizio >>>> >>>> On 25/01/2019 12:26, Jorn Vernee wrote: >>>>> Hi, >>>>> >>>>> Based on JDK-8217664 [1], I have found the following bug with this >>>>> layout: >>>>> >>>>> ``` >>>>> struct Foo { >>>>> struct { >>>>> union { >>>>> int x; >>>>> } Bar; >>>>> }; >>>>> }; >>>>> ``` >>>>> >>>>> To find the offset of the anonymous struct in Foo, we have to find >>>>> the offset of it's first field. But, the current code is looking >>>>> for the first child _cursor_ of the anonymous struct, which is the >>>>> union declaration, and not the field 'Bar'. This again tries to >>>>> look up the first field of 'Bar' which is 'x', so we end up looking >>>>> up the field 'x' in Foo, which throws an error. >>>>> >>>>> The fix is simple, we just have to filter for anonymous >>>>> struct/union and FieldDecl child cursors when doing a recursive >>>>> offset lookup. I've also added an exception message to the >>>>> recursive lookup, which helped debug this. >>>>> >>>>> Please review the following. >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8217781 >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217781/webrev.00/ >>>>> >>>>> Thanks, >>>>> Jorn >>>>> >>>>> [1] : https://bugs.openjdk.java.net/browse/JDK-8217664 From sundararajan.athijegannathan at oracle.com Wed Jan 30 12:08:22 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Wed, 30 Jan 2019 17:38:22 +0530 Subject: [foreign] RFR 8217781: Filter child cursors when doing recursive offset lookup In-Reply-To: <6a7a395d456608e08f14498711226543@xs4all.nl> References: <481fd4e444e00a8cda38d1e7c18f294e@xs4all.nl> <496d5b3d-5bb9-bc04-1ebf-2b3784f344ed@oracle.com> <5a7368f419256b80bf7867d73c300de8@xs4all.nl> <5C5120E5.900@oracle.com> <6a7a395d456608e08f14498711226543@xs4all.nl> Message-ID: <5C5193B6.7070807@oracle.com> +1 -Sundar On 30/01/19, 4:58 PM, Jorn Vernee wrote: > Thanks, > > Updated webrev: > http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217781/webrev.02/ > > I also split the exception message I added in RecordLayoutComputer > since that was also pretty long. > > Jorn > > Sundararajan Athijegannathan schreef op 2019-01-30 04:58: >> +1. >> >> Good fix! subtle bug. >> >> Minor: Cursor.toString is a very long line. Perhaps it can folded >> into 2 lines. >> >> -Sundar >> >> On 30/01/19, 3:08 AM, Maurizio Cimadamore wrote: >>> Looks good - Sundar please also take a look >>> >>> Cheers >>> Maurizio >>> >>> On 25/01/2019 19:45, Jorn Vernee wrote: >>>>> But then, as I was writing this, I realized that RecordLayoutComputer >>>>> seems to have several helper methods which take a Cursor as input, >>>>> when in reality the only input we ever pass is the record cursor >>>>> itself (which is already stashed in a field) - so maybe we should get >>>>> rid of these extra Cursor params and use the field? >>>> >>>> This is not quite possible. `hasIncompleteArray` also calls these >>>> methods with the child cursors of anonymous nested structs and >>>> unions. So we can't just make the helper methods non-static and use >>>> the field. >>>> >>>> It's quite unfortunate the way the bug works. We have to check for >>>> incomplete array fields before trying to process _any_ of the >>>> fields, which means doing a recursive lookup into the anonymous >>>> structs and unions inside the type. >>>> >>>> But there is a simplification possible by replacing isFlattenable >>>> with the method you suggest (which I've named `flattenableChildren` >>>> to show the difference with `children()`) >>>> >>>> Updated webrev: >>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217781/webrev.01/ >>>> >>>> Cheer, >>>> Jorn >>>> >>>> Maurizio Cimadamore schreef op 2019-01-25 19:40: >>>>> Looks good, >>>>> Some minor suggestions - these kind of issues arise because the code >>>>> should never call cursor.children(). >>>>> >>>>> I suggest getting rid of isFlattenable, and have an accessor which >>>>> does this: >>>>> >>>>> Stream children(Cursor c) { >>>>> return c.children() >>>>> .filter(RecordLayoutComputer::isFlattenable) >>>>> } >>>>> >>>>> But then, as I was writing this, I realized that RecordLayoutComputer >>>>> seems to have several helper methods which take a Cursor as input, >>>>> when in reality the only input we ever pass is the record cursor >>>>> itself (which is already stashed in a field) - so maybe we should get >>>>> rid of these extra Cursor params and use the field? >>>>> >>>>> Maurizio >>>>> >>>>> On 25/01/2019 12:26, Jorn Vernee wrote: >>>>>> Hi, >>>>>> >>>>>> Based on JDK-8217664 [1], I have found the following bug with >>>>>> this layout: >>>>>> >>>>>> ``` >>>>>> struct Foo { >>>>>> struct { >>>>>> union { >>>>>> int x; >>>>>> } Bar; >>>>>> }; >>>>>> }; >>>>>> ``` >>>>>> >>>>>> To find the offset of the anonymous struct in Foo, we have to >>>>>> find the offset of it's first field. But, the current code is >>>>>> looking for the first child _cursor_ of the anonymous struct, >>>>>> which is the union declaration, and not the field 'Bar'. This >>>>>> again tries to look up the first field of 'Bar' which is 'x', so >>>>>> we end up looking up the field 'x' in Foo, which throws an error. >>>>>> >>>>>> The fix is simple, we just have to filter for anonymous >>>>>> struct/union and FieldDecl child cursors when doing a recursive >>>>>> offset lookup. I've also added an exception message to the >>>>>> recursive lookup, which helped debug this. >>>>>> >>>>>> Please review the following. >>>>>> >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8217781 >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~jvernee/panama/webrevs/8217781/webrev.00/ >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Jorn >>>>>> >>>>>> [1] : https://bugs.openjdk.java.net/browse/JDK-8217664 From jbvernee at xs4all.nl Wed Jan 30 12:10:47 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Wed, 30 Jan 2019 13:10:47 +0100 Subject: [foreign] RFR 8217989: Consolidate BoundedPointer implementation In-Reply-To: <2b353835-52ea-f6f7-e449-77f71f64bfc3@oracle.com> References: <2b353835-52ea-f6f7-e449-77f71f64bfc3@oracle.com> Message-ID: I agree that Scope and AccessMode belong on BoundedPointer as well. The 3 new helper methods in Util are only being used by BoundedPointer, should the implementation of those 3 methods be moved to BoundedPointer? AFAICT that would also allow you to remove/make private the toUnsafeBase and toUnsafeOffset accessors. Or at least; those 3 methods use BoundedPointer::toUnsafeBase and BoundedPointer::toUnsafeOffset, so imho they should also have 'unsafe' in their name since they don't add any safety. i.e.: Util::copyToUnafe, Util::getBitsUnsafe, and Util::putBitsUnsafe. Cheers, Jorn Maurizio Cimadamore schreef op 2019-01-29 15:38: > Hi, > in the light of recent discussions re. access modes, I decided to go > ahead and cleanup the split between BoundedPointer and > BoundedMemoryRegion, to avoid duplication of functionalities. > > This patch does two things: > > * moves Scope, AccessMode on BoundedPointer (where they belong, IMHO) > * reduce BoundedMemoryRegion to a simple helper class used to check > pointer bounds (and renames it to MemoryBoundInfo) > > I also took care of minor cleanups, such as adding better > encapsulation for pointer/array implementation fields. > > Webrev: > > http://cr.openjdk.java.net/~mcimadamore/panama/8217989/ > > Maurizio From jbvernee at xs4all.nl Wed Jan 30 12:15:56 2019 From: jbvernee at xs4all.nl (jbvernee at xs4all.nl) Date: Wed, 30 Jan 2019 12:15:56 +0000 Subject: hg: panama/dev: 8217781: jextract can not find field offset of field with inline struct/union decl. if it is nested in an anonymous struct/union Message-ID: <201901301215.x0UCFvV7002337@aojmv0008.oracle.com> Changeset: ba49e5d3ae23 Author: jvernee Date: 2019-01-30 13:14 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/ba49e5d3ae23 8217781: jextract can not find field offset of field with inline struct/union decl. if it is nested in an anonymous struct/union Summary: Filter child cursors when doing recursive offset lookup Reviewed-by: mcimadamore, sundar ! src/jdk.internal.clang/share/classes/jdk/internal/clang/Cursor.java ! src/jdk.jextract/share/classes/com/sun/tools/jextract/tree/RecordLayoutComputer.java ! test/jdk/com/sun/tools/jextract/JextractToolProviderTest.java ! test/jdk/com/sun/tools/jextract/nested.h From maurizio.cimadamore at oracle.com Wed Jan 30 12:20:15 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Wed, 30 Jan 2019 12:20:15 +0000 Subject: hg: panama/dev: Automatic merge with foreign Message-ID: <201901301220.x0UCKF0r003775@aojmv0008.oracle.com> Changeset: 9108ae93e749 Author: mcimadamore Date: 2019-01-30 13:20 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/9108ae93e749 Automatic merge with foreign From maurizio.cimadamore at oracle.com Wed Jan 30 12:19:09 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 30 Jan 2019 12:19:09 +0000 Subject: [foreign] RFR 8217989: Consolidate BoundedPointer implementation In-Reply-To: References: <2b353835-52ea-f6f7-e449-77f71f64bfc3@oracle.com> Message-ID: <5482275c-c48a-3e2c-572d-a321f31c2015@oracle.com> On 30/01/2019 12:10, Jorn Vernee wrote: > I agree that Scope and AccessMode belong on BoundedPointer as well. > > The 3 new helper methods in Util are only being used by > BoundedPointer, should the implementation of those 3 methods be moved > to BoundedPointer? AFAICT that would also allow you to remove/make > private the toUnsafeBase and toUnsafeOffset accessors. > > Or at least; those 3 methods use BoundedPointer::toUnsafeBase and > BoundedPointer::toUnsafeOffset, so imho they should also have 'unsafe' > in their name since they don't add any safety. i.e.: > Util::copyToUnafe, Util::getBitsUnsafe, and Util::putBitsUnsafe. Yeah - that seems like a good suggestion; I was moving them back and forth and I ended up there because Util already had a grab of a UNSAFE object. Maurizio > > Cheers, > Jorn > > Maurizio Cimadamore schreef op 2019-01-29 15:38: >> Hi, >> in the light of recent discussions re. access modes, I decided to go >> ahead and cleanup the split between BoundedPointer and >> BoundedMemoryRegion, to avoid duplication of functionalities. >> >> This patch does two things: >> >> * moves Scope, AccessMode on BoundedPointer (where they belong, IMHO) >> * reduce BoundedMemoryRegion to a simple helper class used to check >> pointer bounds (and renames it to MemoryBoundInfo) >> >> I also took care of minor cleanups, such as adding better >> encapsulation for pointer/array implementation fields. >> >> Webrev: >> >> http://cr.openjdk.java.net/~mcimadamore/panama/8217989/ >> >> Maurizio From maurizio.cimadamore at oracle.com Wed Jan 30 16:23:01 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 30 Jan 2019 16:23:01 +0000 Subject: [foreign] RFR 8217989: Consolidate BoundedPointer implementation In-Reply-To: <5482275c-c48a-3e2c-572d-a321f31c2015@oracle.com> References: <2b353835-52ea-f6f7-e449-77f71f64bfc3@oracle.com> <5482275c-c48a-3e2c-572d-a321f31c2015@oracle.com> Message-ID: <0cab7bd3-89a5-bb72-62b6-fc31144d776e@oracle.com> Updated webrev http://cr.openjdk.java.net/~mcimadamore/panama/8217989_v2/ Consolidated code in BoundedPointer and also cleaned up minor issues in Util (e.g. unused fields). Cheers Maurizio On 30/01/2019 12:19, Maurizio Cimadamore wrote: > > On 30/01/2019 12:10, Jorn Vernee wrote: >> I agree that Scope and AccessMode belong on BoundedPointer as well. >> >> The 3 new helper methods in Util are only being used by >> BoundedPointer, should the implementation of those 3 methods be moved >> to BoundedPointer? AFAICT that would also allow you to remove/make >> private the toUnsafeBase and toUnsafeOffset accessors. >> >> Or at least; those 3 methods use BoundedPointer::toUnsafeBase and >> BoundedPointer::toUnsafeOffset, so imho they should also have >> 'unsafe' in their name since they don't add any safety. i.e.: >> Util::copyToUnafe, Util::getBitsUnsafe, and Util::putBitsUnsafe. > > Yeah - that seems like a good suggestion; I was moving them back and > forth and I ended up there because Util already had a grab of a UNSAFE > object. > > Maurizio > >> >> Cheers, >> Jorn >> >> Maurizio Cimadamore schreef op 2019-01-29 15:38: >>> Hi, >>> in the light of recent discussions re. access modes, I decided to go >>> ahead and cleanup the split between BoundedPointer and >>> BoundedMemoryRegion, to avoid duplication of functionalities. >>> >>> This patch does two things: >>> >>> * moves Scope, AccessMode on BoundedPointer (where they belong, IMHO) >>> * reduce BoundedMemoryRegion to a simple helper class used to check >>> pointer bounds (and renames it to MemoryBoundInfo) >>> >>> I also took care of minor cleanups, such as adding better >>> encapsulation for pointer/array implementation fields. >>> >>> Webrev: >>> >>> http://cr.openjdk.java.net/~mcimadamore/panama/8217989/ >>> >>> Maurizio From adam.pocock at oracle.com Wed Jan 30 16:31:39 2019 From: adam.pocock at oracle.com (Adam Pocock) Date: Wed, 30 Jan 2019 11:31:39 -0500 Subject: [vector] Error message in the fromArray length check is misleading Message-ID: I'm vectorising a simple application which operates on fixed length arrays, and due to a mistake in the way I was iterating the arrays I kept triggering the IndexOutOfBoundsException. The message for this exception is pretty misleading, as all my vectors are of length 50, but the error message says: java.lang.IndexOutOfBoundsException: Index 48 out of bounds for length 47 ??? at java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:64) ??? at java.base/jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:70) ??? at java.base/jdk.internal.util.Preconditions.checkIndex(Preconditions.java:248) ??? at java.base/java.util.Objects.checkIndex(Objects.java:372) ??? at jdk.incubator.vector/jdk.incubator.vector.VectorIntrinsics.checkIndex(VectorIntrinsics.java:405) ??? at jdk.incubator.vector/jdk.incubator.vector.Double256Vector$Double256Species.fromArray(Double256Vector.java:1433) I realise my code is blowing past the end of the array & triggering the exception (which is good, as my code was wrong), but I feel like the message should say "Index 50 is out of bounds for length 50", or something similar, rather than saying my array is of a different length to what's actually allocated. Maybe in VectorIntrinsics line 405 ??? case 2: return Objects.checkIndex(ix, length - (vlen - 1)); it could be modified to check if ix + vlen is below length, rather than if ix is below length - (vlen -1)? Thanks, Adam -- Adam Pocock Principal Member of Technical Staff Machine Learning Research Group Oracle Labs, Burlington, MA From jbvernee at xs4all.nl Wed Jan 30 17:05:06 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Wed, 30 Jan 2019 18:05:06 +0100 Subject: [foreign] RFR 8217989: Consolidate BoundedPointer implementation In-Reply-To: <0cab7bd3-89a5-bb72-62b6-fc31144d776e@oracle.com> References: <2b353835-52ea-f6f7-e449-77f71f64bfc3@oracle.com> <5482275c-c48a-3e2c-572d-a321f31c2015@oracle.com> <0cab7bd3-89a5-bb72-62b6-fc31144d776e@oracle.com> Message-ID: Looks good! Jorn Maurizio Cimadamore schreef op 2019-01-30 17:23: > Updated webrev > > http://cr.openjdk.java.net/~mcimadamore/panama/8217989_v2/ > > Consolidated code in BoundedPointer and also cleaned up minor issues > in Util (e.g. unused fields). > > Cheers > Maurizio > > On 30/01/2019 12:19, Maurizio Cimadamore wrote: >> >> On 30/01/2019 12:10, Jorn Vernee wrote: >>> I agree that Scope and AccessMode belong on BoundedPointer as well. >>> >>> The 3 new helper methods in Util are only being used by >>> BoundedPointer, should the implementation of those 3 methods be moved >>> to BoundedPointer? AFAICT that would also allow you to remove/make >>> private the toUnsafeBase and toUnsafeOffset accessors. >>> >>> Or at least; those 3 methods use BoundedPointer::toUnsafeBase and >>> BoundedPointer::toUnsafeOffset, so imho they should also have >>> 'unsafe' in their name since they don't add any safety. i.e.: >>> Util::copyToUnafe, Util::getBitsUnsafe, and Util::putBitsUnsafe. >> >> Yeah - that seems like a good suggestion; I was moving them back and >> forth and I ended up there because Util already had a grab of a UNSAFE >> object. >> >> Maurizio >> >>> >>> Cheers, >>> Jorn >>> >>> Maurizio Cimadamore schreef op 2019-01-29 15:38: >>>> Hi, >>>> in the light of recent discussions re. access modes, I decided to go >>>> ahead and cleanup the split between BoundedPointer and >>>> BoundedMemoryRegion, to avoid duplication of functionalities. >>>> >>>> This patch does two things: >>>> >>>> * moves Scope, AccessMode on BoundedPointer (where they belong, >>>> IMHO) >>>> * reduce BoundedMemoryRegion to a simple helper class used to check >>>> pointer bounds (and renames it to MemoryBoundInfo) >>>> >>>> I also took care of minor cleanups, such as adding better >>>> encapsulation for pointer/array implementation fields. >>>> >>>> Webrev: >>>> >>>> http://cr.openjdk.java.net/~mcimadamore/panama/8217989/ >>>> >>>> Maurizio From henry.jen at oracle.com Wed Jan 30 18:51:41 2019 From: henry.jen at oracle.com (Henry Jen) Date: Wed, 30 Jan 2019 10:51:41 -0800 Subject: [foreign] RFR 8217989: Consolidate BoundedPointer implementation In-Reply-To: <0cab7bd3-89a5-bb72-62b6-fc31144d776e@oracle.com> References: <2b353835-52ea-f6f7-e449-77f71f64bfc3@oracle.com> <5482275c-c48a-3e2c-572d-a321f31c2015@oracle.com> <0cab7bd3-89a5-bb72-62b6-fc31144d776e@oracle.com> Message-ID: <6419E5FE-A7F6-4039-AFE9-55CE5AD21A01@oracle.com> Changes looks good to me. One minor nit, Pointer.java:215 can use a space after cast as line above. Cheers, Henry > On Jan 30, 2019, at 8:23 AM, Maurizio Cimadamore wrote: > > Updated webrev > > http://cr.openjdk.java.net/~mcimadamore/panama/8217989_v2/ > > Consolidated code in BoundedPointer and also cleaned up minor issues in Util (e.g. unused fields). > > Cheers > Maurizio > > On 30/01/2019 12:19, Maurizio Cimadamore wrote: >> >> On 30/01/2019 12:10, Jorn Vernee wrote: >>> I agree that Scope and AccessMode belong on BoundedPointer as well. >>> >>> The 3 new helper methods in Util are only being used by BoundedPointer, should the implementation of those 3 methods be moved to BoundedPointer? AFAICT that would also allow you to remove/make private the toUnsafeBase and toUnsafeOffset accessors. >>> >>> Or at least; those 3 methods use BoundedPointer::toUnsafeBase and BoundedPointer::toUnsafeOffset, so imho they should also have 'unsafe' in their name since they don't add any safety. i.e.: Util::copyToUnafe, Util::getBitsUnsafe, and Util::putBitsUnsafe. >> >> Yeah - that seems like a good suggestion; I was moving them back and forth and I ended up there because Util already had a grab of a UNSAFE object. >> >> Maurizio >> >>> >>> Cheers, >>> Jorn >>> >>> Maurizio Cimadamore schreef op 2019-01-29 15:38: >>>> Hi, >>>> in the light of recent discussions re. access modes, I decided to go >>>> ahead and cleanup the split between BoundedPointer and >>>> BoundedMemoryRegion, to avoid duplication of functionalities. >>>> >>>> This patch does two things: >>>> >>>> * moves Scope, AccessMode on BoundedPointer (where they belong, IMHO) >>>> * reduce BoundedMemoryRegion to a simple helper class used to check >>>> pointer bounds (and renames it to MemoryBoundInfo) >>>> >>>> I also took care of minor cleanups, such as adding better >>>> encapsulation for pointer/array implementation fields. >>>> >>>> Webrev: >>>> >>>> http://cr.openjdk.java.net/~mcimadamore/panama/8217989/ >>>> >>>> Maurizio From richard at openkappa.co.uk Wed Jan 30 20:47:28 2019 From: richard at openkappa.co.uk (Richard Startin) Date: Wed, 30 Jan 2019 20:47:28 +0000 Subject: [vector] Partial permutations In-Reply-To: <9970C4BC-FDC8-49A4-93FE-1B9F72282DFC@oracle.com> References: , <9970C4BC-FDC8-49A4-93FE-1B9F72282DFC@oracle.com> Message-ID: Yes, that's the gist of it. The aim is to avoid horizontal reductions and do as much accumulation in parallel as possible. You are also correct that the upper 16 bytes in counts is superfluous, but C/C++ programmers seem to do this a lot because vpshufb is so cheap. You are correct that there is a bug when all bits are set - there are two ways round it: change the block to 128 or unroll the inner loop so there are two low accumulators and two high accumulators, which avoids overflow. I didn't get round to deciding which way to go before I realised that ByteVector.rearrange wasn't compiling to anything exciting yet (for AVX2 at least). I look forward to revisiting this calculation in the future. The point of sharing this example is not just that I really hope it will one day be possible to use the API to count bits quickly (in which case it would probably be even better implemented within the API, using the AVX-512 vpopcnt extension when available!), but to draw attention to an existing idiom that I suspect may not be served well by the semantics of Vector.rearrange, even if it is currently the most appropriate approximation in the current API. I leave it to the experts to think about whether or how to make this possible, but I hope that it is helpful to draw attention to the kinds of things that would occur to a potential user to implement with the API. It wouldn't matter much to me as a potential user whether this would be expressed as a gather or as a permutation, or even if there were a secret incantation expressible in terms of masks: I just anticipate the need to do parallel lookups without undue overhead. Regarding the SWAR hacks, there's a reason for them: lack of unsigned arithmetic. I don't think it is possible to implement the horizontal reduction in terms of signed byte arithmetic. I think that lack of unsigned arithmetic is another obstacle to lifting and shifting existing vector algorithms to the vector API. I have previously tried and failed to implement StreamVByte compression (which incidentally also requires similar permutations and species changes) with the vector API, for lack of unsigned 8 bit arithmetic. ________________________________ From: John Rose Sent: 30 January 2019 01:34:53 To: Richard Startin Cc: panama-dev at openjdk.java.net Subject: Re: [vector] Partial permutations On Jan 29, 2019, at 2:44 PM, Richard Startin > wrote: lo = lo.add(counts.rearrange(v1.and(0x0F0F0F0F).rebracket(YMM_BYTE).toShuffle()).rebracket(YMM_INT)); Breaking it down: var v1 = ?; // u4[8] var nibblePerByte = v1.and(0x0F0F0F0F).rebracket(YMM_BYTE); // u1[32] var bitCountPerByte = counts.rearrange(nibblePerByte.toShuffle()); // u1[32] var swarLanes = bitCountPerByte.rebracket(YMM_INT); // u4[8] lo = lo.add(expandedCounts); // danger: carry-out across bytes? The second line zeroes out half of the bit positions and breaks out the 8 int lanes into 32 byte lanes. The third line replaces each byte (whose value is in 0..15) with another byte that gives the bit-count (value in 0..4). The fourth line undoes the action of the second line. Now we have four lanes each of which is an integer containing 4 "SIMD across a register" sub-lanes. It seems like the top half of your NIBBLE_COUNTS table could be any old garbage, not a copy of the bottom half. (Did I get it right? This kind of code is tricky!) The final line adds 8x4 lanes and their sub-lanes. I think you have a bug in your block size, since if all the input bits are set, then each nibble count will always be 4, and so each sub-lane count will accumulate that value, 256/4 times, which will cause a carry to propagate across your SWAR sub-lanes. The idiom counts.rearrange(nibblePerByte.toShuffle()) would read better, in this algorithm, as nibblePerByte.selectFrom(counts). It's really akin to a gather operation in this use case, and the similarity could be made more clear by using a gather-like API (which we are still working on). To respond to your question: If I follow you, you note that since the top half of the NIBBLE_COUNTS array is never used, then any instructions which would do the work of loading from that half of the array are useless. To me, this calls the question of how to do strength-reduction optimizations on shuffles. I don't think we want explicit API points for simplified shuffles, any more than we want library routines or operators for division when both operands are positive, even though there are optimizations relevant in both cases. The best way to get improved code for simplified inputs is to work hard to detect those inputs and adjust the instruction selection (or IR generation) accordingly. In this case, the shuffle vector is the logical "and" of 0x0F and some random input, which means that, ideally the optimizer should be able to prove that the selected lanes of "counts" are in the range 0..15. That's probably enough to simplify a complex shuffle idiom in the back end. Failing lanewise type analysis (but why should we fail that?), a pattern match in the C2 backend might also detect that the shuffle was immediately derived from a masking operation. In either case, the optimization might be confounded by the SWAR hacks going on. Perhaps the right answer is to rebracket to YMM_BYTE before the logical "and" (of a byte 0x0F). This is sadly delicate, but can be made more robust by an improvement to the C2 type system to track individual bit positions, or adoption of Graal, which already has that feature. The overall point here is that some of our good code will come from a strength reduction of expensive operations, gated by type and pattern analysis. The design problem is to put enough primitives into the user API but not too many, and enough optimization in the backend. ? John From brian.goetz at oracle.com Wed Jan 30 20:58:52 2019 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 30 Jan 2019 15:58:52 -0500 Subject: [vector] Vector API -- alignment with value types In-Reply-To: References: Message-ID: <5745a703-c239-8487-6835-7c1aadd54d0f@oracle.com> > Part I > ------ > > Here's an idea for simplifying Species, which is: let's drive Species > down to be a simple class that is really just a constant holder for > (element type, shape), and move all the behavior to static methods on > XxxVector. I've started prototyping this, in a rather slash-and-burn manner.? I am about halfway through.? So far, its working, the set of changes I had to make to client code is very small (almost all transforming species.doSomething(args) to XxxVector.doSomething(species, args)). The question, of course, is whether the intrinsification will all survive the transformation.? The most common case is that I have transformed vector intrinsic calls from ??? VectorIntrinsic.blah(Int128Vector.class, ...) to ??? VectorIntrinsics.blah(species.boxType(), ...) The basic assumption is that, under the same conditions that we get inlining now, we'll know the species is a constant, and boxType() will just inline to a concrete box type anyway.? (The goal is to get species to be values, but in the meantime, they can be enum constants.)? This should work, but we'll likely have to do some JIT work to get back to where we see all the inlining and intrinsification.? (Much of this would come free in Valhalla.) There are a few cases where we can't just do the above, and have to do a switch in the lifted method, such as: public static Shuffle shuffle(ByteSpecies species, IntUnaryOperator f) { if (species.boxType() == ByteMaxVector.class) return new ByteMaxVector.ByteMaxShuffle(f); switch (species.bitSize()) { case 64:return new Byte64Vector.Byte64Shuffle(f); case 128:return new Byte128Vector.Byte128Shuffle(f); case 256:return new Byte256Vector.Byte256Shuffle(f); case 512:return new Byte512Vector.Byte512Shuffle(f); default:throw new IllegalArgumentException(Integer.toString(species.bitSize())); } } Because again, species is a constant, this should also just inline down to the right thing.? So far, other than reshape/rebracket, I haven't seen anything requiring more complicated transformations. The only code I found so far that tries to be agnostic to shape and size both is VectorResizeTest; there are strategies for making this work without the combinatorial automated specialization, so I don't see this as a big impediment. Where this leads to is: ?- Vector.Species becomes an interface with a handful of methods (bitSize, etc), and quite possibly one that no one uses directly; ?- IntSpecies and friends become enums, with enum constants for I64, I128, etc (or, values, with static constants for the values); ?- The specialized classes for XxxNnVector.XxxNnSpecies _go away_; ?- Users need not learn about species at all, but if they do care, they are just simple data constants that get fed through the API. I'm not done (and am traveling the next two weeks), but I think I've made progress validating the API transformation.? The real question, then, is when do we do this.? I think it would be best to do before previewing, simply because it is such an intrusive refactoring. But, we'd have to evaluate whether we can mitigate the impact in time. From brian.goetz at oracle.com Wed Jan 30 21:54:08 2019 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 30 Jan 2019 16:54:08 -0500 Subject: [vector] Vector API -- alignment with value types In-Reply-To: <5745a703-c239-8487-6835-7c1aadd54d0f@oracle.com> References: <5745a703-c239-8487-6835-7c1aadd54d0f@oracle.com> Message-ID: I've tried to attach the patches-in-progress for these.? There are two, the speciesFactories.patch I set recently, and then zero.patch.? These only include changes to Vector and the templates; you'll have to re-run the gen-* scripts to get changes to the rest.? They apply to the last version for which the tests pass. Perhaps someone could look at the generated code from typical operations and see how badly this approach perturbs it? On 1/30/2019 3:58 PM, Brian Goetz wrote: > > >> Part I >> ------ >> >> Here's an idea for simplifying Species, which is: let's drive Species >> down to be a simple class that is really just a constant holder for >> (element type, shape), and move all the behavior to static methods on >> XxxVector. > > I've started prototyping this, in a rather slash-and-burn manner. I am > about halfway through.? So far, its working, the set of changes I had > to make to client code is very small (almost all transforming > species.doSomething(args) to XxxVector.doSomething(species, args)). > > The question, of course, is whether the intrinsification will all > survive the transformation.? The most common case is that I have > transformed vector intrinsic calls from > > ??? VectorIntrinsic.blah(Int128Vector.class, ...) > > to > > ??? VectorIntrinsics.blah(species.boxType(), ...) > > The basic assumption is that, under the same conditions that we get > inlining now, we'll know the species is a constant, and boxType() will > just inline to a concrete box type anyway.? (The goal is to get > species to be values, but in the meantime, they can be enum > constants.)? This should work, but we'll likely have to do some JIT > work to get back to where we see all the inlining and > intrinsification.? (Much of this would come free in Valhalla.) > > There are a few cases where we can't just do the above, and have to do > a switch in the lifted method, such as: > > public static Shuffle shuffle(ByteSpecies species, > IntUnaryOperator f) { > ??? if (species.boxType() == ByteMaxVector.class) > ??????? return new ByteMaxVector.ByteMaxShuffle(f); > ??? switch (species.bitSize()) { > ??????? case 64:return new Byte64Vector.Byte64Shuffle(f); > ??????? case 128:return new Byte128Vector.Byte128Shuffle(f); > ??????? case 256:return new Byte256Vector.Byte256Shuffle(f); > ??????? case 512:return new Byte512Vector.Byte512Shuffle(f); > ??????? default:throw new > IllegalArgumentException(Integer.toString(species.bitSize())); > ??? } > } > > > Because again, species is a constant, this should also just inline > down to the right thing.? So far, other than reshape/rebracket, I > haven't seen anything requiring more complicated transformations. > > The only code I found so far that tries to be agnostic to shape and > size both is VectorResizeTest; there are strategies for making this > work without the combinatorial automated specialization, so I don't > see this as a big impediment. > > Where this leads to is: > ?- Vector.Species becomes an interface with a handful of methods > (bitSize, etc), and quite possibly one that no one uses directly; > ?- IntSpecies and friends become enums, with enum constants for I64, > I128, etc (or, values, with static constants for the values); > ?- The specialized classes for XxxNnVector.XxxNnSpecies _go away_; > ?- Users need not learn about species at all, but if they do care, > they are just simple data constants that get fed through the API. > > I'm not done (and am traveling the next two weeks), but I think I've > made progress validating the API transformation.? The real question, > then, is when do we do this.? I think it would be best to do before > previewing, simply because it is such an intrusive refactoring. But, > we'd have to evaluate whether we can mitigate the impact in time. > > -------------- next part -------------- # HG changeset patch # Parent d3b456d84b4ed3b6f4a15a09f9e26433b0d91bd5 diff -r d3b456d84b4e -r 8ab9aad380d0 src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Vector.java --- a/src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Vector.java Wed Jan 09 22:09:11 2019 +0100 +++ b/src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Vector.java Tue Jan 29 16:15:44 2019 -0500 @@ -74,9 +74,9 @@ * factory called a {@link Species}. A Species has an * element type and shape and creates Vector values of the same element type * and shape. - * A species can be {@link #species(Class, Shape)} obtained} given an element + * A species can be {@link Species#of(Class, Shape)} obtained} given an element * type and shape, or a preferred species can be - * {@link #preferredSpecies obtained} given just an element type where the most + * {@link Species#ofPreferred(Class)} obtained} given just an element type where the most * optimal shape is selected for the current platform. It is recommended that * Species instances be held in {@code static final} fields for optimal creation * and usage of Vector values by the runtime compiler. @@ -908,6 +908,32 @@ int length(Species s) { return bitSize() / s.elementSize(); } + + /** + * Finds appropriate shape depending on bitsize. + * + * @param bitSize the size in bits + * @return the shape corresponding to bitsize + * @see #bitSize + */ + public static Shape forBitSize(int bitSize) { + switch (bitSize) { + case 64: + return Shape.S_64_BIT; + case 128: + return Shape.S_128_BIT; + case 256: + return Shape.S_256_BIT; + case 512: + return Shape.S_512_BIT; + default: + if ((bitSize > 0) && (bitSize <= 2048) && (bitSize % 128 == 0)) { + return Shape.S_Max_BIT; + } else { + throw new IllegalArgumentException("Bad vector bit size: " + bitSize); + } + } + } } @@ -962,6 +988,65 @@ // Factory /** + * Finds a species for an element type and shape. + * + * @param c the element type + * @param s the shape + * @param the boxed element type + * @return a species for an element type and shape + * @throws IllegalArgumentException if no such species exists for the + * element type and/or shape + */ + @SuppressWarnings("unchecked") + public static Vector.Species of(Class c, Shape s) { + if (c == float.class) { + return (Vector.Species) FloatVector.species(s); + } + else if (c == double.class) { + return (Vector.Species) DoubleVector.species(s); + } + else if (c == byte.class) { + return (Vector.Species) ByteVector.species(s); + } + else if (c == short.class) { + return (Vector.Species) ShortVector.species(s); + } + else if (c == int.class) { + return (Vector.Species) IntVector.species(s); + } + else if (c == long.class) { + return (Vector.Species) LongVector.species(s); + } + else { + throw new IllegalArgumentException("Bad vector element type: " + c.getName()); + } + } + + /** + * Finds a preferred species for an element type. + *

+ * A preferred species is a species chosen by the platform that has a + * shape of maximal bit size. A preferred species for different element + * types will have the same shape, and therefore vectors created from + * such species will be shape compatible. + * + * @param c the element type + * @param the boxed element type + * @return a preferred species for an element type + * @throws IllegalArgumentException if no such species exists for the + * element type + */ + @SuppressWarnings("unchecked") + public static Vector.Species ofPreferred(Class c) { + Unsafe u = Unsafe.getUnsafe(); + + int vectorLength = u.getMaxVectorSize(c); + int vectorBitSize = bitSizeForVectorLength(c, vectorLength); + Shape s = Shape.forBitSize(vectorBitSize); + return Species.of(c, s); + } + + /** * Returns a vector where all lane elements are set to the default * primitive value. * @@ -1726,30 +1811,6 @@ } /** - * Finds a preferred species for an element type. - *

- * A preferred species is a species chosen by the platform that has a - * shape of maximal bit size. A preferred species for different element - * types will have the same shape, and therefore vectors created from - * such species will be shape compatible. - * - * @param c the element type - * @param the boxed element type - * @return a preferred species for an element type - * @throws IllegalArgumentException if no such species exists for the - * element type - */ - @SuppressWarnings("unchecked") - public static Vector.Species preferredSpecies(Class c) { - Unsafe u = Unsafe.getUnsafe(); - - int vectorLength = u.getMaxVectorSize(c); - int vectorBitSize = bitSizeForVectorLength(c, vectorLength); - Shape s = shapeForVectorBitSize(vectorBitSize); - return species(c, s); - } - - /** * Find bit size based on element type and number of elements. * * @param c the element type @@ -1779,65 +1840,4 @@ throw new IllegalArgumentException("Bad vector type: " + c.getName()); } } - - /** - * Finds appropriate shape depending on bitsize. - * - * @param bitSize the size in bits - * @return the shape corresponding to bitsize - * @see #bitSize - */ - public static Shape shapeForVectorBitSize(int bitSize) { - switch (bitSize) { - case 64: - return Shape.S_64_BIT; - case 128: - return Shape.S_128_BIT; - case 256: - return Shape.S_256_BIT; - case 512: - return Shape.S_512_BIT; - default: - if ((bitSize > 0) && (bitSize <= 2048) && (bitSize % 128 == 0)) { - return Shape.S_Max_BIT; - } else { - throw new IllegalArgumentException("Bad vector bit size: " + bitSize); - } - } - } - - /** - * Finds a species for an element type and shape. - * - * @param c the element type - * @param s the shape - * @param the boxed element type - * @return a species for an element type and shape - * @throws IllegalArgumentException if no such species exists for the - * element type and/or shape - */ - @SuppressWarnings("unchecked") - public static Vector.Species species(Class c, Shape s) { - if (c == float.class) { - return (Vector.Species) FloatVector.species(s); - } - else if (c == double.class) { - return (Vector.Species) DoubleVector.species(s); - } - else if (c == byte.class) { - return (Vector.Species) ByteVector.species(s); - } - else if (c == short.class) { - return (Vector.Species) ShortVector.species(s); - } - else if (c == int.class) { - return (Vector.Species) IntVector.species(s); - } - else if (c == long.class) { - return (Vector.Species) LongVector.species(s); - } - else { - throw new IllegalArgumentException("Bad vector element type: " + c.getName()); - } - } } diff -r d3b456d84b4e -r 8ab9aad380d0 src/jdk.incubator.vector/share/classes/jdk/incubator/vector/X-Vector.java.template --- a/src/jdk.incubator.vector/share/classes/jdk/incubator/vector/X-Vector.java.template Wed Jan 09 22:09:11 2019 +0100 +++ b/src/jdk.incubator.vector/share/classes/jdk/incubator/vector/X-Vector.java.template Tue Jan 29 16:15:44 2019 -0500 @@ -2240,7 +2240,7 @@ */ @SuppressWarnings("unchecked") public static $Type$Species preferredSpecies() { - return ($Type$Species) Vector.preferredSpecies($type$.class); + return ($Type$Species) Species.ofPreferred($type$.class); } /** diff -r d3b456d84b4e -r 8ab9aad380d0 src/jdk.incubator.vector/share/classes/jdk/incubator/vector/X-VectorBits.java.template --- a/src/jdk.incubator.vector/share/classes/jdk/incubator/vector/X-VectorBits.java.template Wed Jan 09 22:09:11 2019 +0100 +++ b/src/jdk.incubator.vector/share/classes/jdk/incubator/vector/X-VectorBits.java.template Tue Jan 29 16:15:44 2019 -0500 @@ -52,8 +52,8 @@ private static final IntVector.IntSpecies INDEX_SPEC; static { int bitSize = Vector.bitSizeForVectorLength(int.class, LENGTH); - Vector.Shape shape = Vector.shapeForVectorBitSize(bitSize); - INDEX_SPEC = (IntVector.IntSpecies) Vector.species(int.class, shape); + Vector.Shape shape = Shape.forBitSize(bitSize); + INDEX_SPEC = (IntVector.IntSpecies) Species.of(int.class, shape); } #end[!longOrDouble64] diff -r d3b456d84b4e -r 8ab9aad380d0 test/jdk/jdk/incubator/vector/PreferredSpeciesTest.java --- a/test/jdk/jdk/incubator/vector/PreferredSpeciesTest.java Wed Jan 09 22:09:11 2019 +0100 +++ b/test/jdk/jdk/incubator/vector/PreferredSpeciesTest.java Tue Jan 29 16:15:44 2019 -0500 @@ -51,7 +51,7 @@ @Test(dataProvider = "classesProvider") void testVectorLength(Class c) { Vector.Species species = - Vector.preferredSpecies(c); + Vector.Species.ofPreferred(c); Assert.assertEquals(species.length(), U.getMaxVectorSize(c)); } -------------- next part -------------- # HG changeset patch # Parent 8ab9aad380d06fcc6831aca7721ee6c5efe9b240 diff --git a/src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Vector.java b/src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Vector.java --- a/src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Vector.java +++ b/src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Vector.java @@ -955,6 +955,20 @@ public abstract Class elementType(); /** + * Returns the vector box type for this species + * + * @return the box type + */ + abstract Class boxType(); + + /** + * Returns the vector mask type for this species + * + * @return the box type + */ + abstract Class maskType(); + + /** * Returns the element size, in bits, of vectors produced by this * species. * @@ -1054,236 +1068,6 @@ */ public abstract Vector zero(); - /** - * Loads a vector from a byte array starting at an offset. - *

- * Bytes are composed into primitive lane elements according to the - * native byte order of the underlying platform - *

- * This method behaves as if it returns the result of calling the - * byte buffer, offset, and mask accepting - * {@link #fromByteBuffer(ByteBuffer, int, Mask) method} as follows: - *

{@code
-         * return this.fromByteBuffer(ByteBuffer.wrap(a), i, this.maskAllTrue());
-         * }
- * - * @param a the byte array - * @param i the offset into the array - * @return a vector loaded from a byte array - * @throws IndexOutOfBoundsException if {@code i < 0} or - * {@code i > a.length - (this.length() * this.elementSize() / Byte.SIZE)} - */ - public abstract Vector fromByteArray(byte[] a, int i); - - /** - * Loads a vector from a byte array starting at an offset and using a - * mask. - *

- * Bytes are composed into primitive lane elements according to the - * native byte order of the underlying platform. - *

- * This method behaves as if it returns the result of calling the - * byte buffer, offset, and mask accepting - * {@link #fromByteBuffer(ByteBuffer, int, Mask) method} as follows: - *

{@code
-         * return this.fromByteBuffer(ByteBuffer.wrap(a), i, m);
-         * }
- * - * @param a the byte array - * @param i the offset into the array - * @param m the mask - * @return a vector loaded from a byte array - * @throws IndexOutOfBoundsException if {@code i < 0} or - * {@code i > a.length - (this.length() * this.elementSize() / Byte.SIZE)} - * @throws IndexOutOfBoundsException if the offset is {@code < 0}, - * or {@code > a.length}, - * for any vector lane index {@code N} where the mask at lane {@code N} - * is set - * {@code i >= a.length - (N * this.elementSize() / Byte.SIZE)} - */ - public abstract Vector fromByteArray(byte[] a, int i, Mask m); - - /** - * Loads a vector from a {@link ByteBuffer byte buffer} starting at an - * offset into the byte buffer. - *

- * Bytes are composed into primitive lane elements according to the - * native byte order of the underlying platform. - *

- * This method behaves as if it returns the result of calling the - * byte buffer, offset, and mask accepting - * {@link #fromByteBuffer(ByteBuffer, int, Mask)} method} as follows: - *

{@code
-         *   return this.fromByteBuffer(b, i, this.maskAllTrue())
-         * }
- * - * @param b the byte buffer - * @param i the offset into the byte buffer - * @return a vector loaded from a byte buffer - * @throws IndexOutOfBoundsException if the offset is {@code < 0}, - * or {@code > b.limit()}, - * or if there are fewer than - * {@code this.length() * this.elementSize() / Byte.SIZE} bytes - * remaining in the byte buffer from the given offset - */ - public abstract Vector fromByteBuffer(ByteBuffer b, int i); - - /** - * Loads a vector from a {@link ByteBuffer byte buffer} starting at an - * offset into the byte buffer and using a mask. - *

- * This method behaves as if the byte buffer is viewed as a primitive - * {@link java.nio.Buffer buffer} for the primitive element type, - * according to the native byte order of the underlying platform, and - * the returned vector is loaded with a mask from a primitive array - * obtained from the primitive buffer. - * The following pseudocode expresses the behaviour, where - * {@coce EBuffer} is the primitive buffer type, {@code e} is the - * primitive element type, and {@code ESpecies} is the primitive - * species for {@code e}: - *

{@code
-         * EBuffer eb = b.duplicate().
-         *     order(ByteOrder.nativeOrder()).position(i).
-         *     asEBuffer();
-         * e[] es = new e[this.length()];
-         * for (int n = 0; n < t.length; n++) {
-         *     if (m.isSet(n))
-         *         es[n] = eb.get(n);
-         * }
-         * Vector r = ((ESpecies)this).fromArray(es, 0, m);
-         * }
- * - * @param b the byte buffer - * @param i the offset into the byte buffer - * @return a vector loaded from a byte buffer - * @throws IndexOutOfBoundsException if the offset is {@code < 0}, - * or {@code > b.limit()}, - * for any vector lane index {@code N} where the mask at lane {@code N} - * is set - * {@code i >= b.limit() - (N * this.elementSize() / Byte.SIZE)} - */ - public abstract Vector fromByteBuffer(ByteBuffer b, int i, Mask m); - - //Mask and shuffle constructions - - /** - * Returns a mask where each lane is set or unset according to a given - * {@code boolean} value. - *

- * For each mask lane, where {@code N} is the mask lane index, - * if the given {@code boolean} value at index {@code N} is {@code true} - * then the mask lane at index {@code N} is set, otherwise it is unset. - * - * @param bits the given {@code boolean} values - * @return a mask where each lane is set or unset according to a given - * {@code boolean} value - * @throws IndexOutOfBoundsException if {@code bits.length < this.length()} - */ - public abstract Mask maskFromValues(boolean... bits); - - /** - * Loads a mask from a {@code boolean} array starting at an offset. - *

- * For each mask lane, where {@code N} is the mask lane index, - * if the array element at index {@code i + N} is {@code true} then the - * mask lane at index {@code N} is set, otherwise it is unset. - * - * @param a the {@code boolean} array - * @param i the offset into the array - * @return the mask loaded from a {@code boolean} array - * @throws IndexOutOfBoundsException if {@code i < 0}, or - * {@code i > a.length - this.length()} - */ - public abstract Mask maskFromArray(boolean[] a, int i); - - /** - * Returns a mask where all lanes are a set. - * - * @return a mask where all lanes are a set - */ - public abstract Mask maskAllTrue(); - - /** - * Returns a mask where all lanes are unset. - * - * @return a mask where all lanes are unset - */ - public abstract Mask maskAllFalse(); - - /** - * Returns a shuffle of mapped indexes where each lane element is - * the result of applying a mapping function to the corresponding lane - * index. - *

- * Care should be taken to ensure Shuffle values produced from this - * method are consumed as constants to ensure optimal generation of - * code. For example, values held in static final fields or values - * held in loop constant local variables. - *

- * This method behaves as if a shuffle is created from an array of - * mapped indexes as follows: - *

{@code
-         *   int[] a = new int[this.length()];
-         *   for (int i = 0; i < a.length; i++) {
-         *       a[i] = f.applyAsInt(i);
-         *   }
-         *   return this.shuffleFromValues(a);
-         * }
- * - * @param f the lane index mapping function - * @return a shuffle of mapped indexes. - */ - public abstract Shuffle shuffle(IntUnaryOperator f); - - /** - * Returns a shuffle where each lane element is the value of its - * corresponding lane index. - *

- * This method behaves as if a shuffle is created from an identity - * index mapping function as follows: - *

{@code
-         *   return this.shuffle(i -> i);
-         * }
- * - * @return a shuffle of lane indexes. - */ - public abstract Shuffle shuffleIota(); - - /** - * Returns a shuffle where each lane element is set to a given - * {@code int} value logically AND'ed by the species length minus one. - *

- * For each shuffle lane, where {@code N} is the shuffle lane index, the - * the {@code int} value at index {@code N} logically AND'ed by - * {@code this.length() - 1} is placed into the resulting shuffle at - * lane index {@code N}. - * - * @param indexes the given {@code int} values - * @return a shuffle where each lane element is set to a given - * {@code int} value - * @throws IndexOutOfBoundsException if the number of int values is - * {@code < this.length()}. - */ - public abstract Shuffle shuffleFromValues(int... indexes); - - /** - * Loads a shuffle from an {@code int} array starting at offset. - *

- * For each shuffle lane, where {@code N} is the shuffle lane index, the - * array element at index {@code i + N} logically AND'ed by - * {@code this.length() - 1} is placed into the resulting shuffle at lane - * index {@code N}. - * - * @param a the {@code int} array - * @param i the offset into the array - * @return a shuffle loaded from an {@code int} array - * @throws IndexOutOfBoundsException if {@code i < 0}, or - * {@code i > a.length - this.length()} - */ - public abstract Shuffle shuffleFromArray(int[] a, int i); - - // Shuffle iota, 0...N - // Vector type/shape transformations /** diff --git a/src/jdk.incubator.vector/share/classes/jdk/incubator/vector/X-Vector.java.template b/src/jdk.incubator.vector/share/classes/jdk/incubator/vector/X-Vector.java.template --- a/src/jdk.incubator.vector/share/classes/jdk/incubator/vector/X-Vector.java.template +++ b/src/jdk.incubator.vector/share/classes/jdk/incubator/vector/X-Vector.java.template @@ -24,13 +24,19 @@ */ package jdk.incubator.vector; -import jdk.internal.vm.annotation.ForceInline; - import java.nio.ByteBuffer; +#if[!byte] +import java.nio.$Type$Buffer; +#end[!byte] import java.nio.ByteOrder; import java.util.Objects; +import java.util.function.IntUnaryOperator; import java.util.concurrent.ThreadLocalRandom; +import jdk.internal.misc.Unsafe; +import jdk.internal.vm.annotation.ForceInline; +import static jdk.incubator.vector.VectorIntrinsics.*; + /** * A specialized {@link Vector} representing an ordered immutable sequence of @@ -41,6 +47,8 @@ $abstractvectortype$() {} + private static final int ARRAY_SHIFT = 31 - Integer.numberOfLeadingZeros(Unsafe.ARRAY_$TYPE$_INDEX_SCALE); + // Unary operator interface FUnOp { @@ -93,7 +101,284 @@ abstract void forEach(Mask<$Boxtype$> m, FUnCon f); - // + // Static factories + + /** + * Returns a vector where all lane elements are set to the default + * primitive value. + * + * @return a zero vector + */ + @ForceInline + @SuppressWarnings("unchecked") + public static $abstractvectortype$ zero($Type$Species species) { + return VectorIntrinsics.broadcastCoerced((Class<$abstractvectortype$>) species.boxType(), $type$.class, species.length(), + 0, + (z -> species.zero())); + } + + /** + * Loads a vector from a byte array starting at an offset. + *

+ * Bytes are composed into primitive lane elements according to the + * native byte order of the underlying platform + *

+ * This method behaves as if it returns the result of calling the + * byte buffer, offset, and mask accepting + * {@link #fromByteBuffer($Type$Species, ByteBuffer, int, Mask) method} as follows: + *

{@code
+     * return this.fromByteBuffer(ByteBuffer.wrap(a), i, this.maskAllTrue());
+     * }
+ * + * @param a the byte array + * @param ix the offset into the array + * @return a vector loaded from a byte array + * @throws IndexOutOfBoundsException if {@code i < 0} or + * {@code i > a.length - (this.length() * this.elementSize() / Byte.SIZE)} + */ + @ForceInline + @SuppressWarnings("unchecked") + public static $abstractvectortype$ fromByteArray($Type$Species species, byte[] a, int ix) { + Objects.requireNonNull(a); + ix = VectorIntrinsics.checkIndex(ix, a.length, species.bitSize() / Byte.SIZE); + return VectorIntrinsics.load((Class<$abstractvectortype$>) species.boxType(), $type$.class, species.length(), + a, ((long) ix) + Unsafe.ARRAY_BYTE_BASE_OFFSET, + a, ix, + (c, idx) -> { + ByteBuffer bbc = ByteBuffer.wrap(c, idx, a.length - idx).order(ByteOrder.nativeOrder()); + $Type$Buffer tb = bbc{#if[byte]?;:.as$Type$Buffer();} + return species.op(i -> tb.get()); + }); + } + + /** + * Loads a vector from a byte array starting at an offset and using a + * mask. + *

+ * Bytes are composed into primitive lane elements according to the + * native byte order of the underlying platform. + *

+ * This method behaves as if it returns the result of calling the + * byte buffer, offset, and mask accepting + * {@link #fromByteBuffer($Type$Species, ByteBuffer, int, Mask) method} as follows: + *

{@code
+     * return this.fromByteBuffer(ByteBuffer.wrap(a), i, m);
+     * }
+ * + * @param a the byte array + * @param ix the offset into the array + * @param m the mask + * @return a vector loaded from a byte array + * @throws IndexOutOfBoundsException if {@code i < 0} or + * {@code i > a.length - (this.length() * this.elementSize() / Byte.SIZE)} + * @throws IndexOutOfBoundsException if the offset is {@code < 0}, + * or {@code > a.length}, + * for any vector lane index {@code N} where the mask at lane {@code N} + * is set + * {@code i >= a.length - (N * this.elementSize() / Byte.SIZE)} + */ + @ForceInline + public static $abstractvectortype$ fromByteArray($Type$Species species, byte[] a, int ix, Mask<$Boxtype$> m) { + return zero(species).blend(fromByteArray(species, a, ix), m); + } + + /** + * Loads a vector from a {@link ByteBuffer byte buffer} starting at an + * offset into the byte buffer. + *

+ * Bytes are composed into primitive lane elements according to the + * native byte order of the underlying platform. + *

+ * This method behaves as if it returns the result of calling the + * byte buffer, offset, and mask accepting + * {@link #fromByteBuffer($Type$Species, ByteBuffer, int, Mask)} method} as follows: + *

{@code
+     *   return this.fromByteBuffer(b, i, this.maskAllTrue())
+     * }
+ * + * @param bb the byte buffer + * @param ix the offset into the byte buffer + * @return a vector loaded from a byte buffer + * @throws IndexOutOfBoundsException if the offset is {@code < 0}, + * or {@code > b.limit()}, + * or if there are fewer than + * {@code this.length() * this.elementSize() / Byte.SIZE} bytes + * remaining in the byte buffer from the given offset + */ + @ForceInline + @SuppressWarnings("unchecked") + public static $abstractvectortype$ fromByteBuffer($Type$Species species, ByteBuffer bb, int ix) { + if (bb.order() != ByteOrder.nativeOrder()) { + throw new IllegalArgumentException(); + } + ix = VectorIntrinsics.checkIndex(ix, bb.limit(), species.bitSize() / Byte.SIZE); + return VectorIntrinsics.load((Class<$abstractvectortype$>) species.boxType(), $type$.class, species.length(), + U.getReference(bb, BYTE_BUFFER_HB), U.getLong(bb, BUFFER_ADDRESS) + ix, + bb, ix, + (c, idx) -> { + ByteBuffer bbc = c.duplicate().position(idx).order(ByteOrder.nativeOrder()); + $Type$Buffer tb = bbc{#if[byte]?;:.as$Type$Buffer();} + return species.op(i -> tb.get()); + }); + } + + /** + * Loads a vector from a {@link ByteBuffer byte buffer} starting at an + * offset into the byte buffer and using a mask. + *

+ * This method behaves as if the byte buffer is viewed as a primitive + * {@link java.nio.Buffer buffer} for the primitive element type, + * according to the native byte order of the underlying platform, and + * the returned vector is loaded with a mask from a primitive array + * obtained from the primitive buffer. + * The following pseudocode expresses the behaviour, where + * {@coce EBuffer} is the primitive buffer type, {@code e} is the + * primitive element type, and {@code ESpecies} is the primitive + * species for {@code e}: + *

{@code
+     * EBuffer eb = b.duplicate().
+     *     order(ByteOrder.nativeOrder()).position(i).
+     *     asEBuffer();
+     * e[] es = new e[this.length()];
+     * for (int n = 0; n < t.length; n++) {
+     *     if (m.isSet(n))
+     *         es[n] = eb.get(n);
+     * }
+     * Vector r = ((ESpecies)this).fromArray(es, 0, m);
+     * }
+ * + * @param bb the byte buffer + * @param ix the offset into the byte buffer + * @return a vector loaded from a byte buffer + * @throws IndexOutOfBoundsException if the offset is {@code < 0}, + * or {@code > b.limit()}, + * for any vector lane index {@code N} where the mask at lane {@code N} + * is set + * {@code i >= b.limit() - (N * this.elementSize() / Byte.SIZE)} + */ + @ForceInline + public static $abstractvectortype$ fromByteBuffer($Type$Species species, ByteBuffer bb, int ix, Mask<$Boxtype$> m) { + return zero(species).blend(fromByteBuffer(species, bb, ix), m); + } + + public static Mask<$Boxtype$> maskFromValues($Type$Species species, boolean... bits) { + if (species.boxType() == $Type$MaxVector.class) + return new $Type$MaxVector.$Type$MaxMask(bits); + switch (species.bitSize()) { + case 64: return new $Type$64Vector.$Type$64Mask(bits); + case 128: return new $Type$128Vector.$Type$128Mask(bits); + case 256: return new $Type$256Vector.$Type$256Mask(bits); + case 512: return new $Type$512Vector.$Type$512Mask(bits); + default: throw new IllegalArgumentException(Integer.toString(species.bitSize())); + } + } + + // @@@ This is a bad implementation -- makes lambdas capturing -- fix this + static Mask<$Boxtype$> trueMask($Type$Species species) { + if (species.boxType() == $Type$MaxVector.class) + return $Type$MaxVector.$Type$MaxMask.TRUE_MASK; + switch (species.bitSize()) { + case 64: return $Type$64Vector.$Type$64Mask.TRUE_MASK; + case 128: return $Type$128Vector.$Type$128Mask.TRUE_MASK; + case 256: return $Type$256Vector.$Type$256Mask.TRUE_MASK; + case 512: return $Type$512Vector.$Type$512Mask.TRUE_MASK; + default: throw new IllegalArgumentException(Integer.toString(species.bitSize())); + } + } + + static Mask<$Boxtype$> falseMask($Type$Species species) { + if (species.boxType() == $Type$MaxVector.class) + return $Type$MaxVector.$Type$MaxMask.FALSE_MASK; + switch (species.bitSize()) { + case 64: return $Type$64Vector.$Type$64Mask.FALSE_MASK; + case 128: return $Type$128Vector.$Type$128Mask.FALSE_MASK; + case 256: return $Type$256Vector.$Type$256Mask.FALSE_MASK; + case 512: return $Type$512Vector.$Type$512Mask.FALSE_MASK; + default: throw new IllegalArgumentException(Integer.toString(species.bitSize())); + } + } + + @ForceInline + @SuppressWarnings("unchecked") + public static Mask<$Boxtype$> maskFromArray($Type$Species species, boolean[] bits, int ix) { + Objects.requireNonNull(bits); + ix = VectorIntrinsics.checkIndex(ix, bits.length, species.length()); + return VectorIntrinsics.load((Class>) species.maskType(), $bitstype$.class, species.length(), + bits, (((long) ix) << ARRAY_SHIFT) + Unsafe.ARRAY_BOOLEAN_BASE_OFFSET, + bits, ix, + (c, idx) -> (Mask<$Boxtype$>) species.opm(n -> c[idx + n])); + } + + @ForceInline + @SuppressWarnings("unchecked") + public static Mask<$Boxtype$> maskAllTrue($Type$Species species) { + return VectorIntrinsics.broadcastCoerced((Class>) species.maskType(), $bitstype$.class, species.length(), + ($bitstype$)-1, + (z -> trueMask(species))); + } + + @ForceInline + @SuppressWarnings("unchecked") + public static Mask<$Boxtype$> maskAllFalse($Type$Species species) { + return VectorIntrinsics.broadcastCoerced((Class>) species.maskType(), $bitstype$.class, species.length(), + 0, + (z -> falseMask(species))); + } + + @ForceInline + public static Shuffle<$Boxtype$> shuffle($Type$Species species, IntUnaryOperator f) { + if (species.boxType() == $Type$MaxVector.class) + return new $Type$MaxVector.$Type$MaxShuffle(f); + switch (species.bitSize()) { + case 64: return new $Type$64Vector.$Type$64Shuffle(f); + case 128: return new $Type$128Vector.$Type$128Shuffle(f); + case 256: return new $Type$256Vector.$Type$256Shuffle(f); + case 512: return new $Type$512Vector.$Type$512Shuffle(f); + default: throw new IllegalArgumentException(Integer.toString(species.bitSize())); + } + } + + @ForceInline + public static Shuffle<$Boxtype$> shuffleIota($Type$Species species) { + if (species.boxType() == $Type$MaxVector.class) + return new $Type$MaxVector.$Type$MaxShuffle(AbstractShuffle.IDENTITY); + switch (species.bitSize()) { + case 64: return new $Type$64Vector.$Type$64Shuffle(AbstractShuffle.IDENTITY); + case 128: return new $Type$128Vector.$Type$128Shuffle(AbstractShuffle.IDENTITY); + case 256: return new $Type$256Vector.$Type$256Shuffle(AbstractShuffle.IDENTITY); + case 512: return new $Type$512Vector.$Type$512Shuffle(AbstractShuffle.IDENTITY); + default: throw new IllegalArgumentException(Integer.toString(species.bitSize())); + } + } + + @ForceInline + public static Shuffle<$Boxtype$> shuffleFromValues($Type$Species species, int... ixs) { + if (species.boxType() == $Type$MaxVector.class) + return new $Type$MaxVector.$Type$MaxShuffle(ixs); + switch (species.bitSize()) { + case 64: return new $Type$64Vector.$Type$64Shuffle(ixs); + case 128: return new $Type$128Vector.$Type$128Shuffle(ixs); + case 256: return new $Type$256Vector.$Type$256Shuffle(ixs); + case 512: return new $Type$512Vector.$Type$512Shuffle(ixs); + default: throw new IllegalArgumentException(Integer.toString(species.bitSize())); + } + } + + @ForceInline + public static Shuffle<$Boxtype$> shuffleFromArray($Type$Species species, int[] ixs, int i) { + if (species.boxType() == $Type$MaxVector.class) + return new $Type$MaxVector.$Type$MaxShuffle(ixs, i); + switch (species.bitSize()) { + case 64: return new $Type$64Vector.$Type$64Shuffle(ixs, i); + case 128: return new $Type$128Vector.$Type$128Shuffle(ixs, i); + case 256: return new $Type$256Vector.$Type$256Shuffle(ixs, i); + case 512: return new $Type$512Vector.$Type$512Shuffle(ixs, i); + default: throw new IllegalArgumentException(Integer.toString(species.bitSize())); + } + } + + + // Ops @Override public abstract $abstractvectortype$ add(Vector<$Boxtype$> v); @@ -2014,7 +2299,7 @@ public abstract $Type$Species species(); /** - * A specialized factory for creating {@link $Type$Vector} value of the same + * A specialized factory for creating {@link $abstractvectortype$} value of the same * shape, and a {@link Mask} and {@link Shuffle} values of the same shape * and {@code int} element type. */ @@ -2198,23 +2483,11 @@ #end[byteOrShort] @Override - public abstract $abstractvectortype$ fromByteArray(byte[] a, int ix); - - @Override - public abstract $abstractvectortype$ fromByteArray(byte[] a, int ix, Mask<$Boxtype$> m); - - @Override - public abstract $abstractvectortype$ fromByteBuffer(ByteBuffer bb, int ix); - - @Override - public abstract $abstractvectortype$ fromByteBuffer(ByteBuffer bb, int ix, Mask<$Boxtype$> m); - - @Override public $abstractvectortype$ reshape(Vector o) { int blen = Math.max(o.species().bitSize(), bitSize()) / Byte.SIZE; ByteBuffer bb = ByteBuffer.allocate(blen).order(ByteOrder.nativeOrder()); o.intoByteBuffer(bb, 0); - return fromByteBuffer(bb, 0); + return $abstractvectortype$.fromByteBuffer(this, bb, 0); } @Override @@ -2253,18 +2526,13 @@ @SuppressWarnings("unchecked") public static $Type$Species species(Vector.Shape s) { Objects.requireNonNull(s); - if (s == Shape.S_64_BIT) { - return $Type$64Vector.SPECIES; - } else if (s == Shape.S_128_BIT) { - return $Type$128Vector.SPECIES; - } else if (s == Shape.S_256_BIT) { - return $Type$256Vector.SPECIES; - } else if (s == Shape.S_512_BIT) { - return $Type$512Vector.SPECIES; - } else if (s == Shape.S_Max_BIT) { - return $Type$MaxVector.SPECIES; - } else { - throw new IllegalArgumentException("Bad shape: " + s); + switch (s) { + case S_64_BIT: return $Type$64Vector.SPECIES; + case S_128_BIT: return $Type$128Vector.SPECIES; + case S_256_BIT: return $Type$256Vector.SPECIES; + case S_512_BIT: return $Type$512Vector.SPECIES; + case S_Max_BIT: return $Type$MaxVector.SPECIES; + default: throw new IllegalArgumentException("Bad shape: " + s); } } } diff --git a/src/jdk.incubator.vector/share/classes/jdk/incubator/vector/X-VectorBits.java.template b/src/jdk.incubator.vector/share/classes/jdk/incubator/vector/X-VectorBits.java.template --- a/src/jdk.incubator.vector/share/classes/jdk/incubator/vector/X-VectorBits.java.template +++ b/src/jdk.incubator.vector/share/classes/jdk/incubator/vector/X-VectorBits.java.template @@ -1051,7 +1051,7 @@ for (int i = 0; i < a.length; i++) { sa[i] = (int) a[i]; } - return SPECIES.shuffleFromArray(sa, 0); + return $abstractvectortype$.shuffleFromArray(SPECIES, sa, 0); } // Memory operations @@ -1129,7 +1129,7 @@ @Override @ForceInline public final void intoByteArray(byte[] a, int ix, Mask<$Boxtype$> m) { - $vectortype$ oldVal = SPECIES.fromByteArray(a, ix); + $vectortype$ oldVal = ($vectortype$) $abstractvectortype$.fromByteArray(SPECIES, a, ix); $vectortype$ newVal = oldVal.blend(this, m); newVal.intoByteArray(a, ix); } @@ -1158,7 +1158,7 @@ @Override @ForceInline public void intoByteBuffer(ByteBuffer bb, int ix, Mask<$Boxtype$> m) { - $vectortype$ oldVal = SPECIES.fromByteBuffer(bb, ix); + $vectortype$ oldVal = ($vectortype$) $abstractvectortype$.fromByteBuffer(SPECIES, bb, ix); $vectortype$ newVal = oldVal.blend(this, m); newVal.intoByteBuffer(bb, ix); } @@ -1571,7 +1571,7 @@ @ForceInline public boolean allTrue() { return VectorIntrinsics.test(COND_carrySet, $masktype$.class, $bitstype$.class, LENGTH, - this, species().maskAllTrue(), + this, $abstractvectortype$.maskAllTrue(species()), (m1, m2) -> super.allTrue()); } } @@ -1662,6 +1662,18 @@ @Override @ForceInline + public Class boxType() { + return $vectortype$.class; + } + + @Override + @ForceInline + public Class maskType() { + return $masktype$.class; + } + + @Override + @ForceInline public int elementSize() { return $Boxtype$.SIZE; } @@ -1704,31 +1716,6 @@ // Factories - @Override - public $masktype$ maskFromValues(boolean... bits) { - return new $masktype$(bits); - } - - @Override - public $shuffletype$ shuffle(IntUnaryOperator f) { - return new $shuffletype$(f); - } - - @Override - public $shuffletype$ shuffleIota() { - return new $shuffletype$(AbstractShuffle.IDENTITY); - } - - @Override - public $shuffletype$ shuffleFromValues(int... ixs) { - return new $shuffletype$(ixs); - } - - @Override - public $shuffletype$ shuffleFromArray(int[] ixs, int i) { - return new $shuffletype$(ixs, i); - } - #if[FP] @Override @ForceInline @@ -1768,22 +1755,6 @@ @Override @ForceInline - public $masktype$ maskAllTrue() { - return VectorIntrinsics.broadcastCoerced($masktype$.class, $bitstype$.class, LENGTH, - ($bitstype$)-1, - (z -> $masktype$.TRUE_MASK)); - } - - @Override - @ForceInline - public $masktype$ maskAllFalse() { - return VectorIntrinsics.broadcastCoerced($masktype$.class, $bitstype$.class, LENGTH, - 0, - (z -> $masktype$.FALSE_MASK)); - } - - @Override - @ForceInline public $vectortype$ scalars($type$... es) { Objects.requireNonNull(es); int ix = VectorIntrinsics.checkIndex(0, es.length, LENGTH); @@ -1795,17 +1766,6 @@ @Override @ForceInline - public $masktype$ maskFromArray(boolean[] bits, int ix) { - Objects.requireNonNull(bits); - ix = VectorIntrinsics.checkIndex(ix, bits.length, LENGTH); - return VectorIntrinsics.load($masktype$.class, $bitstype$.class, LENGTH, - bits, (((long) ix) << ARRAY_SHIFT) + Unsafe.ARRAY_BOOLEAN_BASE_OFFSET, - bits, ix, - (c, idx) -> opm(n -> c[idx + n])); - } - - @Override - @ForceInline public $vectortype$ fromArray($type$[] a, int ix) { Objects.requireNonNull(a); ix = VectorIntrinsics.checkIndex(ix, a.length, LENGTH); @@ -1821,20 +1781,6 @@ return zero().blend(fromArray(a, ax), m); } - @Override - @ForceInline - public $vectortype$ fromByteArray(byte[] a, int ix) { - Objects.requireNonNull(a); - ix = VectorIntrinsics.checkIndex(ix, a.length, bitSize() / Byte.SIZE); - return VectorIntrinsics.load($vectortype$.class, $type$.class, LENGTH, - a, ((long) ix) + Unsafe.ARRAY_BYTE_BASE_OFFSET, - a, ix, - (c, idx) -> { - ByteBuffer bbc = ByteBuffer.wrap(c, idx, a.length - idx).order(ByteOrder.nativeOrder()); - $Type$Buffer tb = bbc{#if[byte]?;:.as$Type$Buffer();} - return op(i -> tb.get()); - }); - } #if[!byteOrShort] @Override @ForceInline @@ -1868,35 +1814,6 @@ @Override @ForceInline - public $vectortype$ fromByteArray(byte[] a, int ix, Mask<$Boxtype$> m) { - return zero().blend(fromByteArray(a, ix), m); - } - - @Override - @ForceInline - public $vectortype$ fromByteBuffer(ByteBuffer bb, int ix) { - if (bb.order() != ByteOrder.nativeOrder()) { - throw new IllegalArgumentException(); - } - ix = VectorIntrinsics.checkIndex(ix, bb.limit(), bitSize() / Byte.SIZE); - return VectorIntrinsics.load($vectortype$.class, $type$.class, LENGTH, - U.getReference(bb, BYTE_BUFFER_HB), U.getLong(bb, BUFFER_ADDRESS) + ix, - bb, ix, - (c, idx) -> { - ByteBuffer bbc = c.duplicate().position(idx).order(ByteOrder.nativeOrder()); - $Type$Buffer tb = bbc{#if[byte]?;:.as$Type$Buffer();} - return op(i -> tb.get()); - }); - } - - @Override - @ForceInline - public $vectortype$ fromByteBuffer(ByteBuffer bb, int ix, Mask<$Boxtype$> m) { - return zero().blend(fromByteBuffer(bb, ix), m); - } - - @Override - @ForceInline @SuppressWarnings("unchecked") public $vectortype$ cast(Vector o) { if (o.length() != LENGTH) diff --git a/test/jdk/jdk/incubator/vector/templates/Kernel-Binary-Masked-op.template b/test/jdk/jdk/incubator/vector/templates/Kernel-Binary-Masked-op.template --- a/test/jdk/jdk/incubator/vector/templates/Kernel-Binary-Masked-op.template +++ b/test/jdk/jdk/incubator/vector/templates/Kernel-Binary-Masked-op.template @@ -2,7 +2,7 @@ $type$[] b = fb.apply(SPECIES.length()); $type$[] r = fr.apply(SPECIES.length()); boolean[] mask = fm.apply(SPECIES.length()); - Vector.Mask<$Wideboxtype$> vmask = SPECIES.maskFromValues(mask); + Vector.Mask<$Wideboxtype$> vmask = $abstractvectortype$.maskFromValues(SPECIES, mask); for (int ic = 0; ic < INVOC_COUNT; ic++) { for (int i = 0; i < a.length; i += SPECIES.length()) { diff --git a/test/jdk/jdk/incubator/vector/templates/Kernel-Blend-op.template b/test/jdk/jdk/incubator/vector/templates/Kernel-Blend-op.template --- a/test/jdk/jdk/incubator/vector/templates/Kernel-Blend-op.template +++ b/test/jdk/jdk/incubator/vector/templates/Kernel-Blend-op.template @@ -2,7 +2,7 @@ $type$[] b = fb.apply(SPECIES.length()); $type$[] r = fr.apply(SPECIES.length()); boolean[] mask = fm.apply(SPECIES.length()); - Vector.Mask<$Wideboxtype$> vmask = SPECIES.maskFromValues(mask); + Vector.Mask<$Wideboxtype$> vmask = $abstractvectortype$.maskFromValues(SPECIES, mask); for (int ic = 0; ic < INVOC_COUNT; ic++) { for (int i = 0; i < a.length; i += SPECIES.length()) { diff --git a/test/jdk/jdk/incubator/vector/templates/Kernel-BoolReduction-op.template b/test/jdk/jdk/incubator/vector/templates/Kernel-BoolReduction-op.template --- a/test/jdk/jdk/incubator/vector/templates/Kernel-BoolReduction-op.template +++ b/test/jdk/jdk/incubator/vector/templates/Kernel-BoolReduction-op.template @@ -3,7 +3,7 @@ for (int ic = 0; ic < INVOC_COUNT; ic++) { for (int i = 0; i < mask.length; i += SPECIES.length()) { - Vector.Mask<$Wideboxtype$> vmask = SPECIES.maskFromArray(mask, i); + Vector.Mask<$Wideboxtype$> vmask = $abstractvectortype$.maskFromArray(SPECIES, mask, i); r[i] = vmask.[[TEST]](); } } diff --git a/test/jdk/jdk/incubator/vector/templates/Kernel-Rearrange.template b/test/jdk/jdk/incubator/vector/templates/Kernel-Rearrange.template --- a/test/jdk/jdk/incubator/vector/templates/Kernel-Rearrange.template +++ b/test/jdk/jdk/incubator/vector/templates/Kernel-Rearrange.template @@ -5,7 +5,7 @@ for (int ic = 0; ic < INVOC_COUNT; ic++) { for (int i = 0; i < a.length; i += SPECIES.length()) { $abstractvectortype$ av = SPECIES.fromArray(a, i); - av.rearrange(SPECIES.shuffleFromArray(order, i)).intoArray(r, i); + av.rearrange($abstractvectortype$.shuffleFromArray(SPECIES, order, i)).intoArray(r, i); } } diff --git a/test/jdk/jdk/incubator/vector/templates/Kernel-Shift-Masked-op.template b/test/jdk/jdk/incubator/vector/templates/Kernel-Shift-Masked-op.template --- a/test/jdk/jdk/incubator/vector/templates/Kernel-Shift-Masked-op.template +++ b/test/jdk/jdk/incubator/vector/templates/Kernel-Shift-Masked-op.template @@ -2,7 +2,7 @@ $type$[] b = fb.apply(SPECIES.length()); $type$[] r = fr.apply(SPECIES.length()); boolean[] mask = fm.apply(SPECIES.length()); - Vector.Mask<$Wideboxtype$> vmask = SPECIES.maskFromValues(mask); + Vector.Mask<$Wideboxtype$> vmask = $abstractvectortype$.maskFromValues(SPECIES, mask); for (int ic = 0; ic < INVOC_COUNT; ic++) { for (int i = 0; i < a.length; i += SPECIES.length()) { diff --git a/test/jdk/jdk/incubator/vector/templates/Kernel-Ternary-Masked-op.template b/test/jdk/jdk/incubator/vector/templates/Kernel-Ternary-Masked-op.template --- a/test/jdk/jdk/incubator/vector/templates/Kernel-Ternary-Masked-op.template +++ b/test/jdk/jdk/incubator/vector/templates/Kernel-Ternary-Masked-op.template @@ -3,7 +3,7 @@ $type$[] c = fc.apply(SPECIES.length()); $type$[] r = fr.apply(SPECIES.length()); boolean[] mask = fm.apply(SPECIES.length()); - Vector.Mask<$Wideboxtype$> vmask = SPECIES.maskFromValues(mask); + Vector.Mask<$Wideboxtype$> vmask = $abstractvectortype$.maskFromValues(SPECIES, mask); for (int ic = 0; ic < INVOC_COUNT; ic++) { for (int i = 0; i < a.length; i += SPECIES.length()) { diff --git a/test/jdk/jdk/incubator/vector/templates/Kernel-Unary-Masked-op.template b/test/jdk/jdk/incubator/vector/templates/Kernel-Unary-Masked-op.template --- a/test/jdk/jdk/incubator/vector/templates/Kernel-Unary-Masked-op.template +++ b/test/jdk/jdk/incubator/vector/templates/Kernel-Unary-Masked-op.template @@ -1,7 +1,7 @@ $type$[] a = fa.apply(SPECIES.length()); $type$[] r = fr.apply(SPECIES.length()); boolean[] mask = fm.apply(SPECIES.length()); - Vector.Mask<$Wideboxtype$> vmask = SPECIES.maskFromValues(mask); + Vector.Mask<$Wideboxtype$> vmask = $abstractvectortype$.maskFromValues(SPECIES, mask); for (int ic = 0; ic < INVOC_COUNT; ic++) { for (int i = 0; i < a.length; i += SPECIES.length()) { diff --git a/test/jdk/jdk/incubator/vector/templates/Perf-Compare.template b/test/jdk/jdk/incubator/vector/templates/Perf-Compare.template old mode 100644 new mode 100755 --- a/test/jdk/jdk/incubator/vector/templates/Perf-Compare.template +++ b/test/jdk/jdk/incubator/vector/templates/Perf-Compare.template @@ -4,7 +4,7 @@ $type$[] a = fa.apply(size); $type$[] b = fb.apply(size); boolean[] ms = fm.apply(size); - Vector.Mask<$Wideboxtype$> m = SPECIES.maskFromArray(ms, 0); + Vector.Mask<$Wideboxtype$> m = $abstractvectortype$.maskFromArray(SPECIES, ms, 0); for (int ic = 0; ic < INVOC_COUNT; ic++) { for (int i = 0; i < a.length; i += SPECIES.length()) { diff --git a/test/jdk/jdk/incubator/vector/templates/X-LoadStoreTest.java.template b/test/jdk/jdk/incubator/vector/templates/X-LoadStoreTest.java.template --- a/test/jdk/jdk/incubator/vector/templates/X-LoadStoreTest.java.template +++ b/test/jdk/jdk/incubator/vector/templates/X-LoadStoreTest.java.template @@ -222,7 +222,7 @@ $type$[] a = fa.apply(SPECIES.length()); $type$[] r = new $type$[a.length]; boolean[] mask = fm.apply(SPECIES.length()); - Vector.Mask<$Boxtype$> vmask = SPECIES.maskFromValues(mask); + Vector.Mask<$Boxtype$> vmask = $abstractvectortype$.maskFromValues(SPECIES, mask); for (int ic = 0; ic < INVOC_COUNT; ic++) { for (int i = 0; i < a.length; i += SPECIES.length()) { @@ -255,7 +255,7 @@ for (int ic = 0; ic < INVOC_COUNT; ic++) { for (int i = 0; i < l; i += s) { - $abstractvectortype$ av = SPECIES.fromByteBuffer(a, i); + $abstractvectortype$ av = $abstractvectortype$.fromByteBuffer(SPECIES, a, i); av.intoByteBuffer(r, i); } } @@ -278,7 +278,7 @@ for (int ic = 0; ic < INVOC_COUNT; ic++) { for (int i = 0; i < l; i += s) { - $abstractvectortype$ av = SPECIES.fromByteBuffer(a, i); + $abstractvectortype$ av = $abstractvectortype$.fromByteBuffer(SPECIES, a, i); av.intoByteBuffer(r, i); } } @@ -296,14 +296,14 @@ ByteBuffer a = toBuffer(fa.apply(SPECIES.length()), fb); ByteBuffer r = fb.apply(a.limit()); boolean[] mask = fm.apply(SPECIES.length()); - Vector.Mask<$Boxtype$> vmask = SPECIES.maskFromValues(mask); + Vector.Mask<$Boxtype$> vmask = $abstractvectortype$.maskFromValues(SPECIES, mask); int l = a.limit(); int s = SPECIES.length() * SPECIES.elementSize() / 8; for (int ic = 0; ic < INVOC_COUNT; ic++) { for (int i = 0; i < l; i += s) { - $abstractvectortype$ av = SPECIES.fromByteBuffer(a, i, vmask); + $abstractvectortype$ av = $abstractvectortype$.fromByteBuffer(SPECIES, a, i, vmask); av.intoByteBuffer(r, i); } } @@ -317,7 +317,7 @@ r = fb.apply(a.limit()); for (int ic = 0; ic < INVOC_COUNT; ic++) { for (int i = 0; i < l; i += s) { - $abstractvectortype$ av = SPECIES.fromByteBuffer(a, i); + $abstractvectortype$ av = $abstractvectortype$.fromByteBuffer(SPECIES, a, i); av.intoByteBuffer(r, i, vmask); } } @@ -336,14 +336,14 @@ a = a.asReadOnlyBuffer().order(a.order()); ByteBuffer r = fb.apply(a.limit()); boolean[] mask = fm.apply(SPECIES.length()); - Vector.Mask<$Boxtype$> vmask = SPECIES.maskFromValues(mask); + Vector.Mask<$Boxtype$> vmask = $abstractvectortype$.maskFromValues(SPECIES, mask); int l = a.limit(); int s = SPECIES.length() * SPECIES.elementSize() / 8; for (int ic = 0; ic < INVOC_COUNT; ic++) { for (int i = 0; i < l; i += s) { - $abstractvectortype$ av = SPECIES.fromByteBuffer(a, i, vmask); + $abstractvectortype$ av = $abstractvectortype$.fromByteBuffer(SPECIES, a, i, vmask); av.intoByteBuffer(r, i); } } From vladimir.x.ivanov at oracle.com Wed Jan 30 22:09:21 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Wed, 30 Jan 2019 14:09:21 -0800 Subject: [vector] Error message in the fromArray length check is misleading In-Reply-To: References: Message-ID: <1e94ce7a-62b3-c155-751e-7cad2b9f6ad6@oracle.com> > I realise my code is blowing past the end of the array & triggering the > exception (which is good, as my code was wrong), but I feel like the > message should say "Index 50 is out of bounds for length 50", or > something similar, rather than saying my array is of a different length > to what's actually allocated. > > Maybe in VectorIntrinsics line 405 > ??? case 2: return Objects.checkIndex(ix, length - (vlen - 1)); > it could be modified to check if ix + vlen is below length, rather than > if ix is below length - (vlen -1)? I fully agree that the message is misleading. It should be much clearer and involve some vector-specific information (vector access spans a range while Objects.checkIndex assumes a single element is accessed). The check is written that way to improve range check elimination (upper bound is invariant in loop body). As previous performance experiments showed there's still some room for improvements, but those are unrelated to the error message. So, I'm in favor of improving error message itself leaving the check as is for now. Best regards, Vladimir Ivanov From sandhya.viswanathan at intel.com Wed Jan 30 22:24:14 2019 From: sandhya.viswanathan at intel.com (Viswanathan, Sandhya) Date: Wed, 30 Jan 2019 22:24:14 +0000 Subject: [vector] Vector API -- alignment with value types In-Reply-To: References: <5745a703-c239-8487-6835-7c1aadd54d0f@oracle.com> Message-ID: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A55C4F@FMSMSX126.amr.corp.intel.com> Thanks a lot Brian, I will try the two patches on some examples and get back to you. Best Regards, Sandhya -----Original Message----- From: panama-dev [mailto:panama-dev-bounces at openjdk.java.net] On Behalf Of Brian Goetz Sent: Wednesday, January 30, 2019 1:54 PM To: panama-dev at openjdk.java.net Subject: Re: [vector] Vector API -- alignment with value types I've tried to attach the patches-in-progress for these.? There are two, the speciesFactories.patch I set recently, and then zero.patch.? These only include changes to Vector and the templates; you'll have to re-run the gen-* scripts to get changes to the rest.? They apply to the last version for which the tests pass. Perhaps someone could look at the generated code from typical operations and see how badly this approach perturbs it? On 1/30/2019 3:58 PM, Brian Goetz wrote: > > >> Part I >> ------ >> >> Here's an idea for simplifying Species, which is: let's drive Species >> down to be a simple class that is really just a constant holder for >> (element type, shape), and move all the behavior to static methods on >> XxxVector. > > I've started prototyping this, in a rather slash-and-burn manner. I am > about halfway through.? So far, its working, the set of changes I had > to make to client code is very small (almost all transforming > species.doSomething(args) to XxxVector.doSomething(species, args)). > > The question, of course, is whether the intrinsification will all > survive the transformation.? The most common case is that I have > transformed vector intrinsic calls from > > ??? VectorIntrinsic.blah(Int128Vector.class, ...) > > to > > ??? VectorIntrinsics.blah(species.boxType(), ...) > > The basic assumption is that, under the same conditions that we get > inlining now, we'll know the species is a constant, and boxType() will > just inline to a concrete box type anyway.? (The goal is to get > species to be values, but in the meantime, they can be enum > constants.)? This should work, but we'll likely have to do some JIT > work to get back to where we see all the inlining and > intrinsification.? (Much of this would come free in Valhalla.) > > There are a few cases where we can't just do the above, and have to do > a switch in the lifted method, such as: > > public static Shuffle shuffle(ByteSpecies species, > IntUnaryOperator f) { > ??? if (species.boxType() == ByteMaxVector.class) > ??????? return new ByteMaxVector.ByteMaxShuffle(f); > ??? switch (species.bitSize()) { > ??????? case 64:return new Byte64Vector.Byte64Shuffle(f); > ??????? case 128:return new Byte128Vector.Byte128Shuffle(f); > ??????? case 256:return new Byte256Vector.Byte256Shuffle(f); > ??????? case 512:return new Byte512Vector.Byte512Shuffle(f); > ??????? default:throw new > IllegalArgumentException(Integer.toString(species.bitSize())); > ??? } > } > > > Because again, species is a constant, this should also just inline > down to the right thing.? So far, other than reshape/rebracket, I > haven't seen anything requiring more complicated transformations. > > The only code I found so far that tries to be agnostic to shape and > size both is VectorResizeTest; there are strategies for making this > work without the combinatorial automated specialization, so I don't > see this as a big impediment. > > Where this leads to is: > ?- Vector.Species becomes an interface with a handful of methods > (bitSize, etc), and quite possibly one that no one uses directly; > ?- IntSpecies and friends become enums, with enum constants for I64, > I128, etc (or, values, with static constants for the values); > ?- The specialized classes for XxxNnVector.XxxNnSpecies _go away_; > ?- Users need not learn about species at all, but if they do care, > they are just simple data constants that get fed through the API. > > I'm not done (and am traveling the next two weeks), but I think I've > made progress validating the API transformation.? The real question, > then, is when do we do this.? I think it would be best to do before > previewing, simply because it is such an intrusive refactoring. But, > we'd have to evaluate whether we can mitigate the impact in time. > > From vladimir.x.ivanov at oracle.com Wed Jan 30 22:29:20 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Wed, 30 Jan 2019 14:29:20 -0800 Subject: [vector] Partial permutations In-Reply-To: References: <9970C4BC-FDC8-49A4-93FE-1B9F72282DFC@oracle.com> Message-ID: > The point of sharing this example is not just that I really hope it will one day be possible to use the API to count > > bits quickly (in which case it would probably be even better implemented within the API, using the AVX-512 vpopcnt > > extension when available!), but to draw attention to an existing idiom that I suspect may not be served well by the > > semantics of Vector.rearrange, even if it is currently the most appropriate approximation in the current API. I leave > > it to the experts to think about whether or how to make this possible, but I hope that it is helpful to draw attention > > to the kinds of things that would occur to a potential user to implement with the API. It wouldn't matter much to me as a > > potential user whether this would be expressed as a gather or as a permutation, or even if there were a secret incantation > > expressible in terms of masks: I just anticipate the need to do parallel lookups without undue overhead. FTR I've been doing a similar exercise recently and ended up with the following examples: PopulationCount [1], SumOfUnsignedBytes [2], SortVector [3], Merge [4]. I ended up with a number of idioms which would be good to optimize on the backend side until there's a way to express them more directly. > Regarding the SWAR hacks, there's a reason for them: lack of unsigned arithmetic. I don't think it is > > possible to implement the horizontal reduction in terms of signed byte arithmetic. I think that lack > > of unsigned arithmetic is another obstacle to lifting and shifting existing vector algorithms to the vector API. > > I have previously tried and failed to implement StreamVByte compression (which incidentally also requires > > similar permutations and species changes) with the vector API, for lack of unsigned 8 bit arithmetic. I had similar experience, plus saturated mode & SAD (sum of absolute differences). Regarding your original question: > It would be nice to be able to hint somehow that only half of the result need be permuted, allowing the permutation to "short circuit" - potentially using faster code. IMO the most promising approach is to split vector in parts, permute some of them, and then merge them back. I believe that's the best way to communicate the semantics of the operation to the compiler. Best regards, Vladimir Ivanov [1] http://hg.openjdk.java.net/panama/dev/raw-file/0447b2b7d707/test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/PopulationCount.java [2] http://hg.openjdk.java.net/panama/dev/raw-file/0447b2b7d707/test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/SumOfUnsignedBytes.java [3] http://hg.openjdk.java.net/panama/dev/raw-file/0447b2b7d707/test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/SortVector.java [4] http://hg.openjdk.java.net/panama/dev/raw-file/0447b2b7d707/test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Merge.java > ________________________________ > From: John Rose > Sent: 30 January 2019 01:34:53 > To: Richard Startin > Cc: panama-dev at openjdk.java.net > Subject: Re: [vector] Partial permutations > > On Jan 29, 2019, at 2:44 PM, Richard Startin > wrote: > > lo = lo.add(counts.rearrange(v1.and(0x0F0F0F0F).rebracket(YMM_BYTE).toShuffle()).rebracket(YMM_INT)); > > Breaking it down: > > var v1 = ?; // u4[8] > var nibblePerByte = v1.and(0x0F0F0F0F).rebracket(YMM_BYTE); // u1[32] > var bitCountPerByte = counts.rearrange(nibblePerByte.toShuffle()); // u1[32] > var swarLanes = bitCountPerByte.rebracket(YMM_INT); // u4[8] > lo = lo.add(expandedCounts); // danger: carry-out across bytes? > > The second line zeroes out half of the bit positions and breaks out the > 8 int lanes into 32 byte lanes. > > The third line replaces each byte (whose value is in 0..15) with another > byte that gives the bit-count (value in 0..4). > > The fourth line undoes the action of the second line. Now we have > four lanes each of which is an integer containing 4 "SIMD across a > register" sub-lanes. > > It seems like the top half of your NIBBLE_COUNTS table could be any > old garbage, not a copy of the bottom half. > > (Did I get it right? This kind of code is tricky!) > > The final line adds 8x4 lanes and their sub-lanes. I think you have > a bug in your block size, since if all the input bits are set, then > each nibble count will always be 4, and so each sub-lane count > will accumulate that value, 256/4 times, which will cause a carry > to propagate across your SWAR sub-lanes. > > The idiom counts.rearrange(nibblePerByte.toShuffle()) would read better, > in this algorithm, as nibblePerByte.selectFrom(counts). It's really akin to > a gather operation in this use case, and the similarity could be made > more clear by using a gather-like API (which we are still working on). > > To respond to your question: If I follow you, you note that since the > top half of the NIBBLE_COUNTS array is never used, then any instructions > which would do the work of loading from that half of the array are useless. > > To me, this calls the question of how to do strength-reduction optimizations > on shuffles. > > I don't think we want explicit API points for simplified shuffles, any more > than we want library routines or operators for division when both operands > are positive, even though there are optimizations relevant in both cases. > > The best way to get improved code for simplified inputs is to work hard > to detect those inputs and adjust the instruction selection (or IR > generation) accordingly. In this case, the shuffle vector is the logical > "and" of 0x0F and some random input, which means that, ideally the > optimizer should be able to prove that the selected lanes of "counts" > are in the range 0..15. That's probably enough to simplify a complex > shuffle idiom in the back end. > > Failing lanewise type analysis (but why should we fail that?), a pattern > match in the C2 backend might also detect that the shuffle was immediately > derived from a masking operation. In either case, the optimization might > be confounded by the SWAR hacks going on. Perhaps the right answer > is to rebracket to YMM_BYTE before the logical "and" (of a byte 0x0F). > This is sadly delicate, but can be made more robust by an improvement > to the C2 type system to track individual bit positions, or adoption of Graal, > which already has that feature. > > The overall point here is that some of our good code will come from a > strength reduction of expensive operations, gated by type and pattern > analysis. The design problem is to put enough primitives into the user > API but not too many, and enough optimization in the backend. > > ? John > From john.r.rose at oracle.com Wed Jan 30 22:39:21 2019 From: john.r.rose at oracle.com (John Rose) Date: Wed, 30 Jan 2019 14:39:21 -0800 Subject: [vector] Error message in the fromArray length check is misleading In-Reply-To: <1e94ce7a-62b3-c155-751e-7cad2b9f6ad6@oracle.com> References: <1e94ce7a-62b3-c155-751e-7cad2b9f6ad6@oracle.com> Message-ID: On Jan 30, 2019, at 2:09 PM, Vladimir Ivanov wrote: > > So, I'm in favor of improving error message itself leaving the check as is for now. > This occasion was anticipated in the design of the Preconditions API. You can plug in your own error reporter as a lambda, without abandoning the intrinsic behavior. From vladimir.x.ivanov at oracle.com Thu Jan 31 02:02:24 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Wed, 30 Jan 2019 18:02:24 -0800 Subject: [Vector] RFR: reshape, resize, rebracket, cast In-Reply-To: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A555A8@FMSMSX126.amr.corp.intel.com> References: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A555A8@FMSMSX126.amr.corp.intel.com> Message-ID: <81f502fb-7636-6f68-56bb-912558fbeb70@oracle.com> Looks fine to me. Best regards, Vladimir Ivanov On 29/01/2019 17:30, Viswanathan, Sandhya wrote: > Hi All, > > Please find the updated webrev for reinterpret, reshape and cast at: > http://cr.openjdk.java.net/~sviswanathan/vectorIntrinsics/ReinterpretReshapeCast/webrev.02/ > > This update implements the two review inputs from Vladimir: > 1) > Update users: > test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/AbstractVectorBenchmark.java > test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/PopulationCount.java > test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/SumOfUnsignedBytes.java > test/jdk/jdk/incubator/vector/VectorHash.java > test/jdk/jdk/incubator/vector/VectorReshapeTests.java > test/jdk/jdk/incubator/vector/CovarOverrideTest.java > > 2) > src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Vector.java: > + public Vector defaultReinterpret(Species s) { > + public abstract Class vectorType(); > Those operations shouldn't be part of public API. > > Please let me know if this looks ok to push. > > Best Regards, > Sandhya > > -----Original Message----- > From: panama-dev [mailto:panama-dev-bounces at openjdk.java.net] On Behalf Of Brian Goetz > Sent: Monday, January 28, 2019 3:38 PM > To: Kharbas, Kishor ; panama-dev at openjdk.java.net > Subject: Re: [Vector] RFR: reshape, resize, rebracket, cast > > The issue you raise is a minor frustration, but overall I think the > direction of movement -- from species to vector -- is the right one.? We > can work on ironing out these small issues later. > > On 1/16/2019 3:20 AM, Kharbas, Kishor wrote: >> >> Hi, >> >> I have a patch which refactors the above methods as discussed in this >> thread - >> https://mail.openjdk.java.net/pipermail/panama-dev/2018-December/003365.html. >> >> Please review the changes at - >> http://cr.openjdk.java.net/~kkharbas/vector-api/webrev-reshape_etc >> >> Generation of intrinsics and correct vector boxing elimination have >> been verified. In the next patch I will add changes to jtreg tests. >> >> I wanted to bring forth one small issue(possibly) which programmers >> might face with this change - >> >> We provide specialized types like IntVector, FloatVector, etc for >> users to define their vectors and they would be able to write code >> like this with previous methods, >> >> /FloatVector float256 = SPECIES_FLOAT256.cast(int256);/ >> >> Here FloatSpecies would always return FloatVector. >> >> However with this change, since cast() is defined on a vector and >> takes species of a generic type, it cannot return a specialized Vector >> like IntVector or FloatVector. User has to explicitly cast the return >> vector from Vector to specialized Vector or use a generic vector of >> corresponding element type. For example, >> >> /? ??FloatVector float256 = (FloatVector)int256.cast(SPECIES_FLOAT256);/ >> >> /? or/ >> >> /? ??Vector float256 = int256.cast(SPECIES_FLOAT256);/ >> >> I am not sure if this is even a problem, but I thought its worth >> mentioning. >> >> Thanks >> >> Kishor >> > From vladimir.x.ivanov at oracle.com Thu Jan 31 02:07:22 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Wed, 30 Jan 2019 18:07:22 -0800 Subject: [vector] RFR: Wrong offset is computed in Mask.maskFromArray() Message-ID: <9b8810ee-3e51-75fe-8009-ba2838a7c66f@oracle.com> http://cr.openjdk.java.net/~vlivanov/panama/vector/mask_mem/webrev.00/ Wrong offset is computed in Mask.maskFromArray() accesses. Best regards, Vladimir Ivanov From vladimir.x.ivanov at oracle.com Thu Jan 31 03:12:19 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Wed, 30 Jan 2019 19:12:19 -0800 Subject: Results: New panama Committer: Wang Zhuo Message-ID: <85913995-5922-94bc-4a70-58a13f87866c@oracle.com> Voting for Wang Zhuo (email: zhuoren.wz at alibaba-inc.com) [1] is now closed. Yes: 4 Veto: 0 Abstain: 0 According to the Bylaws definition of Lazy Consensus, this is sufficient to approve the nomination. Best regards, Vladimir Ivanov [1] http://mail.openjdk.java.net/pipermail/panama-dev/2019-January/003847.html From jbvernee at xs4all.nl Thu Jan 31 12:08:24 2019 From: jbvernee at xs4all.nl (Jorn Vernee) Date: Thu, 31 Jan 2019 13:08:24 +0100 Subject: [foreign] RFR 8218052: Don't throw an exception when encountering a type with a flexible array member Message-ID: <52eed17e59893e3af7ff3595043e7df2@xs4all.nl> Hi, From the bug description: Currently jextract throws an UnsopportedOpperationException when encountering a type that has a flexible array member. As a result of that no class is written for the type, and when loading the enclosing interface this throws a NoClassDefFoundError, prohibiting use of the entire header interface. We could mitigate this problem by emitting an undefined layout reference instead of throwing an exception, which would only make the type with the flexible array unusable, but still allow other things in the same interface to be used. Please review the following: Bug: https://bugs.openjdk.java.net/browse/JDK-8218052 Webrev: http://cr.openjdk.java.net/~jvernee/panama/webrevs/8218052/webrev.00 Thanks, Jorn From maurizio.cimadamore at oracle.com Thu Jan 31 14:10:40 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 31 Jan 2019 14:10:40 +0000 Subject: [foreign] RFR 8218144: Direct invoker does not handle varargs correctly Message-ID: Hi, our CI systems detected a spurious failures on StdLibTest, only on Linux x64. After a lot of digging, I figured out the problem was that SystemV ABI requires the RAX register to be set to the number of vector arguments in a variadic call: %rax: temporary register;with variable arguments passes information about the number of vector registers used So, if we are doing a variadic call, we need to set this register to the number of floating point args we're passing. The universal invoker seems to do that: http://hg.openjdk.java.net/panama/dev/file/tip/src/hotspot/cpu/x86/universalNativeInvoker_x86.cpp#l515 For both variadic and non-variadic calls. So, a natural fix for the direct approach is to tweak the cast to the function pointers in our native stubs - e.g. from this: void DNI_invokeNative_V_JD(JNIEnv *env, jobject _unused, jlong addr, jlong arg0, jdouble arg1) { ???? ((void (*)(jlong, jdouble))addr)(arg0, arg1); } To this: void DNI_invokeNative_V_JD(JNIEnv *env, jobject _unused, jlong addr, jlong arg0, jdouble arg1) { ???? ((void (*)(jlong, jdouble,...))addr)(arg0, arg1); } Which will generate the right set for RAX (even though it will do so also for non-variadic call - but? that's the same as what universal invoker does). I also strengthened the test - I tried first to use sprintf instead of printf, but that didn't work, as that doesn't seem to reproduce the issue (my suspicion here is that 'printf' is optimized to look into RAX). So, instead, I tweaked the test to run for a lot longer, by repeatedly shuffling the permutation array, and I also increased the decimal length of the double formatting string - that's because, unfortunately, we were using 1.23, which has 4 chars, and sometimes the result of printf ended up with "-NaN" which also has 4 chars, despite being very wrong. With both measures, tests are much more likely, so much so that in the affected machine I've never been able to get a pass w/o the proposed patch. I verified that the patch works on all platforms, and I also verified that the patch does not introduce performance regressions, which is not the case (kind of expected, given that the only difference for adding a "..." is the extra MOV to RAX, as shown here [1]). Webrev: http://cr.openjdk.java.net/~mcimadamore/panama/8218144/ Cheers Maurizio [1] - https://gcc.godbolt.org/z/VAgfZc From sundararajan.athijegannathan at oracle.com Thu Jan 31 14:24:33 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Thu, 31 Jan 2019 19:54:33 +0530 Subject: [foreign] RFR 8218144: Direct invoker does not handle varargs correctly In-Reply-To: References: Message-ID: <5C530521.6070804@oracle.com> Looks good. -Sundar On 31/01/19, 7:40 PM, Maurizio Cimadamore wrote: > Hi, > our CI systems detected a spurious failures on StdLibTest, only on > Linux x64. After a lot of digging, I figured out the problem was that > SystemV ABI requires the RAX register to be set to the number of > vector arguments in a variadic call: > > %rax: temporary register;with variable arguments passes information > about the number of vector registers used > > So, if we are doing a variadic call, we need to set this register to > the number of floating point args we're passing. The universal invoker > seems to do that: > > http://hg.openjdk.java.net/panama/dev/file/tip/src/hotspot/cpu/x86/universalNativeInvoker_x86.cpp#l515 > > > For both variadic and non-variadic calls. > > So, a natural fix for the direct approach is to tweak the cast to the > function pointers in our native stubs - e.g. from this: > > void DNI_invokeNative_V_JD(JNIEnv *env, jobject _unused, jlong addr, > jlong arg0, jdouble arg1) { > ((void (*)(jlong, jdouble))addr)(arg0, arg1); > } > > To this: > > void DNI_invokeNative_V_JD(JNIEnv *env, jobject _unused, jlong addr, > jlong arg0, jdouble arg1) { > ((void (*)(jlong, jdouble,...))addr)(arg0, arg1); > } > > Which will generate the right set for RAX (even though it will do so > also for non-variadic call - but that's the same as what universal > invoker does). > > I also strengthened the test - I tried first to use sprintf instead of > printf, but that didn't work, as that doesn't seem to reproduce the > issue (my suspicion here is that 'printf' is optimized to look into > RAX). So, instead, I tweaked the test to run for a lot longer, by > repeatedly shuffling the permutation array, and I also increased the > decimal length of the double formatting string - that's because, > unfortunately, we were using 1.23, which has 4 chars, and sometimes > the result of printf ended up with "-NaN" which also has 4 chars, > despite being very wrong. With both measures, tests are much more > likely, so much so that in the affected machine I've never been able > to get a pass w/o the proposed patch. > > I verified that the patch works on all platforms, and I also verified > that the patch does not introduce performance regressions, which is > not the case (kind of expected, given that the only difference for > adding a "..." is the extra MOV to RAX, as shown here [1]). > > Webrev: > > http://cr.openjdk.java.net/~mcimadamore/panama/8218144/ > > Cheers > Maurizio > > [1] - https://gcc.godbolt.org/z/VAgfZc > > > > > From sundararajan.athijegannathan at oracle.com Thu Jan 31 15:17:27 2019 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Thu, 31 Jan 2019 20:47:27 +0530 Subject: [foreign] RFR 8218150: jextract should optionally generate a jmod file Message-ID: <5C531187.7000007@oracle.com> Please review. RFE: https://bugs.openjdk.java.net/browse/JDK-8218150 Webrev: https://cr.openjdk.java.net/~sundar/8218150/webrev.00/ Thanks, -Sundar From henry.jen at oracle.com Thu Jan 31 15:53:15 2019 From: henry.jen at oracle.com (Henry Jen) Date: Thu, 31 Jan 2019 07:53:15 -0800 Subject: [foreign] RFR 8218144: Direct invoker does not handle varargs correctly In-Reply-To: References: Message-ID: <73D25AF5-C04B-48C9-BAE6-4BBB1C07427F@oracle.com> Fix looks good, wondering why not just put down expected string in the test case instead of using String.format, which assumes format string is same for C and Java, while it maybe true, seems error prone to me. Also I am curious if varargs would change any performance on the DNI? Cheers, Henry > On Jan 31, 2019, at 6:10 AM, Maurizio Cimadamore wrote: > > Hi, > our CI systems detected a spurious failures on StdLibTest, only on Linux x64. After a lot of digging, I figured out the problem was that SystemV ABI requires the RAX register to be set to the number of vector arguments in a variadic call: > > %rax: temporary register;with variable arguments passes information about the number of vector registers used > > So, if we are doing a variadic call, we need to set this register to the number of floating point args we're passing. The universal invoker seems to do that: > > http://hg.openjdk.java.net/panama/dev/file/tip/src/hotspot/cpu/x86/universalNativeInvoker_x86.cpp#l515 > > For both variadic and non-variadic calls. > > So, a natural fix for the direct approach is to tweak the cast to the function pointers in our native stubs - e.g. from this: > > void DNI_invokeNative_V_JD(JNIEnv *env, jobject _unused, jlong addr, jlong arg0, jdouble arg1) { > ((void (*)(jlong, jdouble))addr)(arg0, arg1); > } > > To this: > > void DNI_invokeNative_V_JD(JNIEnv *env, jobject _unused, jlong addr, jlong arg0, jdouble arg1) { > ((void (*)(jlong, jdouble,...))addr)(arg0, arg1); > } > > Which will generate the right set for RAX (even though it will do so also for non-variadic call - but that's the same as what universal invoker does). > > I also strengthened the test - I tried first to use sprintf instead of printf, but that didn't work, as that doesn't seem to reproduce the issue (my suspicion here is that 'printf' is optimized to look into RAX). So, instead, I tweaked the test to run for a lot longer, by repeatedly shuffling the permutation array, and I also increased the decimal length of the double formatting string - that's because, unfortunately, we were using 1.23, which has 4 chars, and sometimes the result of printf ended up with "-NaN" which also has 4 chars, despite being very wrong. With both measures, tests are much more likely, so much so that in the affected machine I've never been able to get a pass w/o the proposed patch. > > I verified that the patch works on all platforms, and I also verified that the patch does not introduce performance regressions, which is not the case (kind of expected, given that the only difference for adding a "..." is the extra MOV to RAX, as shown here [1]). > > Webrev: > > http://cr.openjdk.java.net/~mcimadamore/panama/8218144/ > > Cheers > Maurizio > > [1] - https://gcc.godbolt.org/z/VAgfZc > > > > > From maurizio.cimadamore at oracle.com Thu Jan 31 16:17:20 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 31 Jan 2019 16:17:20 +0000 Subject: [foreign] RFR 8218144: Direct invoker does not handle varargs correctly In-Reply-To: <73D25AF5-C04B-48C9-BAE6-4BBB1C07427F@oracle.com> References: <73D25AF5-C04B-48C9-BAE6-4BBB1C07427F@oracle.com> Message-ID: <37487178-d2a6-d18d-4f92-8b9d095c8e54@oracle.com> On 31/01/2019 15:53, Henry Jen wrote: > Fix looks good, wondering why not just put down expected string in the test case instead of using String.format, which assumes format string is same for C and Java, while it maybe true, seems error prone to me. The test used to do that before - but then I realized it was more obscure than anything. For the simple strings generated by the test, String.format and printf produce same results. > > Also I am curious if varargs would change any performance on the DNI? As said in the RFR, I re-tested perfs after the patch and noticed no negative impact. Maurizio > > Cheers, > Henry > > >> On Jan 31, 2019, at 6:10 AM, Maurizio Cimadamore wrote: >> >> Hi, >> our CI systems detected a spurious failures on StdLibTest, only on Linux x64. After a lot of digging, I figured out the problem was that SystemV ABI requires the RAX register to be set to the number of vector arguments in a variadic call: >> >> %rax: temporary register;with variable arguments passes information about the number of vector registers used >> >> So, if we are doing a variadic call, we need to set this register to the number of floating point args we're passing. The universal invoker seems to do that: >> >> http://hg.openjdk.java.net/panama/dev/file/tip/src/hotspot/cpu/x86/universalNativeInvoker_x86.cpp#l515 >> >> For both variadic and non-variadic calls. >> >> So, a natural fix for the direct approach is to tweak the cast to the function pointers in our native stubs - e.g. from this: >> >> void DNI_invokeNative_V_JD(JNIEnv *env, jobject _unused, jlong addr, jlong arg0, jdouble arg1) { >> ((void (*)(jlong, jdouble))addr)(arg0, arg1); >> } >> >> To this: >> >> void DNI_invokeNative_V_JD(JNIEnv *env, jobject _unused, jlong addr, jlong arg0, jdouble arg1) { >> ((void (*)(jlong, jdouble,...))addr)(arg0, arg1); >> } >> >> Which will generate the right set for RAX (even though it will do so also for non-variadic call - but that's the same as what universal invoker does). >> >> I also strengthened the test - I tried first to use sprintf instead of printf, but that didn't work, as that doesn't seem to reproduce the issue (my suspicion here is that 'printf' is optimized to look into RAX). So, instead, I tweaked the test to run for a lot longer, by repeatedly shuffling the permutation array, and I also increased the decimal length of the double formatting string - that's because, unfortunately, we were using 1.23, which has 4 chars, and sometimes the result of printf ended up with "-NaN" which also has 4 chars, despite being very wrong. With both measures, tests are much more likely, so much so that in the affected machine I've never been able to get a pass w/o the proposed patch. >> >> I verified that the patch works on all platforms, and I also verified that the patch does not introduce performance regressions, which is not the case (kind of expected, given that the only difference for adding a "..." is the extra MOV to RAX, as shown here [1]). >> >> Webrev: >> >> http://cr.openjdk.java.net/~mcimadamore/panama/8218144/ >> >> Cheers >> Maurizio >> >> [1] - https://gcc.godbolt.org/z/VAgfZc >> >> >> >> >> From henry.jen at oracle.com Thu Jan 31 16:21:34 2019 From: henry.jen at oracle.com (Henry Jen) Date: Thu, 31 Jan 2019 08:21:34 -0800 Subject: [foreign] RFR 8218153: Read/Write of Value layout should honor declared endianness Message-ID: Hi, Please review the webrev[1] that perform necessary conversion based on declared endianness of Value layout. In the webrev, also fixed some issues: 1. enum constant need to have ACC_FINAL 2. union field offset is not correct [1] https://cr.openjdk.java.net/~henryjen/panama/endianness/03/webrev/ Cheers, Henry From maurizio.cimadamore at oracle.com Thu Jan 31 16:22:01 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 31 Jan 2019 16:22:01 +0000 Subject: [foreign] RFR 8218150: jextract should optionally generate a jmod file In-Reply-To: <5C531187.7000007@oracle.com> References: <5C531187.7000007@oracle.com> Message-ID: <3926d0ef-3019-8a24-cbbc-ab8f819fc05f@oracle.com> I like it! Very polished patch, and very good functionality too. This will enable packagers to deliver self contained JDK images which can play nicely with e.g. Docker. Maurizio On 31/01/2019 15:17, Sundararajan Athijegannathan wrote: > Please review. > > RFE: https://bugs.openjdk.java.net/browse/JDK-8218150 > Webrev: https://cr.openjdk.java.net/~sundar/8218150/webrev.00/ > > Thanks, > -Sundar From maurizio.cimadamore at oracle.com Thu Jan 31 16:30:18 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Thu, 31 Jan 2019 16:30:18 +0000 Subject: hg: panama/dev: Automatic merge with foreign Message-ID: <201901311630.x0VGUI2r017151@aojmv0008.oracle.com> Changeset: 2b0f3116896a Author: mcimadamore Date: 2019-01-31 17:30 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/2b0f3116896a Automatic merge with foreign From sundararajan.athijegannathan at oracle.com Thu Jan 31 16:28:10 2019 From: sundararajan.athijegannathan at oracle.com (sundararajan.athijegannathan at oracle.com) Date: Thu, 31 Jan 2019 16:28:10 +0000 Subject: hg: panama/dev: 8218150: jextract should optionally generate a jmod file Message-ID: <201901311628.x0VGSAlX016162@aojmv0008.oracle.com> Changeset: 72d6684b8cb1 Author: sundar Date: 2019-01-31 21:58 +0530 URL: http://hg.openjdk.java.net/panama/dev/rev/72d6684b8cb1 8218150: jextract should optionally generate a jmod file Reviewed-by: mcimadamore ! doc/panama_foreign.html ! doc/panama_foreign.md ! src/jdk.jextract/share/classes/com/sun/tools/jextract/Context.java + src/jdk.jextract/share/classes/com/sun/tools/jextract/JModWriter.java + src/jdk.jextract/share/classes/com/sun/tools/jextract/JarWriter.java ! src/jdk.jextract/share/classes/com/sun/tools/jextract/Main.java ! src/jdk.jextract/share/classes/com/sun/tools/jextract/SymbolFilter.java ! src/jdk.jextract/share/classes/com/sun/tools/jextract/Writer.java ! src/jdk.jextract/share/classes/com/sun/tools/jextract/resources/Messages.properties ! test/jdk/com/sun/tools/jextract/JextractToolProviderTest.java ! test/jdk/com/sun/tools/jextract/Runner.java From adam.pocock at oracle.com Thu Jan 31 16:45:20 2019 From: adam.pocock at oracle.com (Adam Pocock) Date: Thu, 31 Jan 2019 11:45:20 -0500 Subject: [vector] Error message in the fromArray length check is misleading In-Reply-To: References: <1e94ce7a-62b3-c155-751e-7cad2b9f6ad6@oracle.com> Message-ID: So that would mean replacing VectorIntrinsics line 405: ??? case 2: return Objects.checkIndex(ix, length - (vlen - 1)); with: ??? case 2: return Preconditions.checkIndex(ix, length - (vlen - 1), (String kind, List vals) -> new IndexOutOfBoundsException(String.format("Index %d out of bounds for length %d",vals.get(0)+(vlen-1),vals.get(1)+(vlen-1)))); I think you'd need to capture vlen to recover the correct end values for the indices, which makes it rather ugly. Maybe it could be moved onto the vectors themselves, that way the lambda could be static and it would pick up a static vlen? Adam On 30/01/2019 17:39, John Rose wrote: > On Jan 30, 2019, at 2:09 PM, Vladimir Ivanov > > > wrote: >> >> So, I'm in favor of improving error message itself leaving the check >> as is for now. >> > > This occasion was anticipated in the design of the Preconditions API. > You can plug in your own error reporter as a lambda, without abandoning > the intrinsic behavior. -- Adam Pocock Principal Member of Technical Staff Machine Learning Research Group Oracle Labs, Burlington, MA From maurizio.cimadamore at oracle.com Thu Jan 31 16:48:00 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Thu, 31 Jan 2019 16:48:00 +0000 Subject: hg: panama/dev: 8218144: Direct invoker does not handle varargs correctly Message-ID: <201901311648.x0VGm00b024113@aojmv0008.oracle.com> Changeset: ee90acb65346 Author: mcimadamore Date: 2019-01-31 16:47 +0000 URL: http://hg.openjdk.java.net/panama/dev/rev/ee90acb65346 8218144: Direct invoker does not handle varargs correctly Reviewed-by: sundar, hjen ! src/hotspot/share/prims/directNativeInvoker.cpp ! test/jdk/java/foreign/StdLibTest.java From maurizio.cimadamore at oracle.com Thu Jan 31 16:50:17 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Thu, 31 Jan 2019 16:50:17 +0000 Subject: hg: panama/dev: Automatic merge with foreign Message-ID: <201901311650.x0VGoHG2025206@aojmv0008.oracle.com> Changeset: 3759c761018e Author: mcimadamore Date: 2019-01-31 17:50 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/3759c761018e Automatic merge with foreign From maurizio.cimadamore at oracle.com Thu Jan 31 17:09:44 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Thu, 31 Jan 2019 17:09:44 +0000 Subject: hg: panama/dev: 8217989: Consolidate BoundedPointer implementation Message-ID: <201901311709.x0VH9jsP004468@aojmv0008.oracle.com> Changeset: 0a3d5d4c81a2 Author: mcimadamore Date: 2019-01-31 17:09 +0000 URL: http://hg.openjdk.java.net/panama/dev/rev/0a3d5d4c81a2 8217989: Consolidate BoundedPointer implementation Reviewed-by: sundar, hjen ! src/java.base/share/classes/java/foreign/memory/Array.java ! src/java.base/share/classes/java/foreign/memory/Pointer.java ! src/java.base/share/classes/java/foreign/memory/Struct.java ! src/java.base/share/classes/jdk/internal/foreign/ScopeImpl.java ! src/java.base/share/classes/jdk/internal/foreign/Util.java ! src/java.base/share/classes/jdk/internal/foreign/abi/DirectSignatureShuffler.java ! src/java.base/share/classes/jdk/internal/foreign/abi/UniversalUpcallHandler.java ! src/java.base/share/classes/jdk/internal/foreign/abi/x64/sysv/SysVx64ABI.java ! src/java.base/share/classes/jdk/internal/foreign/abi/x64/windows/Windowsx64ABI.java ! src/java.base/share/classes/jdk/internal/foreign/memory/BoundedArray.java ! src/java.base/share/classes/jdk/internal/foreign/memory/BoundedPointer.java ! src/java.base/share/classes/jdk/internal/foreign/memory/References.java ! test/jdk/java/foreign/ScopeTest.java From maurizio.cimadamore at oracle.com Thu Jan 31 17:08:37 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 31 Jan 2019 17:08:37 +0000 Subject: [foreign] RFR 8218153: Read/Write of Value layout should honor declared endianness In-Reply-To: References: Message-ID: Hi Henry, the code looks good, but stepping back I think there's an overall theme here of what the right choice for the default should be, and the tension between implicit and explicit. So far we have erred on the side of explicit-ness, arguing that implicit info is typically a place where subtle bugs can hide. I think endianness put this assumption to the test, in a way that I will try to capture in this email. tl;dr; the name of the game is _inferring_ sensible defaults; this doesn't mean that the underlying layouts have no explicit information, just that the explicit information is _inferred_. First, let's remove jextract from the discussion - this discussion is NOT about having the binder magically guessing what the right layout should be - we assume that jextract will construct the fully explicit layouts, as per the descriptions in the annotations (which are fully explicit). This discussion is then mostly about programmatic access - and what defaults should the API provide (if any); I see mainly two places that would be affected by this: *) constants such as NativeTypes.INT32 - should this be LE, BE or PE ? *) When clients build a value layout w/o specifying endianness, should the result be LE, BE or PE? I think that we have three options: a) default means platform endianness b) default means something specific, either LE or BE c) we remove defaults - e.g. not possible to construct value layout w/o endianness, no NativeTypes constants w/o endianness - everything explicit Let's consider the options: I believe (b) will be problematic -? as visible in the memory region code, if a user creates/uses a int native type for int he would expect to be able to read an int - example: try (Scope sc = Scope.newNativeScope) { ??? Pointer pi = sc.allocate(NativeTypes.INT32); ??? pi.set(42); ??? assertEquals(pi.get(), 42); } If the above test fails in a platform-dependent way, I think users will be very confused! Which leaves us to either (a) or (c). (c) seems a bit of a nuclear option - of course it is a possibility that is self-consistent - but now all users will have to reason about endianness when writing simple snippets such as the one above. If you have jextract to help you, this is not an issue, as the correct information will be extracted from the C API, but here we're talking about a piece of Java code, and its ability to be portable across platforms. So this decision affects deeply the usability of our off-heap API. (a) seems of course the most user-friendly option - if the user doesn't specify anything, we pick something that makes sense (e.g. where ints can be written and re-read on a memory location w/o extra swaps :-)). The cost this approach has is more evident when different pieces of code are interacting using different endiannes - let's imagine: try (Scope sc = Scope.newNativeScope) { ??? Pointer pi = sc.allocate(NativeTypes.INT32); ??? Pointer pi_le = sc.allocate(NativeTypes.LE_INT32); ??? pi.set(42); ??? Pointer.copy(pi, pi_le); ??? assertEquals(pi_le.get(), 42); //?? } The result of this operation is, again, going to be platform-dependent - for platforms which are LE, this test passes - for BE platforms this test fails (as our pointer copy cannot take care of swapping LE/BE bits). Overall, I think there's no 'right' solution here. It is pretty much a 'pick your poison scenario' - do we care more about usability? Or do we care more about correctness? The former implies (a), the latter implies (c). As much as I'd like to strive for correctness, I'm well aware that if we go down the (c) path, and make it impossible for people to say INT32 w/o specifying endianness, (1) people will be mad and (2) after finished fuming, they will immediately write code like this: static LayoutType INT32 =? ByteOrder.nativeOrder() == ByteOrder.BIG_ENDIAN ? ???????? NativeTypes.BE_INT32 : NativeTypes.LE_INT32; // why, oh why? And the resulting code might not be that much better off. So, this might be one of those cases where being too paternalistic could backfire. So, out on a limb, I'd say let's pick (a). Also, as discussed offline with Henry, I think it's also important that, if we go down the (a) path, "inferring" the right endianness is something that happens when we create the layout - after which we should be able to see which endianness has been inferred by querying the layout API. This IMHO makes swallowing (a) easier, as it's always possible to recover the info to diagnose what is going on. If we like this idea of 'inferring' information to offer a more usable API, then I'd say that we should also provide _unsized_ constants in NativeTypes like INT, LONG etc. whose size will be inferred at construction time (again the user can query that size if he wants to). This would also eliminate the need for having the Type enum in the SystemABI interface. Thoughts? Maurizio On 31/01/2019 16:21, Henry Jen wrote: > Hi, > > Please review the webrev[1] that perform necessary conversion based on declared endianness of Value layout. > > In the webrev, also fixed some issues: > > 1. enum constant need to have ACC_FINAL > 2. union field offset is not correct > > [1] https://cr.openjdk.java.net/~henryjen/panama/endianness/03/webrev/ > > Cheers, > Henry > From maurizio.cimadamore at oracle.com Thu Jan 31 17:15:21 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Thu, 31 Jan 2019 17:15:21 +0000 Subject: hg: panama/dev: Automatic merge with foreign Message-ID: <201901311715.x0VHFLKO006999@aojmv0008.oracle.com> Changeset: bdd7502c62c1 Author: mcimadamore Date: 2019-01-31 18:15 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/bdd7502c62c1 Automatic merge with foreign From john.r.rose at oracle.com Thu Jan 31 18:41:16 2019 From: john.r.rose at oracle.com (John Rose) Date: Thu, 31 Jan 2019 10:41:16 -0800 Subject: [foreign] RFR 8218153: Read/Write of Value layout should honor declared endianness In-Reply-To: References: Message-ID: On Jan 31, 2019, at 9:08 AM, Maurizio Cimadamore wrote: > > Thoughts? You already know my main thought here: It's easy to create deep future troubles with these choices. Some more thoughts on this thorny problem: I think it's reasonable to opt in explicitly to platform polarity and even platform sizes. So, yes, there should be constants that provide all of that. But there also need to be ways to get precision, without having the platform in the way. One key question is what should be the *first* set of types that a Panama user encounters? That first set is the set of types which sets the overall tone of either precision or magical platform dependence. The magic in the latter case feels like a creature comfort at first. And then your system size grows to include some portability requirement, such as WORA or network protocols. At that point all of your creature comforts become crawling bugs. You hit a wall until you find all of them. If you didn't opt into them explicitly, then it takes a very long time to find them all, and your portability story keeps failing until you find them all. This took *years* to do in HotSpot when we went from x86 to x86+SPARC. It was miserable. There were many bad fixes due to some engineer hopefully swapping bytes at one point, and later finding out the bad order was somewhere else. I think there are still bad spots where we have a double-swap somewhere, or a poorly named hi/lo or first/second distinction that no longer makes sense. Moral of the story: You can't take back a decision to ignore byte order or integer size, without spending months of reengineering and bug chasing. Let's not do that in Panama. One reason I care about portability here is that portability is one of the values that Java adds to C, and Panama is the place where C libraries can be upgraded to play in Java's world. Once you are coding in Java, you (usually) don't have to worry platform dependencies. In Panama, jextract is where the dependencies are injected, and it's clear that a jextract-generated API is platform dependent. But writing Java code from scratch needs to be platform-independent until proven otherwise (which means an explicit opt-in). (Similar points can be made about safety. Java is safe routinely where C is unsafe routinely. Java code from scratch should be as safe and portable as we can make it.) So, my conviction is that when a user programs with the Java API, as opposed with jextract, we need to make sure the first names that user encounters are the solid names. Solid names don't have secret platform dependencies on size or byte order. Does the user not care about platform neutrality? That's fine, just opt into the platform specific names. I would prefer to do this with an import of a nested class, so there's evidence at the top of the source file, and in the classfile, that platform dependencies are being injected into the code. import static java.foreign.NativeType.*; // solid types only import static java.foreign.CurrentPlatform.*; // platform-endian types import static java.foreign.CurrentCType.*; // platform types defined by C (I just noticed that endian polarity and int-size work like locale do in string operations. Regarding locale, I think our overall practice is to back away from "magic" APIs which vary their behavior based on what country the JVM woke up in. IIRC in the early days of Java there were more such "magic" APIs, because who could object to helping the programmer make an easy decision? Let's learn from our past mistakes!) By the way, platform sizes are not CPU-specific but ABI-specific and even C-language specific. The endian polarity of the platform is visible even if you are staying inside of Java, because of the byte order of objects on the heap. But there's never any doubt about the size of Java values. That's why I posit three sets of names in the import above. Now for the portable names, there's the question of whether LE or BE should be the default, or whether both should be explicit. I'd be fine either any of those three answers, because, given an assurance that the names are not "magic" and don't secretly change their meanings, a programmer can reasonably learn any fixed convention. I think little-endian is a graceful choice for a fixed convention, but I would hate to waste time replaying a tedious flame war between endian advocates. Anybody familiar with assembly-level programming in both polarities can form a pretty clear opinion as to which convention is slightly more natural than the other, depending on their own personal definition of "natural". And they should probably keep it to themselves. One way to please everybody on polarity would be to (again) supply a way to make an explicit import, at the header of the source file, to show exactly what's going one: import static java.foreign.NativeType.LE.*; //or import static java.foreign.NativeType.BE.*; And then we have NativeType.LE_INT32 and *also* NativeType.BE.INT32. A funny naming convention invented, and everybody queues up at their chosen window. No confusion, because every source file (and every classfile) says exactly what are the ground rules. ? John From john.r.rose at oracle.com Thu Jan 31 18:42:55 2019 From: john.r.rose at oracle.com (John Rose) Date: Thu, 31 Jan 2019 10:42:55 -0800 Subject: [vector] Error message in the fromArray length check is misleading In-Reply-To: References: <1e94ce7a-62b3-c155-751e-7cad2b9f6ad6@oracle.com> Message-ID: <8345414D-9B29-4D8D-98E4-F5FFBEB6880E@oracle.com> On Jan 31, 2019, at 8:45 AM, Adam Pocock wrote: > > I think you'd need to capture vlen to recover the correct end values for the indices, which makes it rather ugly. Maybe it could be moved onto the vectors themselves, that way the lambda could be static and it would pick up a static vlen? Yes, you cottoned onto the capture problem quickly. To display vlen in the error message, the right way to go is statically create on lambda for each possible vlen. From sandhya.viswanathan at intel.com Thu Jan 31 18:45:13 2019 From: sandhya.viswanathan at intel.com (sandhya.viswanathan at intel.com) Date: Thu, 31 Jan 2019 18:45:13 +0000 Subject: hg: panama/dev: Summary: API changes for rebracket, reshape, resize and cast. Message-ID: <201901311845.x0VIjEXL011554@aojmv0008.oracle.com> Changeset: f15b67c4b503 Author: kkharbas Date: 2019-01-31 10:50 -0800 URL: http://hg.openjdk.java.net/panama/dev/rev/f15b67c4b503 Summary: API changes for rebracket, reshape, resize and cast. Reviewed-by: vlivanov, goetz ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Byte128Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Byte256Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Byte512Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Byte64Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/ByteMaxVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/ByteVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Double128Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Double256Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Double512Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Double64Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/DoubleMaxVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/DoubleVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Float128Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Float256Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Float512Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Float64Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/FloatMaxVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/FloatVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Int128Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Int256Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Int512Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Int64Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/IntMaxVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/IntVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Long128Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Long256Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Long512Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Long64Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/LongMaxVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/LongVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Short128Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Short256Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Short512Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Short64Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/ShortMaxVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/ShortVector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/Vector.java ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/X-Vector.java.template ! src/jdk.incubator.vector/share/classes/jdk/incubator/vector/X-VectorBits.java.template ! test/jdk/jdk/incubator/vector/CovarOverrideTest.java ! test/jdk/jdk/incubator/vector/VectorHash.java ! test/jdk/jdk/incubator/vector/VectorReshapeTests.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/AbstractVectorBenchmark.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Byte128Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Byte256Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Byte512Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Byte64Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/ByteMaxVector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/ByteScalar.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Double128Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Double256Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Double512Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Double64Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/DoubleMaxVector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/DoubleScalar.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Float128Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Float256Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Float512Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Float64Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/FloatMaxVector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/FloatScalar.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Int128Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Int256Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Int512Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Int64Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/IntMaxVector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/IntScalar.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Long128Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Long256Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Long512Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Long64Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/LongMaxVector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/LongScalar.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/PopulationCount.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Short128Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Short256Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Short512Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/Short64Vector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/ShortMaxVector.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/ShortScalar.java ! test/jdk/jdk/incubator/vector/benchmark/src/main/java/benchmark/jdk/incubator/vector/SumOfUnsignedBytes.java From maurizio.cimadamore at oracle.com Thu Jan 31 18:50:27 2019 From: maurizio.cimadamore at oracle.com (maurizio.cimadamore at oracle.com) Date: Thu, 31 Jan 2019 18:50:27 +0000 Subject: hg: panama/dev: Automatic merge with vectorIntrinsics Message-ID: <201901311850.x0VIoSKh013324@aojmv0008.oracle.com> Changeset: 5216c920ce97 Author: mcimadamore Date: 2019-01-31 19:50 +0100 URL: http://hg.openjdk.java.net/panama/dev/rev/5216c920ce97 Automatic merge with vectorIntrinsics From maurizio.cimadamore at oracle.com Thu Jan 31 18:59:42 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 31 Jan 2019 18:59:42 +0000 Subject: [foreign] RFR 8218153: Read/Write of Value layout should honor declared endianness In-Reply-To: References: Message-ID: <7dfef1b6-decd-5084-7b29-9fb5e1945d19@oracle.com> I think all this goes for the nuclear option (c), with some static import reliefs. I also think that inferring LE or BE always (as per my option b) is as wrong as inferring platform endianness - in terms of place where bugs can hide, and difficulty in terms of finding where such bugs are coming from (because it's implicit). So, I think the Value layout API should NOT have a default constructor w/o endianness. Maurizio On 31/01/2019 18:41, John Rose wrote: > On Jan 31, 2019, at 9:08 AM, Maurizio Cimadamore > > wrote: >> >> Thoughts? > > You already know my main thought here: ?It's easy to > create deep future troubles with these choices. > > Some more thoughts on this thorny problem: ?I think > it's reasonable to opt in explicitly to platform polarity > and even platform sizes. ?So, yes, there should be constants > that provide all of that. > > But there also need to be ways to get precision, without > having the platform in the way. > > One key question is what should be the *first* set of > types that a Panama user encounters? ?That first set > is the set of types which sets the overall tone of either > precision or magical platform dependence. > > The magic in the latter case feels like a creature comfort > at first. ?And then your system size grows to include some > portability requirement, such as WORA or network protocols. > At that point all of your creature comforts become crawling > bugs. ?You hit a wall until you find all of them. ?If you didn't > opt into them explicitly, then it takes a very long time to > find them all, and your portability story keeps failing until > you find them all. > > This took *years* to do in HotSpot when we went from x86 > to x86+SPARC. ?It was miserable. ?There were many bad > fixes due to some engineer hopefully swapping bytes at > one point, and later finding out the bad order was > somewhere else. ?I think there are still bad spots where > we have a double-swap somewhere, or a poorly named > hi/lo or first/second distinction that no longer makes sense. > Moral of the story: ?You can't take back a decision to ignore > byte order or integer size, without spending months of > reengineering and bug chasing. ?Let's not do that in Panama. > > One reason I care about portability here is that portability > is one of the values that Java adds to C, and Panama is the > place where C libraries can be upgraded to play in Java's > world. ?Once you are coding in Java, you (usually) don't > have to worry platform?dependencies. ?In Panama, jextract > is where the dependencies are injected, and it's clear that > a jextract-generated API is platform dependent. ?But writing > Java code from scratch needs to be platform-independent > until proven otherwise (which means an explicit opt-in). > > (Similar points can be made about safety. ?Java is safe > routinely where C is unsafe routinely. ?Java code from > scratch should be as safe and portable as we can make it.) > > So, my conviction is that when a user programs with the > Java API, as opposed with jextract, we need to make sure > the first names that user encounters are the solid names. > Solid names don't have secret platform dependencies on > size or byte order. ?Does the user not care about platform > neutrality? ?That's fine, just opt into the platform specific > names. ?I would prefer to do this with an import of a > nested class, so there's evidence at the top of the source > file, and in the classfile, that platform dependencies are > being injected into the code. > > import static java.foreign.NativeType.*; ?// solid types only > import static java.foreign.CurrentPlatform.*; ?// platform-endian types > import static java.foreign.CurrentCType.*; ?// platform types defined by C > > (I just noticed that endian polarity and int-size work > like locale do in string operations. ?Regarding locale, > I think our overall practice is to back away from "magic" > APIs which vary their behavior based on what country > the JVM woke up in. ?IIRC in the early days of Java there > were more such "magic" APIs, because who could object > to helping the programmer make an easy decision? > Let's learn from our past mistakes!) > > By the way, platform sizes are not CPU-specific but > ABI-specific and even C-language specific. ?The > endian polarity of the platform is visible even if you > are staying inside of Java, because of the byte order > of objects on the heap. ?But there's never any doubt > about the size of Java values. ?That's why I posit three > sets of names in the import above. > > Now for the portable names, there's the question of > whether LE or BE should be the default, or whether > both should be explicit. ?I'd be fine either any of > those three answers, because, given an assurance > that the names are not "magic" and don't secretly > change their meanings, a programmer can reasonably > learn any fixed convention. ?I think little-endian > is a graceful choice for a fixed convention, but I > would hate to waste time replaying a tedious flame > war between endian advocates. ?Anybody familiar > with assembly-level programming in both polarities > can form a pretty clear opinion as to which convention > is slightly more natural than the other, depending > on their own personal definition of "natural". ?And > they should probably keep it to themselves. > > One way to please everybody on polarity would be > to (again) supply a way to make an explicit import, > at the header of the source file, to show exactly > what's going one: > > import static java.foreign.NativeType.LE.*; > //or import static java.foreign.NativeType.BE.*; > > And then we have NativeType.LE_INT32 and > *also* NativeType.BE.INT32. ?A funny naming > convention invented, and everybody queues > up at their chosen window. ?No confusion, > because every source file (and every classfile) > says exactly what are the ground rules. > > ? John From sandhya.viswanathan at intel.com Thu Jan 31 19:15:34 2019 From: sandhya.viswanathan at intel.com (Viswanathan, Sandhya) Date: Thu, 31 Jan 2019 19:15:34 +0000 Subject: [vector] RFR: Wrong offset is computed in Mask.maskFromArray() In-Reply-To: <9b8810ee-3e51-75fe-8009-ba2838a7c66f@oracle.com> References: <9b8810ee-3e51-75fe-8009-ba2838a7c66f@oracle.com> Message-ID: <02FCFB8477C4EF43A2AD8E0C60F3DA2BB1A560A8@FMSMSX126.amr.corp.intel.com> Looks good to me. Best Regards, Sandhya -----Original Message----- From: panama-dev [mailto:panama-dev-bounces at openjdk.java.net] On Behalf Of Vladimir Ivanov Sent: Wednesday, January 30, 2019 6:07 PM To: panama-dev at openjdk.java.net Subject: [vector] RFR: Wrong offset is computed in Mask.maskFromArray() http://cr.openjdk.java.net/~vlivanov/panama/vector/mask_mem/webrev.00/ Wrong offset is computed in Mask.maskFromArray() accesses. Best regards, Vladimir Ivanov From john.r.rose at oracle.com Thu Jan 31 19:51:54 2019 From: john.r.rose at oracle.com (John Rose) Date: Thu, 31 Jan 2019 11:51:54 -0800 Subject: [foreign] RFR 8218153: Read/Write of Value layout should honor declared endianness In-Reply-To: <7dfef1b6-decd-5084-7b29-9fb5e1945d19@oracle.com> References: <7dfef1b6-decd-5084-7b29-9fb5e1945d19@oracle.com> Message-ID: I would be very happy with this outcome, and I think we can also make the users happy (in their various kinds and factions), by providing a tidy menu of import options. On Jan 31, 2019, at 10:59 AM, Maurizio Cimadamore wrote: > > I think all this goes for the nuclear option (c), with some static import reliefs. > > I also think that inferring LE or BE always (as per my option b) is as wrong as inferring platform endianness - in terms of place where bugs can hide, and difficulty in terms of finding where such bugs are coming from (because it's implicit). > > So, I think the Value layout API should NOT have a default constructor w/o endianness. > > From henry.jen at oracle.com Thu Jan 31 21:56:06 2019 From: henry.jen at oracle.com (Henry Jen) Date: Thu, 31 Jan 2019 13:56:06 -0800 Subject: [foreign] RFR 8218153: Read/Write of Value layout should honor declared endianness In-Reply-To: References: <7dfef1b6-decd-5084-7b29-9fb5e1945d19@oracle.com> Message-ID: Great, I?ll get rid of default endianness construction in Value.java. What?s left is how we divide those LayoutType constants for import(a bike shed), So here is what I think, java.foreign.NativeTypes.*; // Those are for portable types, explicit length, endianness same as JVM(platform) java.foreign.NativeTypes.LE.*; // Portable Little Endian types java.foreign.NativeTypes.BE.*; // Portable Big Endian types java.foreign.CTypes..*; // Types that are matching C types and ABI dependent If we want to be even more strict, further split NativeTypes.* into java.foreign.NativeTypes.*; // // Those are for completely portable types, explicit length, endianness is not relevant, like VOID, INT8 java.foreign.Platform.*; // Those are for portable types safe on same VM architecture, explicit length and same endianness as JVM, INT16, INT32, INT64 I think we need separate class to make the import required for platform specific types, so that as John suggested, we will have clear evidence on top knowing the class maybe platform specific. Same reason we probably should use nested class for BE/LE instead of prefixed type names, as we don?t need both. Note that we might want to have LE/BE variants for CTypes as well since endianness and size are orthogonal. As what we have in https://hg.openjdk.java.net/panama/dev/file/5216c920ce97/src/java.base/share/classes/jdk/internal/foreign/memory/Types.java btw, which is only valid for x86 at the moment. Suggestions? Cheers, Henry > On Jan 31, 2019, at 11:51 AM, John Rose wrote: > > I would be very happy with this outcome, and I think we can also > make the users happy (in their various kinds and factions), > by providing a tidy menu of import options. > > On Jan 31, 2019, at 10:59 AM, Maurizio Cimadamore wrote: >> >> I think all this goes for the nuclear option (c), with some static import reliefs. >> >> I also think that inferring LE or BE always (as per my option b) is as wrong as inferring platform endianness - in terms of place where bugs can hide, and difficulty in terms of finding where such bugs are coming from (because it's implicit). >> >> So, I think the Value layout API should NOT have a default constructor w/o endianness. >> >> > From samuel.audet at gmail.com Thu Jan 31 22:40:38 2019 From: samuel.audet at gmail.com (Samuel Audet) Date: Fri, 1 Feb 2019 07:40:38 +0900 Subject: reusing more of LLVM In-Reply-To: <20528665-d4be-e67d-83bb-2060344ce9e0@oracle.com> References: <6fbff2eb-b100-0c03-7663-c732a3904302@oracle.com> <3e4546e7-f768-6550-3a31-b38dae835d7b@oracle.com> <6E570AB5-88D3-43E6-B58C-E2297CD6D5CA@oracle.com> <20528665-d4be-e67d-83bb-2060344ce9e0@oracle.com> Message-ID: <42ab4785-02b1-1bfc-ba28-6052c7497d2f@gmail.com> Sure, I will start a new thread. I figured my main concern is that Panama appears to be reimplementing portions of LLVM in Java as part of SystemABI. Why not reuse this functionality from LLVM? What are the issues? Why is this not documented anywhere? From discussions I see on, for example, http://openjdk.java.net/jeps/8189173 it sounds to me like OpenJDK simply does not have what it takes to deal with the LLVM community, but am I missing something? Are there technical limitations as well? Frankly though, just looking at what https://www.azul.com/products/zing/ has achieved, I don't see any, but am I mistaken? Samuel On 1/30/19 7:59 PM, Maurizio Cimadamore wrote: > Hi Samuel, > the point you raise has nothing to do with the scope discussion. Please > kindly consider moving it to a separate thread. > > Cheers > Maurizio > > On 30/01/2019 02:21, Samuel Audet wrote: >> Hi, >> >> I would like to hear more about what John mentions below about being >> "challenging enough" and leaving work for the future. I would go one >> step further. The Substrate VM team already has something working as >> part of their C interface, and they did not need to parse anything to >> get it working. It does not try to do as much as Panama, but they did >> what is in my opinion most important: A simpler more efficient variant >> of JNI. If we could get this bit of Panama stabilized first, instead >> of trying to parse everything, that would be great I think. What do >> you think? Or if that is not possible, how is Panama different from >> Substrate VM such that the same approach would not work? >> >> I am starting the get the impression that Panama is basically >> reimplementing portions of LLVM in Java as part of SystemABI. How much >> effort would it take to, for example, implement the required subset of >> the Java ABI in LLVM? Doing it that way, we would not need to test the >> C/C++ ABI, at least, making it possible instead to reuse tests from >> the JDK itself, or am I talking nonsense here? Looks like that might >> be challenging as well: http://openjdk.java.net/jeps/8189173. In any >> case, it would be great if we could have a discussion about these >> things instead of having OpenJDK dictate everything! >> >> Samuel >> >> On 1/30/19 5:03 AM, John Rose wrote: >>> Currently we are focusing on accurately extracting all possible >>> raw APIs, and providing efficient access to them.? This is >>> challenging enough for now.? So you won't see much help >>> for civilizing yet.? I expect that folks will start to experiment >>> with civilizing layers when the raw extraction mechanisms >>> stabilize. From john.r.rose at oracle.com Thu Jan 31 23:00:52 2019 From: john.r.rose at oracle.com (John Rose) Date: Thu, 31 Jan 2019 15:00:52 -0800 Subject: [foreign] RFR 8218153: Read/Write of Value layout should honor declared endianness In-Reply-To: References: <7dfef1b6-decd-5084-7b29-9fb5e1945d19@oracle.com> Message-ID: On Jan 31, 2019, at 1:56 PM, Henry Jen wrote: > > Great, I?ll get rid of default endianness construction in Value.java. What?s left is how we divide those LayoutType constants for import(a bike shed), > > So here is what I think, > > java.foreign.NativeTypes.*; // Those are for portable types, explicit length, endianness same as JVM(platform) I'd rather have a prefix on those too, so I prefer your suggestion further down. > java.foreign.NativeTypes.LE.*; // Portable Little Endian types > java.foreign.NativeTypes.BE.*; // Portable Big Endian types > java.foreign.CTypes..*; // Types that are matching C types and ABI dependent Yes. Or possibly ABITypes, with the understanding that C and the ABI are interdependent. > If we want to be even more strict, further split NativeTypes.* into > > java.foreign.NativeTypes.*; // // Those are for completely portable types, explicit length, endianness is not relevant, like VOID, INT8 There are only a few of these; they can be duplicated on both sides. > java.foreign.Platform.*; // Those are for portable types safe on same VM architecture, explicit length and same endianness as JVM, INT16, INT32, INT64 Yes, platform is OK. I expect many users will want to say import static java.foreign.Platform.* and be done. There might be a place to use static inheritance to organize subsets and supersets of these names. I'm thinking maybe CTypes <: Platform for example, since the ABI types are a superset of the platform types. But maybe the names are all disjoint, in which case it's moot. > I think we need separate class to make the import required for platform specific types, so that as John suggested, we will have clear evidence on top knowing the class maybe platform specific. Same reason we probably should use nested class for BE/LE instead of prefixed type names, as we don?t need both. Yes, I suppose we don't need both kinds of names, and given a choice we should have more name spaces rather than fewer, to allow selective imports that distinguish between Platform (which is pure Java) and ABI (which brings in dependencies on C definitions). Also BE and LE for people working with protocols and other rigidly defined formats. For the record, I don't mind the fact that BE.INT32 and LE.INT32 have the same name, even though it could potentially be confusing to the reader of code far from the import statement. It's a stylistic choice for the writer of the code: Either declare "house rules" for endian polarity and use the ambiguous names, or else require a BE or LE prefix everywhere. (Or a mix; that's possible also.) For this particular application, having two meanings for one importable name is OK, since the point of ambiguity is well known and there are several conventions for dealing with it. We allow coders to pick their convention. > Note that we might want to have LE/BE variants for CTypes as well since endianness and size are orthogonal. As what we have in > https://hg.openjdk.java.net/panama/dev/file/5216c920ce97/src/java.base/share/classes/jdk/internal/foreign/memory/Types.java > btw, which is only valid for x86 at the moment. Rather than adding even more names, I would prefer to have an operator on the constants to switch polarity. Something like: assert BE.INT32.endian(Endian.LE) == LE.INT32; This is the sort of degree of freedom that the intrinsic API is designed to support: https://openjdk.java.net/jeps/348 The idea is that a call to an intrinsic method on a constant will also be a constant. ? John > > Suggestions? > > Cheers, > Henry > > >> On Jan 31, 2019, at 11:51 AM, John Rose wrote: >> >> I would be very happy with this outcome, and I think we can also >> make the users happy (in their various kinds and factions), >> by providing a tidy menu of import options. >> >> On Jan 31, 2019, at 10:59 AM, Maurizio Cimadamore wrote: >>> >>> I think all this goes for the nuclear option (c), with some static import reliefs. >>> >>> I also think that inferring LE or BE always (as per my option b) is as wrong as inferring platform endianness - in terms of place where bugs can hide, and difficulty in terms of finding where such bugs are coming from (because it's implicit). >>> >>> So, I think the Value layout API should NOT have a default constructor w/o endianness. >>> >>> >> > From maurizio.cimadamore at oracle.com Thu Jan 31 23:41:13 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 31 Jan 2019 23:41:13 +0000 Subject: [foreign] RFR 8218153: Read/Write of Value layout should honor declared endianness In-Reply-To: References: <7dfef1b6-decd-5084-7b29-9fb5e1945d19@oracle.com> Message-ID: <6f250562-b7ef-21cb-1a43-aff449a45b20@oracle.com> Can I suggest that, from a logistical point of view, we split the original changeset from the static import discussion? Which kind of imports we need in practice is a question that will be much better answered by looking at concrete use cases, and using the API in anger, rather than trying to abstractly find the best possible split. Also, where to put these extra constants depend, in my view, on what we do with SystemABI - if that would to become a public type, I think that would be a natural place where to put some of these. Maurizio On 31/01/2019 21:56, Henry Jen wrote: > Great, I?ll get rid of default endianness construction in Value.java. What?s left is how we divide those LayoutType constants for import(a bike shed), > > So here is what I think, > > java.foreign.NativeTypes.*; // Those are for portable types, explicit length, endianness same as JVM(platform) > java.foreign.NativeTypes.LE.*; // Portable Little Endian types > java.foreign.NativeTypes.BE.*; // Portable Big Endian types > java.foreign.CTypes..*; // Types that are matching C types and ABI dependent > > If we want to be even more strict, further split NativeTypes.* into > > java.foreign.NativeTypes.*; // // Those are for completely portable types, explicit length, endianness is not relevant, like VOID, INT8 > java.foreign.Platform.*; // Those are for portable types safe on same VM architecture, explicit length and same endianness as JVM, INT16, INT32, INT64 > > I think we need separate class to make the import required for platform specific types, so that as John suggested, we will have clear evidence on top knowing the class maybe platform specific. Same reason we probably should use nested class for BE/LE instead of prefixed type names, as we don?t need both. > > Note that we might want to have LE/BE variants for CTypes as well since endianness and size are orthogonal. As what we have in > https://hg.openjdk.java.net/panama/dev/file/5216c920ce97/src/java.base/share/classes/jdk/internal/foreign/memory/Types.java > btw, which is only valid for x86 at the moment. > > Suggestions? > > Cheers, > Henry > > >> On Jan 31, 2019, at 11:51 AM, John Rose wrote: >> >> I would be very happy with this outcome, and I think we can also >> make the users happy (in their various kinds and factions), >> by providing a tidy menu of import options. >> >> On Jan 31, 2019, at 10:59 AM, Maurizio Cimadamore wrote: >>> I think all this goes for the nuclear option (c), with some static import reliefs. >>> >>> I also think that inferring LE or BE always (as per my option b) is as wrong as inferring platform endianness - in terms of place where bugs can hide, and difficulty in terms of finding where such bugs are coming from (because it's implicit). >>> >>> So, I think the Value layout API should NOT have a default constructor w/o endianness. >>> >>>