Generic (void *)int
Maurizio Cimadamore
maurizio.cimadamore at oracle.com
Wed Feb 20 00:44:54 UTC 2019
On 20/02/2019 00:40, Maurizio Cimadamore wrote:
>
>
> On 20/02/2019 00:35, Giuseppe Barbieri wrote:
>> elect at elect-NUC8i5BEK:~/IdeaProjects/panama-dev$ bash configure
>> --disable-warnings-as-errors
>> --with-libclang=../../Documents/clang+llvm-7.0.1-x86_64-linux-gnu-ubuntu-18.04
>>
>> I dont think `..` may be the problem, because for example
>> `clang-c/Index.h` is found without issues
>>
>> this is my tree:
>> https://gist.github.com/elect86/74c9c95fd9327fb83ffefa0af3a3ec48
>>
>> Do you also use the very same snapshot?
>
> Lastest one I tried was 7.0.0 (Ubuntu 16.04 version, but that should
> hardly make a difference?)
>
> Have you tried without '..' (e.g. absolute path) ?
>
If the absolute path is not an issue, then I think it could be a
'configure' issue with 18.10 - you have a very recent GCC compiler - and
I bet that if you try to compile a sample program which call that
function (clang_getClangVersion), it will fail to link or something like
that. That's what configure does - it create a small snippet which calls
a function and verifies that it compiles & link correctly. The other
checks are simpler - they just check for the presence/absence of certain
files.
Maurizio
> Maurizio
>
>>
>> Il giorno mer 20 feb 2019 alle ore 01:31 Maurizio Cimadamore
>> <maurizio.cimadamore at oracle.com
>> <mailto:maurizio.cimadamore at oracle.com>> ha scritto:
>>
>>
>> On 20/02/2019 00:22, Giuseppe Barbieri wrote:
>>> configure: error: Cannot locate libclang or headers at the
>>> specified locations:
>>> /home/elect/Documents/clang+llvm-7.0.1-x86_64-linux-gnu-ubuntu-18.04/lib
>>> /home/elect/Documents/clang+llvm-7.0.1-x86_64-linux-gnu-ubuntu-18.04/include
>>>
>>> Note, I had to choose 18.04 although I have 18.10 because it was
>>> the newest available
>>
>> That's ok, I think.
>>
>> And, are you passing to --with-libclang the following?
>>
>> /home/elect/Documents/clang+llvm-7.0.1-x86_64-linux-gnu-ubuntu-18.04
>>
>> If so, do the include/ and lib/ folder exist under such folder?
>>
>> e.g. in my machine:
>>
>> $ tree -L 1 /opt/clang/clang+llvm-6.0.1-x86_64-linux-gnu-ubuntu-16.04
>> /opt/clang/clang+llvm-6.0.1-x86_64-linux-gnu-ubuntu-16.04
>> ├── bin
>> ├── include
>> ├── lib
>> ├── libexec
>> └── share
>>
>> Maurizio
>>
>>>
>>> Anyway, I'd not like to abuse your time, I can give up for the
>>> moment waiting until you or someone else will try and write down
>>> a guide for building on an updated ubuntu/linux system.
>>>
>>> Il giorno mer 20 feb 2019 alle ore 00:37 Maurizio Cimadamore
>>> <maurizio.cimadamore at oracle.com
>>> <mailto:maurizio.cimadamore at oracle.com>> ha scritto:
>>>
>>>
>>> On 19/02/2019 22:33, Giuseppe Barbieri wrote:
>>>> I'll do as you said
>>>>
>>>> By "snapshot", do you mean the pre-built binaries?
>>> Yes
>>>>
>>>> Just tried but apparently there seems to be no such option
>>>
>>> you need to update the branch
>>>
>>> $ hg update foreign
>>>
>>> Now the repo will be in sync with the Panama/foreign branch.
>>>
>>>
>>> Maurizio
>>>
>>>>
>>>> $ bash configure --disable-warnings-as-errors
>>>> --with-libclang
>>>> ../../Documents/clang+llvm-7.0.1-x86_64-linux-gnu-ubuntu-18.04/
>>>> configure: WARNING: you should use --build, --host, --target
>>>> configure: WARNING: invalid host type:
>>>> ../../Documents/clang+llvm-7.0.1-x86_64-linux-gnu-ubuntu-18.04/
>>>> configure: error: unrecognized options: --with-libclang
>>>> configure exiting with result code 1
>>>>
>>>> then by:
>>>>
>>>> $ bash configure --help
>>>>
>>>> I went through all the `--with` and there is no such option
>>>>
>>>> The only llvm reference I found was
>>>>
>>>> /The following toolchains are available as arguments to
>>>> --with-toolchain-type.
>>>> Which are valid to use depends on the build platform.
>>>> gcc GNU Compiler Collection
>>>> clang clang/LLVM/
>>>>
>>>> But I dont know if it's the same thing.. I guess not
>>>>
>>>>
>>>> Il giorno mar 19 feb 2019 alle ore 22:28 Maurizio
>>>> Cimadamore <maurizio.cimadamore at oracle.com
>>>> <mailto:maurizio.cimadamore at oracle.com>> ha scritto:
>>>>
>>>>
>>>> On 19/02/2019 20:55, Giuseppe Barbieri wrote:
>>>>> That document is really enlightening, I think it
>>>>> should be added under "Links" here
>>>>> https://openjdk.java.net/projects/panama/
>>>>>
>>>>> I'll read it again and again in order to deeply grasp
>>>>> the concepts there expressed
>>>>>
>>>>> In the meanwhile, I'd love to being able to build it
>>>>> to try the very last changes, like `|Scope.globalScope()`|
>>>>>
>>>>> In this regards, the problem I get is the following
>>>>> https://gist.github.com/elect86/3365904008c5e7b0e9e58c9e31def671
>>>>> if I do simply:
>>>>
>>>> There's a new build we issued just few hours ago which
>>>> has this:
>>>>
>>>> https://jdk.java.net/panama/
>>>>
>>>> That said... comments below:
>>>>
>>>>>
>>>>> - bash configure
>>>>> - make images
>>>>>
>>>>> It looks to be the same case of this user:
>>>>> https://mail.openjdk.java.net/pipermail/jdk-dev/2019-January/002520.html
>>>>>
>>>>> Ubuntu 18.10, gcc version 8.2.0 (Ubuntu 8.2.0-7ubuntu1)
>>>>
>>>> Ok, so that seems an issue that has to do with the gcc
>>>> compiler being to new - I'm on 7.3 on Ubuntu 18.04 and
>>>> I don't have this issue - this is likely to be a
>>>> JDK-wide issue as the email thread points out.
>>>>
>>>> If disabling warnings works for you, that's probably
>>>> the best path until the underlying issue is fixed by
>>>> the build team in mainline jdk/jdk.
>>>>
>>>>> Also, it seems like I already have what seems
>>>>> necessary about llvm, or?
>>>>>
>>>>> $ sudo apt-get install clang-7 lldb-7 lld-7
>>>>> clang-7 is already the newest version
>>>>> (1:7.0.1~svn348686-1~exp1~20190113235558.10).
>>>>> lldb-7 is already the newest version
>>>>> (1:7.0.1~svn348686-1~exp1~20190113235558.10).
>>>>> lld-7 is already the newest version
>>>>> (1:7.0.1~svn348686-1~exp1~20190113235558.10).
>>>>
>>>> I'd advise *against* using the bundled clang that comes
>>>> with Ubuntu - you can make that work, eventually, but
>>>> it's a lot harder than just grab the latest LLVM
>>>> snapshot at the link I provided and set a single build
>>>> variable (instead of three). That's the way I build
>>>> every day on Ubuntu (both 16.04 and 18.04).
>>>>
>>>> But you still have to pass LLVM to configure:
>>>>
>>>> $ sh configure --with-libclang <path to LLVM install> ...
>>>>
>>>> Otherwise the clang native library won't be built and
>>>> your build will be missing jextract. A quick way to see
>>>> if clang has been picked up by the build is to do this
>>>> (from the JDK root):
>>>>
>>>> $ cat build/linux-x86_64-server-release/spec.gmk | grep
>>>> CLANG
>>>>
>>>> If you see that the variables printed out do not
>>>> contain any path pointing to your LLVM install, then it
>>>> didn't work :-)
>>>>
>>>> Note to self: we should make the Panama/foreign
>>>> configure step fail if LLVM is not found, rather than
>>>> silently fail and then fail to produce jextract.
>>>>
>>>> Maurizio
>>>>
>>>>
>>>>>
>>>>>
>>>>> Giuseppe
>>>>>
>>>>>
>>>>> Il giorno mar 19 feb 2019 alle ore 12:09 Maurizio
>>>>> Cimadamore <maurizio.cimadamore at oracle.com
>>>>> <mailto:maurizio.cimadamore at oracle.com>> ha scritto:
>>>>>
>>>>>
>>>>> On 19/02/2019 07:17, Giuseppe Barbieri wrote:
>>>>>> Ok, so first of all, thanks for your effort into
>>>>>> this project
>>>>>>
>>>>>> Tthis is something extremely cool that I wished
>>>>>> we had already from some time
>>>>>>
>>>>>> JNI is quite tedious to work with (sorry), and on
>>>>>> the other side JNA is easier but can be quite
>>>>>> slower for some applications.
>>>>>>
>>>>>> JNA is, however, nowhere so immediate and fast as
>>>>>> using jextractor, actually is extremely verbose..
>>>>>>
>>>>>> Coming back to feedback and observation, I think
>>>>>> it's kind of critical to understand exactly how
>>>>>> shall `Scope` be used, which is the general
>>>>>> strategy to follow.
>>>>>>
>>>>>> Because I see a couple of options here (which are
>>>>>> not ortogonal with each other, a combination may
>>>>>> be possible):
>>>>>>
>>>>>> - create a scope every method where you need
>>>>>> allocation
>>>>>> - global scope for the whole project (and re-use
>>>>>> all you can)
>>>>>> - class scope
>>>>>> - [3d real-time graphics or other performance
>>>>>> critical app] (create it once per frame and) pass
>>>>>> it down the line
>>>>>
>>>>> Yep - all of this is possible, and it is
>>>>> ultimately up to the application to do the right
>>>>> thing.
>>>>>
>>>>> Note that the Scope API is is being refined as we
>>>>> speak, according to this proposal:
>>>>>
>>>>> https://cr.openjdk.java.net/~mcimadamore/panama/scopes.html
>>>>>
>>>>> Which introduces a stronger ownership model, and
>>>>> the concept of 'library scope'. So you could also
>>>>> just use the library scope and allocate everything
>>>>> there.
>>>>>
>>>>>>
>>>>>> Also, what about multithreading? How Scope
>>>>>> interact with that/which properties has in this
>>>>>> regard?
>>>>> This is covered in the document above - short
>>>>> answer: certain scopes are more multi-thread
>>>>> friendly than others, and can have a fastest
>>>>> implementation which just validates that the
>>>>> owning thread doesn't change from creation time.
>>>>> But if you want to stash a scope into a field and
>>>>> keep reusing it, you will run into races, so the
>>>>> implementation has to take care of that (it
>>>>> doesn't now).
>>>>>>
>>>>>> For example, I can say that GL, AL and CL require
>>>>>> a fair amount of native memory operations.
>>>>>>
>>>>>> VK, instead, is another planet: the amount of
>>>>>> native structs and other is massive, each method
>>>>>> essentially create some info struct (kind of
>>>>>> kotlin data classes) just to hold a lot of
>>>>>> parameters and then pass them as argument to some
>>>>>> function and retrieve an opaque object.
>>>>>>
>>>>>> How much is going to cost using Scope? Do you
>>>>>> have any performance metrics?
>>>>>
>>>>> We don't have concrete numbers for this specific
>>>>> axis yet - of course if you end up allocating one
>>>>> scope per struct you are not much better off than
>>>>> you would be allocating on heap, so some balance
>>>>> will have to be found by the application. In terms
>>>>> of the VM, most of these scope instances can be
>>>>> proven to be non-escaping, so their allocation
>>>>> shouldn't cost that much (but escape analysis
>>>>> mileage is very conservative in Hotspot, and
>>>>> mileage can vary - Graal might be better at that,
>>>>> with its partial escape analsysis logic).
>>>>>
>>>>> Also, we are considering optimizations if the only
>>>>> thing a scope does is allocating a _single_ resource.
>>>>>
>>>>>>
>>>>>> Have you considered alternatives ways, such as
>>>>>> thread-local/stack allocations?
>>>>>>
>>>>>> One thing I noticed immediately is that there is
>>>>>> no possibility to deallocate pointer and other
>>>>>> native resources, other than let the Scope be
>>>>>> GC"ed and with it all the corresponding native
>>>>>> resources allocated with, I guess.
>>>>> This is covered in the document - the short answer
>>>>> is that de-allocating a single resource is a
>>>>> 'false friend' - if you go down that path, you end
>>>>> up making pointers 'mutable' which will prevent
>>>>> them from being true value types (which is our
>>>>> ultimate target - we want pointer creation to be
>>>>> super cheap). Better to have immutable
>>>>> pointers-as-cursors, and to put the mutable state
>>>>> elsewhere (scope).
>>>>>>
>>>>>> So, this leads me think that the Scope class is
>>>>>> supposed to be short lived? But at the same time
>>>>>> I'm afraid this may cause some pressure on the GC
>>>>>> under specific cases. I don't know.
>>>>>>
>>>>>> Nevertheless, I'd like to have some way to
>>>>>> deallocate objects.
>>>>>>
>>>>>> Let's take as an example that hello triangle
>>>>>>
>>>>>> In the `init` (constructor) I do need to allocate
>>>>>> for example several resources, some of them can
>>>>>> be trashed and freed right at the end of the
>>>>>> constructor, while some other instead shall
>>>>>> survive since they are going to be used in the
>>>>>> render method, like the pMVP (to accomodate the
>>>>>> matrix) and the vertexArray (to hold the name).
>>>>>
>>>>> That seems to suggest that you need two scopes -
>>>>> one for the stuff that is short-lived (scope can
>>>>> be created in the constructor) and another, more
>>>>> global one, for the application. In the new model,
>>>>> described in the document, you can start from the
>>>>> application scope:
>>>>>
>>>>> Scope appScope = Scope.globalScope().fork();
>>>>>
>>>>> And then, inside the constructor, you can 'fork'
>>>>> another scope:
>>>>>
>>>>> Scope constructorScope = appScope.fork();
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>> A little note aside: I tried also to build it
>>>>>> from source, I had to
>>>>>>
>>>>>> `bash configure --disable-warnings-as-errors`
>>>>> that's weird - this should not be needed... esp.
>>>>> on Linux - what problem are you getting?
>>>>>>
>>>>>> to avoid treating warnings like errors and it
>>>>>> worked. But I don't see any jextractor under
>>>>>> ./build/linux-x86_64-server-release/images/jdk/bin/
>>>>>
>>>>> I wonder if this is related to the issue above -
>>>>> note that to build jextract you need to point the
>>>>> build at clang - the easiest way to do that is to
>>>>> download a snapshot here:
>>>>>
>>>>> http://releases.llvm.org/
>>>>>
>>>>> And then point the build at that:
>>>>>
>>>>> sh configure --with-libclang <path to LLVM install>
>>>>>
>>>>> This will generate the jextract library and,
>>>>> hopefully, will also get rid of the warning.
>>>>>
>>>>> Maurizio
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>> Giuseppe
>>>>>>
>>>>>> Il giorno lun 18 feb 2019 alle ore 15:33 Maurizio
>>>>>> Cimadamore <maurizio.cimadamore at oracle.com
>>>>>> <mailto:maurizio.cimadamore at oracle.com>> ha scritto:
>>>>>>
>>>>>> Whohoo!
>>>>>>
>>>>>> Looking forward to the feedback - and thanks
>>>>>> for the patience!
>>>>>>
>>>>>> Maurizio
>>>>>>
>>>>>> On 18/02/2019 14:30, Giuseppe Barbieri wrote:
>>>>>>> Uh, sorry, my bad, you are right,
>>>>>>>
>>>>>>> ptr1.set(Vec3.lenght.L)
>>>>>>>
>>>>>>> should have been
>>>>>>>
>>>>>>> ptr1.set(Vec3.size.L)
>>>>>>>
>>>>>>> it works flawless! Thanks you all, guys
>>>>>>>
>>>>>>> I'll write later a list with some feedbacks to smooth the experience for gl devs
>>>>>>>
>>>>>>>
>>>>>>> Il giorno lun 18 feb 2019 alle ore 15:27
>>>>>>> Maurizio Cimadamore
>>>>>>> <maurizio.cimadamore at oracle.com
>>>>>>> <mailto:maurizio.cimadamore at oracle.com>> ha
>>>>>>> scritto:
>>>>>>>
>>>>>>> Uhm - the code snippet for converting
>>>>>>> long into pointer is correct; I
>>>>>>> checked this on my machine:
>>>>>>>
>>>>>>> class Test {
>>>>>>> public static void main(String[]
>>>>>>> args) throws Throwable {
>>>>>>> try (Scope sc =
>>>>>>> Scope.globalScope().fork()) {
>>>>>>> var ptr1 = sc.allocate(NativeTypes.LONG);
>>>>>>> ptr1.set(42L);
>>>>>>> var p1 =
>>>>>>> ptr1.cast(NativeTypes.VOID).cast(NativeTypes.VOID.pointer()).get();
>>>>>>> System.err.println(p1.addr());
>>>>>>> }
>>>>>>> }
>>>>>>> }
>>>>>>>
>>>>>>>
>>>>>>> And this prints 42 as expected.
>>>>>>>
>>>>>>> Are you sure the problem is coming from
>>>>>>> that pointer, and not from
>>>>>>> somewhere else?
>>>>>>>
>>>>>>> Maurizio
>>>>>>>
>>>>>>> On 18/02/2019 14:16, Giuseppe Barbieri
>>>>>>> wrote:
>>>>>>> > I just had a chance to try, but that,
>>>>>>> unfortunately, doesnt seem to work..
>>>>>>> >
>>>>>>> https://github.com/elect86/panama/blob/master/src/test/kotlin/hello-triangle.kt#L100-L102
>>>>>>> >
>>>>>>> > I get a black triangle (I shall get a
>>>>>>> colored one instead)
>>>>>>> >
>>>>>>> > I guess I get at least the vertex
>>>>>>> positions (the triangle, although black,
>>>>>>> > is there) because in my case
>>>>>>> `semantic.attr.POSITION` is luckily zero,
>>>>>>> > which is the same index where the
>>>>>>> attributes start from.
>>>>>>> >
>>>>>>> > Other ideas?
>>>>>>> >
>>>>>>> > Il giorno lun 18 feb 2019 alle ore
>>>>>>> 10:56 Jorn Vernee <jbvernee at xs4all.nl
>>>>>>> <mailto:jbvernee at xs4all.nl>>
>>>>>>> > ha scritto:
>>>>>>> >
>>>>>>> >> Hi Giuseppe,
>>>>>>> >>
>>>>>>> >> You should be able to use this trick:
>>>>>>> >>
>>>>>>> >> long pointerValue = 12L; // e.g.
>>>>>>> >> Scope scope = Scope.newNativeScope;
>>>>>>> // or Scope.globalScope().fork()
>>>>>>> >> depending on which version you are
>>>>>>> >> Pointer<Long> ptr =
>>>>>>> scope.allocate(NativeTypes.UINT64);
>>>>>>> >> ptr.set(pointerValue);
>>>>>>> >> Pointer<?> result =
>>>>>>> >>
>>>>>>> ptr.cast(NativeTypes.VOID).cast(NativeTypes.VOID.pointer()).get();
>>>>>>> >> // use 'result'
>>>>>>> >>
>>>>>>> >> Be aware that this does do an
>>>>>>> allocation of a 64 bit int, so you might
>>>>>>> >> want to reuse the allocated space if
>>>>>>> you create a lot of pointers from
>>>>>>> >> literals.
>>>>>>> >>
>>>>>>> >> Maybe in the future we can add an API
>>>>>>> for creating pointers from long
>>>>>>> >> literals directly.
>>>>>>> >>
>>>>>>> >> Jorn
>>>>>>> >>
>>>>>>> >> Giuseppe Barbieri schreef op
>>>>>>> 2019-02-18 10:44:
>>>>>>> >>> Thanks Sundar,
>>>>>>> >>>
>>>>>>> >>> that works flawless for `(void *)0`
>>>>>>> >>>
>>>>>>> >>> But now I also would need a generic
>>>>>>> solution, for example:
>>>>>>> >>>
>>>>>>> >>> (void *)12
>>>>>>> >>>
>>>>>>> >>> There is a sill opengl call (
>>>>>>> glVertexPointer ) requesting explicitely
>>>>>>> >>> that:
>>>>>>> https://stackoverflow.com/a/8283855/1047713
>>>>>>> >>>
>>>>>>> >>> Signature:
>>>>>>> >>>
>>>>>>> >>> void glVertexAttribPointer(int
>>>>>>> index, int size, int type, byte
>>>>>>> >>> normalized, int stride, Pointer<?>
>>>>>>> pointer);
>>>>>>> >>>
>>>>>>> >>>
>>>>>>> >>> is there actually a way?
>>>>>>> >>>
>>>>>>> >>>
>>>>>>> >>>
>>>>>>> >>>
>>>>>>> >>>
>>>>>>> >>>
>>>>>>> >>> You can use Pointer.nullPointer()
>>>>>>> method.
>>>>>>> >>>
>>>>>>> >>> -Sundar
>>>>>>> >>>
>>>>>>> >>> On 18/02/19, 8:09 AM, Giuseppe
>>>>>>> Barbieri wrote:
>>>>>>> >>>> Hi,
>>>>>>> >>>>
>>>>>>> >>>> I'm looking for a way to convert this:
>>>>>>> >>>>
>>>>>>> >>>> (void*)0
>>>>>>> >>>>
>>>>>>> >>>> in Java.
>>>>>>> >>>>
>>>>>>> >>>> I tried to allocate a pointer and
>>>>>>> set its value to 0, but it didnt
>>>>>>> >>>> work
>>>>>>> >>>>
>>>>>>> >>>> Any ideas, guys?
>>>>>>> >>>>
>>>>>>> >>>> Thanks in advance
>>>>>>>
More information about the panama-dev
mailing list