Generic (void *)int
Maurizio Cimadamore
maurizio.cimadamore at oracle.com
Wed Feb 20 00:40:02 UTC 2019
On 20/02/2019 00:35, Giuseppe Barbieri wrote:
> elect at elect-NUC8i5BEK:~/IdeaProjects/panama-dev$ bash configure
> --disable-warnings-as-errors
> --with-libclang=../../Documents/clang+llvm-7.0.1-x86_64-linux-gnu-ubuntu-18.04
>
> I dont think `..` may be the problem, because for example
> `clang-c/Index.h` is found without issues
>
> this is my tree:
> https://gist.github.com/elect86/74c9c95fd9327fb83ffefa0af3a3ec48
>
> Do you also use the very same snapshot?
Lastest one I tried was 7.0.0 (Ubuntu 16.04 version, but that should
hardly make a difference?)
Have you tried without '..' (e.g. absolute path) ?
Maurizio
>
> Il giorno mer 20 feb 2019 alle ore 01:31 Maurizio Cimadamore
> <maurizio.cimadamore at oracle.com
> <mailto:maurizio.cimadamore at oracle.com>> ha scritto:
>
>
> On 20/02/2019 00:22, Giuseppe Barbieri wrote:
>> configure: error: Cannot locate libclang or headers at the
>> specified locations:
>> /home/elect/Documents/clang+llvm-7.0.1-x86_64-linux-gnu-ubuntu-18.04/lib
>> /home/elect/Documents/clang+llvm-7.0.1-x86_64-linux-gnu-ubuntu-18.04/include
>>
>> Note, I had to choose 18.04 although I have 18.10 because it was
>> the newest available
>
> That's ok, I think.
>
> And, are you passing to --with-libclang the following?
>
> /home/elect/Documents/clang+llvm-7.0.1-x86_64-linux-gnu-ubuntu-18.04
>
> If so, do the include/ and lib/ folder exist under such folder?
>
> e.g. in my machine:
>
> $ tree -L 1 /opt/clang/clang+llvm-6.0.1-x86_64-linux-gnu-ubuntu-16.04
> /opt/clang/clang+llvm-6.0.1-x86_64-linux-gnu-ubuntu-16.04
> ├── bin
> ├── include
> ├── lib
> ├── libexec
> └── share
>
> Maurizio
>
>>
>> Anyway, I'd not like to abuse your time, I can give up for the
>> moment waiting until you or someone else will try and write down
>> a guide for building on an updated ubuntu/linux system.
>>
>> Il giorno mer 20 feb 2019 alle ore 00:37 Maurizio Cimadamore
>> <maurizio.cimadamore at oracle.com
>> <mailto:maurizio.cimadamore at oracle.com>> ha scritto:
>>
>>
>> On 19/02/2019 22:33, Giuseppe Barbieri wrote:
>>> I'll do as you said
>>>
>>> By "snapshot", do you mean the pre-built binaries?
>> Yes
>>>
>>> Just tried but apparently there seems to be no such option
>>
>> you need to update the branch
>>
>> $ hg update foreign
>>
>> Now the repo will be in sync with the Panama/foreign branch.
>>
>>
>> Maurizio
>>
>>>
>>> $ bash configure --disable-warnings-as-errors
>>> --with-libclang
>>> ../../Documents/clang+llvm-7.0.1-x86_64-linux-gnu-ubuntu-18.04/
>>> configure: WARNING: you should use --build, --host, --target
>>> configure: WARNING: invalid host type:
>>> ../../Documents/clang+llvm-7.0.1-x86_64-linux-gnu-ubuntu-18.04/
>>> configure: error: unrecognized options: --with-libclang
>>> configure exiting with result code 1
>>>
>>> then by:
>>>
>>> $ bash configure --help
>>>
>>> I went through all the `--with` and there is no such option
>>>
>>> The only llvm reference I found was
>>>
>>> /The following toolchains are available as arguments to
>>> --with-toolchain-type.
>>> Which are valid to use depends on the build platform.
>>> gcc GNU Compiler Collection
>>> clang clang/LLVM/
>>>
>>> But I dont know if it's the same thing.. I guess not
>>>
>>>
>>> Il giorno mar 19 feb 2019 alle ore 22:28 Maurizio Cimadamore
>>> <maurizio.cimadamore at oracle.com
>>> <mailto:maurizio.cimadamore at oracle.com>> ha scritto:
>>>
>>>
>>> On 19/02/2019 20:55, Giuseppe Barbieri wrote:
>>>> That document is really enlightening, I think it should
>>>> be added under "Links" here
>>>> https://openjdk.java.net/projects/panama/
>>>>
>>>> I'll read it again and again in order to deeply grasp
>>>> the concepts there expressed
>>>>
>>>> In the meanwhile, I'd love to being able to build it to
>>>> try the very last changes, like `|Scope.globalScope()`|
>>>>
>>>> In this regards, the problem I get is the following
>>>> https://gist.github.com/elect86/3365904008c5e7b0e9e58c9e31def671
>>>> if I do simply:
>>>
>>> There's a new build we issued just few hours ago which
>>> has this:
>>>
>>> https://jdk.java.net/panama/
>>>
>>> That said... comments below:
>>>
>>>>
>>>> - bash configure
>>>> - make images
>>>>
>>>> It looks to be the same case of this user:
>>>> https://mail.openjdk.java.net/pipermail/jdk-dev/2019-January/002520.html
>>>>
>>>> Ubuntu 18.10, gcc version 8.2.0 (Ubuntu 8.2.0-7ubuntu1)
>>>
>>> Ok, so that seems an issue that has to do with the gcc
>>> compiler being to new - I'm on 7.3 on Ubuntu 18.04 and I
>>> don't have this issue - this is likely to be a JDK-wide
>>> issue as the email thread points out.
>>>
>>> If disabling warnings works for you, that's probably the
>>> best path until the underlying issue is fixed by the
>>> build team in mainline jdk/jdk.
>>>
>>>> Also, it seems like I already have what seems necessary
>>>> about llvm, or?
>>>>
>>>> $ sudo apt-get install clang-7 lldb-7 lld-7
>>>> clang-7 is already the newest version
>>>> (1:7.0.1~svn348686-1~exp1~20190113235558.10).
>>>> lldb-7 is already the newest version
>>>> (1:7.0.1~svn348686-1~exp1~20190113235558.10).
>>>> lld-7 is already the newest version
>>>> (1:7.0.1~svn348686-1~exp1~20190113235558.10).
>>>
>>> I'd advise *against* using the bundled clang that comes
>>> with Ubuntu - you can make that work, eventually, but
>>> it's a lot harder than just grab the latest LLVM
>>> snapshot at the link I provided and set a single build
>>> variable (instead of three). That's the way I build
>>> every day on Ubuntu (both 16.04 and 18.04).
>>>
>>> But you still have to pass LLVM to configure:
>>>
>>> $ sh configure --with-libclang <path to LLVM install> ...
>>>
>>> Otherwise the clang native library won't be built and
>>> your build will be missing jextract. A quick way to see
>>> if clang has been picked up by the build is to do this
>>> (from the JDK root):
>>>
>>> $ cat build/linux-x86_64-server-release/spec.gmk | grep
>>> CLANG
>>>
>>> If you see that the variables printed out do not contain
>>> any path pointing to your LLVM install, then it didn't
>>> work :-)
>>>
>>> Note to self: we should make the Panama/foreign
>>> configure step fail if LLVM is not found, rather than
>>> silently fail and then fail to produce jextract.
>>>
>>> Maurizio
>>>
>>>
>>>>
>>>>
>>>> Giuseppe
>>>>
>>>>
>>>> Il giorno mar 19 feb 2019 alle ore 12:09 Maurizio
>>>> Cimadamore <maurizio.cimadamore at oracle.com
>>>> <mailto:maurizio.cimadamore at oracle.com>> ha scritto:
>>>>
>>>>
>>>> On 19/02/2019 07:17, Giuseppe Barbieri wrote:
>>>>> Ok, so first of all, thanks for your effort into
>>>>> this project
>>>>>
>>>>> Tthis is something extremely cool that I wished we
>>>>> had already from some time
>>>>>
>>>>> JNI is quite tedious to work with (sorry), and on
>>>>> the other side JNA is easier but can be quite
>>>>> slower for some applications.
>>>>>
>>>>> JNA is, however, nowhere so immediate and fast as
>>>>> using jextractor, actually is extremely verbose..
>>>>>
>>>>> Coming back to feedback and observation, I think
>>>>> it's kind of critical to understand exactly how
>>>>> shall `Scope` be used, which is the general
>>>>> strategy to follow.
>>>>>
>>>>> Because I see a couple of options here (which are
>>>>> not ortogonal with each other, a combination may
>>>>> be possible):
>>>>>
>>>>> - create a scope every method where you need
>>>>> allocation
>>>>> - global scope for the whole project (and re-use
>>>>> all you can)
>>>>> - class scope
>>>>> - [3d real-time graphics or other performance
>>>>> critical app] (create it once per frame and) pass
>>>>> it down the line
>>>>
>>>> Yep - all of this is possible, and it is ultimately
>>>> up to the application to do the right thing.
>>>>
>>>> Note that the Scope API is is being refined as we
>>>> speak, according to this proposal:
>>>>
>>>> https://cr.openjdk.java.net/~mcimadamore/panama/scopes.html
>>>>
>>>> Which introduces a stronger ownership model, and
>>>> the concept of 'library scope'. So you could also
>>>> just use the library scope and allocate everything
>>>> there.
>>>>
>>>>>
>>>>> Also, what about multithreading? How Scope
>>>>> interact with that/which properties has in this
>>>>> regard?
>>>> This is covered in the document above - short
>>>> answer: certain scopes are more multi-thread
>>>> friendly than others, and can have a fastest
>>>> implementation which just validates that the owning
>>>> thread doesn't change from creation time. But if
>>>> you want to stash a scope into a field and keep
>>>> reusing it, you will run into races, so the
>>>> implementation has to take care of that (it doesn't
>>>> now).
>>>>>
>>>>> For example, I can say that GL, AL and CL require
>>>>> a fair amount of native memory operations.
>>>>>
>>>>> VK, instead, is another planet: the amount of
>>>>> native structs and other is massive, each method
>>>>> essentially create some info struct (kind of
>>>>> kotlin data classes) just to hold a lot of
>>>>> parameters and then pass them as argument to some
>>>>> function and retrieve an opaque object.
>>>>>
>>>>> How much is going to cost using Scope? Do you have
>>>>> any performance metrics?
>>>>
>>>> We don't have concrete numbers for this specific
>>>> axis yet - of course if you end up allocating one
>>>> scope per struct you are not much better off than
>>>> you would be allocating on heap, so some balance
>>>> will have to be found by the application. In terms
>>>> of the VM, most of these scope instances can be
>>>> proven to be non-escaping, so their allocation
>>>> shouldn't cost that much (but escape analysis
>>>> mileage is very conservative in Hotspot, and
>>>> mileage can vary - Graal might be better at that,
>>>> with its partial escape analsysis logic).
>>>>
>>>> Also, we are considering optimizations if the only
>>>> thing a scope does is allocating a _single_ resource.
>>>>
>>>>>
>>>>> Have you considered alternatives ways, such as
>>>>> thread-local/stack allocations?
>>>>>
>>>>> One thing I noticed immediately is that there is
>>>>> no possibility to deallocate pointer and other
>>>>> native resources, other than let the Scope be
>>>>> GC"ed and with it all the corresponding native
>>>>> resources allocated with, I guess.
>>>> This is covered in the document - the short answer
>>>> is that de-allocating a single resource is a 'false
>>>> friend' - if you go down that path, you end up
>>>> making pointers 'mutable' which will prevent them
>>>> from being true value types (which is our ultimate
>>>> target - we want pointer creation to be super
>>>> cheap). Better to have immutable
>>>> pointers-as-cursors, and to put the mutable state
>>>> elsewhere (scope).
>>>>>
>>>>> So, this leads me think that the Scope class is
>>>>> supposed to be short lived? But at the same time
>>>>> I'm afraid this may cause some pressure on the GC
>>>>> under specific cases. I don't know.
>>>>>
>>>>> Nevertheless, I'd like to have some way to
>>>>> deallocate objects.
>>>>>
>>>>> Let's take as an example that hello triangle
>>>>>
>>>>> In the `init` (constructor) I do need to allocate
>>>>> for example several resources, some of them can be
>>>>> trashed and freed right at the end of the
>>>>> constructor, while some other instead shall
>>>>> survive since they are going to be used in the
>>>>> render method, like the pMVP (to accomodate the
>>>>> matrix) and the vertexArray (to hold the name).
>>>>
>>>> That seems to suggest that you need two scopes -
>>>> one for the stuff that is short-lived (scope can be
>>>> created in the constructor) and another, more
>>>> global one, for the application. In the new model,
>>>> described in the document, you can start from the
>>>> application scope:
>>>>
>>>> Scope appScope = Scope.globalScope().fork();
>>>>
>>>> And then, inside the constructor, you can 'fork'
>>>> another scope:
>>>>
>>>> Scope constructorScope = appScope.fork();
>>>>
>>>>
>>>>>
>>>>>
>>>>> A little note aside: I tried also to build it from
>>>>> source, I had to
>>>>>
>>>>> `bash configure --disable-warnings-as-errors`
>>>> that's weird - this should not be needed... esp. on
>>>> Linux - what problem are you getting?
>>>>>
>>>>> to avoid treating warnings like errors and it
>>>>> worked. But I don't see any jextractor under
>>>>> ./build/linux-x86_64-server-release/images/jdk/bin/
>>>>
>>>> I wonder if this is related to the issue above -
>>>> note that to build jextract you need to point the
>>>> build at clang - the easiest way to do that is to
>>>> download a snapshot here:
>>>>
>>>> http://releases.llvm.org/
>>>>
>>>> And then point the build at that:
>>>>
>>>> sh configure --with-libclang <path to LLVM install>
>>>>
>>>> This will generate the jextract library and,
>>>> hopefully, will also get rid of the warning.
>>>>
>>>> Maurizio
>>>>
>>>>
>>>>>
>>>>>
>>>>> Giuseppe
>>>>>
>>>>> Il giorno lun 18 feb 2019 alle ore 15:33 Maurizio
>>>>> Cimadamore <maurizio.cimadamore at oracle.com
>>>>> <mailto:maurizio.cimadamore at oracle.com>> ha scritto:
>>>>>
>>>>> Whohoo!
>>>>>
>>>>> Looking forward to the feedback - and thanks
>>>>> for the patience!
>>>>>
>>>>> Maurizio
>>>>>
>>>>> On 18/02/2019 14:30, Giuseppe Barbieri wrote:
>>>>>> Uh, sorry, my bad, you are right,
>>>>>>
>>>>>> ptr1.set(Vec3.lenght.L)
>>>>>>
>>>>>> should have been
>>>>>>
>>>>>> ptr1.set(Vec3.size.L)
>>>>>>
>>>>>> it works flawless! Thanks you all, guys
>>>>>>
>>>>>> I'll write later a list with some feedbacks to smooth the experience for gl devs
>>>>>>
>>>>>>
>>>>>> Il giorno lun 18 feb 2019 alle ore 15:27
>>>>>> Maurizio Cimadamore
>>>>>> <maurizio.cimadamore at oracle.com
>>>>>> <mailto:maurizio.cimadamore at oracle.com>> ha
>>>>>> scritto:
>>>>>>
>>>>>> Uhm - the code snippet for converting
>>>>>> long into pointer is correct; I
>>>>>> checked this on my machine:
>>>>>>
>>>>>> class Test {
>>>>>> public static void main(String[]
>>>>>> args) throws Throwable {
>>>>>> try (Scope sc =
>>>>>> Scope.globalScope().fork()) {
>>>>>> var ptr1 = sc.allocate(NativeTypes.LONG);
>>>>>> ptr1.set(42L);
>>>>>> var p1 =
>>>>>> ptr1.cast(NativeTypes.VOID).cast(NativeTypes.VOID.pointer()).get();
>>>>>> System.err.println(p1.addr());
>>>>>> }
>>>>>> }
>>>>>> }
>>>>>>
>>>>>>
>>>>>> And this prints 42 as expected.
>>>>>>
>>>>>> Are you sure the problem is coming from
>>>>>> that pointer, and not from
>>>>>> somewhere else?
>>>>>>
>>>>>> Maurizio
>>>>>>
>>>>>> On 18/02/2019 14:16, Giuseppe Barbieri wrote:
>>>>>> > I just had a chance to try, but that,
>>>>>> unfortunately, doesnt seem to work..
>>>>>> >
>>>>>> https://github.com/elect86/panama/blob/master/src/test/kotlin/hello-triangle.kt#L100-L102
>>>>>> >
>>>>>> > I get a black triangle (I shall get a
>>>>>> colored one instead)
>>>>>> >
>>>>>> > I guess I get at least the vertex
>>>>>> positions (the triangle, although black,
>>>>>> > is there) because in my case
>>>>>> `semantic.attr.POSITION` is luckily zero,
>>>>>> > which is the same index where the
>>>>>> attributes start from.
>>>>>> >
>>>>>> > Other ideas?
>>>>>> >
>>>>>> > Il giorno lun 18 feb 2019 alle ore
>>>>>> 10:56 Jorn Vernee <jbvernee at xs4all.nl
>>>>>> <mailto:jbvernee at xs4all.nl>>
>>>>>> > ha scritto:
>>>>>> >
>>>>>> >> Hi Giuseppe,
>>>>>> >>
>>>>>> >> You should be able to use this trick:
>>>>>> >>
>>>>>> >> long pointerValue = 12L; // e.g.
>>>>>> >> Scope scope = Scope.newNativeScope;
>>>>>> // or Scope.globalScope().fork()
>>>>>> >> depending on which version you are
>>>>>> >> Pointer<Long> ptr =
>>>>>> scope.allocate(NativeTypes.UINT64);
>>>>>> >> ptr.set(pointerValue);
>>>>>> >> Pointer<?> result =
>>>>>> >>
>>>>>> ptr.cast(NativeTypes.VOID).cast(NativeTypes.VOID.pointer()).get();
>>>>>> >> // use 'result'
>>>>>> >>
>>>>>> >> Be aware that this does do an
>>>>>> allocation of a 64 bit int, so you might
>>>>>> >> want to reuse the allocated space if
>>>>>> you create a lot of pointers from
>>>>>> >> literals.
>>>>>> >>
>>>>>> >> Maybe in the future we can add an API
>>>>>> for creating pointers from long
>>>>>> >> literals directly.
>>>>>> >>
>>>>>> >> Jorn
>>>>>> >>
>>>>>> >> Giuseppe Barbieri schreef op
>>>>>> 2019-02-18 10:44:
>>>>>> >>> Thanks Sundar,
>>>>>> >>>
>>>>>> >>> that works flawless for `(void *)0`
>>>>>> >>>
>>>>>> >>> But now I also would need a generic
>>>>>> solution, for example:
>>>>>> >>>
>>>>>> >>> (void *)12
>>>>>> >>>
>>>>>> >>> There is a sill opengl call (
>>>>>> glVertexPointer ) requesting explicitely
>>>>>> >>> that:
>>>>>> https://stackoverflow.com/a/8283855/1047713
>>>>>> >>>
>>>>>> >>> Signature:
>>>>>> >>>
>>>>>> >>> void glVertexAttribPointer(int index,
>>>>>> int size, int type, byte
>>>>>> >>> normalized, int stride, Pointer<?>
>>>>>> pointer);
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> is there actually a way?
>>>>>> >>>
>>>>>> >>>
>>>>>> >>>
>>>>>> >>>
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> You can use Pointer.nullPointer() method.
>>>>>> >>>
>>>>>> >>> -Sundar
>>>>>> >>>
>>>>>> >>> On 18/02/19, 8:09 AM, Giuseppe
>>>>>> Barbieri wrote:
>>>>>> >>>> Hi,
>>>>>> >>>>
>>>>>> >>>> I'm looking for a way to convert this:
>>>>>> >>>>
>>>>>> >>>> (void*)0
>>>>>> >>>>
>>>>>> >>>> in Java.
>>>>>> >>>>
>>>>>> >>>> I tried to allocate a pointer and
>>>>>> set its value to 0, but it didnt
>>>>>> >>>> work
>>>>>> >>>>
>>>>>> >>>> Any ideas, guys?
>>>>>> >>>>
>>>>>> >>>> Thanks in advance
>>>>>>
More information about the panama-dev
mailing list