Generic (void *)int
Maurizio Cimadamore
maurizio.cimadamore at oracle.com
Tue Feb 19 21:28:18 UTC 2019
On 19/02/2019 20:55, Giuseppe Barbieri wrote:
> That document is really enlightening, I think it should be added under
> "Links" here https://openjdk.java.net/projects/panama/
>
> I'll read it again and again in order to deeply grasp the concepts
> there expressed
>
> In the meanwhile, I'd love to being able to build it to try the very
> last changes, like `|Scope.globalScope()`|
>
> In this regards, the problem I get is the following
> https://gist.github.com/elect86/3365904008c5e7b0e9e58c9e31def671 if I
> do simply:
There's a new build we issued just few hours ago which has this:
https://jdk.java.net/panama/
That said... comments below:
>
> - bash configure
> - make images
>
> It looks to be the same case of this user:
> https://mail.openjdk.java.net/pipermail/jdk-dev/2019-January/002520.html
>
> Ubuntu 18.10, gcc version 8.2.0 (Ubuntu 8.2.0-7ubuntu1)
Ok, so that seems an issue that has to do with the gcc compiler being to
new - I'm on 7.3 on Ubuntu 18.04 and I don't have this issue - this is
likely to be a JDK-wide issue as the email thread points out.
If disabling warnings works for you, that's probably the best path until
the underlying issue is fixed by the build team in mainline jdk/jdk.
> Also, it seems like I already have what seems necessary about llvm, or?
>
> $ sudo apt-get install clang-7 lldb-7 lld-7
> clang-7 is already the newest version
> (1:7.0.1~svn348686-1~exp1~20190113235558.10).
> lldb-7 is already the newest version
> (1:7.0.1~svn348686-1~exp1~20190113235558.10).
> lld-7 is already the newest version
> (1:7.0.1~svn348686-1~exp1~20190113235558.10).
I'd advise *against* using the bundled clang that comes with Ubuntu -
you can make that work, eventually, but it's a lot harder than just grab
the latest LLVM snapshot at the link I provided and set a single build
variable (instead of three). That's the way I build every day on Ubuntu
(both 16.04 and 18.04).
But you still have to pass LLVM to configure:
$ sh configure --with-libclang <path to LLVM install> ...
Otherwise the clang native library won't be built and your build will be
missing jextract. A quick way to see if clang has been picked up by the
build is to do this (from the JDK root):
$ cat build/linux-x86_64-server-release/spec.gmk | grep CLANG
If you see that the variables printed out do not contain any path
pointing to your LLVM install, then it didn't work :-)
Note to self: we should make the Panama/foreign configure step fail if
LLVM is not found, rather than silently fail and then fail to produce
jextract.
Maurizio
>
>
> Giuseppe
>
>
> Il giorno mar 19 feb 2019 alle ore 12:09 Maurizio Cimadamore
> <maurizio.cimadamore at oracle.com
> <mailto:maurizio.cimadamore at oracle.com>> ha scritto:
>
>
> On 19/02/2019 07:17, Giuseppe Barbieri wrote:
>> Ok, so first of all, thanks for your effort into this project
>>
>> Tthis is something extremely cool that I wished we had already
>> from some time
>>
>> JNI is quite tedious to work with (sorry), and on the other side
>> JNA is easier but can be quite slower for some applications.
>>
>> JNA is, however, nowhere so immediate and fast as using
>> jextractor, actually is extremely verbose..
>>
>> Coming back to feedback and observation, I think it's kind of
>> critical to understand exactly how shall `Scope` be used, which
>> is the general strategy to follow.
>>
>> Because I see a couple of options here (which are not ortogonal
>> with each other, a combination may be possible):
>>
>> - create a scope every method where you need allocation
>> - global scope for the whole project (and re-use all you can)
>> - class scope
>> - [3d real-time graphics or other performance critical app]
>> (create it once per frame and) pass it down the line
>
> Yep - all of this is possible, and it is ultimately up to the
> application to do the right thing.
>
> Note that the Scope API is is being refined as we speak, according
> to this proposal:
>
> https://cr.openjdk.java.net/~mcimadamore/panama/scopes.html
>
> Which introduces a stronger ownership model, and the concept of
> 'library scope'. So you could also just use the library scope and
> allocate everything there.
>
>>
>> Also, what about multithreading? How Scope interact with
>> that/which properties has in this regard?
> This is covered in the document above - short answer: certain
> scopes are more multi-thread friendly than others, and can have a
> fastest implementation which just validates that the owning thread
> doesn't change from creation time. But if you want to stash a
> scope into a field and keep reusing it, you will run into races,
> so the implementation has to take care of that (it doesn't now).
>>
>> For example, I can say that GL, AL and CL require a fair amount
>> of native memory operations.
>>
>> VK, instead, is another planet: the amount of native structs and
>> other is massive, each method essentially create some info struct
>> (kind of kotlin data classes) just to hold a lot of parameters
>> and then pass them as argument to some function and retrieve an
>> opaque object.
>>
>> How much is going to cost using Scope? Do you have any
>> performance metrics?
>
> We don't have concrete numbers for this specific axis yet - of
> course if you end up allocating one scope per struct you are not
> much better off than you would be allocating on heap, so some
> balance will have to be found by the application. In terms of the
> VM, most of these scope instances can be proven to be
> non-escaping, so their allocation shouldn't cost that much (but
> escape analysis mileage is very conservative in Hotspot, and
> mileage can vary - Graal might be better at that, with its partial
> escape analsysis logic).
>
> Also, we are considering optimizations if the only thing a scope
> does is allocating a _single_ resource.
>
>>
>> Have you considered alternatives ways, such as thread-local/stack
>> allocations?
>>
>> One thing I noticed immediately is that there is no possibility
>> to deallocate pointer and other native resources, other than let
>> the Scope be GC"ed and with it all the corresponding native
>> resources allocated with, I guess.
> This is covered in the document - the short answer is that
> de-allocating a single resource is a 'false friend' - if you go
> down that path, you end up making pointers 'mutable' which will
> prevent them from being true value types (which is our ultimate
> target - we want pointer creation to be super cheap). Better to
> have immutable pointers-as-cursors, and to put the mutable state
> elsewhere (scope).
>>
>> So, this leads me think that the Scope class is supposed to be
>> short lived? But at the same time I'm afraid this may cause some
>> pressure on the GC under specific cases. I don't know.
>>
>> Nevertheless, I'd like to have some way to deallocate objects.
>>
>> Let's take as an example that hello triangle
>>
>> In the `init` (constructor) I do need to allocate for example
>> several resources, some of them can be trashed and freed right at
>> the end of the constructor, while some other instead shall
>> survive since they are going to be used in the render method,
>> like the pMVP (to accomodate the matrix) and the vertexArray (to
>> hold the name).
>
> That seems to suggest that you need two scopes - one for the stuff
> that is short-lived (scope can be created in the constructor) and
> another, more global one, for the application. In the new model,
> described in the document, you can start from the application scope:
>
> Scope appScope = Scope.globalScope().fork();
>
> And then, inside the constructor, you can 'fork' another scope:
>
> Scope constructorScope = appScope.fork();
>
>
>>
>>
>> A little note aside: I tried also to build it from source, I had to
>>
>> `bash configure --disable-warnings-as-errors`
> that's weird - this should not be needed... esp. on Linux - what
> problem are you getting?
>>
>> to avoid treating warnings like errors and it worked. But I don't
>> see any jextractor under
>> ./build/linux-x86_64-server-release/images/jdk/bin/
>
> I wonder if this is related to the issue above - note that to
> build jextract you need to point the build at clang - the easiest
> way to do that is to download a snapshot here:
>
> http://releases.llvm.org/
>
> And then point the build at that:
>
> sh configure --with-libclang <path to LLVM install>
>
> This will generate the jextract library and, hopefully, will also
> get rid of the warning.
>
> Maurizio
>
>
>>
>>
>> Giuseppe
>>
>> Il giorno lun 18 feb 2019 alle ore 15:33 Maurizio Cimadamore
>> <maurizio.cimadamore at oracle.com
>> <mailto:maurizio.cimadamore at oracle.com>> ha scritto:
>>
>> Whohoo!
>>
>> Looking forward to the feedback - and thanks for the patience!
>>
>> Maurizio
>>
>> On 18/02/2019 14:30, Giuseppe Barbieri wrote:
>>> Uh, sorry, my bad, you are right,
>>>
>>> ptr1.set(Vec3.lenght.L)
>>>
>>> should have been
>>>
>>> ptr1.set(Vec3.size.L)
>>>
>>> it works flawless! Thanks you all, guys
>>>
>>> I'll write later a list with some feedbacks to smooth the experience for gl devs
>>>
>>>
>>> Il giorno lun 18 feb 2019 alle ore 15:27 Maurizio Cimadamore
>>> <maurizio.cimadamore at oracle.com
>>> <mailto:maurizio.cimadamore at oracle.com>> ha scritto:
>>>
>>> Uhm - the code snippet for converting long into pointer
>>> is correct; I
>>> checked this on my machine:
>>>
>>> class Test {
>>> public static void main(String[] args) throws
>>> Throwable {
>>> try (Scope sc = Scope.globalScope().fork()) {
>>> var ptr1 = sc.allocate(NativeTypes.LONG);
>>> ptr1.set(42L);
>>> var p1 =
>>> ptr1.cast(NativeTypes.VOID).cast(NativeTypes.VOID.pointer()).get();
>>> System.err.println(p1.addr());
>>> }
>>> }
>>> }
>>>
>>>
>>> And this prints 42 as expected.
>>>
>>> Are you sure the problem is coming from that pointer,
>>> and not from
>>> somewhere else?
>>>
>>> Maurizio
>>>
>>> On 18/02/2019 14:16, Giuseppe Barbieri wrote:
>>> > I just had a chance to try, but that, unfortunately,
>>> doesnt seem to work..
>>> >
>>> https://github.com/elect86/panama/blob/master/src/test/kotlin/hello-triangle.kt#L100-L102
>>> >
>>> > I get a black triangle (I shall get a colored one instead)
>>> >
>>> > I guess I get at least the vertex positions (the
>>> triangle, although black,
>>> > is there) because in my case `semantic.attr.POSITION`
>>> is luckily zero,
>>> > which is the same index where the attributes start from.
>>> >
>>> > Other ideas?
>>> >
>>> > Il giorno lun 18 feb 2019 alle ore 10:56 Jorn Vernee
>>> <jbvernee at xs4all.nl <mailto:jbvernee at xs4all.nl>>
>>> > ha scritto:
>>> >
>>> >> Hi Giuseppe,
>>> >>
>>> >> You should be able to use this trick:
>>> >>
>>> >> long pointerValue = 12L; // e.g.
>>> >> Scope scope = Scope.newNativeScope; // or
>>> Scope.globalScope().fork()
>>> >> depending on which version you are
>>> >> Pointer<Long> ptr =
>>> scope.allocate(NativeTypes.UINT64);
>>> >> ptr.set(pointerValue);
>>> >> Pointer<?> result =
>>> >>
>>> ptr.cast(NativeTypes.VOID).cast(NativeTypes.VOID.pointer()).get();
>>> >> // use 'result'
>>> >>
>>> >> Be aware that this does do an allocation of a 64 bit
>>> int, so you might
>>> >> want to reuse the allocated space if you create a lot
>>> of pointers from
>>> >> literals.
>>> >>
>>> >> Maybe in the future we can add an API for creating
>>> pointers from long
>>> >> literals directly.
>>> >>
>>> >> Jorn
>>> >>
>>> >> Giuseppe Barbieri schreef op 2019-02-18 10:44:
>>> >>> Thanks Sundar,
>>> >>>
>>> >>> that works flawless for `(void *)0`
>>> >>>
>>> >>> But now I also would need a generic solution, for
>>> example:
>>> >>>
>>> >>> (void *)12
>>> >>>
>>> >>> There is a sill opengl call ( glVertexPointer )
>>> requesting explicitely
>>> >>> that: https://stackoverflow.com/a/8283855/1047713
>>> >>>
>>> >>> Signature:
>>> >>>
>>> >>> void glVertexAttribPointer(int index, int size, int
>>> type, byte
>>> >>> normalized, int stride, Pointer<?> pointer);
>>> >>>
>>> >>>
>>> >>> is there actually a way?
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>> You can use Pointer.nullPointer() method.
>>> >>>
>>> >>> -Sundar
>>> >>>
>>> >>> On 18/02/19, 8:09 AM, Giuseppe Barbieri wrote:
>>> >>>> Hi,
>>> >>>>
>>> >>>> I'm looking for a way to convert this:
>>> >>>>
>>> >>>> (void*)0
>>> >>>>
>>> >>>> in Java.
>>> >>>>
>>> >>>> I tried to allocate a pointer and set its value to
>>> 0, but it didnt
>>> >>>> work
>>> >>>>
>>> >>>> Any ideas, guys?
>>> >>>>
>>> >>>> Thanks in advance
>>>
More information about the panama-dev
mailing list