How To Create Array of Structs?
Maurizio Cimadamore
maurizio.cimadamore at oracle.com
Wed Sep 8 13:28:47 UTC 2021
This fix for this issue has been integrated.
If you are able to build the Panama repo manually, you should be able to
re-extract Vulkan, and hopefully you should see the layout for
VkSemaphore in there.
Cheers
Maurizio
On 06/09/2021 13:03, Sundararajan Athijegannathan wrote:
> Filed: https://bugs.openjdk.java.net/browse/JDK-8273382
> <https://bugs.openjdk.java.net/browse/JDK-8273382>
>
> -Sundar
> ------------------------------------------------------------------------
> *From:* panama-dev <panama-dev-retn at openjdk.java.net> on behalf of
> Sundararajan Athijegannathan <sundararajan.athijegannathan at oracle.com>
> *Sent:* 06 September 2021 10:20
> *To:* Maurizio Cimadamore <maurizio.cimadamore at oracle.com>; Michael
> Ennen <mike.ennen at gmail.com>; Jorn Vernee <jorn.vernee at oracle.com>
> *Cc:* panama-dev at openjdk.java.net <panama-dev at openjdk.java.net>
> *Subject:* Re: How To Create Array of Structs?
> Hi,
>
> jextract does not seem to generate layout for any pointer type. Only
> non-point typedefs are handled.
>
>
> Example:
>
>
> typedef int* intptr;
>
>
>
> struct Point {
>
> int x, y;
>
> };
>
>
> typedef struct Point* pointptr;
>
>
> Nothing generated for both intptr and pointptr;
>
>
> -Sundar
> ________________________________
> From: panama-dev <panama-dev-retn at openjdk.java.net> on behalf of
> Maurizio Cimadamore <maurizio.cimadamore at oracle.com>
> Sent: 04 September 2021 03:42
> To: Michael Ennen <mike.ennen at gmail.com>; Jorn Vernee
> <jorn.vernee at oracle.com>
> Cc: panama-dev at openjdk.java.net <panama-dev at openjdk.java.net>
> Subject: Re: How To Create Array of Structs?
>
> Hi Michael,
> I don't think the problem is lack of support for function-like macro.
>
> Yes, VK_DEFINE_NON_DISPATCHABLE_HANDLE is a function-like macro, but
> this declaration:
>
> ```
> VK_DEFINE_NON_DISPATCHABLE_HANDLE(VkSemaphore)
> ```
>
> is contained in the header, and preprocessed, so libclang sees something
> like what you describe:
>
> ```
> typedef struct VkSemaphore_T *VkSemaphore;
> ```
>
> The only problem is that jextract is (because of an issue we're
> investigating) not generating the layout for that typedef.
>
> That said, layout or no layout, things are not too different - for all
> intents and purposes, you can assume that the layout of the handle is
> just C_POINTER.
>
> But you are never supposed to _dereference_ that handle - instead you
> are supposed to use other functions to create and use semaphores, such as:
>
> https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/vkSignalSemaphore.html
> <https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/vkSignalSemaphore.html>
> https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/vkWaitSemaphores.html
> <https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/vkWaitSemaphores.html>
> https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/vkCreateSemaphore.html
> <https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/vkCreateSemaphore.html>
> https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/vkDestroySemaphore.html
> <https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/vkDestroySemaphore.html>
>
> etc.
>
> In other words, inside the handle there's an opaque pointer to memory
> that is managed by the vulkan library. You are not supposed to know
> what's inside that memory. Nor you need to know what the size of
> VkSemaphore_T is.
>
> So, if you want to create an array of N semaphores, you just do this
>
> ```
> MemorySegment semaphores = allocator.allocateArray(C_POINTER, N);
> ```
>
> Then you will probably have to initialize the pointers in the array,
> possibly using vkCreateSemaphore. As Radu pointed out, the
> vkCreateSemaphore API wants you to pass a pointer to a VkSemaphore
> (which is already a pointer). So, something like this:
>
> ```
> for (int i = 0 ; i < N ; i++) {
> vkCreateSemaphore( ... , semaphores.asSlice(C_POINTER.byteSize() *
> i) , ...);
> }
> ```
>
> This effectively iterates over the semaphore array, gets a slice of the
> "semaphores" array which starts at the element which needs to be
> initialized (hence you get a pointer to a VkSemaphore - e.g. a pointer
> to a pointer), and passes this slice pointer to the vkCreateSemaphore
> routine. By the time the routine finishes, it should have written
> something in the array element (the opaque handle).
>
> Hope this helps.
>
> Maurizio
>
>
>
> On 03/09/2021 22:50, Michael Ennen wrote:
> > Let me try and expand upon what I was attempting to communicate
> yesterday:
> >
> > I know that jextract does not currently support function-like macros
> > so the handles like VkSemaphore are not exposed.
> >
> > That's why I tried to first declare it (the array of VkSemaphores) as
> > a pointer to a pointer.
> >
> > But then I cannot get the size of VkSemaphore (really VkSemaphore_T)
> > so I am unsure how to use the technique you described which does work
> > in the case
> > of the VkPipelineShaderStageCreateInfo struct - because it is not an
> > "opaque" handle. Vulkan uses these a lot, things like VkInstance,
> > VkDevice, etc. Specifically:
> >
> > VkSemaphore is defined by this macro:
> >
> > VK_DEFINE_NON_DISPATCHABLE_HANDLE(VkSemaphore)
> >
> > Which is how Vulkan does opaque handles (so one cannot inspect the
> > contents of the VkSemaphore struct and/or rely on it).
> >
> > If jextract did support function-like macros then it would expand to
> > the typedef:
> >
> > typedef struct VkSemaphore_T *VkSemaphore;
> >
> > and then one could possibly just do VkSemaphore.$LAYOUT() ? Or even
> > still that may not work because of the "opaque"ness.
> >
> > Thanks for any assistance.
> >
> > On Thu, Sep 2, 2021 at 11:16 AM Michael Ennen <mike.ennen at gmail.com
> > <mailto:mike.ennen at gmail.com <mailto:mike.ennen at gmail.com>>> wrote:
> >
> > Quick question that is now happening while trying to replicate the
> > fix when I don't have an explicit struct handle like in the case
> > of VkSemaphore:
> >
> >
> https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkSemaphore.html
> <https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkSemaphore.html>
> >
> <https://urldefense.com/v3/__https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkSemaphore.html__;!!ACWV5N9M2RV99hQ!faDzh_GMQ92kXooRnDu53BXY9QKXlxdt-C6CYOnU9F6mvczo0318RPY2V4kVrykMhUK0Wlw$
> <https://urldefense.com/v3/__https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkSemaphore.html__;!!ACWV5N9M2RV99hQ!faDzh_GMQ92kXooRnDu53BXY9QKXlxdt-C6CYOnU9F6mvczo0318RPY2V4kVrykMhUK0Wlw$>>
> >
> > For those types I don't have a struct layout to work with so I
> > have been just declaring a double pointer:
> >
> > var pVkPipeline =
> > SegmentAllocator.ofScope(scope).allocate(C_POINTER.byteSize());
> >
> > and then using MemoryAccess.getAddress(pVkPipeline).
> >
> > So in this case how can I create an array of VkSemaphore structs
> > without knowing anything about the struct (it is an opaque handle?).
> >
> > Hope that makes sense.
> >
> > On Thu, Sep 2, 2021 at 11:06 AM Michael Ennen
> > <mike.ennen at gmail.com <mailto:mike.ennen at gmail.com
> <mailto:mike.ennen at gmail.com>>> wrote:
> >
> > Jorn,
> >
> > Thank you so much - that was exactly what I was looking for. I
> > don't know why I had such trouble trying to create an array of
> > structs rather than an array of points to structs.
> >
> > Thank you both.
> >
> > On Thu, Sep 2, 2021 at 4:54 AM Jorn Vernee
> > <jorn.vernee at oracle.com <mailto:jorn.vernee at oracle.com
> <mailto:jorn.vernee at oracle.com>>> wrote:
> >
> > Additionally, it looks like you're allocating an array of
> > pointers for
> > pStages, and it looks like the API expects an array of
> > structs.
> >
> > Instead of allocating the different
> > VkPipelineShaderStageCreateInfo
> > individually, you should allocate the array of struct
> > upfront, and then
> > fill in the different structs in-place.
> >
> > Like:
> >
> > diff --git a/src/main/java/com/brcolow/game/Vulkan.java
> > b/src/main/java/com/brcolow/game/Vulkan.java
> > index f0cd278..520513b 100644
> > --- a/src/main/java/com/brcolow/game/Vulkan.java
> > +++ b/src/main/java/com/brcolow/game/Vulkan.java
> > @@ -429,17 +429,17 @@ public class Vulkan {
> > var pVertShaderModule =
> > getShaderModule(MemoryAccess.getAddress(pVkDevice),
> > vertShaderBytes, scope);
> > var pFragShaderModule =
> > getShaderModule(MemoryAccess.getAddress(pVkDevice),
> > fragShaderBytes, scope);
> >
> > - var pVertShaderStageInfo =
> > VkPipelineShaderStageCreateInfo.allocate(scope);
> > -
> > VkPipelineShaderStageCreateInfo.sType$set(pVertShaderStageInfo,
> >
> > vulkan_h.VK_STRUCTURE_TYPE_PIPELINE_SHADER_STAGE_CREATE_INFO());
> > -
> > VkPipelineShaderStageCreateInfo.stage$set(pVertShaderStageInfo,
> >
> > vulkan_h.VK_SHADER_STAGE_VERTEX_BIT());
> > -
> > VkPipelineShaderStageCreateInfo.module$set(pVertShaderStageInfo,
> >
> > MemoryAccess.getAddress(pVertShaderModule));
> > -
> > VkPipelineShaderStageCreateInfo.pName$set(pVertShaderStageInfo,
> >
> > CLinker.toCString("main", scope).address());
> > -
> > - var pFragShaderStageInfo =
> > VkPipelineShaderStageCreateInfo.allocate(scope);
> > -
> > VkPipelineShaderStageCreateInfo.sType$set(pFragShaderStageInfo,
> >
> > vulkan_h.VK_STRUCTURE_TYPE_PIPELINE_SHADER_STAGE_CREATE_INFO());
> > -
> > VkPipelineShaderStageCreateInfo.stage$set(pFragShaderStageInfo,
> >
> > vulkan_h.VK_SHADER_STAGE_FRAGMENT_BIT());
> > -
> > VkPipelineShaderStageCreateInfo.module$set(pFragShaderStageInfo,
> >
> > MemoryAccess.getAddress(pFragShaderModule));
> > -
> > VkPipelineShaderStageCreateInfo.pName$set(pFragShaderStageInfo,
> >
> > CLinker.toCString("main", scope).address());
> > + MemorySegment pStages =
> >
> SegmentAllocator.ofScope(scope).allocateArray(VkPipelineShaderStageCreateInfo.$LAYOUT(),
> >
> > 2);
> > +
> > + VkPipelineShaderStageCreateInfo.sType$set(pStages, 0,
> > vulkan_h.VK_STRUCTURE_TYPE_PIPELINE_SHADER_STAGE_CREATE_INFO());
> > + VkPipelineShaderStageCreateInfo.stage$set(pStages, 0,
> > vulkan_h.VK_SHADER_STAGE_VERTEX_BIT());
> > + VkPipelineShaderStageCreateInfo.module$set(pStages, 0,
> > MemoryAccess.getAddress(pVertShaderModule));
> > + VkPipelineShaderStageCreateInfo.pName$set(pStages, 0,
> > CLinker.toCString("main", scope).address());
> > +
> > + VkPipelineShaderStageCreateInfo.sType$set(pStages, 1,
> > vulkan_h.VK_STRUCTURE_TYPE_PIPELINE_SHADER_STAGE_CREATE_INFO());
> > + VkPipelineShaderStageCreateInfo.stage$set(pStages, 1,
> > vulkan_h.VK_SHADER_STAGE_FRAGMENT_BIT());
> > + VkPipelineShaderStageCreateInfo.module$set(pStages, 1,
> > MemoryAccess.getAddress(pFragShaderModule));
> > + VkPipelineShaderStageCreateInfo.pName$set(pStages, 1,
> > CLinker.toCString("main", scope).address());
> >
> > var pVertexInputStateInfo =
> > VkPipelineVertexInputStateCreateInfo.allocate(scope);
> > VkPipelineVertexInputStateCreateInfo.sType$set(pVertexInputStateInfo,
> >
> > vulkan_h.VK_STRUCTURE_TYPE_PIPELINE_VERTEX_INPUT_STATE_CREATE_INFO());
> > @@ -581,16 +581,14 @@ public class Vulkan {
> > // If we change stageCount to 2 we get a
> > crash....
> > // I think we are creating the pStages array
> > incorrectly.
> > This is also manifesting further down when trying
> > // to create arrays pWaitSemaphores,
> > pSignalSemaphores, etc.
> > -
> > VkGraphicsPipelineCreateInfo.stageCount$set(pPipelineCreateInfo,
> > 1);
> > - MemorySegment pStages =
> > SegmentAllocator.ofScope(scope).allocateArray(C_POINTER,
> > new Addressable[]{
> > - pVertShaderStageInfo,
> > pFragShaderStageInfo});
> > - System.out.println("pStages: " + pStages);
> > - System.out.println("pStages[0]: " +
> >
> VkPipelineShaderStageCreateInfo.ofAddress(MemoryAccess.getAddressAtIndex(pStages,
> >
> > 0), scope));
> > - System.out.println("pStages[0].sType: " +
> >
> VkPipelineShaderStageCreateInfo.sType$get(VkPipelineShaderStageCreateInfo.ofAddress(MemoryAccess.getAddressAtIndex(pStages,
> >
> > 0), scope)));
> > - System.out.println("pStages[1]: " +
> >
> VkPipelineShaderStageCreateInfo.ofAddress(MemoryAccess.getAddressAtIndex(pStages,
> >
> > 1), scope));
> > - System.out.println("pStages[1].sType: " +
> >
> VkPipelineShaderStageCreateInfo.sType$get(VkPipelineShaderStageCreateInfo.ofAddress(MemoryAccess.getAddressAtIndex(pStages,
> >
> > 1), scope)));
> > -
> > -
> > VkGraphicsPipelineCreateInfo.pStages$set(pPipelineCreateInfo,
> > MemoryAccess.getAddress(pStages));
> > +
> > VkGraphicsPipelineCreateInfo.stageCount$set(pPipelineCreateInfo,
> > 2);
> > + System.out.println("pStages: " +
> > pStages.address());
> > + System.out.println("pStages[0]: " +
> > pStages.asSlice(0,
> > VkPipelineShaderStageCreateInfo.$LAYOUT().byteSize()));
> > + System.out.println("pStages[0].sType: " +
> > VkPipelineShaderStageCreateInfo.sType$get(pStages, 0));
> > + System.out.println("pStages[1]: " +
> > pStages.asSlice(VkPipelineShaderStageCreateInfo.$LAYOUT().byteSize(),
> >
> > VkPipelineShaderStageCreateInfo.$LAYOUT().byteSize()));
> > + System.out.println("pStages[1].sType: " +
> > VkPipelineShaderStageCreateInfo.sType$get(pStages, 1));
> > +
> > +
> > VkGraphicsPipelineCreateInfo.pStages$set(pPipelineCreateInfo,
> > pStages.address());
> > VkGraphicsPipelineCreateInfo.pVertexInputState$set(pPipelineCreateInfo,
> >
> > pVertexInputStateInfo.address());
> >
> VkGraphicsPipelineCreateInfo.pInputAssemblyState$set(pPipelineCreateInfo,
> >
> > pPipelineInputAssemblyStateInfo.address());
> > VkGraphicsPipelineCreateInfo.pViewportState$set(pPipelineCreateInfo,
> >
> > pPipelineViewportStateInfo.address());
> >
> > HTH,
> > Jorn
> >
> > On 02/09/2021 11:56, Maurizio Cimadamore wrote:
> > > Hi Michael,
> > > very interesting experiments - thanks for reaching out.
> > >
> > > Eyeballing your code, I zoomed into this line:
> > >
> > >
> >
> https://github.com/brcolow/java-vulkan/blob/master/src/main/java/com/brcolow/game/Vulkan.java#L593
> <https://github.com/brcolow/java-vulkan/blob/master/src/main/java/com/brcolow/game/Vulkan.java#L593>
> >
> <https://urldefense.com/v3/__https://github.com/brcolow/java-vulkan/blob/master/src/main/java/com/brcolow/game/Vulkan.java*L593__;Iw!!ACWV5N9M2RV99hQ!faDzh_GMQ92kXooRnDu53BXY9QKXlxdt-C6CYOnU9F6mvczo0318RPY2V4kVrykM2LZrvQ0$
> <https://urldefense.com/v3/__https://github.com/brcolow/java-vulkan/blob/master/src/main/java/com/brcolow/game/Vulkan.java*L593__;Iw!!ACWV5N9M2RV99hQ!faDzh_GMQ92kXooRnDu53BXY9QKXlxdt-C6CYOnU9F6mvczo0318RPY2V4kVrykM2LZrvQ0$>>
> >
> > >
> > >
> > > Here you want to set the `pStages` pointer, a pointer to
> > a memory
> > > region which presumably has to contain exactly
> > `stageCount` structs of
> > > type `VkPipelineShaderStageCreateInfo`.
> > >
> > > I believe there's an extra getAddress in there; this:
> > >
> > > ```
> > >
> > VkGraphicsPipelineCreateInfo.pStages$set(pPipelineCreateInfo,
> > > MemoryAccess.getAddress(pStages));
> > > ```
> > >
> > > should be:
> > >
> > > ```
> > >
> > VkGraphicsPipelineCreateInfo.pStages$set(pPipelineCreateInfo,
> > pStages);
> > > ```
> > >
> > > That is, you want to pass the pointer to the `pStages`
> > region you just
> > > created, so that Vulkan will be able to see all
> > elements. If you call
> > > "MemoryAccess::getAddress" your code will read the first
> > 64 bits from
> > > the `pStages` region, turn those into a MemoryAddress,
> > and set that
> > > into the enclosing `pPipelineCreateInfo` struct. Which I
> > don't think
> > > is what you want?
> > >
> > > Let me know if this works.
> > >
> > > Cheers
> > > Maurizio
> > >
> > >
> > > On 02/09/2021 06:35, Michael Ennen wrote:
> > >> I have been doing more exploring with the
> > panama-foreign, ffi, and
> > >> jextract
> > >> branches - this time messing around with seeing if I
> > can get a rainbow
> > >> triangle to be rendered on the screen with Vulkan.
> > >>
> > >> I am really close. There is one problem I am having
> > which is how to
> > >> create
> > >> an array of structs?
> > >>
> > >> See this, for example:
> > >>
> > >>
> >
> https://github.com/brcolow/java-vulkan/blob/master/src/main/java/com/brcolow/game/Vulkan.java#L581
> <https://github.com/brcolow/java-vulkan/blob/master/src/main/java/com/brcolow/game/Vulkan.java#L581>
> >
> <https://urldefense.com/v3/__https://github.com/brcolow/java-vulkan/blob/master/src/main/java/com/brcolow/game/Vulkan.java*L581__;Iw!!ACWV5N9M2RV99hQ!faDzh_GMQ92kXooRnDu53BXY9QKXlxdt-C6CYOnU9F6mvczo0318RPY2V4kVrykMp_8smQQ$
> <https://urldefense.com/v3/__https://github.com/brcolow/java-vulkan/blob/master/src/main/java/com/brcolow/game/Vulkan.java*L581__;Iw!!ACWV5N9M2RV99hQ!faDzh_GMQ92kXooRnDu53BXY9QKXlxdt-C6CYOnU9F6mvczo0318RPY2V4kVrykMp_8smQQ$>>
> >
> > >>
> > >>
> > >> If I set the stageCount to 2, then I get a validation
> > exception about
> > >> the
> > >> second shader module having bogus values. I believe
> > this is caused by me
> > >> creating the array of structs incorrectly.
> > >>
> > >> I am trying to set the pStages element of this struct:
> > >>
> >
> https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkGraphicsPipelineCreateInfo.html
> <https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkGraphicsPipelineCreateInfo.html>
> >
> <https://urldefense.com/v3/__https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkGraphicsPipelineCreateInfo.html__;!!ACWV5N9M2RV99hQ!faDzh_GMQ92kXooRnDu53BXY9QKXlxdt-C6CYOnU9F6mvczo0318RPY2V4kVrykMguxb9-s$
> <https://urldefense.com/v3/__https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkGraphicsPipelineCreateInfo.html__;!!ACWV5N9M2RV99hQ!faDzh_GMQ92kXooRnDu53BXY9QKXlxdt-C6CYOnU9F6mvczo0318RPY2V4kVrykMguxb9-s$>>
> >
> > >>
> > >>
> > >> to an array containing pVertShaderStageInfo and
> > pFragShaderStageInfo.
> > >>
> > >> My attempt (I have tried other ways that also fail) to
> > do that is:
> > >>
> > >> MemorySegment pStages =
> > SegmentAllocator.ofScope(scope).allocateArray(
> > >> C_POINTER, new Addressable[]{
> > >> pVertShaderStageInfo, pFragShaderStageInfo});
> > >>
> > VkGraphicsPipelineCreateInfo.pStages$set(pPipelineCreateInfo,
> > >> MemoryAccess.getAddress(pStages));
> > >>
> > >>
> > >> If anyone could assist me with the proper way to
> create the
> > >> VkPipelineShaderStageCreateInfo* array containing
> > >> pVertShaderStageInfoand
> > >> pFragShaderStageInfo - it would be greatly appreciated!
> > >>
> >
> >
> >
> > --
> > Michael Ennen
> >
> >
> >
> > --
> > Michael Ennen
> >
> >
> >
> > --
> > Michael Ennen
More information about the panama-dev
mailing list