How To Create Array of Structs?
Sundararajan Athijegannathan
sundararajan.athijegannathan at oracle.com
Mon Sep 6 12:03:55 UTC 2021
Filed: https://bugs.openjdk.java.net/browse/JDK-8273382
-Sundar
________________________________
From: panama-dev <panama-dev-retn at openjdk.java.net> on behalf of Sundararajan Athijegannathan <sundararajan.athijegannathan at oracle.com>
Sent: 06 September 2021 10:20
To: Maurizio Cimadamore <maurizio.cimadamore at oracle.com>; Michael Ennen <mike.ennen at gmail.com>; Jorn Vernee <jorn.vernee at oracle.com>
Cc: panama-dev at openjdk.java.net <panama-dev at openjdk.java.net>
Subject: Re: How To Create Array of Structs?
Hi,
jextract does not seem to generate layout for any pointer type. Only non-point typedefs are handled.
Example:
typedef int* intptr;
struct Point {
int x, y;
};
typedef struct Point* pointptr;
Nothing generated for both intptr and pointptr;
-Sundar
________________________________
From: panama-dev <panama-dev-retn at openjdk.java.net> on behalf of Maurizio Cimadamore <maurizio.cimadamore at oracle.com>
Sent: 04 September 2021 03:42
To: Michael Ennen <mike.ennen at gmail.com>; Jorn Vernee <jorn.vernee at oracle.com>
Cc: panama-dev at openjdk.java.net <panama-dev at openjdk.java.net>
Subject: Re: How To Create Array of Structs?
Hi Michael,
I don't think the problem is lack of support for function-like macro.
Yes, VK_DEFINE_NON_DISPATCHABLE_HANDLE is a function-like macro, but
this declaration:
```
VK_DEFINE_NON_DISPATCHABLE_HANDLE(VkSemaphore)
```
is contained in the header, and preprocessed, so libclang sees something
like what you describe:
```
typedef struct VkSemaphore_T *VkSemaphore;
```
The only problem is that jextract is (because of an issue we're
investigating) not generating the layout for that typedef.
That said, layout or no layout, things are not too different - for all
intents and purposes, you can assume that the layout of the handle is
just C_POINTER.
But you are never supposed to _dereference_ that handle - instead you
are supposed to use other functions to create and use semaphores, such as:
https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/vkSignalSemaphore.html
https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/vkWaitSemaphores.html
https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/vkCreateSemaphore.html
https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/vkDestroySemaphore.html
etc.
In other words, inside the handle there's an opaque pointer to memory
that is managed by the vulkan library. You are not supposed to know
what's inside that memory. Nor you need to know what the size of
VkSemaphore_T is.
So, if you want to create an array of N semaphores, you just do this
```
MemorySegment semaphores = allocator.allocateArray(C_POINTER, N);
```
Then you will probably have to initialize the pointers in the array,
possibly using vkCreateSemaphore. As Radu pointed out, the
vkCreateSemaphore API wants you to pass a pointer to a VkSemaphore
(which is already a pointer). So, something like this:
```
for (int i = 0 ; i < N ; i++) {
vkCreateSemaphore( ... , semaphores.asSlice(C_POINTER.byteSize() *
i) , ...);
}
```
This effectively iterates over the semaphore array, gets a slice of the
"semaphores" array which starts at the element which needs to be
initialized (hence you get a pointer to a VkSemaphore - e.g. a pointer
to a pointer), and passes this slice pointer to the vkCreateSemaphore
routine. By the time the routine finishes, it should have written
something in the array element (the opaque handle).
Hope this helps.
Maurizio
On 03/09/2021 22:50, Michael Ennen wrote:
> Let me try and expand upon what I was attempting to communicate yesterday:
>
> I know that jextract does not currently support function-like macros
> so the handles like VkSemaphore are not exposed.
>
> That's why I tried to first declare it (the array of VkSemaphores) as
> a pointer to a pointer.
>
> But then I cannot get the size of VkSemaphore (really VkSemaphore_T)
> so I am unsure how to use the technique you described which does work
> in the case
> of the VkPipelineShaderStageCreateInfo struct - because it is not an
> "opaque" handle. Vulkan uses these a lot, things like VkInstance,
> VkDevice, etc. Specifically:
>
> VkSemaphore is defined by this macro:
>
> VK_DEFINE_NON_DISPATCHABLE_HANDLE(VkSemaphore)
>
> Which is how Vulkan does opaque handles (so one cannot inspect the
> contents of the VkSemaphore struct and/or rely on it).
>
> If jextract did support function-like macros then it would expand to
> the typedef:
>
> typedef struct VkSemaphore_T *VkSemaphore;
>
> and then one could possibly just do VkSemaphore.$LAYOUT() ? Or even
> still that may not work because of the "opaque"ness.
>
> Thanks for any assistance.
>
> On Thu, Sep 2, 2021 at 11:16 AM Michael Ennen <mike.ennen at gmail.com
> <mailto:mike.ennen at gmail.com>> wrote:
>
> Quick question that is now happening while trying to replicate the
> fix when I don't have an explicit struct handle like in the case
> of VkSemaphore:
>
> https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkSemaphore.html
> <https://urldefense.com/v3/__https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkSemaphore.html__;!!ACWV5N9M2RV99hQ!faDzh_GMQ92kXooRnDu53BXY9QKXlxdt-C6CYOnU9F6mvczo0318RPY2V4kVrykMhUK0Wlw$>
>
> For those types I don't have a struct layout to work with so I
> have been just declaring a double pointer:
>
> var pVkPipeline =
> SegmentAllocator.ofScope(scope).allocate(C_POINTER.byteSize());
>
> and then using MemoryAccess.getAddress(pVkPipeline).
>
> So in this case how can I create an array of VkSemaphore structs
> without knowing anything about the struct (it is an opaque handle?).
>
> Hope that makes sense.
>
> On Thu, Sep 2, 2021 at 11:06 AM Michael Ennen
> <mike.ennen at gmail.com <mailto:mike.ennen at gmail.com>> wrote:
>
> Jorn,
>
> Thank you so much - that was exactly what I was looking for. I
> don't know why I had such trouble trying to create an array of
> structs rather than an array of points to structs.
>
> Thank you both.
>
> On Thu, Sep 2, 2021 at 4:54 AM Jorn Vernee
> <jorn.vernee at oracle.com <mailto:jorn.vernee at oracle.com>> wrote:
>
> Additionally, it looks like you're allocating an array of
> pointers for
> pStages, and it looks like the API expects an array of
> structs.
>
> Instead of allocating the different
> VkPipelineShaderStageCreateInfo
> individually, you should allocate the array of struct
> upfront, and then
> fill in the different structs in-place.
>
> Like:
>
> diff --git a/src/main/java/com/brcolow/game/Vulkan.java
> b/src/main/java/com/brcolow/game/Vulkan.java
> index f0cd278..520513b 100644
> --- a/src/main/java/com/brcolow/game/Vulkan.java
> +++ b/src/main/java/com/brcolow/game/Vulkan.java
> @@ -429,17 +429,17 @@ public class Vulkan {
> var pVertShaderModule =
> getShaderModule(MemoryAccess.getAddress(pVkDevice),
> vertShaderBytes, scope);
> var pFragShaderModule =
> getShaderModule(MemoryAccess.getAddress(pVkDevice),
> fragShaderBytes, scope);
>
> - var pVertShaderStageInfo =
> VkPipelineShaderStageCreateInfo.allocate(scope);
> -
> VkPipelineShaderStageCreateInfo.sType$set(pVertShaderStageInfo,
>
> vulkan_h.VK_STRUCTURE_TYPE_PIPELINE_SHADER_STAGE_CREATE_INFO());
> -
> VkPipelineShaderStageCreateInfo.stage$set(pVertShaderStageInfo,
>
> vulkan_h.VK_SHADER_STAGE_VERTEX_BIT());
> -
> VkPipelineShaderStageCreateInfo.module$set(pVertShaderStageInfo,
>
> MemoryAccess.getAddress(pVertShaderModule));
> -
> VkPipelineShaderStageCreateInfo.pName$set(pVertShaderStageInfo,
>
> CLinker.toCString("main", scope).address());
> -
> - var pFragShaderStageInfo =
> VkPipelineShaderStageCreateInfo.allocate(scope);
> -
> VkPipelineShaderStageCreateInfo.sType$set(pFragShaderStageInfo,
>
> vulkan_h.VK_STRUCTURE_TYPE_PIPELINE_SHADER_STAGE_CREATE_INFO());
> -
> VkPipelineShaderStageCreateInfo.stage$set(pFragShaderStageInfo,
>
> vulkan_h.VK_SHADER_STAGE_FRAGMENT_BIT());
> -
> VkPipelineShaderStageCreateInfo.module$set(pFragShaderStageInfo,
>
> MemoryAccess.getAddress(pFragShaderModule));
> -
> VkPipelineShaderStageCreateInfo.pName$set(pFragShaderStageInfo,
>
> CLinker.toCString("main", scope).address());
> + MemorySegment pStages =
> SegmentAllocator.ofScope(scope).allocateArray(VkPipelineShaderStageCreateInfo.$LAYOUT(),
>
> 2);
> +
> + VkPipelineShaderStageCreateInfo.sType$set(pStages, 0,
> vulkan_h.VK_STRUCTURE_TYPE_PIPELINE_SHADER_STAGE_CREATE_INFO());
> + VkPipelineShaderStageCreateInfo.stage$set(pStages, 0,
> vulkan_h.VK_SHADER_STAGE_VERTEX_BIT());
> + VkPipelineShaderStageCreateInfo.module$set(pStages, 0,
> MemoryAccess.getAddress(pVertShaderModule));
> + VkPipelineShaderStageCreateInfo.pName$set(pStages, 0,
> CLinker.toCString("main", scope).address());
> +
> + VkPipelineShaderStageCreateInfo.sType$set(pStages, 1,
> vulkan_h.VK_STRUCTURE_TYPE_PIPELINE_SHADER_STAGE_CREATE_INFO());
> + VkPipelineShaderStageCreateInfo.stage$set(pStages, 1,
> vulkan_h.VK_SHADER_STAGE_FRAGMENT_BIT());
> + VkPipelineShaderStageCreateInfo.module$set(pStages, 1,
> MemoryAccess.getAddress(pFragShaderModule));
> + VkPipelineShaderStageCreateInfo.pName$set(pStages, 1,
> CLinker.toCString("main", scope).address());
>
> var pVertexInputStateInfo =
> VkPipelineVertexInputStateCreateInfo.allocate(scope);
> VkPipelineVertexInputStateCreateInfo.sType$set(pVertexInputStateInfo,
>
> vulkan_h.VK_STRUCTURE_TYPE_PIPELINE_VERTEX_INPUT_STATE_CREATE_INFO());
> @@ -581,16 +581,14 @@ public class Vulkan {
> // If we change stageCount to 2 we get a
> crash....
> // I think we are creating the pStages array
> incorrectly.
> This is also manifesting further down when trying
> // to create arrays pWaitSemaphores,
> pSignalSemaphores, etc.
> -
> VkGraphicsPipelineCreateInfo.stageCount$set(pPipelineCreateInfo,
> 1);
> - MemorySegment pStages =
> SegmentAllocator.ofScope(scope).allocateArray(C_POINTER,
> new Addressable[]{
> - pVertShaderStageInfo,
> pFragShaderStageInfo});
> - System.out.println("pStages: " + pStages);
> - System.out.println("pStages[0]: " +
> VkPipelineShaderStageCreateInfo.ofAddress(MemoryAccess.getAddressAtIndex(pStages,
>
> 0), scope));
> - System.out.println("pStages[0].sType: " +
> VkPipelineShaderStageCreateInfo.sType$get(VkPipelineShaderStageCreateInfo.ofAddress(MemoryAccess.getAddressAtIndex(pStages,
>
> 0), scope)));
> - System.out.println("pStages[1]: " +
> VkPipelineShaderStageCreateInfo.ofAddress(MemoryAccess.getAddressAtIndex(pStages,
>
> 1), scope));
> - System.out.println("pStages[1].sType: " +
> VkPipelineShaderStageCreateInfo.sType$get(VkPipelineShaderStageCreateInfo.ofAddress(MemoryAccess.getAddressAtIndex(pStages,
>
> 1), scope)));
> -
> -
> VkGraphicsPipelineCreateInfo.pStages$set(pPipelineCreateInfo,
> MemoryAccess.getAddress(pStages));
> +
> VkGraphicsPipelineCreateInfo.stageCount$set(pPipelineCreateInfo,
> 2);
> + System.out.println("pStages: " +
> pStages.address());
> + System.out.println("pStages[0]: " +
> pStages.asSlice(0,
> VkPipelineShaderStageCreateInfo.$LAYOUT().byteSize()));
> + System.out.println("pStages[0].sType: " +
> VkPipelineShaderStageCreateInfo.sType$get(pStages, 0));
> + System.out.println("pStages[1]: " +
> pStages.asSlice(VkPipelineShaderStageCreateInfo.$LAYOUT().byteSize(),
>
> VkPipelineShaderStageCreateInfo.$LAYOUT().byteSize()));
> + System.out.println("pStages[1].sType: " +
> VkPipelineShaderStageCreateInfo.sType$get(pStages, 1));
> +
> +
> VkGraphicsPipelineCreateInfo.pStages$set(pPipelineCreateInfo,
> pStages.address());
> VkGraphicsPipelineCreateInfo.pVertexInputState$set(pPipelineCreateInfo,
>
> pVertexInputStateInfo.address());
> VkGraphicsPipelineCreateInfo.pInputAssemblyState$set(pPipelineCreateInfo,
>
> pPipelineInputAssemblyStateInfo.address());
> VkGraphicsPipelineCreateInfo.pViewportState$set(pPipelineCreateInfo,
>
> pPipelineViewportStateInfo.address());
>
> HTH,
> Jorn
>
> On 02/09/2021 11:56, Maurizio Cimadamore wrote:
> > Hi Michael,
> > very interesting experiments - thanks for reaching out.
> >
> > Eyeballing your code, I zoomed into this line:
> >
> >
> https://github.com/brcolow/java-vulkan/blob/master/src/main/java/com/brcolow/game/Vulkan.java#L593
> <https://urldefense.com/v3/__https://github.com/brcolow/java-vulkan/blob/master/src/main/java/com/brcolow/game/Vulkan.java*L593__;Iw!!ACWV5N9M2RV99hQ!faDzh_GMQ92kXooRnDu53BXY9QKXlxdt-C6CYOnU9F6mvczo0318RPY2V4kVrykM2LZrvQ0$>
>
> >
> >
> > Here you want to set the `pStages` pointer, a pointer to
> a memory
> > region which presumably has to contain exactly
> `stageCount` structs of
> > type `VkPipelineShaderStageCreateInfo`.
> >
> > I believe there's an extra getAddress in there; this:
> >
> > ```
> >
> VkGraphicsPipelineCreateInfo.pStages$set(pPipelineCreateInfo,
> > MemoryAccess.getAddress(pStages));
> > ```
> >
> > should be:
> >
> > ```
> >
> VkGraphicsPipelineCreateInfo.pStages$set(pPipelineCreateInfo,
> pStages);
> > ```
> >
> > That is, you want to pass the pointer to the `pStages`
> region you just
> > created, so that Vulkan will be able to see all
> elements. If you call
> > "MemoryAccess::getAddress" your code will read the first
> 64 bits from
> > the `pStages` region, turn those into a MemoryAddress,
> and set that
> > into the enclosing `pPipelineCreateInfo` struct. Which I
> don't think
> > is what you want?
> >
> > Let me know if this works.
> >
> > Cheers
> > Maurizio
> >
> >
> > On 02/09/2021 06:35, Michael Ennen wrote:
> >> I have been doing more exploring with the
> panama-foreign, ffi, and
> >> jextract
> >> branches - this time messing around with seeing if I
> can get a rainbow
> >> triangle to be rendered on the screen with Vulkan.
> >>
> >> I am really close. There is one problem I am having
> which is how to
> >> create
> >> an array of structs?
> >>
> >> See this, for example:
> >>
> >>
> https://github.com/brcolow/java-vulkan/blob/master/src/main/java/com/brcolow/game/Vulkan.java#L581
> <https://urldefense.com/v3/__https://github.com/brcolow/java-vulkan/blob/master/src/main/java/com/brcolow/game/Vulkan.java*L581__;Iw!!ACWV5N9M2RV99hQ!faDzh_GMQ92kXooRnDu53BXY9QKXlxdt-C6CYOnU9F6mvczo0318RPY2V4kVrykMp_8smQQ$>
>
> >>
> >>
> >> If I set the stageCount to 2, then I get a validation
> exception about
> >> the
> >> second shader module having bogus values. I believe
> this is caused by me
> >> creating the array of structs incorrectly.
> >>
> >> I am trying to set the pStages element of this struct:
> >>
> https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkGraphicsPipelineCreateInfo.html
> <https://urldefense.com/v3/__https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkGraphicsPipelineCreateInfo.html__;!!ACWV5N9M2RV99hQ!faDzh_GMQ92kXooRnDu53BXY9QKXlxdt-C6CYOnU9F6mvczo0318RPY2V4kVrykMguxb9-s$>
>
> >>
> >>
> >> to an array containing pVertShaderStageInfo and
> pFragShaderStageInfo.
> >>
> >> My attempt (I have tried other ways that also fail) to
> do that is:
> >>
> >> MemorySegment pStages =
> SegmentAllocator.ofScope(scope).allocateArray(
> >> C_POINTER, new Addressable[]{
> >> pVertShaderStageInfo, pFragShaderStageInfo});
> >>
> VkGraphicsPipelineCreateInfo.pStages$set(pPipelineCreateInfo,
> >> MemoryAccess.getAddress(pStages));
> >>
> >>
> >> If anyone could assist me with the proper way to create the
> >> VkPipelineShaderStageCreateInfo* array containing
> >> pVertShaderStageInfoand
> >> pFragShaderStageInfo - it would be greatly appreciated!
> >>
>
>
>
> --
> Michael Ennen
>
>
>
> --
> Michael Ennen
>
>
>
> --
> Michael Ennen
More information about the panama-dev
mailing list