RFR: 7903615: Jextract should generate javadoc contents using libclang pretty printer [v3]

[Maurizio Cimadamore]

On Thu, 21 Dec 2023 15:39:28 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> This PR removes CDeclarationPrinter and, instead, uses libclang pretty-printing capabilities to print the javadoc contents for a given declaration.
>> 
>> The new logic will compute the comment string at parse time, in `TreeMaker`. The comment is then attached to the declaration using an attribute (and later retrieved when it's time to generate code).
>> 
>> I've used some custom printing policy options, after testing several of them to try and find the best results. With these options, there are two main differences from what we used to generate before:
>> 
>> 1. named structs/unions are re-expanded in function declarations (e.g. the javadoc of a function accepting a `struct Foo` will also contain the definition of `Foo`)
>> 2. there is no pretty-printing equivalent for types. This means that, for functional interfaces, we instead print the "parent" declaration, e.g. the declaration whose type is a function pointer.
>> 
>> It would be nice if we could improve on (1) by adding a new policy e.g. `IncludeAnonymousTagDefinition` - e.g. it is useful to have tag definitions inlined when the definition is anonymous (otherwise clang will generate a blob of text containing the source location of the anon definition), but it would be better perhaps not to inline the definition if it's not anonymous.
>> 
>> Example:
>> 
>> 
>> struct Point { int x; int y; };
>> 
>> void func(struct Point *pp);
>> 
>> 
>> This generates the following strings:
>> 
>> 
>> struct Point {
>>     int x;
>>     int y;
>> }
>> 
>> void func(struct Point {
>>     int x;
>>     int y;
>> } *pp)
>> 
>> 
>> I don't think it's a deal breaker, but it does make the javadoc more verbose.
>> 
>> Overall the changes are quite straightforward, and the code is simplified considerably. Also, this fixes a lot of little issues that arise when we try to generate javadoc for anonymous structs for which before only `struct` was printed.
>> 
>> There are only two situations in which I had to manually improve the clang output:
>> 
>> * macros - in this case pretty printer prints nothing :-) so I use the macro tokens to generate the corresponding doc string;
>> * enum constant - in this case pretty printer only prints the enum constant name, but the previous output had also the enum name and the value. I tweaked enum constants manually (also in `TreeMaker`). By doing that, I could then also remove `EnumConstantLifter`.
>
> Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix issue with comments in macro values

When testing with windows.h, I encountered a case where a macro constant had some comments in it, e.g.


#define FOO 42 /* a value */


Unfortunately, since my initial approach just copied all the tokens into the resulting javadoc, this fails. I've uploaded a fix for this where now macro values are generated in a much simpler fashion (w/o blindly copying the tokens). With this fix, windows.h is back to normal. I've also added a test for this sneaky issue.

-------------

PR Comment: https://git.openjdk.org/jextract/pull/169#issuecomment-1866520451