RFR: 7903615: Jextract should generate javadoc contents using libclang pretty printer [v2]

[Maurizio Cimadamore]

On Thu, 21 Dec 2023 14:59:41 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> This PR removes CDeclarationPrinter and, instead, uses libclang pretty-printing capabilities to print the javadoc contents for a given declaration.
>> 
>> The new logic will compute the comment string at parse time, in `TreeMaker`. The comment is then attached to the declaration using an attribute (and later retrieved when it's time to generate code).
>> 
>> I've used some custom printing policy options, after testing several of them to try and find the best results. With these options, there are two main differences from what we used to generate before:
>> 
>> 1. named structs/unions are re-expanded in function declarations (e.g. the javadoc of a function accepting a `struct Foo` will also contain the definition of `Foo`)
>> 2. there is no pretty-printing equivalent for types. This means that, for functional interfaces, we instead print the "parent" declaration, e.g. the declaration whose type is a function pointer.
>> 
>> It would be nice if we could improve on (1) by adding a new policy e.g. `IncludeAnonymousTagDefinition` - e.g. it is useful to have tag definitions inlined when the definition is anonymous (otherwise clang will generate a blob of text containing the source location of the anon definition), but it would be better perhaps not to inline the definition if it's not anonymous.
>> 
>> Example:
>> 
>> 
>> struct Point { int x; int y; };
>> 
>> void func(struct Point *pp);
>> 
>> 
>> This generates the following strings:
>> 
>> 
>> struct Point {
>>     int x;
>>     int y;
>> }
>> 
>> void func(struct Point {
>>     int x;
>>     int y;
>> } *pp)
>> 
>> 
>> I don't think it's a deal breaker, but it does make the javadoc more verbose.
>> 
>> Overall the changes are quite straightforward, and the code is simplified considerably. Also, this fixes a lot of little issues that arise when we try to generate javadoc for anonymous structs for which before only `struct` was printed.
>> 
>> There are only two situations in which I had to manually improve the clang output:
>> 
>> * macros - in this case pretty printer prints nothing :-) so I use the macro tokens to generate the corresponding doc string;
>> * enum constant - in this case pretty printer only prints the enum constant name, but the previous output had also the enum name and the value. I tweaked enum constants manually (also in `TreeMaker`). By doing that, I could then also remove `EnumConstantLifter`.
>
> Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Add heuristic to shorten javadoc when named struct/unions/enums are referenced

src/main/java/org/openjdk/jextract/impl/TreeMaker.java line 468:

> 466:         String declString = switch (decl) {
> 467:             case Declaration.Constant _ -> null; // do nothing for enum constants
> 468:             case Typedef _ -> declarationString(cursor, true); // always expand typedefs

typedefs are always printed in the expanded form because (a) it's not too annoying anyway and (b) in the more compact form, if the entity being typedeffed is an anonymous struct/union/enum, the output of the pretty printer can be weird. For instance in this case:


typedef struct { int x; int y; } Y;


The pretty printer generates:


typedef struct Y Y


Which is not great. The expanded form instead prints:


typedef struct {
    int x;
    int y;
} Y


as expected.

-------------

PR Review Comment: https://git.openjdk.org/jextract/pull/169#discussion_r1434185865