[foreign-jextract] RFR: 8263515: Generate flatter bindings

[Maurizio Cimadamore]

On Mon, 15 Mar 2021 12:34:43 GMT, Athijegannathan Sundararajan <sundar at openjdk.org> wrote:

>> As mentioned in [1], while the current generation strategy is handy (it only requires a single static import from the user to get started), it also generates huge, nested source files. This has a cost on tooling (as IDE can sometimes have issues with hugh source files) and on class loading, as the generated classfiles are full of innerclass/nested attributes. It is also not clear how much the current generation strategy will scale once javadoc support will be added to generated bindings.
>> 
>> This patch tweaks jextract so that flatter bindings are generated. In other words, all toplevel structs/union/typedef will now be emitted in a separate toplevel class. Structs that appear nested in the C source code will still be nested in the resulted bindings (although that might be revisited - after all in C there's only one namespace).
>> 
>> The obvious negative side-effect of the change is that a single static import will no longer be sufficient to e.g. import all functional interfaces and structs defined in a header - additional imports (or a single, additional star import) might be required too (see tests). The assumption is that doing so is an acceptable trade off.
>> 
>> The implementation had to change in two, main ways; first, the split between JavaSourceBuilder and NestedSourceBuilder was problematic, since with the flatter scheme a struct can appear both as a nested entity, or as a toplevel one.
>> 
>> The second change is that JavaSourceBuilder has been split into two: a general, abstract class (JavaSourceBuilder) and a more concrete one (ClassSourceBuilder). The latter is a builder for Java classes/interfaces, while the former is the root of all builders - and can also be used by builders that do not really map to any specific class (e.g. TopLevelBuilder).
>> 
>> Now, all concrete builders are linked (via the enclosing field) to the TopLevelBuilder. This chain is useful, as it allows up to keep "nesting" (this is visible in how the code in OutputFactory::visitScoped is shaped).
>> 
>> I've tested this patch on a bunch of libraries on Linux, and also on Windows.h and I did not notice any footprint change - in both system jextract feels faster and seems to use less memory (likley due to the fact that sources generated - stored in a string builder - are smaller). Startup seems to have improved too, at least in my OpenGL test, where the time to complete the first native call went from 80ms to 60ms. The only explanation I have for this fact is that the generated classes have much much less inner class/nestmates attributes - but could also be a coincidence (e.g. maybe the patch accidentally rearranges constants in a "better" way, for this particular test).
>> 
>> [1] - https://bugs.openjdk.java.net/browse/JDK-8263515
>
> test/jdk/tools/jextract/test8254983/LibTest8254983Test.java line 30:
> 
>> 28: import static test.jextract.test8254983.test8254983_h.*;
>> 29: import test.jextract.test8254983.*;
>> 30: 
> 
> One side effect of this non-nesting .. is that now we need two imports (import static * for functions, constants etc. and import * for structs, functional interfaces..).

Yes - this was discussed in the mailing list. I will edit the PR comment to reflect that too, so that it becomes more evident.

-------------

PR: https://git.openjdk.java.net/panama-foreign/pull/470