Refactoring JavaFX Builds & Sources

[Mario Torre]

> Further more, you should be able to do "find usages" in your IDE and get EVERYTHING IN THE UNIVERSE, rather than right now where in NB it doesn't have enough information about the 160 other projects in our system to be able to do this. This is bad.
>
> Right now we build with ant. Many of you have suggested Maven. Some of you at JavaOne suggested Gradle. I am presently looking into Gradle. The key things I like about Gradle are:
>
>         1) It isn't XML
>         2) Like Ivy & Maven, it handles dependencies well
>
> We presently have a pile of XML files and nasty build logic for downloading dependencies (this is all in the closed repo, but you would be exposed to the horror if we just made it all public). I wanted instead to use Ivy at the very least, but rather if I can get better build happiness from another setup entirely (like Gradle) then I might as well give it a try.

I kind of like Gradle, but let me put the cart before the horse here
(I don't know if this sounds as well in English as it is in Italian,
where we actually use the oxen, but well :)

While Maven is a well understood tool, Gradle is fairly new. This
turns to be especially problematic when building and shipping on Linux
distributions because there are some constraints and limitation on
what can go or not in a given release (btw, I think this could be
considered a problem on Windows and OSX installations in some
environments where you can't simply install any tool you want).

Maven, on the other end, is pretty likely sitting on your machine
already (as are Make, GCC and Configure most of the time on developers
machines), so I suspect it will be easier for the average folk as well
as heavily filled packager to take care of the build bits.

Oh my, I would have never thought I would try to sell Maven myself! Of
course, don't make this stop evolutionary attempts at solving the
build process, but I thought it's one consideration to take into
account when deciding what build system to use, since it's such a
fundamental part of the code.

Ultimately, the build infrastructure should be what makes more sense
for the project obviously.

>         1) I'll make sure cross builds all continue to work
>         2) Partial builds (for native code particularly) will continue to work such that you don't HAVE to build native code
>         3) It will produce bit-for-bit compatible output in the artifacts/ directory, so all our RE (Release Engineering) scripts continue to work with little or no modification
>
> Also, in my tests so far the speed of the build is dramatically faster -- but then I don't have everything in yet so that might just be wishful thinking. I hope and expect it to be faster though.
>
> We generate code in our build system for the following reasons:
>
>         1) We have a VersionInfo class which the build system will update with version information. It is a com.sun.something class
>         2) We have an annotation processor which generates all of the builders
>         3) We have two grammars that Antlr has process to produce parsers -- one for JSL (for our shaders) and one for CSS
>         4) We have a "decora compiler" which takes JSL files and feeds it through the JSL parser and produces code / shaders
>
> In addition to code generation, we also build a ton of native code (gstreamer, webkit, prism, fonts, glass, image loading). So that has to fit into the build system. Building native code is SLOW, so being able to avoid it for "normal" developers, and being able to avoid native builds when nothing changed in the native code, is important.

Speaking about this, I wonder if we will end up doing like IcedTea
with OpenJDK, since most of those probably have an OS counterpart we
would rather want to use (in other words, make those elements somehow
pluggable).

> I have found that some of our unit tests are written such that they require more of the platform, rather than just one class. So for example, there are tests in the scene graph which might want to use a UI control for some purpose. I'm thinking the easiest thing to do is to have a test directory at the top level which houses all the tests, and break them up based on whether they are part of the smoke test, integration test, or manual test suite. Smoke tests are those that are run continuously. They must be headless, and they must be completely automated, and they must execute within some reasonable amount of time (since hudson is going to be doing it continuously). The integration tests are the rest of the automated tests that need to be run between every integration with "master". Finally the manual tests are not run as part of the integration / smoke tests, but really are just little "toys" or apps we've written that are useful when developing a feature or whatnot. We've got a ton of these little guys (mostly of the "HelloWorld" variety), and these would go into "manual".

This is one advantage of maven, in that you are somehow forced to have
this layout. Maven supports the idea of modules, they are a hierarchy
that pretty much resembles what the project structure you depict later
looks, and for each of those modules you would have unit tests.
Integration tests could simply go into an high level module, the same
for the manual tests.

> We also will have an apps directory, where we will place all the samples (like Ensemble) and experiments (like the JavaOne Schedule Builder).
>
> And I want to have a "benchmarks" directory where all of our benchmarks will live. The only problem with this one is that we have a dependency on JRockit benchmark harness, and I'm not sure how to resolve that with the open source bits. Maybe the JRockit benchmark harness is already open source, I haven't dug into it yet to find out. However I believe we *must* have benchmarks open, as well as the SQE tests. My goal is to run OpenJFX in the open and to have meaningful contributions -- how can that happen if we're keeping back essential verification tools? How can somebody submit a patch and not know whether it is going to impact performance? So I want to get the benchmarks all out, but we need to figure out what to do about this dependency. Another option is to replace this dependency with one of the other open source benchmark harnesses, but this would need to be carefully considered as it would disrupt our benchmark reporting for some period of time. But ultimately having public numbers and public tests would, I think, be very beneficial across the board.

As a side note, we're developing a monitoring tool for OpenJDK
(Thermostat). I don't want to put Thermostat in competition with
JRockit (well, I did so already!), but by the time the full code is
out, we will be quite likely ready for prime time. There are also
other tools that do pretty good job, like VisualVM and JConsole.
Depending on the harness you have (which I assume means loads of
binary files with dumps and performance related data), we can either
try to make them compatible with all those tools, or better yet,
develop some kind exchange format for Performance Analysis (so that we
are not really depending on any tool). This can be a cool
collaboration between some other teams (I suspect such an effort would
go beyond JavaFX of course). In any case, having the harness in the
code doesn't hurt (as soon as we don't depend on them to build the
final code).

Cheers,
Mario

-- 
pgp key: http://subkeys.pgp.net/ PGP Key ID: 80F240CF
Fingerprint: BA39 9666 94EC 8B73 27FA  FC7C 4086 63E3 80F2 40CF

IcedRobot: www.icedrobot.org
Proud GNU Classpath developer: http://www.classpath.org/
Read About us at: http://planet.classpath.org
OpenJDK: http://openjdk.java.net/projects/caciocavallo/

Please, support open standards:
http://endsoftpatents.org/