Refactoring JavaFX Builds & Sources

Fri Oct 19 07:24:02 PDT 2012

On Fri, Oct 19, 2012 at 7:31 AM, Mario Torre <neugens.limasoftware at gmail.com
> wrote:

> > Further more, you should be able to do "find usages" in your IDE and get
> EVERYTHING IN THE UNIVERSE, rather than right now where in NB it doesn't
> have enough information about the 160 other projects in our system to be
> able to do this. This is bad.
> >
> > Right now we build with ant. Many of you have suggested Maven. Some of
> you at JavaOne suggested Gradle. I am presently looking into Gradle. The
> key things I like about Gradle are:
> >
> >         1) It isn't XML
> >         2) Like Ivy & Maven, it handles dependencies well
> >
> > We presently have a pile of XML files and nasty build logic for
> downloading dependencies (this is all in the closed repo, but you would be
> exposed to the horror if we just made it all public). I wanted instead to
> use Ivy at the very least, but rather if I can get better build happiness
> from another setup entirely (like Gradle) then I might as well give it a
> try.
>
> I kind of like Gradle, but let me put the cart before the horse here
> (I don't know if this sounds as well in English as it is in Italian,
> where we actually use the oxen, but well :)
>
> While Maven is a well understood tool, Gradle is fairly new. This
> turns to be especially problematic when building and shipping on Linux
> distributions because there are some constraints and limitation on
> what can go or not in a given release (btw, I think this could be
> considered a problem on Windows and OSX installations in some
> environments where you can't simply install any tool you want).
>
> Maven, on the other end, is pretty likely sitting on your machine
> already (as are Make, GCC and Configure most of the time on developers
> machines), so I suspect it will be easier for the average folk as well
> as heavily filled packager to take care of the build bits.
>

Gradle has a "wrapper" script that can be checked in with the build (the
gradlew script), it will download the specific version of gradle and build
it in place, from the build. So having Gradle on your machine is not a
requirement, the gradle script will build with it in place like it was an
external library.

Plus, Android has just shifted to building user projects with Gradle
instead of Ant.  So there will be many Linux distros that will ship and
approve Gradle just for that reason.

>
> Oh my, I would have never thought I would try to sell Maven myself! Of
> course, don't make this stop evolutionary attempts at solving the
> build process, but I thought it's one consideration to take into
> account when deciding what build system to use, since it's such a
> fundamental part of the code.
>
> Ultimately, the build infrastructure should be what makes more sense
> for the project obviously.
>
> >         1) I'll make sure cross builds all continue to work
> >         2) Partial builds (for native code particularly) will continue
> to work such that you don't HAVE to build native code
> >         3) It will produce bit-for-bit compatible output in the
> artifacts/ directory, so all our RE (Release Engineering) scripts continue
> to work with little or no modification
> >
> > Also, in my tests so far the speed of the build is dramatically faster
> -- but then I don't have everything in yet so that might just be wishful
> thinking. I hope and expect it to be faster though.
> >
> > We generate code in our build system for the following reasons:
> >
> >         1) We have a VersionInfo class which the build system will
> update with version information. It is a com.sun.something class
> >         2) We have an annotation processor which generates all of the
> builders
> >         3) We have two grammars that Antlr has process to produce
> parsers -- one for JSL (for our shaders) and one for CSS
> >         4) We have a "decora compiler" which takes JSL files and feeds
> it through the JSL parser and produces code / shaders
> >
> > In addition to code generation, we also build a ton of native code
> (gstreamer, webkit, prism, fonts, glass, image loading). So that has to fit
> into the build system. Building native code is SLOW, so being able to avoid
> it for "normal" developers, and being able to avoid native builds when
> nothing changed in the native code, is important.
>
> Speaking about this, I wonder if we will end up doing like IcedTea
> with OpenJDK, since most of those probably have an OS counterpart we
> would rather want to use (in other words, make those elements somehow
> pluggable).
>
> > I have found that some of our unit tests are written such that they
> require more of the platform, rather than just one class. So for example,
> there are tests in the scene graph which might want to use a UI control for
> some purpose. I'm thinking the easiest thing to do is to have a test
> directory at the top level which houses all the tests, and break them up
> based on whether they are part of the smoke test, integration test, or
> manual test suite. Smoke tests are those that are run continuously. They
> must be headless, and they must be completely automated, and they must
> execute within some reasonable amount of time (since hudson is going to be
> doing it continuously). The integration tests are the rest of the automated
> tests that need to be run between every integration with "master". Finally
> the manual tests are not run as part of the integration / smoke tests, but
> really are just little "toys" or apps we've written that are useful when
> developing a feature or whatnot. We've got a ton of these little guys
> (mostly of the "HelloWorld" variety), and these would go into "manual".
>
> This is one advantage of maven, in that you are somehow forced to have
> this layout. Maven supports the idea of modules, they are a hierarchy
> that pretty much resembles what the project structure you depict later
> looks, and for each of those modules you would have unit tests.
> Integration tests could simply go into an high level module, the same
> for the manual tests.
>

The problem with Maven is when you get off the beaten path (like running
your own annotation processor, shipping source code in the jar, custom
build languages) the build script gets very verbose and hard to read.  All
the XML creates more noise.  Yes, you can do it, but it becomes difficult
to maintain.  Build scripts for Maven projects that follow the maven
conventions are dead simple, but OpenJFX is not one of those run of the
mill builds.

>
> > We also will have an apps directory, where we will place all the samples
> (like Ensemble) and experiments (like the JavaOne Schedule Builder).
> >
> > And I want to have a "benchmarks" directory where all of our benchmarks
> will live. The only problem with this one is that we have a dependency on
> JRockit benchmark harness, and I'm not sure how to resolve that with the
> open source bits. Maybe the JRockit benchmark harness is already open
> source, I haven't dug into it yet to find out. However I believe we *must*
> have benchmarks open, as well as the SQE tests. My goal is to run OpenJFX
> in the open and to have meaningful contributions -- how can that happen if
> we're keeping back essential verification tools? How can somebody submit a
> patch and not know whether it is going to impact performance? So I want to
> get the benchmarks all out, but we need to figure out what to do about this
> dependency. Another option is to replace this dependency with one of the
> other open source benchmark harnesses, but this would need to be carefully
> considered as it would disrupt our benchmark reporting for some period of
> time. But ultimately having public numbers and public tests would, I think,
> be very beneficial across the board.
>
> As a side note, we're developing a monitoring tool for OpenJDK
> (Thermostat). I don't want to put Thermostat in competition with
> JRockit (well, I did so already!), but by the time the full code is
> out, we will be quite likely ready for prime time. There are also
> other tools that do pretty good job, like VisualVM and JConsole.
> Depending on the harness you have (which I assume means loads of
> binary files with dumps and performance related data), we can either
> try to make them compatible with all those tools, or better yet,
> develop some kind exchange format for Performance Analysis (so that we
> are not really depending on any tool). This can be a cool
> collaboration between some other teams (I suspect such an effort would
> go beyond JavaFX of course). In any case, having the harness in the
> code doesn't hurt (as soon as we don't depend on them to build the
> final code).
>
> Cheers,
> Mario
>
> --
> pgp key: http://subkeys.pgp.net/ PGP Key ID: 80F240CF
> Fingerprint: BA39 9666 94EC 8B73 27FA  FC7C 4086 63E3 80F2 40CF
>
> IcedRobot: www.icedrobot.org
> Proud GNU Classpath developer: http://www.classpath.org/
> Read About us at: http://planet.classpath.org
> OpenJDK: http://openjdk.java.net/projects/caciocavallo/
>
> Please, support open standards:
> http://endsoftpatents.org/
>

-- 
There is nothing that will hold me back.  I know who I am....
I remember wher I came from, and I feel stronger for knowing.
Zane, Ninja of Ice.  Ninjago S01E07