Refactoring JavaFX Builds & Sources
Sven Reimers
sven.reimers at gmail.com
Fri Feb 1 09:25:30 PST 2013
Sound great!
-Sven
On Fri, Feb 1, 2013 at 6:07 PM, Richard Bair <richard.bair at oracle.com>wrote:
> I just pushed to the rt repo my two "work in progress" gradle scripts,
> build.gradle and settings.gradle. You will find that these don't work. The
> reason is that I have another (still closed) "generator" gradle script
> which takes the existing sources and munges them around into a directory
> structure that the new gradle scripts work against.
>
> I'm working with the full set of sources and trying to get the new build
> to work with the full set of sources. I'm targeting the end of the month
> (Feb) for having the build system sorted out.
>
> For example my "generator" script looks like:
>
> task generate << {
> // Copy over the new gradle build file
> copy {
> from "."
> into "javafx"
> include "build.gradle", "settings.gradle"
> }
>
> // Create the build-tools project
> copy {
> from "rt/javafx-annotation-processor/src",
> "rt/decora-compiler/src"
> into "javafx/build-tools/src/main/java"
> exclude "META-INF/**", "**/*.properties", "**/*.stg", "**/JSL.g"
> }
> copy {
> from "rt/javafx-annotation-processor/src",
> "rt/decora-compiler/src"
> into "javafx/build-tools/src/main/resources"
> include "META-INF/**", "**/*.properties", "**/*.stg"
> }
> copy {
> from
> "rt/decora-compiler/src/com/sun/scenario/effect/compiler/JSL.g"
> into "javafx/build-tools/src/main/antlr"
> }
> copy {
> from "rt-closed/javafx-beans/src"
> into "javafx/build-tools/src/main/java"
> include "com/sun/javafx/beans/annotations/**",
> "com/sun/javafx/collections/annotations/**"
> }
>
> …. and lots more
> }
>
> You can see some paths are "rt/" and some are "rt-closed/". What I can do
> is create a generator.gradle file in rt that contains only the portion that
> is already open so that people in the community can try out the new
> structure / contribute / etc? The main gradle build file then needs to just
> include jfxrt.jar on the build path as the binary plug.
>
> Thoughts?
> Richard
>
> On Oct 19, 2012, at 7:24 AM, Danno Ferrin <danno.ferrin at shemnon.com>
> wrote:
>
> >
> >
> > On Fri, Oct 19, 2012 at 7:31 AM, Mario Torre <
> neugens.limasoftware at gmail.com> wrote:
> > > Further more, you should be able to do "find usages" in your IDE and
> get EVERYTHING IN THE UNIVERSE, rather than right now where in NB it
> doesn't have enough information about the 160 other projects in our system
> to be able to do this. This is bad.
> > >
> > > Right now we build with ant. Many of you have suggested Maven. Some of
> you at JavaOne suggested Gradle. I am presently looking into Gradle. The
> key things I like about Gradle are:
> > >
> > > 1) It isn't XML
> > > 2) Like Ivy & Maven, it handles dependencies well
> > >
> > > We presently have a pile of XML files and nasty build logic for
> downloading dependencies (this is all in the closed repo, but you would be
> exposed to the horror if we just made it all public). I wanted instead to
> use Ivy at the very least, but rather if I can get better build happiness
> from another setup entirely (like Gradle) then I might as well give it a
> try.
> >
> > I kind of like Gradle, but let me put the cart before the horse here
> > (I don't know if this sounds as well in English as it is in Italian,
> > where we actually use the oxen, but well :)
> >
> > While Maven is a well understood tool, Gradle is fairly new. This
> > turns to be especially problematic when building and shipping on Linux
> > distributions because there are some constraints and limitation on
> > what can go or not in a given release (btw, I think this could be
> > considered a problem on Windows and OSX installations in some
> > environments where you can't simply install any tool you want).
> >
> > Maven, on the other end, is pretty likely sitting on your machine
> > already (as are Make, GCC and Configure most of the time on developers
> > machines), so I suspect it will be easier for the average folk as well
> > as heavily filled packager to take care of the build bits.
> >
> > Gradle has a "wrapper" script that can be checked in with the build (the
> gradlew script), it will download the specific version of gradle and build
> it in place, from the build. So having Gradle on your machine is not a
> requirement, the gradle script will build with it in place like it was an
> external library.
> >
> > Plus, Android has just shifted to building user projects with Gradle
> instead of Ant. So there will be many Linux distros that will ship and
> approve Gradle just for that reason.
> >
> >
> > Oh my, I would have never thought I would try to sell Maven myself! Of
> > course, don't make this stop evolutionary attempts at solving the
> > build process, but I thought it's one consideration to take into
> > account when deciding what build system to use, since it's such a
> > fundamental part of the code.
> >
> > Ultimately, the build infrastructure should be what makes more sense
> > for the project obviously.
> >
> > > 1) I'll make sure cross builds all continue to work
> > > 2) Partial builds (for native code particularly) will continue
> to work such that you don't HAVE to build native code
> > > 3) It will produce bit-for-bit compatible output in the
> artifacts/ directory, so all our RE (Release Engineering) scripts continue
> to work with little or no modification
> > >
> > > Also, in my tests so far the speed of the build is dramatically faster
> -- but then I don't have everything in yet so that might just be wishful
> thinking. I hope and expect it to be faster though.
> > >
> > > We generate code in our build system for the following reasons:
> > >
> > > 1) We have a VersionInfo class which the build system will
> update with version information. It is a com.sun.something class
> > > 2) We have an annotation processor which generates all of the
> builders
> > > 3) We have two grammars that Antlr has process to produce
> parsers -- one for JSL (for our shaders) and one for CSS
> > > 4) We have a "decora compiler" which takes JSL files and feeds
> it through the JSL parser and produces code / shaders
> > >
> > > In addition to code generation, we also build a ton of native code
> (gstreamer, webkit, prism, fonts, glass, image loading). So that has to fit
> into the build system. Building native code is SLOW, so being able to avoid
> it for "normal" developers, and being able to avoid native builds when
> nothing changed in the native code, is important.
> >
> > Speaking about this, I wonder if we will end up doing like IcedTea
> > with OpenJDK, since most of those probably have an OS counterpart we
> > would rather want to use (in other words, make those elements somehow
> > pluggable).
> >
> > > I have found that some of our unit tests are written such that they
> require more of the platform, rather than just one class. So for example,
> there are tests in the scene graph which might want to use a UI control for
> some purpose. I'm thinking the easiest thing to do is to have a test
> directory at the top level which houses all the tests, and break them up
> based on whether they are part of the smoke test, integration test, or
> manual test suite. Smoke tests are those that are run continuously. They
> must be headless, and they must be completely automated, and they must
> execute within some reasonable amount of time (since hudson is going to be
> doing it continuously). The integration tests are the rest of the automated
> tests that need to be run between every integration with "master". Finally
> the manual tests are not run as part of the integration / smoke tests, but
> really are just little "toys" or apps we've written that are useful when
> developing a feature or whatnot. We've got a ton of these little guys
> (mostly of the "HelloWorld" variety), and these would go into "manual".
> >
> > This is one advantage of maven, in that you are somehow forced to have
> > this layout. Maven supports the idea of modules, they are a hierarchy
> > that pretty much resembles what the project structure you depict later
> > looks, and for each of those modules you would have unit tests.
> > Integration tests could simply go into an high level module, the same
> > for the manual tests.
> >
> > The problem with Maven is when you get off the beaten path (like running
> your own annotation processor, shipping source code in the jar, custom
> build languages) the build script gets very verbose and hard to read. All
> the XML creates more noise. Yes, you can do it, but it becomes difficult
> to maintain. Build scripts for Maven projects that follow the maven
> conventions are dead simple, but OpenJFX is not one of those run of the
> mill builds.
> >
> >
> >
> > > We also will have an apps directory, where we will place all the
> samples (like Ensemble) and experiments (like the JavaOne Schedule Builder).
> > >
> > > And I want to have a "benchmarks" directory where all of our
> benchmarks will live. The only problem with this one is that we have a
> dependency on JRockit benchmark harness, and I'm not sure how to resolve
> that with the open source bits. Maybe the JRockit benchmark harness is
> already open source, I haven't dug into it yet to find out. However I
> believe we *must* have benchmarks open, as well as the SQE tests. My goal
> is to run OpenJFX in the open and to have meaningful contributions -- how
> can that happen if we're keeping back essential verification tools? How can
> somebody submit a patch and not know whether it is going to impact
> performance? So I want to get the benchmarks all out, but we need to figure
> out what to do about this dependency. Another option is to replace this
> dependency with one of the other open source benchmark harnesses, but this
> would need to be carefully considered as it would disrupt our benchmark
> reporting for some period of time. But ultimately having public numbers and
> public tests would, I think, be very beneficial across the board.
> >
> > As a side note, we're developing a monitoring tool for OpenJDK
> > (Thermostat). I don't want to put Thermostat in competition with
> > JRockit (well, I did so already!), but by the time the full code is
> > out, we will be quite likely ready for prime time. There are also
> > other tools that do pretty good job, like VisualVM and JConsole.
> > Depending on the harness you have (which I assume means loads of
> > binary files with dumps and performance related data), we can either
> > try to make them compatible with all those tools, or better yet,
> > develop some kind exchange format for Performance Analysis (so that we
> > are not really depending on any tool). This can be a cool
> > collaboration between some other teams (I suspect such an effort would
> > go beyond JavaFX of course). In any case, having the harness in the
> > code doesn't hurt (as soon as we don't depend on them to build the
> > final code).
> >
> > Cheers,
> > Mario
> >
> > --
> > pgp key: http://subkeys.pgp.net/ PGP Key ID: 80F240CF
> > Fingerprint: BA39 9666 94EC 8B73 27FA FC7C 4086 63E3 80F2 40CF
> >
> > IcedRobot: www.icedrobot.org
> > Proud GNU Classpath developer: http://www.classpath.org/
> > Read About us at: http://planet.classpath.org
> > OpenJDK: http://openjdk.java.net/projects/caciocavallo/
> >
> > Please, support open standards:
> > http://endsoftpatents.org/
> >
> >
> >
> > --
> > There is nothing that will hold me back. I know who I am....
> > I remember wher I came from, and I feel stronger for knowing.
> > Zane, Ninja of Ice. Ninjago S01E07
> >
>
>
--
Sven Reimers
* Senior Expert Software Architect
* NetBeans Dream Team Member: http://dreamteam.netbeans.org
* NetBeans Governance Board Member:
http://netbeans.org/about/os/governance.html
* Community Leader NetBeans: http://community.java.net/netbeans
Desktop Java:
http://community.java.net/javadesktop
* Duke's Choice Award Winner 2009
* Blog: http://nbguru.blogspot.com
* XING: https://www.xing.com/profile/Sven_Reimers8
* LinkedIn: http://www.linkedin.com/in/svenreimers
Join the NetBeans Groups:
* XING: http://www.xing.com/group-20148.82db20
* NUGM: http://haug-server.dyndns.org/display/NUGM/Home
* LinkedIn: http://www.linkedin.com/groups?gid=1860468
http://www.linkedin.com/groups?gid=107402
http://www.linkedin.com/groups?gid=1684717
* Oracle: https://mix.oracle.com/groups/18497
More information about the openjfx-dev
mailing list