Building frustration

[Stefan Marr]

Hi Raffaella:

> On 24 Dec 2015, at 19:08, Raffaello Giulietti <raffaello.giulietti at supsi.ch> wrote:
> 
> There are two kind of Truffle/Graal users:
> 
> (1) Those who like or need to be in control of every aspect of Truffle
> and Graal. They have come to master configurations, builds, flags,
> environment variables and can tweak with every detail of the product.
> They know Mercurial, mx, the C++ compilers and all sort of other great
> tools.
> 
> (2) And those who would like to use Truffle/Graal to experiment with
> language implementations. They tend to avoid diving into low level
> aspects of building Truffle and Graal, not because of lack of interest
> but because of the limited time they can devote in fighting against
> unset flags, missing libraries, obsolete documentation, undocumented
> dependencies and so on.

Generally, I absolutely agree with your concerns and had the same issues in the beginning.

But at least for Linux, I’d think that the situation improved quite a bit since the requirement of having two JDKs installed was dropped.

Based on my last write up [1], I’d think that on a Ubuntu system you only need the build-essential  package (`sudo apt-get install build-essential`) and a recent JDK 8 [2].
I don’t have any experience with other Linux system, but I’d assume it is similar.

With that foundation, mx should take care of the rest. And it is also easily automatized. I am using for instance the script here [3] to keep the build server updated and managed via git (it uses hg only internally) 


> In the past I've tried on Windows, with a low rate of success. Hence, I
> decided to switch to Linux (a couple of distributions, currently on
> Oracle Linux 7.2). I often stumble against similar problems.

What are your concrete problems, could you provide more details? 

> I conclude that building Truffle/Graal from source is currently simply
> not robust enough because of the many moving parts: the OSes, the
> standard tools, the standard libraries, the user environment, the
> documentation, the dependencies, etc.

Without more details, I am not sure what could be done.
Actually, I’d find the list of instructions on the Graal wiki already pretty minimal [4].
And I am not aware of any ‘unusual’ software requirements/library dependencies. 

> * Much more frequently updated binaries for various platforms. I
> understand this means more work for Oracle Labs and/or JKU.

Hm, we got some internal infrastructure with nightly builds. But I don’t think that covers your request with ‘various platforms’.

My personal approach would be to make building from source as reliable, robust, and easy as possible. Especially for researchers. Because only then you really know what’s in the system.
But if there are issues with it, I think, we could really use more details.

> * Or a robust build system for dummies like me, interested in making
> good use of the product but unwilling to spend a long time in
> understanding obscure error messages on failed builds to discover that
> some undocumented and unknown environment variable was set incorrectly.

Yeah, well, we need bug reports for that :)
Please, send a mail to the list as soon as you hit a stumbling block! In many cases, we might be able to do something in the build system (mx) or at least give guidance.

> * How many of you are experiencing similar frustrating build problems?

In the beginning, yes. But since we only need a single JDK, I didn’t have problems.
Also, we fixed mx to set standard settings for problematic platforms like OS X, which wasn’t automatic in the beginning. Since then, I didn’t have issues I think.

> * How many of you have invested or still invest hours adjusting and
> tuning your working environment so as to make something like "mx spull ;
> mx build” work without fuss?

In very very rare cases there were issues with mx and project configurations in the past (if I remember correctly). Don’t remember any concrete blocking issues thought.

> * What can we do in practical terms to document the many tweaks one must
> be aware of to turn frustration into joy?

Well :) I am not aware of any, but they should all go on [4].
If you got something that is missing, please report it.


> Ideally, we should strive to a build system that can check the user
> environment and suggest everything s/he shall undertake preemptively to
> make a build succeed without troubles.

At least from my experience, mx already does a pretty good job with that.
But please report missing things :)

Best regards
Stefan


[1] http://stefan-marr.de/papers/oopsla-marr-ducasse-meta-tracing-vs-partial-evaluation-artifacts/#user-content-32-software-dependencies
[2] http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html
[3] https://github.com/smarr/GraalBasic/blob/master/checkout.sh
[4] https://wiki.openjdk.java.net/display/Graal/Instructions



-- 
Stefan Marr
Johannes Kepler Universität Linz
http://stefan-marr.de/research/