Re: RFR: JDK-8147795 Build system support for BSD
On 2016-01-20 13:06, David Holmes wrote:
On 20/01/2016 9:15 PM, Alan Bateman wrote:
On 20/01/2016 10:54, Magnus Ihse Bursie wrote:
During my spare time last autumn, and in the holidays, I've been playing around with the three major BSDs (FreeBSD, OpenBSD and NetBSD), trying to learn something new. (Yeah, I know, this proves that I have no life :-)).
And what better way to learn an operating system than to try and build OpenJDK on it? :-) It quickly turned out that while there is a bsd-port/jdk9 forest, it is empty (that is, it's identical to jdk9/jdk9 with no BSD-specific patches in it). And building jdk8 is sooo 2014. :)
So I started hacking around, focusing on improving issues in the build system that prevented the build to succeed. I also needed to fix issues in the source code (of course), but not as much as I'd expected. My total solution builds and runs (I've tested "javac HelloWorld.java") on the three BSDs, but some workarounds are needed, mostly likely due to incomplete fixes in the source code.
The build changes turned out to also be an improvement for all platforms in some areas, and I'd like to integrate it into the mainline. While it is not enough in itself to build on BSD, it's a (necessary) step on the way. I'll post a second review later on for my source code changes, which still need some more cleanup to be presentable.
Bug: https://bugs.openjdk.java.net/browse/JDK-8147795 WebRev: http://cr.openjdk.java.net/~ihse/JDK-8147795-build-system-support-for-bsd/we...
Clearly some good work here but who is going to maintain this? For the AIX port then the SAP engineers maintain the port and keeping it working, it's not clear what will happen here. Also support for new ports feels like it should have a JEP.
My thoughts exactly. I was under the impression that what we had for OSX already worked on some of the BSD's as we have had some patches in the past from people building on that platform. But an attempt to produce complete support for the various BSD's needs to come with some level of support and commitment.
I think I need to elaborate a bit on my thinking here. First of all, I agree that adding full support for a complete new port would, at the very least, require a JEP. This is not what I intended with this change. A reflection, though: If the requirement for such a port is that a company provides continuous testing and support, then I believe it's unlikely that any BSD port will ever reach the mainline, due to the community based nature of the BSD projects. (I'll leave aside the question if that is a reasonable requirement.) We already have the BSD Port Project [1], sponsored by the Porters Group [2]. They maintain their own forests. For jdk8, the forest contains a number of patches for BSD. These have not been included upstream, for reasons I can only speculate in. For jdk9, as I wrote in my mail, the forest has been created but no BSD-specific patches has been submitted. From my cursory inspection, the source code changes in bsd-port/jdk8 could probably be moved to bsd-port/jdk9 with considerable less effort than the makefile changes. It is my general impression that developers working with porting is good at fixing C/C++ changes that relate to the target platform of their port, but considerably less so in working with the makefile changes that might be needed. (That's not just about porting; developers in general are often more fluent and comfortable working in C/C++ than in makefiles or shell scripts.) My idea was to provide a sane basis in the build system, on top of which it is easier to create a full port. Note that I'm not *introducing* BSD support in the OpenJDK build system. We already have that, but only partially. I'm just filling in the holes. Most of the changes in my patch relate to the configure script. The configure script is by it's very nature adapted to running in various environments, officially supported as well as other, common as well as rare. Much of the peculiarities of the BSDs from the configure scripts point of view, is not very different from e.g. the recent request to support freetype on OS X El Capitan. I assumed that a bening change to the build system that does not affect any existing platforms, and which provides a good foundation for bringing the bsd-port to jdk9, would be a bit of a no-brainer to integrate. Even if it would break later on due to lack of continous testing, it's often a task of an order of magnitute simpler to fix a previously working code (even in Makefiles, and even to developers that are not fluent in make), than to add the initial code. I've cc:ed the bsd-port mailing list in this discussion. Maybe someone from that group has anything to add? /Magnus [1] http://openjdk.java.net/projects/bsd-port/ [2] http://openjdk.java.net/groups/porters/
David -----
Maybe I need to wait for the "source code webrev" but I could imagine some refactoring needed to avoid duplication with OS X specific code. Also when the Mac port was brought into OpenJDK then it came with a lot of #ifdef _ALLBSD_SOURCE patches and a lot of inconsistencies. General PITA when trying to move OS X specific patches forward and I know of several areas where the BSD specific code was just dropped.
-Alan.
On 20.01.2016 22:17, Magnus Ihse Bursie wrote:
A reflection, though: If the requirement for such a port is that a company provides continuous testing and support, then I believe it's unlikely that any BSD port will ever reach the mainline, due to the community based nature of the BSD projects.
Community supported ports have been merged into mainline in the past when they have passed the TCK with the expectation that they are kept in shape by the corresponding (sub)community in OpenJDK, and if they aren't, that they'd get dropped out of mainline again. A JEP would be a second item to look at, of course, now that we have a JEP process in place. Typically, they'd go through the JDK Release Project in development first (i.e. JDK 9 now), and then potentially get backported to an Update release Project. With respect to the BSD Port, the FreeBSD Foundation is listed here: http://openjdk.java.net/groups/conformance/JckAccess/jck-access.html but I'm not sure if they have completed their respective efforts yet.
We already have the BSD Port Project [1], sponsored by the Porters Group [2]. They maintain their own forests. For jdk8, the forest contains a number of patches for BSD. These have not been included upstream, for reasons I can only speculate in.
I think it unfortunately came down to lack of man power among BSD Port developers at the time when the dust after the OS X Port's integration into JDK 8 settled. [1] Making it easier for the BSD port to integrate JDK 9 changes sounds fine to me, but I'm not a JDK 9/build area Reviewer. cheers, dalibor topic [1] http://mail.openjdk.java.net/pipermail/bsd-port-dev/2014-April/002245.html -- <http://www.oracle.com> Dalibor Topic | Principal Product Manager Phone: +494089091214 <tel:+494089091214> | Mobile: +491737185961 <tel:+491737185961> ORACLE Deutschland B.V. & Co. KG | Kühnehöfe 5 | 22761 Hamburg ORACLE Deutschland B.V. & Co. KG Hauptverwaltung: Riesstr. 25, D-80992 München Registergericht: Amtsgericht München, HRA 95603 Komplementärin: ORACLE Deutschland Verwaltung B.V. Hertogswetering 163/167, 3543 AS Utrecht, Niederlande Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 Geschäftsführer: Alexander van der Ven, Astrid Kepper, Val Maher <http://www.oracle.com/commitment> Oracle is committed to developing practices and products that help protect the environment
Hi Magnus, I am just an observer here but good work. I have been on this list for a long time because of my interest in MacOSX and that port so I naturally thought that BSD would become part of mainline Java too. More platforms for Java is better. Because of your work I took a look at BSD and the history along with the birth of GNU Linux. I really didn't know what a great server platform BSD is or how much it is used. I always see Greg Lewis pulling changes into BSD and wondered why it wasn't part of the main Java repo. It appears that people must use Java but build from source. Not sure if that is just standard procedure on FreeBSD for instance. https://www.freebsd.org/java/ I guess it would only take one variant of BSD to pass the TCK - not sure if BSD was ever in the mainline. Anyway, hope you manage to get your changes accepted and you can make more progress. Eric On Thu, Jan 21, 2016 at 4:44 AM, dalibor topic <dalibor.topic@oracle.com> wrote:
On 20.01.2016 22:17, Magnus Ihse Bursie wrote:
A reflection, though: If the requirement for such a port is that a company provides continuous testing and support, then I believe it's unlikely that any BSD port will ever reach the mainline, due to the community based nature of the BSD projects.
Community supported ports have been merged into mainline in the past when they have passed the TCK with the expectation that they are kept in shape by the corresponding (sub)community in OpenJDK, and if they aren't, that they'd get dropped out of mainline again.
A JEP would be a second item to look at, of course, now that we have a JEP process in place.
Typically, they'd go through the JDK Release Project in development first (i.e. JDK 9 now), and then potentially get backported to an Update release Project.
With respect to the BSD Port, the FreeBSD Foundation is listed here: http://openjdk.java.net/groups/conformance/JckAccess/jck-access.html but I'm not sure if they have completed their respective efforts yet.
We already have the BSD Port Project [1], sponsored by the Porters Group
[2]. They maintain their own forests. For jdk8, the forest contains a number of patches for BSD. These have not been included upstream, for reasons I can only speculate in.
I think it unfortunately came down to lack of man power among BSD Port developers at the time when the dust after the OS X Port's integration into JDK 8 settled. [1]
Making it easier for the BSD port to integrate JDK 9 changes sounds fine to me, but I'm not a JDK 9/build area Reviewer.
cheers, dalibor topic
[1] http://mail.openjdk.java.net/pipermail/bsd-port-dev/2014-April/002245.html -- <http://www.oracle.com> Dalibor Topic | Principal Product Manager Phone: +494089091214 <tel:+494089091214> | Mobile: +491737185961 <tel:+491737185961>
ORACLE Deutschland B.V. & Co. KG | Kühnehöfe 5 | 22761 Hamburg
ORACLE Deutschland B.V. & Co. KG Hauptverwaltung: Riesstr. 25, D-80992 München Registergericht: Amtsgericht München, HRA 95603
Komplementärin: ORACLE Deutschland Verwaltung B.V. Hertogswetering 163/167, 3543 AS Utrecht, Niederlande Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 Geschäftsführer: Alexander van der Ven, Astrid Kepper, Val Maher
<http://www.oracle.com/commitment> Oracle is committed to developing practices and products that help protect the environment
I've replied to Magnus separately on the freebsd-java mailing list. This is great and I'd love to see it in either the jdk9 mainline or the bsd-port repo of jdk9. The big barrier has always been the volunteer effort required to get past the TCK and fix whatever relatively non-mainstream bugs that are lurking for BSD. I'm aware of people running openjdk8, for instance, for production purposes on FreeBSD and the OpenBSD and NetBSD likewise seem to be kept relatively up to date (or were last time I looked). I haven't had time to work on the full jdk9 port but have been keeping the repo up to date for when time became available or others were able to contribute. It is nice to see the latter starting to happen. - Greg On Thu, Jan 21, 2016 at 11:41:43AM -0800, Eric Richardson wrote:
Hi Magnus,
I am just an observer here but good work. I have been on this list for a long time because of my interest in MacOSX and that port so I naturally thought that BSD would become part of mainline Java too. More platforms for Java is better. Because of your work I took a look at BSD and the history along with the birth of GNU Linux. I really didn't know what a great server platform BSD is or how much it is used.
I always see Greg Lewis pulling changes into BSD and wondered why it wasn't part of the main Java repo. It appears that people must use Java but build from source. Not sure if that is just standard procedure on FreeBSD for instance. https://www.freebsd.org/java/ I guess it would only take one variant of BSD to pass the TCK - not sure if BSD was ever in the mainline.
Anyway, hope you manage to get your changes accepted and you can make more progress. Eric
On Thu, Jan 21, 2016 at 4:44 AM, dalibor topic <dalibor.topic@oracle.com> wrote:
On 20.01.2016 22:17, Magnus Ihse Bursie wrote:
A reflection, though: If the requirement for such a port is that a company provides continuous testing and support, then I believe it's unlikely that any BSD port will ever reach the mainline, due to the community based nature of the BSD projects.
Community supported ports have been merged into mainline in the past when they have passed the TCK with the expectation that they are kept in shape by the corresponding (sub)community in OpenJDK, and if they aren't, that they'd get dropped out of mainline again.
A JEP would be a second item to look at, of course, now that we have a JEP process in place.
Typically, they'd go through the JDK Release Project in development first (i.e. JDK 9 now), and then potentially get backported to an Update release Project.
With respect to the BSD Port, the FreeBSD Foundation is listed here: http://openjdk.java.net/groups/conformance/JckAccess/jck-access.html but I'm not sure if they have completed their respective efforts yet.
We already have the BSD Port Project [1], sponsored by the Porters Group
[2]. They maintain their own forests. For jdk8, the forest contains a number of patches for BSD. These have not been included upstream, for reasons I can only speculate in.
I think it unfortunately came down to lack of man power among BSD Port developers at the time when the dust after the OS X Port's integration into JDK 8 settled. [1]
Making it easier for the BSD port to integrate JDK 9 changes sounds fine to me, but I'm not a JDK 9/build area Reviewer.
cheers, dalibor topic
[1] http://mail.openjdk.java.net/pipermail/bsd-port-dev/2014-April/002245.html -- <http://www.oracle.com> Dalibor Topic | Principal Product Manager Phone: +494089091214 <tel:+494089091214> | Mobile: +491737185961 <tel:+491737185961>
ORACLE Deutschland B.V. & Co. KG | K??hneh??fe 5 | 22761 Hamburg
ORACLE Deutschland B.V. & Co. KG Hauptverwaltung: Riesstr. 25, D-80992 M??nchen Registergericht: Amtsgericht M??nchen, HRA 95603
Komplement??rin: ORACLE Deutschland Verwaltung B.V. Hertogswetering 163/167, 3543 AS Utrecht, Niederlande Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 Gesch??ftsf??hrer: Alexander van der Ven, Astrid Kepper, Val Maher
<http://www.oracle.com/commitment> Oracle is committed to developing practices and products that help protect the environment
-- Greg Lewis Email : glewis@eyesbeyond.com Eyes Beyond Web : http://www.eyesbeyond.com Information Technology FreeBSD : glewis@FreeBSD.org
Thus wrote Greg Lewis (glewis@eyesbeyond.com):
[...] I'm aware of people running openjdk8, for instance, for production purposes on FreeBSD and the OpenBSD and NetBSD likewise seem to be kept relatively up to date (or were last time I looked).
blog.NetBSD.org is Apache Roller on openjdk8, on NetBSD of course. (Thanks all involved :)) regards, spz
participants (5)
-
dalibor topic
-
Eric Richardson
-
Greg Lewis
-
Magnus Ihse Bursie
-
S.P.Zeidler