Odd problem building current 8u snapshot

Tue Jun 25 16:39:41 UTC 2019

Hi,

While trying to find a strange problem with Shenandoah-enabled builds
by Aleksey Shipilev (now resolved :) I ended up trying to build a current
8u snapshot in order to reproduce/verify the original problem.
During this I encountered a rather unexpected problem with the build
system, and Aleksey recommended to post here - maybe one of the 8u-Andrews
knows what's going on.

All this is using the shenandoah-jdk8.tar.xz tarball from
https://builds.shipilev.net/source-snapshots/, and yes, it also
happens with the vanilla 8u tarball (which seems good, since it means
the problem is generic).

Apologies for the verbosity but IMHO it's useful to see everything as
it happened. The only thing I do here is acommodate gcc9 and disable
the -Werror setting, otherwise it's all defaults and plain Linux,
with IcedTea-3-12 for bootstrap:

---snip---
holger>unset JAVAC
holger>unset JAVA_HOME
holger>tar xf shenandoah-jdk8.tar.xz
holger>cd shenandoah-jdk8
holger>sh configure --with-extra-cflags=-Wno-error
Running generated-configure.sh
configure: Configuration created at Tue Jun 25 10:06:20 CEST 2019.
configure: configure script generated at timestamp 1556533111.
checking for basename... /usr/bin/basename
checking for bash... /bin/bash
checking for cat... /bin/cat
checking for chmod... /bin/chmod
checking for cmp... /usr/bin/cmp
checking for comm... /usr/bin/comm
checking for cp... /bin/cp
checking for cut... /usr/bin/cut
checking for date... /bin/date
checking for gdiff... no
checking for diff... /usr/bin/diff
checking for dirname... /usr/bin/dirname
checking for echo... /bin/echo
checking for expr... /usr/bin/expr
checking for file... /usr/bin/file
checking for find... /usr/bin/find
checking for head... /usr/bin/head
checking for ln... /bin/ln
checking for ls... /bin/ls
checking for mkdir... /bin/mkdir
checking for mktemp... /usr/bin/mktemp
checking for mv... /bin/mv
checking for nawk... no
checking for gawk... /usr/bin/gawk
checking for printf... /usr/bin/printf
checking for rm... /bin/rm
checking for sh... /bin/sh
checking for sort... /usr/bin/sort
checking for tail... /usr/bin/tail
checking for tar... /bin/tar
checking for tee... /usr/bin/tee
checking for touch... /usr/bin/touch
checking for tr... /usr/bin/tr
checking for uname... /usr/bin/uname
checking for uniq... /usr/bin/uniq
checking for wc... /usr/bin/wc
checking for which... /usr/bin/which
checking for xargs... /usr/bin/xargs
checking for gawk... gawk
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for fgrep... /bin/grep -F
checking for a sed that does not truncate output... /bin/sed
checking for cygpath... no
checking for greadlink... no
checking for readlink... /usr/bin/readlink
checking for df... /bin/df
checking for SetFile... no
checking for cpio... /bin/cpio
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking target system type... x86_64-unknown-linux-gnu
checking openjdk-build os-cpu... linux-x86_64
checking openjdk-target os-cpu... linux-x86_64
checking compilation type... native
checking for top-level directory... /tmp/shenandoah-jdk8
checking for presence of closed sources... no
checking if closed source is suppressed (openjdk-only)... no
checking which variant of the JDK to build... normal
checking which interpreter of the JVM to build... template
checking which variants of the JVM to build... server
checking which debug level to use... release
checking for sysroot...
checking for toolchain path...
checking for extra path...
checking where to store configuration... in default location
checking what configuration name to use... linux-x86_64-normal-server-release
checking for apt-get... no
checking for yum... no
checking for port... no
checking for pkgutil... no
checking for pkgadd... no
checking for gmake... /usr/bin/gmake
configure: Testing potential make at /usr/bin/gmake, found using gmake in PATH
configure: Using GNU make 3.81 (or later) at /usr/bin/gmake (version: GNU Make 4.2.1)
checking if find supports -delete... yes
checking for unzip... /usr/bin/unzip
checking for zip... /usr/bin/zip
checking for ldd... /usr/bin/ldd
checking for readelf... /usr/bin/readelf
checking for hg... no
checking for stat... /usr/bin/stat
checking for time... /usr/bin/time
checking for pkg-config... /usr/bin/pkg-config
checking pkg-config is at least version 0.9.0... yes
checking for 7z... 7z
checking for wget... wget
checking headful support... include support for both headful and headless
checking for javac... /usr/bin/javac
checking for java... /usr/bin/java
configure: Found potential Boot JDK using well-known locations (in /usr/lib/jvm/openjdk-bin-12)
configure: Potential Boot JDK found at /usr/lib/jvm/openjdk-bin-12 did not contain an rt.jar; ignoring
configure: Found potential Boot JDK using well-known locations (in /usr/lib/jvm/icedtea-8)
checking for Boot JDK... /usr/lib/jvm/icedtea-8
checking Boot JDK version... openjdk version "1.8.0_212" OpenJDK Runtime Environment (IcedTea 3.12.0) (Gentoo icedtea-3.12.0) OpenJDK 64-Bit Server VM (build 25.212-b04, mixed mode)
checking for java in Boot JDK... ok
checking for javac in Boot JDK... ok
checking for javah in Boot JDK... ok
checking for javap in Boot JDK... ok
checking for jar in Boot JDK... ok
checking for rmic in Boot JDK... ok
checking for native2ascii in Boot JDK... ok
checking if Boot JDK is 32 or 64 bits... 64
checking flags for boot jdk java command ...
checking flags for boot jdk java command for big workloads...  -Xms64M -Xmx1600M -XX:ThreadStackSize=1536
checking flags for boot jdk java command for small workloads...  -XX:+UseSerialGC -Xms32M -Xmx512M
configure: Using default toolchain gcc (GNU Compiler Collection)
checking for gcc... /usr/bin/gcc
checking resolved symbolic links for CC... /usr/x86_64-pc-linux-gnu/gcc-bin/9.1.0/x86_64-pc-linux-gnu-gcc
checking if CC is disguised ccache... no, keeping CC
configure: Using gcc C compiler version 9.1.0 [gcc (Gentoo 9.1.0-r1 p1.1) 9.1.0]
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether /usr/bin/gcc accepts -g... yes
checking for /usr/bin/gcc option to accept ISO C89... none needed
checking for g++... /usr/bin/g++
checking resolved symbolic links for CXX... /usr/x86_64-pc-linux-gnu/gcc-bin/9.1.0/x86_64-pc-linux-gnu-g++
checking if CXX is disguised ccache... no, keeping CXX
configure: Using gcc C++ compiler version 9.1.0 [g++ (Gentoo 9.1.0-r1 p1.1) 9.1.0]
checking whether we are using the GNU C++ compiler... yes
checking whether /usr/bin/g++ accepts -g... yes
checking how to run the C preprocessor... /usr/bin/gcc -E
checking how to run the C++ preprocessor... /usr/bin/g++ -E
checking for ar... ar
configure: Rewriting AR to "/usr/bin/ar"
checking for strip... strip
configure: Rewriting STRIP to "/usr/bin/strip"
checking for otool... no
checking for nm... nm
configure: Rewriting NM to "/usr/bin/nm"
checking for gobjcopy... no
checking for objcopy... objcopy
configure: Rewriting OBJCOPY to "/usr/bin/objcopy"
checking for gobjdump... no
checking for objdump... objdump
configure: Rewriting OBJDUMP to "/usr/bin/objdump"
checking for broken SuSE 'ld' which only understands anonymous version tags in executables... no
checking for jtreg... no
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking stdio.h usability... yes
checking stdio.h presence... yes
checking for stdio.h... yes
checking size of int *... 8
checking for target address size... 64 bits
checking whether byte ordering is bigendian... no
checking if the C++ compiler supports "-std=gnu++98 -Werror"... yes
checking if the C compiler supports "-fno-delete-null-pointer-checks -Werror"... yes
checking if the C++ compiler supports "-fno-delete-null-pointer-checks -Werror"... yes
checking if both compilers support "-fno-delete-null-pointer-checks -Werror"... yes
checking if the C compiler supports "-fno-lifetime-dse -Werror"... yes
checking if the C++ compiler supports "-fno-lifetime-dse -Werror"... yes
checking if both compilers support "-fno-lifetime-dse -Werror"... yes
checking if the C++ compiler supports "-ffp-contract=off -Werror"... yes
checking if the C compiler supports "-m64"... yes
checking if the C++ compiler supports "-m64"... yes
checking if both compilers support "-m64"... yes
checking if the C compiler supports "-m64"... yes
checking if the C++ compiler supports "-m64"... yes
checking if both compilers support "-m64"... yes
checking if we should generate debug symbols... true
checking if we should zip debug-info files... yes
checking what type of native debug symbols to use (this will override previous settings)... not specified
configure: --with-native-debug-symbols not specified. Using values from --disable-debug-symbols and --disable-zip-debug-info
checking what is not needed on Linux?... pulse
checking for X... libraries , headers
checking for gethostbyname... yes
checking for connect... yes
checking for remove... yes
checking for shmat... yes
checking for IceConnectionNumber in -lICE... yes
checking for X11/extensions/shape.h... yes
checking for X11/extensions/Xrender.h... yes
checking for X11/extensions/XTest.h... yes
checking for X11/Intrinsic.h... yes
checking cups/cups.h usability... yes
checking cups/cups.h presence... yes
checking for cups/cups.h... yes
checking cups/ppd.h usability... yes
checking cups/ppd.h presence... yes
checking for cups/ppd.h... yes
checking for FREETYPE... yes
checking for freetype... yes (using pkg-config)
checking if we can compile and link with freetype... yes
checking if we should bundle freetype... no
checking for ALSA... yes
checking fontconfig/fontconfig.h usability... yes
checking fontconfig/fontconfig.h presence... yes
checking for fontconfig/fontconfig.h... yes
checking for main in -ljpeg... yes
checking for which giflib to use... bundled
checking for compress in -lz... yes
checking for which zlib to use... bundled
checking for cos in -lm... yes
checking for dlopen in -ldl... yes
checking if dynamic link of stdc++ is possible... yes
checking if static link of stdc++ is possible... yes
checking how to link with libstdc++... static
checking if elliptic curve crypto implementation is present... yes
checking for number of cores... 8
checking for memory size... 24041 MB
checking for appropriate number of jobs to run in parallel... 7
checking whether to use sjavac... no
checking that precompiled headers work... yes
checking is ccache enabled... no
checking if build directory is on local disk... yes
configure: creating /tmp/shenandoah-jdk8/build/linux-x86_64-normal-server-release/config.status
config.status: creating /tmp/shenandoah-jdk8/build/linux-x86_64-normal-server-release/spec.gmk
config.status: creating /tmp/shenandoah-jdk8/build/linux-x86_64-normal-server-release/hotspot-spec.gmk
config.status: creating /tmp/shenandoah-jdk8/build/linux-x86_64-normal-server-release/bootcycle-spec.gmk
config.status: creating /tmp/shenandoah-jdk8/build/linux-x86_64-normal-server-release/compare.sh
config.status: creating /tmp/shenandoah-jdk8/build/linux-x86_64-normal-server-release/spec.sh
config.status: creating /tmp/shenandoah-jdk8/build/linux-x86_64-normal-server-release/Makefile
config.status: creating /tmp/shenandoah-jdk8/build/linux-x86_64-normal-server-release/config.h

====================================================
A new configuration has been successfully created in
/tmp/shenandoah-jdk8/build/linux-x86_64-normal-server-release
using configure arguments '--with-extra-cflags=-Wno-error'.

Configuration summary:
* Debug level:    release
* JDK variant:    normal
* JVM variants:   server
* OpenJDK target: OS: linux, CPU architecture: x86, address length: 64

Tools summary:
* Boot JDK:       openjdk version "1.8.0_212" OpenJDK Runtime Environment (IcedTea 3.12.0) (Gentoo icedtea-3.12.0) OpenJDK 64-Bit Server VM (build 25.212-b04, mixed mode)  (at /usr/lib/jvm/icedtea-8)
* Toolchain:      gcc (GNU Compiler Collection)
* C Compiler:     Version 9.1.0 (at /usr/bin/gcc)
* C++ Compiler:   Version 9.1.0 (at /usr/bin/g++)

Build performance summary:
* Cores to use:   7
* Memory limit:   24041 MB
---snip---

--> so that looks good, let's build:

holger>make images
<building>
..
## Starting jdk
Importing CORBA classes.jar
Importing CORBA src.zip
Importing CORBA bin.zip

zip error: Nothing to do! (/tmp/shenandoah-jdk8/build/linux-x86_64-normal-server-release/jdk/lib/amd64/server/libjsig.diz.tmp)
gmake[2]: *** [Import.gmk:189: /tmp/shenandoah-jdk8/build/linux-x86_64-normal-server-release/jdk/lib/amd64/server/libjsig.diz] Error 12
gmake[2]: *** Waiting for unfinished jobs....
gmake[1]: *** [BuildJdk.gmk:51: import-only] Error 2
make: *** [/tmp/shenandoah-jdk8//make/Main.gmk:117: jdk-only] Error 2

holger>cd /tmp/shenandoah-jdk8/build/linux-x86_64-normal-server-release/jdk/lib/amd64/server
holger>ll
total 169M
-rw-r--r-- 1 holger users 1.4K Jun 25 10:11 Xusage.txt
lrwxrwxrwx 1 holger users   20 Jun 25 10:11 libjsig.debuginfo -> ../libjsig.debuginfo
lrwxrwxrwx 1 holger users   13 Jun 25 10:11 libjsig.so -> ../libjsig.so*
-rw-r--r-- 1 holger users 153M Jun 25 10:11 libjvm.diz
-rwxr-xr-x 1 holger users  17M Jun 25 10:11 libjvm.so*

holger>ll ..
total 160K
-rw-r--r-- 1 holger users 5.4K Jun 25 10:11 libjsig.diz
-rwxr-xr-x 1 holger users  17K Jun 25 10:11 libjsig.so*
-rw-r--r-- 1 holger users  73K Jun 25 10:11 libsaproc.diz
-rwxr-xr-x 1 holger users  55K Jun 25 10:11 libsaproc.so*
drwxr-xr-x 2 holger users  140 Jun 25 10:11 server/

--> libjsig.debuginfo is wrong: fix the stale symlink

holger>rm libjsig.debuginfo
holger>ln -s ../libjsig.diz .
holger>cd ../../../../../../
holger>make images

--> build continues successfully & the result works \o/
holger>/tmp/shenandoah-jdk8/build/linux-x86_64-normal-server-release/images/j2sdk-image/bin/java -version
openjdk version "1.8.0-internal"
OpenJDK Runtime Environment (build 1.8.0-internal-holger_2019_06_25_10_06-b00)
OpenJDK 64-Bit Server VM (build 25.71-b00, mixed mode)

As far as I can tell this is not something that should happen in a default
build configuration. Why it happens or if there's a backport fix missing I
can't say, but I figured it's worth reporting. :)
I can gladly test any recommended patches/fixes.

thanks,
Holger