RFR: 8326583: Remove over-generalized DefineNativeToolchain solution [v4]

Sun Mar 3 23:30:51 UTC 2024

On Tue, 27 Feb 2024 11:19:59 GMT, Magnus Ihse Bursie <ihse at openjdk.org> wrote:

>> The idea of setting up general "toolchains" in the native build was good, but it turned out that we really only need a single toolchain, with a single twist: if it should use CC or CPP to link. This is better described by a specific argument to SetupNativeCompilation, LANG := C++ or LANG := C (the default).
>> 
>> There is a single exception to this rule, and that is if we want to compile for the build platform rather than the target platform. (This is only done for adlc) To keep expressing this difference, introduce TARGET_TYPE := BUILD (or TARGET_TYPE := TARGET, the default).
>> 
>> The final odd-case was the hack for building hsdis/bin on mingw. This can be resolved using direct variables to SetupNativeCompilation, instead of first creating a toolchain.
>> 
>> Doing this refactoring will simplify the SetupNativeCompilation code, and make it much clearer if we use the C or C++ linker.
>
> Magnus Ihse Bursie has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits:
> 
>  - Merge branch 'master' into remove-toolchain-define
>  - Rename LANG to LINK_TYPE
>  - Reword "lib" comment to fit in better
>  - Merge branch 'master' into remove-toolchain-define
>  - 8326583: Remove over-generalized DefineNativeToolchain solution

Indeed hsdis or no hsdis does not matter at all. --enable-ccache is broken with this commit in any case. I executed the much simplified script below, which builds once using this commit and once the parent commit. Seems, that with hsdis enabled HSDIS was just the first thing that needed to be compiled. Now it is failing when "Creating support/modules_libs/java.base/server/libjvm.dylib".

Regarding why I use ccache: I frankly didn't time it to see if there is an advantage. I just read in doc/building.md (lines 1517 ff) "The JDK build supports building with ccache when using gcc or clang. Using ccache can radically speed up compilation of native code if you often rebuild the same sources." so I included it in my personal script to build OpenJDK. But I can easily live without using ccache and personally wouldn't mind if OpenJDK stops supporting ccache. 

Below the content of `build/macosx-aarch64-server-fastdebug/hotspot/variant-server/libjvm/objs/precompiled/precompiled.hpp.pch.cmdline` for this commit ac3ce2aa where you can see the main issue, that the command `/opt/homebrew/bin/ccache /usr/bin/clang` is missing in front of the first compiler option -D__STDC_FORMAT_MACROS. Additionally some environment variables are missing/different.

cd /tmp/ac3ce2aa/jdk && CCACHE_COMPRESS=1 CCACHE_SLOPPINESS=pch_defines -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -D_GNU_SOURCE -pipe -fno-rtti -fno-exceptions -fvisibility=hidden -fno-strict-aliasing -fno-omit-frame-pointer -flimit-debug-info -mno-omit-leaf-frame-pointer -mstack-alignment=16 -std=c++14 -DMAC_OS_X_VERSION_MIN_REQUIRED=110000 -mmacosx-version-min=11.00.00 -DLIBC=default -D_ALLBSD_SOURCE -D_DARWIN_C_SOURCE -D_XOPEN_SOURCE -Wall -Wextra -Wformat=2 -Wpointer-arith -Wsign-compare -Wreorder -Wunused-function -Wundef -Wunused-value -Woverloaded-virtual -fPIC -DVM_LITTLE_ENDIAN -D_LP64=1 -arch arm64 -DASSERT -DCHECK_UNHANDLED_OOPS -DTARGET_ARCH_aarch64 -DINCLUDE_SUFFIX_OS=_bsd -DINCLUDE_SUFFIX_CPU=_aarch64 -DINCLUDE_SUFFIX_COMPILER=_gcc -DTARGET_COMPILER_gcc -DAARCH64 -DHOTSPOT_LIB_ARCH='"aarch64"' -DCOMPILER1 -DCOMPILER2 -DDTRACE_ENABLED -Ibuild/macosx-aarch64-server-fastdebug/hotspot/variant-server/gensrc/adfiles -Isrc/hotspot/share -Isrc/hotspo
 t/os/bsd -Isrc/hotspot/os/posix -Isrc/hotspot/cpu/aarch64 -Isrc/hotspot/os_cpu/bsd_aarch64 -Ibuild/macosx-aarch64-server-fastdebug/hotspot/variant-server/gensrc -Isrc/hotspot/share/precompiled -Isrc/hotspot/share/include -Isrc/hotspot/os/posix/include -Ibuild/macosx-aarch64-server-fastdebug/support/modules_include/java.base -Ibuild/macosx-aarch64-server-fastdebug/support/modules_include/java.base/darwin -Isrc/java.base/share/native/libjimage -arch arm64 -Ibuild/macosx-aarch64-server-fastdebug/hotspot/variant-server/gensrc/adfiles -Isrc/hotspot/share -Isrc/hotspot/os/bsd -Isrc/hotspot/os/posix -Isrc/hotspot/cpu/aarch64 -Isrc/hotspot/os_cpu/bsd_aarch64 -Ibuild/macosx-aarch64-server-fastdebug/hotspot/variant-server/gensrc -g -gdwarf-4 -gdwarf-aranges -Wno-unknown-warning-option -Wno-unused-parameter -Wno-unused -Wno-sometimes-uninitialized -Wno-missing-braces -Wno-delete-non-abstract-non-virtual-dtor -Wno-unknown-pragmas -Wno-tautological-constant-out-of-range-compare -isysroot /Applic
 ations/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.2.sdk -iframework /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.2.sdk/System/Library/Frameworks -O3 -x c++-header -c -MMD -MF build/macosx-aarch64-server-fastdebug/hotspot/variant-server/libjvm/objs/precompiled/precompiled.hpp.pch.d.tmp src/hotspot/share/precompiled/precompiled.hpp -o build/macosx-aarch64-server-fastdebug/hotspot/variant-server/libjvm/objs/precompiled/precompiled.hpp.pch

and here the content of `build/macosx-aarch64-server-fastdebug/hotspot/variant-server/libjvm/objs/precompiled/precompiled.hpp.pch.cmdline` for the parent commit bceaed6d

cd /tmp/bceaed6d/jdk && CCACHE_COMPRESS=1 CCACHE_SLOPPINESS=pch_defines,time_macros CCACHE_BASEDIR=/tmp/bceaed6d/jdk/ /opt/homebrew/bin/ccache /usr/bin/clang -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -D_GNU_SOURCE -pipe -fno-rtti -fno-exceptions -fvisibility=hidden -fno-strict-aliasing -fno-omit-frame-pointer -flimit-debug-info -mno-omit-leaf-frame-pointer -mstack-alignment=16 -std=c++14 -DMAC_OS_X_VERSION_MIN_REQUIRED=110000 -mmacosx-version-min=11.00.00 -DLIBC=default -D_ALLBSD_SOURCE -D_DARWIN_C_SOURCE -D_XOPEN_SOURCE -Wall -Wextra -Wformat=2 -Wpointer-arith -Wsign-compare -Wreorder -Wunused-function -Wundef -Wunused-value -Woverloaded-virtual -fPIC -DVM_LITTLE_ENDIAN -D_LP64=1 -arch arm64 -DASSERT -DCHECK_UNHANDLED_OOPS -DTARGET_ARCH_aarch64 -DINCLUDE_SUFFIX_OS=_bsd -DINCLUDE_SUFFIX_CPU=_aarch64 -DINCLUDE_SUFFIX_COMPILER=_gcc -DTARGET_COMPILER_gcc -DAARCH64 -DHOTSPOT_LIB_ARCH='"aarch64"' -DCOMPILER1 -DCOMPILER2 -DDTRACE_ENABLED -Ibuild/macosx-aarch64-s
 erver-fastdebug/hotspot/variant-server/gensrc/adfiles -Isrc/hotspot/share -Isrc/hotspot/os/bsd -Isrc/hotspot/os/posix -Isrc/hotspot/cpu/aarch64 -Isrc/hotspot/os_cpu/bsd_aarch64 -Ibuild/macosx-aarch64-server-fastdebug/hotspot/variant-server/gensrc -Isrc/hotspot/share/precompiled -Isrc/hotspot/share/include -Isrc/hotspot/os/posix/include -Ibuild/macosx-aarch64-server-fastdebug/support/modules_include/java.base -Ibuild/macosx-aarch64-server-fastdebug/support/modules_include/java.base/darwin -Isrc/java.base/share/native/libjimage -arch arm64 -Ibuild/macosx-aarch64-server-fastdebug/hotspot/variant-server/gensrc/adfiles -Isrc/hotspot/share -Isrc/hotspot/os/bsd -Isrc/hotspot/os/posix -Isrc/hotspot/cpu/aarch64 -Isrc/hotspot/os_cpu/bsd_aarch64 -Ibuild/macosx-aarch64-server-fastdebug/hotspot/variant-server/gensrc -g -gdwarf-4 -gdwarf-aranges -Wno-unknown-warning-option -Wno-unused-parameter -Wno-unused -Wno-sometimes-uninitialized -Wno-missing-braces -Wno-delete-non-abstract-non-virtual-dtor 
 -Wno-unknown-pragmas -Wno-tautological-constant-out-of-range-compare -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.2.sdk -iframework /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.2.sdk/System/Library/Frameworks -O3 -x c++-header -c -MMD -MF build/macosx-aarch64-server-fastdebug/hotspot/variant-server/libjvm/objs/precompiled/precompiled.hpp.pch.d.tmp src/hotspot/share/precompiled/precompiled.hpp -o build/macosx-aarch64-server-fastdebug/hotspot/variant-server/libjvm/objs/precompiled/precompiled.hpp.pch

Script used to test

#!/bin/bash

export LANG=en_US
export LC_ALL=en_US

suspectedCommit=ac3ce2aa
preceedingCommit=bceaed6d

function build() {
 bash ./configure --enable-ccache --disable-warnings-as-errors --with-debug-level=fastdebug
 make clean
 make images
}

cd /tmp
rm -rf $preceedingCommit
mkdir $preceedingCommit
cd $preceedingCommit
git clone "https://github.com/openjdk/jdk.git"
cd jdk
git checkout $preceedingCommit
build

cd /tmp
rm -rf $suspectedCommit
mkdir $suspectedCommit
cd $suspectedCommit
git clone "https://github.com/openjdk/jdk.git"
cd jdk
git checkout $suspectedCommit
build

-------------

PR Comment: https://git.openjdk.org/jdk/pull/17986#issuecomment-1975396416