RFR: 8307858: [REDO] JDK-8307194 Add make target for optionally building a complete set of all JDK and hotspot libjvm static libraries [v4]

[Jiangli Zhou]

On Thu, 1 Jun 2023 16:18:24 GMT, Erik Joelsson <erikj at openjdk.org> wrote:

> > > Another possibility might be the user provided `BUILD_LDCXX` includes extra options in the testing build (?). If that's the case, we probably could define a separate `BUILD_LD_PARTIAL` with no added options. In our prototype on JDK 11, we define a separate one for partial linking.
> > 
> > 
> > I tried building `static-libs-image` for linux-aarch64 on Ubuntu machine using `devkit` to reproduce the failure. Also tried building for `linux-ppc64le-server-release` target using `devkit` on Ubuntu. Both built successfully with using `devkit`. I could not build a `devkit` for arm32 (with `make TARGETS="arm-linux-gnueabihf" BASE_OS=Fedora BASE_OS_VERSION=17`, also tried with BASE_OS_VERSION=17).
> > @erikj79 Could you please help provide additional insight for the build failure you found for arm32? `BUILD_LIBJVM_partial_link.cmdline` might help shed some light.
> 
> There are no extra options. I suspect it's an issue with the older GCC version. Here is one failing command line:
> 
> ```
> $ ( cd /home/erik/git/jdk/build/linux-arm32 ; /var/tmp/jib-erik/install/jpg/infra/builddeps/devkit-linux_x64-to-linux_arm/gcc8.2.0-Fedora27+1.0/devkit-linux_x64-to-linux_arm-gcc8.2.0-Fedora27+1.0.tar.gz/x86_64-linux-gnu-to-arm-linux-gnueabihf/bin/arm-linux-gnueabihf-gcc -r --sysroot=/var/tmp/jib-erik/install/jpg/infra/builddeps/devkit-linux_x64-to-linux_arm/gcc8.2.0-Fedora27+1.0/devkit-linux_x64-to-linux_arm-gcc8.2.0-Fedora27+1.0.tar.gz/x86_64-linux-gnu-to-arm-linux-gnueabihf/arm-linux-gnueabihf/sysroot -o /home/erik/git/jdk/build/linux-arm32/support/native/java.rmi/librmi/static/librmi_relocatable.o /home/erik/git/jdk/build/linux-arm32/support/native/java.rmi/librmi/static/GC.o )
> /var/tmp/jib-erik/install/jpg/infra/builddeps/devkit-linux_x64-to-linux_arm/gcc8.2.0-Fedora27+1.0/devkit-linux_x64-to-linux_arm-gcc8.2.0-Fedora27+1.0.tar.gz/x86_64-linux-gnu-to-arm-linux-gnueabihf/bin/../lib/gcc/arm-linux-gnueabihf/8.2.0/../../../../arm-linux-gnueabihf/bin/ld: fatal error: cannot mix -r with dynamic object /var/tmp/jib-erik/install/jpg/infra/builddeps/devkit-linux_x64-to-linux_arm/gcc8.2.0-Fedora27+1.0/devkit-linux_x64-to-linux_arm-gcc8.2.0-Fedora27+1.0.tar.gz/x86_64-linux-gnu-to-arm-linux-gnueabihf/bin/../lib/gcc/arm-linux-gnueabihf/8.2.0/../../../../arm-linux-gnueabihf/lib/libgcc_s.so.1
> ```
> 

Thanks a lot for looking into the issue.

> > The partial linking was originally suggested by C++/Clang toolchain folks to mitigate linker overhead that was observed during final executable link time. For a static library containing any object file (`.o`) that was compiled with ThinLTO (https://clang.llvm.org/docs/ThinLTO.html) enabled, linking an executable using the static library without distributed ThinLTO could experience more overhead and slow down linking. Solving the macosx `ar` limitation is a side-effect/benefit of using partial linking. We probably would want to include the partial linking even without the `ar` limitation.
> 
> This seems to be Clang specific, so maybe only apply this to Clang and not GCC? Have you measured the difference in link time and concluded that this workaround is needed? If so, how big were the differences? 

It was noticeably slower (which prompted me looked into it at that time) when linking the executable using the static libraries created without partial linking workaround. In my experiments, it took >2m when linking final executable. With the mitigation the final linking took a few seconds.

> Given that it prints a lot of warnings on mac, I would suggest leaving the partial linking out of this patch to get something in that is actually working. It was included in this patch because of the side effect it had with handling long path names. However, since looking closer at that issue, we were using a different workaround for Clang already and that same workaround should work fine for AR on mac as well.

Sounds good for solving the macosx `ar` limitation differently. I'll change that. We can also exclude the partial linking part for gcc (due the older tool issue that you've found). Any concerns with including partial linking step for clang on Linux?

-------------

PR Comment: https://git.openjdk.org/jdk/pull/14064#issuecomment-1572922666