RFR: 8368250: [AIX] now ubsan vptr check is also possible (follow up of JDK-8354686)

[Erik Joelsson]

On Wed, 24 Sep 2025 13:23:00 GMT, Joachim Kern <jkern at openjdk.org> wrote:

> After [JDK-8354686](https://bugs.openjdk.org/browse/JDK-8354686) the ubsan vptr checks still did not work. We finally found out, that they only work, if all linkage units are linked with the C++ compiler frontend as linker, especially the main executables (java, javac, ...) which are linked with the C compiler frontend.
> So for a ubsan enabled build on AIX we let everything link with the C++ compiler frontend.
> Background: The C++ compiler frontend inherently adds special static libraries which the C compiler does not. This results in missing symbols when trying to start a program and its shared libraries, when they were linked in mixed mode.

make/autoconf/jdk-options.m4 line 570:

> 568:   # In the ubsan case we have to link every binary with the C++-compiler as linker, because inherently
> 569:   # the C-Compiler and the C++-compiler used as linker provide a different set of ubsan exports.
> 570:   # Linkung an executable with the C-compiler and one of its shared libraries with the C++-compiler

Suggestion:

  # Linking an executable with the C-compiler and one of its shared libraries with the C++-compiler

make/autoconf/jdk-options.m4 line 575:

> 573:       UBSAN_CFLAGS="$UBSAN_CFLAGS -DLLVM_SYMBOLIZER=$(dirname $(dirname $CC))/tools/ibm-llvm-symbolizer"
> 574:       UBSAN_LDFLAGS="$UBSAN_LDFLAGS -Wl,-bbigtoc"
> 575:       LD="$LDCXX"

Indentation for blocks should be 2 spaces. https://openjdk.org/groups/build/doc/code-conventions.html
Suggestion:

    UBSAN_CFLAGS="$UBSAN_CFLAGS -DLLVM_SYMBOLIZER=$(dirname $(dirname $CC))/tools/ibm-llvm-symbolizer"
    UBSAN_LDFLAGS="$UBSAN_LDFLAGS -Wl,-bbigtoc"
    LD="$LDCXX"

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/27468#discussion_r2376669448
PR Review Comment: https://git.openjdk.org/jdk/pull/27468#discussion_r2376675889