JNI oddity in OpenJDK

[Andrew Haley]

On 06/14/2012 09:21 AM, Joe Osborne wrote:

> I've been recommended by a couple of members far wiser than I to bring this
> issue to this mailing list.
>
> We've recently switched from using Sun's JDK to OpenJDK6, and found
> something a little troubling when using JNI.
>
> In our native code, we do various forms of memory analysis for debugging,
> and somewhat unsurprisingly, this includes overriding the global 'operator
> new' set of methods.
> However, running with the OpenJDK6 JRE, our methods are never hit. Does the
> JVM deliberately redirect these when linking, or is something else going on?

This is a matter of link order.

operator new(unsigned long) is defined by libstdc++ and by your library.
LD_DEBUG reveals:

      8933:     symbol=operator new(unsigned long);  lookup in file=java [0]
      8933:     symbol=operator new(unsigned long);  lookup in file=/lib64/libpthread.so.0 [0]
      8933:     symbol=operator new(unsigned long);  lookup in file=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.3.x86_64/jre/bin/../lib/amd64/jli/libjli.so [0]
      8933:     symbol=operator new(unsigned long);  lookup in file=/lib64/libz.so.1 [0]
      8933:     symbol=operator new(unsigned long);  lookup in file=/lib64/libdl.so.2 [0]
      8933:     symbol=operator new(unsigned long);  lookup in file=/lib64/libc.so.6 [0]
      8933:     symbol=operator new(unsigned long);  lookup in file=/lib64/ld-linux-x86-64.so.2 [0]
      8933:     symbol=operator new(unsigned long);  lookup in file=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.3.x86_64/jre/lib/amd64/server/libjvm.so [0]
      8933:     symbol=operator new(unsigned long);  lookup in file=/usr/lib64/libstdc++.so.6 [0]
      8933:     binding file /home/aph/JNITest/libJNITest.so [0] to /usr/lib64/libstdc++.so.6 [0]: normal symbol `operator new(unsigned long)'

This happens because on most GNU/Linux systems, we link with the system's
libstdc++.so, but your JNI library is loaded last.  On Oracle's binary
distribution, they link statically with libstdc++ and remove all of its
symbols.

Do this:
 $ g++ -o libJNITest.so *.cpp -fpic -shared
 $ LD_PRELOAD=./libJNITest.so java JNITest
Calling custom operator new, should now abort
Aborted (core dumped)

Andrew.