RFR (XS) JDK-8010389 - After fix for 7107135 a failed dlopen() call results in a VM crash

[Christian Tornqvist]

When I debugged this I looked for other calls to dlopen but didn't see one,
there may have been a call to other dlsym/dlclose functions that I didn't
see though. However, when I stepped past the failed dlopen() I did a call
(with gdb) to dlerror() but got null back there too.  

Talked to Per Liden (in the Exajava team) and he told me that the dlerror
information is saved in thread locals (seems like this is similar to how it
is done with GetLastError on Windows where this information is saved in the
TEB).

Thanks,
Christian

-----Original Message-----
From: David Holmes [mailto:david.holmes at oracle.com] 
Sent: den 21 mars 2013 06:13
To: Ioi Lam
Cc: hotspot-runtime-dev at openjdk.java.net; Christian Törnqvist
Subject: Re: RFR (XS) JDK-8010389 - After fix for 7107135 a failed dlopen()
call results in a VM crash

On 21/03/2013 2:21 PM, Ioi Lam wrote:
> Please review:
>
> http://cr.openjdk.java.net/~iklam/8010389/dlopen_crash_001/
> <http://cr.openjdk.java.net/%7Eiklam/8010389/dlopen_crash_001/>
>
> Bug: After fix for 7107135 a failed dlopen() call results in a VM 
> crash
>
> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=8010389
> https://jbs.oracle.com/bugs/browse/JDK-8010389
>
>      (The bugs.sun.com page may be unavailable for a couple of days)
>
> Summary of fix:
>
> Previously, if dlopen() fails inside the VM Thread, the Java thread 
> would call dlerror() to try to find out what happened, but would get a 
> NULL return instead. Attempts to use this NULL value causes the JVM to 
> crash.

That is interesting and annoying. POSIX does not require ::dlerror() to be
thread-safe. Linux does not specify that it is per-thread either. Is the
problem that this occurs in a different thread, or simply that there may be
an interleaving dl*() call that clears dlerror() ?

David
-----

>
>      The fix is to read dlerror() in the VM Thread when necessary.
>
> To Christian:
>
>      I integrated your test into jtreg, but I needed to write a shell 
> test to set
>      the LD_LIBRARY_PATH. Is there a way to avoid the shell test?
>
> Tests executed:
>
>      * JPRT (linux only)
>      * UTE (vm.quick.testlist)
>      * JTREG
>
> Thanks,
> Ioi
>