OpenJDK 18 Linux Bug when Wrapping Clang, Crashes Outside Native Frames

[Maurizio Cimadamore]

Hi again,
I've tried your Panama-only reproducer and I can confirm that the crash 
disappears if you prepend `LIBCLANG_DISABLE_CRASH_RECOVERY=false` to 
your launcher, as in:

LIBCLANG_DISABLE_CRASH_RECOVERY=false java 
-Djava.library.path=<path-to-LLVM-libs> 
--enable-native-access=ALL-UNNAMED --add-modules jdk.incubator.foreign 
MinimalReproductionCase complex.cpp

Maurizio


On 18/07/2022 17:43, Maurizio Cimadamore wrote:
>
> Hi Joshua,
> I know about coffi, and very excited to see stuff like these happening 
> - kudos!
>
> As for clang, my hunch is that you are hitting the dreaded "libclang 
> crash recovery" issue - e.g.
>
> https://reviews.llvm.org/D23662
>
> Basically, clang installs its own signal handlers, which end up 
> overriding (at least on Linux) the signal handlers installed by the JVM.
>
> Our Jextract implementation (which is based on a Panama port of 
> libclang) has also to workaround this:
>
> https://github.com/openjdk/jextract/blob/master/src/main/java/org/openjdk/jextract/clang/LibClang.java#L54
>
> (On windows, the recovery logic seems to work ok, but on Linux it 
> causes spurious crashes, pretty much all over the place).
>
> In principle, setting the LIBCLANG_DISABLE_CRASH_RECOVERY variable 
> should be a quick way for you to check if that's indeed the issue.
>
> Cheers
> Maurizio
>
> On 18/07/2022 17:16, Joshua Suskalo wrote:
>> This is the first time I've posted to the panama-dev mailing list, so 
>> if this isn't the right place for this, please forgive me.
>>
>> I've been working fairly happily with Panama (sans not liking that 
>> Addressable was made sealed in JDK 18) since JDK 17 building a 
>> wrapper for it in Clojure called coffi[1], but I've recently run into 
>> a bug that's a bit outside of what I think I can solve. I'm fairly 
>> sure it's not an error in how I'm using Panama (although tracking it 
>> down has helped me find and fix some bugs in coffi), and I've been 
>> able to reproduce it in a short plain Java file.
>>
>> I have a full listing for a Clojure reproduction case using coffi, a 
>> Java reproduction case using Panama directly, and a C version that 
>> appears to work just fine, as well as some test cpp files to use as 
>> inputs, all available in a paste on sourcehut[2]. The first argument 
>> to running them is the filename to parse. To run the Clojure version 
>> you only need to have the Clojure CLI installed (available from most 
>> package managers), mark the file executable, and run it as a script.
>>
>> The problem is that when calling functions from clang's C api[3], the 
>> JVM appears to enter a corrupted state that will eventually crash 
>> with a SIGSEGV. Usually in my experience this happens outside of 
>> native stack frames, and when working locally in a REPL in Clojure 
>> the actual crash that occurred seemed non-deterministic, though that 
>> likely just had to do with slightly different inputs to the system, 
>> as the reproduction case I've included[2] appears to have 
>> deterministic behavior on my machine.
>>
>> Notably I have not observed similar behavior with other C libraries, 
>> including ones which use upcalls, which means that this *may* simply 
>> be an issue of clang corrupting memory through its normal use that 
>> causes problems with the JVM but which does not affect the C runtime. 
>> Unfortunately I don't know how to test that theory. I also believe I 
>> have determined that this is not caused by the native threads in 
>> clang, as disabling threading by passing the arguments 
>> `-mthread-model single` to the parsing does not appear to prevent a 
>> crash, although in my limited testing it *did* appear to increase the 
>> delay between running the native code and the JVM crashing.
>>
>> Thanks for your time and for reading,
>> Joshua Suskalo
>>
>> [1]: https://github.com/IGJoshua/coffi
>> [2]: https://paste.sr.ht/~srasu/80750a5513bb5e175169465875a155136aad44d7
>> [3]: https://clang.llvm.org/doxygen/index.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/panama-dev/attachments/20220718/53dd4f86/attachment-0001.htm>