RFR: 8356643: Switching focus causes Segmentation Fault on zLinux

[Amit Kumar]

On Fri, 16 May 2025 11:29:26 GMT, Amit Kumar <amitkumar at openjdk.org> wrote:

> In OpenJKD code, for big endian architecture, `XIMPreeditState` is considered `unsigned int` whereas in `xlib` code it is `unsigned long` [source](https://github.com/mirror/libX11/blob/ff8706a5eae25b8bafce300527079f68a201d27f/include/X11/Xlib.h#L1267).
> 
> Now, openjdk compiled code pushes a frame and allots the address, just 4 bytes away, because it considers state to be a int, from the store `callers_sp` : 
> 
> 
> 2: /x $r15 = 0x3ff8137da20
> 3: x/2xg 0x3ff8137da20
> 0x3ff8137da20:	0x000003ff8137daf0	0x000003fffa2a9a86
> 
> 
> `0x000003ff8137daf0` is callers sp for s390x.  And this is address for `state` : 
> 
> (gdb) p &state
> $9 = (unsigned int *) 0x3ff8137da1c
> 
> 
> which is just 4 byte away from the callers sp: 
> 
> (gdb) p/x 0x3ff8137da20 - 4 
> $10 = 0x3ff8137da1c
> 
> 
> Now while executing `_XimDecodePreeditState` method from xlib library where `state` is of type `unsigned long` we are going to emit a 8-byte instruction.: 
> 
> (gdb) disassemble _XimDecodePreeditState
> Dump of assembler code for function _XimDecodePreeditState:
> => 0x000003ff989f3360 <+0>:	  llgf	%r1,8(%r2)
>    0x000003ff989f3366 <+6>:	  lghi	%r2,1
>    0x000003ff989f336a <+10>:	lg	%r1,0(%r3,%r1)
>    0x000003ff989f3370 <+16>:	stg	%r1,0(%r4)  
>    0x000003ff989f3376 <+22>:	br	%r14
> End of assembler dump.
> 
> 
> `0x000003ff989f3370 <+16>:	stg	%r1,0(%r4)  ` here `r4` is pointing the `state` variable. And the moment this write happens we will corrupt the first 4 byte of callers_sp, which will cause issue while the stack-unwinding. And at the end JVM will crash with segmentation fault. This is always reproducible with the test case provided in the JBS. And test only failed with compiler (c1 & c2) but passed always with interpreter.

closing in favour of https://github.com/openjdk/jdk/pull/24197

-------------

PR Comment: https://git.openjdk.org/jdk/pull/25266#issuecomment-2886572360