RFR: 8332863: Crash in JPEG decoder if we enable MEM_STATS

[Kevin Rushforth]

On Fri, 24 May 2024 06:48:50 GMT, Jayathirth D V <jdv at openjdk.org> wrote:

> In IJG library's jmemmgr.c file we can define MEM_STATS(by default this flag is not defined and we don't see any issue) to enable printing of memory statistics log. But if we enable it, we get crash while disposing IJG stored objects in jmemmgr->free-pool() function. 
> 
> 
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> # SIGSEGV (0xb) at pc=0x00000001269d5164, pid=47784, tid=259
> #
> # JRE version: Java(TM) SE Runtime Environment (21.0+35) (build 21+35-LTS-2513)
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (21+35-LTS-2513, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, bsd-aarch64)
> # Problematic frame:
> # C [libjavafx_iio.dylib+0x49164] free_pool+0x88
> #
> # No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
> #
> # If you would like to submit a bug report, please visit:
> # https://bugreport.java.com/bugreport/crash.jsp
> # The crash happened outside the Java Virtual Machine in native code.
> # See problematic frame for where to report the bug.
> 
> --------------- T H R E A D ---------------
> 
> Current thread (0x0000000121a42c00): JavaThread "JavaFX Application Thread" [_thread_in_native, id=259, stack(0x000000016d11c000,0x000000016d918000) (8176K)]
> 
> Stack: [0x000000016d11c000,0x000000016d918000], sp=0x000000016d912780, free space=8153k
> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
> C [libjavafx_iio.dylib+0x49164] free_pool+0x88
> C [libjavafx_iio.dylib+0x49410] self_destruct+0x3c
> C [libjavafx_iio.dylib+0xe888] jpeg_destroy+0x3c
> C [libjavafx_iio.dylib+0x4bb1c] imageio_dispose+0x98
> C [libjavafx_iio.dylib+0x4b178] disposeIIO+0x2c
> C [libjavafx_iio.dylib+0x4b140] Java_com_sun_javafx_iio_jpeg_JPEGImageLoader_disposeNative+0x2c
> 
> 
> This is happening because we delete the error handler before we actually start deleting IJG stored objects and while freeing the IJG objects we try to access cinfo->err->trace_level of error handler. This early deletion of error handler is happening in jpegloader.c->imageio_dispose() function. 
> 
> I have moved deletion of error handler logic after we destroy IJG stored objects in jpegloader.c->imageio_dispose(). This resolves this issue.
> There is no regression test case because we need to enable MEM_STATS flag to see this issue.
> Ran graphics unit tests also and i don't see any issues with this change.

Reviewer: @arapte

-------------

PR Comment: https://git.openjdk.org/jfx/pull/1463#issuecomment-2129398988