RFR: 8373626: [asan] read past end of buffer in sun.awt.image.ImagingLib.convolveBI [v3]

[Sergey Bylokhov]

On Thu, 22 Jan 2026 17:57:42 GMT, Phil Race <prr at openjdk.org> wrote:

>> Some of the medialib native functions implementing Convolve read data from arrays when it is not needed or used instead of reading just what is needed and used.
>> This is detected as a read out of bounds. It is limited and hasn't been seen to result in any crashes without ASAN, and the OOB values that are read are never used so there's a very limited problem.
>> The changes here make the mlib_ImageConv_*nw.c files match what happens in the mlib_ImageConv_*ext.c files which read just the data they need.
>> The changes are fairly mechanical but there could be copy/paste errors for a reviewer to find.
>> 
>> Not easy to provide a test case, building with --enable-asan is needed and for me it works only on macOS.
>> I did that and ran all our existing automated tests on our CI systems.
>
> Phil Race has updated the pull request incrementally with one additional commit since the last revision:
> 
>   8373626

src/java.desktop/share/native/libmlib_image/mlib_ImageConv_u16nw.c line 922:

> 920:           off += kw;
> 921: 
> 922:           sp += (kw - 1)*chan1;

Just to check: before the patch, the code read data from sp to set pXX, then increased sp. After the patch, the order is different. Is that correct?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/29257#discussion_r2776761075