RFR: 8373626: [asan] read past end of buffer in sun.awt.image.ImagingLib.convolveBI [v3]

[Phil Race]

On Thu, 22 Jan 2026 17:57:42 GMT, Phil Race <prr at openjdk.org> wrote:

>> Some of the medialib native functions implementing Convolve read data from arrays when it is not needed or used instead of reading just what is needed and used.
>> This is detected as a read out of bounds. It is limited and hasn't been seen to result in any crashes without ASAN, and the OOB values that are read are never used so there's a very limited problem.
>> The changes here make the mlib_ImageConv_*nw.c files match what happens in the mlib_ImageConv_*ext.c files which read just the data they need.
>> The changes are fairly mechanical but there could be copy/paste errors for a reviewer to find.
>> 
>> Not easy to provide a test case, building with --enable-asan is needed and for me it works only on macOS.
>> I did that and ran all our existing automated tests on our CI systems.
>
> Phil Race has updated the pull request incrementally with one additional commit since the last revision:
> 
>   8373626

@mrserb I updated last week.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/29257#issuecomment-3803184282