RFR: 8289697: buffer overflow in MTLVertexCache.m: MTLVertexCache_AddGlyphQuad [v3]

[Ajit Ghaisas]

On Tue, 5 Jul 2022 21:26:21 GMT, Vladimir Kempik <vkempik at openjdk.org> wrote:

>> Please review this simple patch. When running idea on jdk17 with asan I have found this buffer overflow.
>> The code checks the cache for at least one free element, while placing 6 elements to the cache.
>> The fix checks the presence of 6 free elements.
>
> Vladimir Kempik has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Replace constant with define

I will submit a full test run with this patch and then approve on it's successful completion.

src/java.desktop/macosx/native/libawt_lwawt/java2d/metal/MTLVertexCache.m line 69:

> 67: 
> 68: //Next define should exactly match to the amount of MTLVC_ADD_VERTEX in MTLVC_ADD_TRIANGLES
> 69: #define MTL_TRIS_IN_VERTEX 6

Constants starting with `MTL_` should be avoided as they are used in Metal Framework library.
I think, this should be `VERTS_FOR_A_QUAD`.

-------------

PR: https://git.openjdk.org/jdk/pull/9368