RFR: 8377937: [macos] GlyphMetrics advance does not consider font rotation

[Daniel Gredler]

On Mon, 23 Feb 2026 17:51:38 GMT, Sergey Bylokhov <serb at openjdk.org> wrote:

>>> Another difference is handing invisible INVISIBLE_GLYPHS, which I assume works fine on macOS as is.
>> 
>> Yes, the test exercises this scenario because the new line and tab chars in the test text are mapped to invisible glyph IDs. However, it might not hurt to add an early check here to avoid entering native code entirely in this scenario -- although this should happen rarely, so I'm not sure how effective this will be as a mini-optimization. I'm happy to add this check though, if the consensus is that it's a good idea.
>> 
>>> The FileFontStrike.getGlyphMetrics( ) has a check if (glyphPtr != 0L) before calling StrikeCache.getGlyphXAdvance. Do not we need the same check in the new code?
>> 
>> It doesn't look like it to me, mainly because `CGGI_CreateNewGlyphInfoFrom` (see call sequence below) expects that malloc never fails. That's maybe a bit odd, but I think if we get NULL / 0 back as a pointer, we have problems elsewhere (in `CGGI_CreateNewGlyphInfoFrom`, not here).
>> 
>> Sequence: getGlyphImagePtr -> getGlyphImagePtrs -> getFilteredGlyphImagePtrs -> getGlyphImagePtrsNative -> Java_sun_font_CStrike_getGlyphImagePtrsNative -> CGGlyphImages_GetGlyphImagePtrs -> CGGI_CreateGlyphsAndScanForComplexities -> CGGI_CreateGlyphInfos -> **CGGI_CreateNewGlyphInfoFrom** /// OR /// CGGI_FillImagesForGlyphs -> CGGI_FillImagesForGlyphsWithSizedCanvas -> CGGI_CreateImageForUnicode -> **CGGI_CreateNewGlyphInfoFrom**
>
>>Sequence: getGlyphImagePtr -> getGlyphImagePtrs -> getFilteredGlyphImagePtrs -> getGlyphImagePtrsNative -> Java_sun_font_CStrike_getGlyphImagePtrsNative -> CGGlyphImages_GetGlyphImagePtrs -> CGGI_CreateGlyphsAndScanForComplexities -> CGGI_CreateGlyphInfos -> CGGI_CreateNewGlyphInfoFrom /// OR /// CGGI_FillImagesForGlyphs -> CGGI_FillImagesForGlyphsWithSizedCanvas -> CGGI_CreateImageForUnicode -> CGGI_CreateNewGlyphInfoFrom
> 
> This is some objc code that may throw an NSException. It will be caught in Java_sun_font_CStrike_getGlyphImagePtrsNative by JNI_COCOA_ENTER / JNI_COCOA_EXIT.
> 
> And that assumption that malloc always succeed is suspicion.

@mrserb I've created a separate issue [JDK-8378667](https://bugs.openjdk.org/browse/JDK-8378667) for the missing malloc failure check, and added a defensive null pointer check in the Java code being changed here.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/29726#issuecomment-3958791228