RFR: 8330936: [ubsan] exclude function BilinearInterp and ShapeSINextSpan in libawt java2d from ubsan checks

[Phil Race]

On Thu, 23 Jan 2025 09:23:52 GMT, Matthias Baesken <mbaesken at openjdk.org> wrote:

> In java2d coding there are a few overflows (those are shown when running jtreg tests with ubsan enabled binaries) 
> jtreg test java/awt/Scrollbar/AquaLFScrollbarTest/ScrollBarBorderTest.java shows
> 
> jdk/src/java.desktop/share/native/libawt/java2d/loops/TransformHelper.c:683:16: runtime error: signed integer overflow: 1651910497 + 660764199 cannot be represented in type 'int'
>     #0 0x7efe59e6ece8 in BilinearInterp src/java.desktop/share/native/libawt/java2d/loops/TransformHelper.c:683
>     #1 0x7efe59e75e21 in Java_sun_java2d_loops_TransformHelper_Transform src/java.desktop/share/native/libawt/java2d/loops/TransformHelper.c:499
>     #2 0x7efe9b8dee7b (<unknown module>)
> 
> java/awt/BasicStroke/DashStrokeTest.java shows
> 
> src/java.desktop/share/native/libawt/java2d/pipe/ShapeSpanIterator.c:1366:21: runtime error: signed integer overflow: 128253951 + 2118518271 cannot be represented in type 'int'
>     #0 0x7fb97d7daf21 in ShapeSINextSpan src/java.desktop/share/native/libawt/java2d/pipe/ShapeSpanIterator.c:1366
>     #1 0x7fb97d62fa7e in AnyIntSetSpans src/java.desktop/share/native/libawt/java2d/loops/AnyInt.c:75
>     #2 0x7fb97d6a8816 in Java_sun_java2d_loops_FillSpans_FillSpans src/java.desktop/share/native/libawt/java2d/loops/FillSpans.c:92
>     #3 0x7fba12d07e7b (<unknown module>)
> 
> 
> There is currently no need seen to adjust this coding, so exclude the methods from ubsan checking to avoid unneeded warnings.

Marked as reviewed by prr (Reviewer).

I looked at the native code, and it seems to me that overflow is actually by design, when working with fractional pixel positions and color components.
So I think this points out that whilst ubsan findings might be an interesting data point, it isn't an automatic "go update the code".

The mechanism being used to silence the warnings might be palatable in very small quantities, but I sincerely hope it is not the start of a pattern. If it is, then something less intrusive to the source code would be better.
For example, something that is part of the build system. This would become a "must" if there's a 3rd party library involved.
Eg, when we find a compiler warning against 3rd party code, we just selectively disable the warning, and do not change the imported sources.

-------------

PR Review: https://git.openjdk.org/jdk/pull/23255#pullrequestreview-2662521582
PR Comment: https://git.openjdk.org/jdk/pull/23255#issuecomment-2702106349