malloc/calloc return value NULL check

[Baesken, Matthias]

Regarding calloc – checking   ,

jdk.incubator.vector/unix/native/libsleef

seems to have quite a lot  of callocs without a NULL check afterwards , but afaik it is  3rd party coding so we will probably not touch it.


For sunFont.c  I see no direct handling  after calloc in the C code  :

java.desktop/share/native/libfontmanager/sunFont.c-67- */
java.desktop/share/native/libfontmanager/sunFont.c-68-JNIEXPORT jlong JNICALL Java_sun_font_NullFontScaler_getGlyphImage
java.desktop/share/native/libfontmanager/sunFont.c-69-  (JNIEnv *env, jobject scaler, jlong pContext, jint glyphCode) {
java.desktop/share/native/libfontmanager/sunFont.c:70:    void *nullscaler = calloc(1, sizeof(GlyphInfo));
java.desktop/share/native/libfontmanager/sunFont.c-71-    return ptr_to_jlong(nullscaler);
java.desktop/share/native/libfontmanager/sunFont.c-72-}
java.desktop/share/native/libfontmanager/sunFont.c-73-
--
java.desktop/share/native/libfontmanager/sunFont.c-303-JNIEXPORT jlong JNICALL
java.desktop/share/native/libfontmanager/sunFont.c-304-Java_sun_font_StrikeCache_getInvisibleGlyphPtr(JNIEnv *env, jclass cls) {
java.desktop/share/native/libfontmanager/sunFont.c-305-
java.desktop/share/native/libfontmanager/sunFont.c:306:    GlyphInfo *info = (GlyphInfo*) calloc(1, sizeof(GlyphInfo));
java.desktop/share/native/libfontmanager/sunFont.c-307-    return (jlong)(uintptr_t)info; /* invisible glyph */
java.desktop/share/native/libfontmanager/sunFont.c-308-}


(but in the Java coding  calling this,  we seem to have some special  checks for 0, so maybe it is fine)


Here

java.base/windows/native/libjli/java_md.c:951:    appArgIdx = calloc(argc, sizeof(int));
java.base/windows/native/libjli/java_md.c-952-    for (i = idx, j = 0; i < stdargc; i++) {
java.base/windows/native/libjli/java_md.c-953-        if (isTool) { // filter -J used by tools to pass JVM options
java.base/windows/native/libjli/java_md.c-954-            arg = stdargs[i].arg;

we seem to miss a check, should I open a JBS issue ?


Best regards, Matthias


From: Thomas Stüfe <thomas.stuefe at gmail.com>
Sent: Friday, 11 July 2025 18:19
To: Baesken, Matthias <matthias.baesken at sap.com>
Cc: build-dev at openjdk.org
Subject: Re: malloc/calloc return value NULL check

Absolutely, yes.

The larger the allocated size, the more important. Linux kernel, by default, only protects a small area against NULL accesses; depending on distro, 4KB or 64 (?) KB. And the JVM, at various places, allocates in low-area ranges. So accessing NULL+<large offset> can actually land you at a valid unrelated address instead of faulting.

/Thomas

On Fri, Jul 11, 2025 at 2:57 PM Baesken, Matthias <matthias.baesken at sap.com<mailto:matthias.baesken at sap.com>> wrote:
Hi, when playing around with the  GCC static analyzer  ( https://developers.redhat.com/articles/2022/04/12/state-static-analysis-gcc-12-compiler )   I noticed
a lot of complaints about  missing  NULL checks of  malloc/calloc  return values in the code base.

While we check these return values for NULL at a lot of places in the codebase,  it is not done always.
Should we do it always  (except 3rd party code probably where we do not want to have large diffs to upstream) ?

Or is it considered not important enough  to do it always?

Best regards, Matthias
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/build-dev/attachments/20250715/96f72d64/attachment-0001.htm>