RFR: 8283225: [AIX] ClassLoader.c produces incorrect OutOfMemory Exception when length is 0 [v2]

Tyler Steele duke at openjdk.java.net
Wed Mar 16 17:45:16 UTC 2022


> As described in the linked issue, NullClassBytesTest fails due an OutOfMemoryError produced on AIX when the test calls defineClass with a byte array of size of 0. The native implementation of defineClass then calls  malloc with a size of 0. On AIX malloc(0) returns NULL, while on other platforms it return a valid address. When NULL is produced by malloc for this reason, ClassLoader.c incorrectly interprets this as a failure due to a lack of memory.
> 
> This PR modifies ClassLoader.c to produce an OutOfMemoryError only when `errno == ENOMEM` and to produce a ClassFormatError with the message "ClassLoader internal allocation failure" in all other cases (in which malloc returns NULL). 
> 
> In addition, I performed some minor tidy-up work in ClassLoader.c by changing instances of `return 0` to `return NULL`, and `if (some_ptr == 0)` to `if (some_ptr == NULL)`. This was done to improve the clarity of the code in ClassLoader.c, but didn't feel worthy of opening a separate issue.
> 
> ### Alternatives
> 
> It would be possible to address this failure by modifying the test to accept the OutOfMemoryError on AIX. I thought it was a better solution to modify ClassLoader.c to produce an OutOfMemoryError only when the system is actually out of memory.
> 
> ### Testing
> 
> This change has been tested on AIX and Linux/x86.

Tyler Steele has updated the pull request incrementally with four additional commits since the last revision:

 - Fixes type warning.
 - Removes unneeded ClassFormatError from ClassLoader.c
 - Revert "Extract memory error logic to helper procedure"
   
   This reverts commit b631eb0ccd5f3748c2010c864f8ccef0c1da9c42.
 - Avoid calling malloc with size zero.

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/7829/files
  - new: https://git.openjdk.java.net/jdk/pull/7829/files/b631eb0c..de946b41

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=7829&range=01
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=7829&range=00-01

  Stats: 41 lines in 2 files changed: 10 ins; 21 del; 10 mod
  Patch: https://git.openjdk.java.net/jdk/pull/7829.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/7829/head:pull/7829

PR: https://git.openjdk.java.net/jdk/pull/7829


More information about the core-libs-dev mailing list