StringBuilder OOMs earlier with JRE 11 compared to JRE 8

[Remi Forax]

----- Original Message -----
> From: "David Holmes" <david.holmes at oracle.com>
> To: "S" <simeon.danailov.andreev at gmail.com>, "core-libs-dev" <core-libs-dev at openjdk.java.net>
> Cc: "Andrey Loskutov" <loskutov at gmx.de>
> Sent: Vendredi 27 Août 2021 15:25:25
> Subject: Re: StringBuilder OOMs earlier with JRE 11 compared to JRE 8

> Hi Simeon,
> 
> Redirecting this to core-libs-dev as it is not a serviceability issue.
> (bcc serviceabillity-dev)

Hi Simeon,
in Java 9, the String representation was changed to use less memory if the characters can be encoded in ISO Latin 1,
a side effect is that StringBuilder now uses a byte[] instead of a char[], hence the maximum size being half the one it was previously.

> 
> Thanks,
> David

[1] https://github.com/openjdk/jdk/blob/master/src/java.base/share/classes/java/lang/AbstractStringBuilder.java#L63

> 
> On 27/08/2021 8:53 pm, S A wrote:
>> Hi all,
>> 
>> while working on https://bugs.eclipse.org/bugs/show_bug.cgi?id=575641
>> <https://bugs.eclipse.org/bugs/show_bug.cgi?id=575641>, I noticed that
>> StringBuilder throws an OOM "half as early" in OpenJDK 11 (and 17 early
>> access), when compared to OpenJDK 8.
>> 
>> In particular, I ran the following snippet to see where the limit of
>> appending to a StringBuilder is:
>> 
>> StringBuilder sb = new StringBuilder();
>> long n = 8L * 1024L * 1024L;
>> for (long i = 0; i < n; ++i) {
>> int m = 1024;
>> for (int j = 0; j < m; ++j) {
>> int length = sb.length();
>> try {
>> sb.append((char) j);
>> } catch (Error e) {
>> System.out.println("Error occurred at length=" + length + " [i=" + i +
>> ", j=" + j + "]");
>> throw e;
>> }
>> 
>> }
>> }
>> 
>> JRE 8:
>> 
>> Error occurred at length=2147483645 [i=2097151, j=1021]
>> Exception in thread "main" java.lang.OutOfMemoryError: Requested array
>> size exceeds VM limit
>> at java.util.Arrays.copyOf(Arrays.java:3332)
>> at
>> java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:124)
>> at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:649)
>> at java.lang.StringBuilder.append(StringBuilder.java:202)
>> at
>> test.stringbuilder.TestStringbuilderOOM.main(TestStringbuilderOOM.java:13)
>> 
>> JRE 11:
>> 
>> Error occurred at length=1073741822 [i=1048575, j=1022]
>> Exception in thread "main" java.lang.OutOfMemoryError: Requested array
>> size exceeds VM limit
>> at java.base/java.util.Arrays.copyOf(Arrays.java:3745)
>> at
>> java.base/java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:172)
>> at
>> java.base/java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:748)
>> at java.base/java.lang.StringBuilder.append(StringBuilder.java:241)
>> at
>> TestJava11/test.stringbuilder.TestStringbuilderOOM.main(TestStringbuilderOOM.java:13)
>> 
>> While StringBuilder is not a good choice for holding GBs of text, I
>> wonder that no effort is made to clamp the capacity to its limit when (I
>> assume) it is being doubled upon array expansion.
>> 
>> Is this something that should be looked into or it can be safely ignored
>> (from JDK users POV)?
>> 
>> Best regards,
> > Simeon