OOM in json writer

Erich Schubert erich at debian.org
Tue Feb 15 09:47:50 UTC 2022


Hi,

I have been running a rather large set of benchmarks, and it eventually 
caused an OOM in the JSON writer:

Exception in thread "main" java.lang.OutOfMemoryError: Requested array 
size exceeds VM limit
         at java.base/java.util.Arrays.copyOf(Arrays.java:3537)
         at 
java.base/java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:228)
         at 
java.base/java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:582)
         at java.base/java.lang.StringBuilder.append(StringBuilder.java:175)
         at 
org.openjdk.jmh.results.format.JSONResultFormat.tidy(JSONResultFormat.java:321)
         at 
org.openjdk.jmh.results.format.JSONResultFormat.writeOut(JSONResultFormat.java:156)
         at 
org.openjdk.jmh.results.format.ResultFormatFactory$1.writeOut(ResultFormatFactory.java:50)
         at org.openjdk.jmh.runner.Runner.internalRun(Runner.java:317)
         at org.openjdk.jmh.runner.Runner.run(Runner.java:209)
         at org.openjdk.jmh.Main.main(Main.java:71)

I will now try -Djmh.json.rawData=false, an seemingly undocumented option.

Could you please consider:

a) compressed json.gz output format - these files compress very well

b) less/optional whitespace in json - right now, it has 4 spaces per 
depth inserted by the tidy() function. Maybe no whitespace at all in 
lengthy raw value arrays? Most of the time, json files are meant to be 
machine readable, not human readable - if desired, a user can pretty 
print them if desired (e.g., using  python3 -m json.tool). As you can 
see, the tidy() function does cause the problems here.

c) directly writing the output rather than building it entirely 
in-memory to conserve memory

d) maybe eventually use an actual json writer library than the current 
replaceAll() hack? Building JSON this way is strongly discouraged IMHO.

Thank you.

Best regards,
Erich Schubert



More information about the jmh-dev mailing list