OOM in json writer
Erich Schubert
erich at debian.org
Tue Feb 15 09:47:50 UTC 2022
Hi,
I have been running a rather large set of benchmarks, and it eventually
caused an OOM in the JSON writer:
Exception in thread "main" java.lang.OutOfMemoryError: Requested array
size exceeds VM limit
at java.base/java.util.Arrays.copyOf(Arrays.java:3537)
at
java.base/java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:228)
at
java.base/java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:582)
at java.base/java.lang.StringBuilder.append(StringBuilder.java:175)
at
org.openjdk.jmh.results.format.JSONResultFormat.tidy(JSONResultFormat.java:321)
at
org.openjdk.jmh.results.format.JSONResultFormat.writeOut(JSONResultFormat.java:156)
at
org.openjdk.jmh.results.format.ResultFormatFactory$1.writeOut(ResultFormatFactory.java:50)
at org.openjdk.jmh.runner.Runner.internalRun(Runner.java:317)
at org.openjdk.jmh.runner.Runner.run(Runner.java:209)
at org.openjdk.jmh.Main.main(Main.java:71)
I will now try -Djmh.json.rawData=false, an seemingly undocumented option.
Could you please consider:
a) compressed json.gz output format - these files compress very well
b) less/optional whitespace in json - right now, it has 4 spaces per
depth inserted by the tidy() function. Maybe no whitespace at all in
lengthy raw value arrays? Most of the time, json files are meant to be
machine readable, not human readable - if desired, a user can pretty
print them if desired (e.g., using python3 -m json.tool). As you can
see, the tidy() function does cause the problems here.
c) directly writing the output rather than building it entirely
in-memory to conserve memory
d) maybe eventually use an actual json writer library than the current
replaceAll() hack? Building JSON this way is strongly discouraged IMHO.
Thank you.
Best regards,
Erich Schubert
More information about the jmh-dev
mailing list