Enable compiler memory limits by default?

[Thomas Stüfe]

Thank you, Vladimir.

Issue: https://bugs.openjdk.org/browse/JDK-8331185 and proposed
PR: https://github.com/openjdk/jdk/pull/18969

I am curious if testing shakes any pre-existing bugs loose.

Cheers, Thomas


On Mon, Apr 22, 2024 at 7:35 PM Vladimir Kozlov <vladimir.kozlov at oracle.com>
wrote:

> Hi Thomas,
>
> I like option 1).
>
> I think 1Gb is reasonable starting value - I know that C2 may ease eat
> 512Kb of memory.
> But before we set exact value we need to test it in all our tiers.
> I don't want to create a lot of failures which we will not have time to
> fix fast. It should be rare case as you stated.
>
> We also need to decide how we fix/avoid such failure:
>
> 1. Recompile with some optimizations off (I assume we can tell which
> optimization triggers big memory consumption and safely bailout from
> compilation)
> 2. Recompile with some inlining off
> 3. Mark method not compilable by corresponding compiler
> ...
>
> Please file RFE and PR. We will help with testing.
>
> Thanks,
> Vladimir K
>
> On 4/12/24 12:30 AM, Thomas Stüfe wrote:
> > Hi,
> >
> > Issues like https://bugs.openjdk.org/browse/JDK-8330103
> > <https://bugs.openjdk.org/browse/JDK-8330103> show that compiler memory
> > consumption can be an issue.
> >
> > Since https://bugs.openjdk.org/browse/JDK-8318016
> > <https://bugs.openjdk.org/browse/JDK-8318016>, we have an optional
> > per-compilation memory limit. If we reach that limit, one of two things
> > (configurable) happens: we either assert or abort the compilation.
> >
> > These memory limits build on the compiler memory statistic added with
> > https://bugs.openjdk.org/browse/JDK-8317683
> > <https://bugs.openjdk.org/browse/JDK-8317683>. Enabling
> > memory limits also enables memory statistics.
> >
> > Some ideas:
> >
> > 1) We could enable a reasonable memory limit per default for debug
> > builds. Preferably combined with the assert option. That way, we run all
> > tests on a debug VM with memory limits enabled. If there are
> > pathological compilations during testing, we will notice them.
> >
> > (I don't know if we would notice them today; even if testers let JVMs
> > run with outside ulimits, these limits are typically very high to allow
> > for the total expected memory consumption of the test JVM).
> >
> > Such a memory limit could be set at whatever we feel is pathological,
> > e.g., several hundred MB. Even set at 1GB, we would hopefully see cases
> > like 8318016 in our tests.
> >
> > 2) If we don't want (1), we could at least enable memory statistics by
> > default for debug builds and print it out to hs-err files.
> >
> > 3) We could also enable memory limits in release builds and bail out of
> > the compilations. A small cost is involved, probably negligible: on
> > Arena enlargement, we increase several thread local counters.
> > Unfortunately, there is a small risk, too, in that bailout paths in C2
> > may be broken, leading to follow-up errors. We fixed them all, I think,
> > but there is a remaining risk. OTOH, using up excessive amounts of
> > memory is also not optimal.
> >
> > What do you think? Would this make sense? If (1) makes sense to you,
> > what limit would be reasonable?
> >
> > Cheers, Thomas
> >
> >
> >
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-compiler-dev/attachments/20240426/72514c95/attachment.htm>