Linux, hugetlbfs, UseSHM removal?

[Thomas Stüfe]

On Mon, Jun 26, 2023 at 9:19 AM Stefan Karlsson <stefan.karlsson at oracle.com>
wrote:

> I'm asking around to see if I can get an answer how important UseSHM is.
>
> StefanK
>

Thank you, Stefan.

I did some deeper digging into its history:

Huge page support using System V shmget predates OpenJDK. The moniker
"special", as in "reserve_memory_special()", is also from the initial load.

Huge page support using mmap had been introduced by JDK-7034464 [1] into
JDK 6 [2]. This patch also introduced UseSHM and UseHugeTLBFS. From that
point on, mmap-based reservation had been the default, shmget the fallback.

Comments during review [3] and in the patch suggest that mmap support for
TLBFS had been unreliable, and therefore SHM support had been retained as
fallback. Looking at the API manpages (shmget and mmap), TLBFS support,
there is a version difference (shmget: Linux 2.6, mmap: Linux 2.6.32). If
that is not a documentation fluke, there may have been a time window within
which mmap support for MAP_HUGETLBFS was flaky.

I also found mentions of "mmap not being able to use huge pages for
anonymous mappings"; this is not true anymore. I looked through the kernel
changelogs for kernel 2.6, but could find no explicit mentioning of mmap
hugetlb support fixes wrt anonymous mappings. There was, however, a lot of
activity for hugetlbfs support so that it may be the 2.6.xx kernel was
stabilizing tlbfs support and things had been flaky.

As an interesting side note, one of the advantages cited for mmap was the
ability to uncommit and re-commit arbitrary pages from the range. While
true, we stopped doing that with JDK-8007074 [5][6], since the re-commit of
these pages cannot be guaranteed (technically also true for small pages,
but much more probable for explicit hugepages). Since then, we keep
hugepages as committed, and since then today's "special" flag found in
various places is synonymous for "precommitted, don't uncommit".

Linux 2.6 is EOL. The lowest still supported kernel seems to be 3.16. I
believe cutting SHM support from JDK 22 onwards should be safe.

[1] https://bugs.openjdk.org/browse/JDK-7034464
[2]
https://github.com/openjdk/jdk6/commit/c796d80b281c81bee156919378d9afec8c85c220
[3] https://mail.openjdk.org/pipermail/hotspot-dev/2011-April/004079.html
[4] https://mirrors.edge.kernel.org/pub/linux/kernel/v2.6/
[5] https://bugs.openjdk.org/browse/JDK-8007074
[6]
https://github.com/openjdk/jdk8u/commit/7ade7987a0a8ccfde017811623d338b4c4b80237



>
> On 2023-06-23 14:11, Thomas Stüfe wrote:
> > Hi all,
> >
> > We talked about this [1] and StefanJ opened an issue open for it [2],
> > and also I had private talks with various people. I am currently
> > digging around in LP initialization again, and getting rid of SHM mode
> > would be so nice. Perfect time for it too, just after LTS fork-off.
> >
> > SHM tlbfs mode causes a lot of extra complexity. Since SHM segments
> > can only be detached as a whole, they behave more like Windows
> > VirtualSpace instead of mmap. As in, the whole mapping cannot be
> > split, cannot be partially uncommitted etc. As StefanJ pointed out in
> > [2], it also requires more permissions than mmap-tlbfs mode.
> >
> > I have always had the suspicion that this is a dusty corner of hotspot
> > that is rarely used and probably broken in many corner cases. For
> > example, the sanity checks at VM startup are not correctly
> > implemented, see JDK-8310262
> >
> > Therefore, unless someone knows a good reason against removal, I would
> > like to remove SHM mode from hotspot.
> >
> > We could get rid of both -XX:+UseSHM and -XX:+UseHugeTLBFS, since one
> > is the negation of the other. Large page support on Linux would become
> > a lot simpler then and just a matter of: +UseLargePages and +-UseTHP.
> > A nice complexity reduction, and we could get rid of a many tests too.
> >
> > (Obviously, this would need a CSR and Release notes etc).
> >
> > Any objections?
> >
> > Thanks, Thomas
> >
> > [1]
> > https://mail.openjdk.org/pipermail/hotspot-dev/2020-December/046885.html
> > [2] https://bugs.openjdk.org/browse/JDK-8261894
> > [3] https://bugs.openjdk.org/browse/JDK-8310262
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-dev/attachments/20230626/903a1b5b/attachment.htm>