Linux large pages: why do we have UseSHM?

[Volker Simonis]

Thomas Stüfe <thomas.stuefe at gmail.com> schrieb am Sa., 5. Dez. 2020, 07:26:

> Hi David,
>
> thanks a lot, David. This made a lot of things clearer (BTW I wish the
> patch links would work for these old bugs in JBS).
>

You probably already know this, so just for reference. The patch hashes are
still valid in all existing Mercurial repositories, just some of the old
team repos have disappeared.

The easiest and quickest way to view such a change is to click on the
commit link and than replace the team repo with the corresponding main
repo. E.g. instead of:

http://hg.openjdk.java.net/jdk7/hotspot-rt/hotspot/rev/139667d9836a

use:

http://hg.openjdk.java.net/jdk7/jdk7/hotspot/rev/139667d9836a



> So here is what I understand:
>
> UseSHM had been there first.
>
> Then JDK-7034464 introduced Transparent Huge Pages. Those needs
> madvise(MADV_HUGEPAGE). For TPH, allocating via mmap(MAP_HUGETLB) was added
> alongside the old shmget(). I am not sure why, would TPH not have worked
> with shmget()? But anyway, for a while TLBFS was synonymous with TPH.
>
> Later we had https://bugs.openjdk.java.net/browse/JDK-8024838, which
> claimed TPH were responsible for performance problems and switched support
> for them off by default. This left the mmap(MAP_HUGETLB) allocation intact
> but avoided now using TPH. So it gave us the new explicit-TLBFS mode
> (UseHugeTLBFS).
>
> Interestingly, one of the reasons JDK-7034464 gave for using mmap() instead
> of SHM was that mmap can be committed on demand. That is true (you can call
> mmap on every part of a mapping), but later with
> https://bugs.openjdk.java.net/browse/JDK-8007074 this capability had been
> switched off. Commit on demand would need to mmap(MAP_HUGETLBFS) to commit
> and uncommit a mapping, but that may fail if the huge page pool is
> exhausted, which would make the mmap call fail, which would introduce a
> hole in the reserved space which concurrently running code could
> accidentally mmap into... so, in effect, TLBFS is always committed now.
>
> There are still some holes in my understanding. But mainly, I wonder
> whether we still need UseSHM. Would love to hear opinions.
>
> Thanks! Thomas
>
> On Sat, Dec 5, 2020 at 6:33 AM David Holmes <david.holmes at oracle.com>
> wrote:
>
> > Hi Thomas,
> >
> > On 4/12/2020 4:48 am, Thomas Stüfe wrote:
> > > Hi,
> > >
> > > I wonder why we support two different ways (discounting THPs) to
> allocate
> > > huge pages on Linux.
> > >
> > > We support allocating huge pages both via System V shm APIs (UseSHM)
> and
> > > via the "newer" mmap Posix APIs (UseHugeTLBFS). The latter is default
> for
> > > +UseLargePages.
> >
> > We had the UseSHM stuff first. The UseHugeTLBFS stuff came later. See
> > this discussion:
> >
> >
> http://mail.openjdk.java.net/pipermail/hotspot-dev/2011-April/004063.html
> >
> > and the related bug that made the changes:
> >
> > https://bugs.openjdk.java.net/browse/JDK-7034464
> >
> > HTH.
> >
> > David
> >
> > > I do not see a practical difference beside the fact that:
> > > - System V shm API is slightly fussier to use (eg no partial unmap
> > possible)
> > > - it requires kernel.shmmax to be correctly set, in addition to the
> > > vm.nr_xxx_hugepages
> > > - it requires more user permissions
> > >
> > > But in the end, we end up with kernel huge pages, one way or the other.
> > > Both give access to 2M and 1G pages.
> > >
> > > I looked through mailing list archives and bug descriptions, but beyond
> > > vague mentionings of "one way could fail, then try the other" I did not
> > > find anything.
> > >
> > > I must be missing something. Does anyone have any closer knowledge
> > > about this?
> > >
> > > Thank you!
> > >
> > > Thomas
> > >
> >
>