[aarch64-port-dev ] Question about ccs reservation, CDS and aarch64 specifics

[Thomas Stüfe]

Hi all,

I am currently trying to wrap my head around the various ways the
CompressedClassSpace is reserved. Coding has grown a bit in complexity with
the advent of CDS/AppCDS and recently some aarch64 changes atop of that,
changing behavior for aarch64 and ppc64. Maybe someone can enlighten me a
bit.

Specifically, I am looking at

Metaspace::reserve_space

and its aarch64-specific outgrow

Metaspace::reserve_preferred_space

Despite its generic-sounding name, these functions can only be used to
allocate ccs. They lack any interface description, so I parsed the code to
understand their behavior.

So I tried and here is how I think Metaspace::reserve_space works for the
various combinations of input parameters:

A) requested_addr == NULL && use_requested_addr == false:
[aarch64, ppc64]: Attempt to reserve at one of the preferred OS dependent
allocation points. Failing that, return an unreserved space.
[others]: Reserve a space anywhere.

B) requested_addr == NULL && use_requested_addr == true:
[aarch64, ppc64]: Does nothing, returns an unreserved space immediately. I
assume this would be an invalid combination, but since it is not asserted I
am not sure.
[others]: Reserve a space anywhere (use_requested_addr is ignored).

C) requested_addr != NULL && use_requested_addr == false:
[aarch64, ppc64]: First attempt to reserve at the requested address, but
only if that would cause the space to falls into the lower 4G. Failing
that, allocate at one of the preferred OS dependent allocation points.
Failing that, return an unreserved space.
[others]: Attempt to reserve at requested_addr. . Failing that, return an
unreserved space.

D) requested_addr != NULL && use_requested_addr == true:
[aarch64, ppc64]: First attempt to reserve at the requested address, but
only if that would cause the space to falls into the lower 4G. Failing
that, return an unreserved space.
[others]: Attempt to reserve at requested_addr. . Failing that, return an
unreserved space. (use_requested_addr is ignored).

Note the many subtle platform differences. E.g. on aarch64 we honor the
requested address only if ccs would fall below 4G, for all other platforms
we always honor them. Or how for most platforms the parameter
"use_requested_addr" is just ignored.

Or that on aarch64 we never seem to "try anywhere", we just try a fixed set
of attachment points and if these are all occupied we fail. Is this a bug
or by design? Can we always rely at least one  of the attachment points
being unoccupied? Looking at the options for
MacroAssembler::KlassDecodeMode on aarch64, a simple decoding using x +
base >> shift seems not to be wanted? There seems to be no fall back mode
which would work with any value of base/shift?

About reserve_preferred_space(), I was confused why a separate
"use_requested_addr" was even needed - requested_addr!=NULL would be a
perfectly valid way to communicate that the requested address should be
used. I wish we could simplify the coding to just two cases:
- hand down a requested address, which is to be taken-or-fail (somewhat
like case D)
- hand down NULL, which means "try whatever": which for most OSes would be
really anywhere, for aarch64 could be the fixed set of attachment points.
This would be case (A).

About case (C): under which circumstances does it happen that caller code
hands down a requested address below 4G which happens to be free? Does that
make sense? In other words, if the whole point of
Metaspace::reserve_preferred_space() is "OS knows better, let it try to
find a good address", would it not make sense to just try a low address as
part of the try-addresses-loop?

Hope these questions make sense, and thanks a lot!

..Thomas