RFR(S) 8241071 Generation of classes.jsa is not deterministic

[Thomas Stüfe]

Hi Ioi,

Please don't do this :)

First off, how would this work when dumping with UseCompressedClassPointers
off? In that case allocation would be relegated to non-class metaspace
which cannot guarantee that kind of address stability.

Even in class space, I do not think you can guarantee addresses growing
monotonously. Class unloading could happens during dump time, so space may
be returned to class freelist and later reused. Metadata can be prematurely
deallocated, e.g. if a class load error occurs or byte code is rewritten by
some agent. Remainder of Metachunks are used up in a delayed fashion. All
these cases will present you with pointers which are not growing
monotonously.

I also believe this problem of non-deterministic placement is not limited
to Symbols, but that you should see it for Klass structures too, albeit
rarely. I believe the fact that you do not see this is an accident, or we
are not looking that closely. E.g. if we were to change the frequency at
which we retrieve MetaBlocks from the freeblocklist (see
http://hg.openjdk.java.net/jdk/jdk/file/534855a30ef5/src/hotspot/share/memory/metaspace/spaceManager.cpp#l414),
you would get more reuse of deallocated blocks and would certainly see more
volatility in the addresses.

This is all true with the current implementation; the upcoming new one uses
a buddy style allocator behind the scenes where it is by no means
guaranteed that the first chunks get used first. I think this is what still
happens, by sheer accident, but I am hesitant to promise such a behavior in
the future. It removes freedom from the implementation in a lot of ways.

Small examples, we might want to shepherd certain allocations into separate
parts of class space (e.g. Klass structures from hidden classes) to
minimize fragmentation. Or add a mode, for testing, where we would allocate
Klass at the very end of ccs, or at certain "round" addresses, to shake
loose errors in the calling layers which rely too much on how Klass
pointers look like.

Bottomline I think the assumption that ccs allocates in
monotonously ascending order is not even correct today, and may break very
easily, and we should not rely on this.

I think instead of misusing ccs for this, it would be cleaner to just
allocate a large C heap area as backing storage for the symbols? How much
space are we talking about? If memory is a concern, we could just reserve a
range and commit it manually as we go.

Or could we not order the placement of Klass and Symbol at dump time? Dump
time is not that time critical, no?

Thanks & Sorry,

Thomas



On Mon, Apr 27, 2020 at 7:31 AM Ioi Lam <ioi.lam at oracle.com> wrote:

> https://bugs.openjdk.java.net/browse/JDK-8241071
>
> http://cr.openjdk.java.net/~iklam/jdk15/8241071-deterministic-cds-archive.v02/
>
> The goal is to for "java -Xshare:dump" to produce deterministic contents in
> the CDS archive that depend only on the contents of $JAVA_HOME/lib/modules.
>
> Problems:
> [1] Symbols in the CDS archive may have non-deterministic order because
>      Arena allocation is non-deterministic.
> [2] The contents of the CDS shared heap region may be randomized due to
>      ImmutableCollections.SALT32L.
>
> Fixes:
> [1] With -Xshare:dump, allocate Symbols from the class space (64-bit only).
>      See changes in symbol.cpp for details.
> [2] When running the VM with -Xshare:dump, ImmutableCollections.SALT32L is
>      initialized with a deterministic seed. NOTE: this affects ONLY when
> the
>      VM is running with the special flag -Xshare:dump to dump the CDS
> archive.
>      It does NOT affect normal execution of Java programs.
>
> ---
> I also cleaned up the -Xlog:cds output and print out the CRC of each
> CDS region, to help diagnose why two CDS archives may have different
> contents.
>
> Thanks
> - Ioi
>