RFR: 8343699: [aarch64] Bug in MacroAssembler::klass_decode_mode()

[Thomas Stuefe]

In `MacroAssembler::klass_decode_mode(),` there is a subtle bug in the xor part.


  if (operand_valid_for_logical_immediate(
        /*is32*/false, (uint64_t)CompressedKlassPointers::base())) {
(1) const size_t range = CompressedKlassPointers::klass_range_end() - CompressedKlassPointers::base();
(2) const uint64_t range_mask = (1ULL << log2i(range)) - 1;
(3) if (((uint64_t)CompressedKlassPointers::base() & range_mask) == 0) {
      log_debug(metaspace)("MacroAssembler::klass_decode_mode xor");
      return (_klass_decode_mode = KlassDecodeXor);
    }
  }


We first determine if the encoding base is encodable as immediate. If it is, then we check if it intersects with the value range of a narrow Klass ID.

(Note: the code ignores encoding shift since the XOR will be applied to the pre-shifted narrow Klass value later.)

The test is done by
1) calculating the range the encoding needs to cover
2) calculating a corresponding bit mask
3) checking that this mask does not intersect with the encoding base.

(2) contains a wrongness: `range_mask = (1ULL << log2i(range)) - 1` . `log2i` returns the largest log2 value *smaller* or equal to input. So, for `range` values that are not a pow2 value, the resulting mask will be one bit too short. As an effect, the code may chose XOR for cases where the lowest encoding base bit can intersect the highest narrow Klass ID bit, thus making the xor lossy.

----

Example:

Let range be 80MB (`-XX:CompressedClassSpaceSize=80m -Xshare:off`).

Then, range_mask = `1 << log2i(80m) - 1` => `(1 << 26) - 1` => `0x3ff_ffff`
The largest possible nKlass value, however, sits just below 80MB => `0x500_0000`. As we see, the mask does not cover the full extent of the narrow Klass ID value range (bit 26 is not covered).
Hence, if we have a base with bit 26 set, its bit 26 intersects a possible bit 26 in high-value narrow Klass ID. The xor would not be lossless.

----

The error is very unlikely because
- we try to reserve the klass range at addresses that are guaranteed not to intersect with the narrow Klass range. Only if that fails - very unlikely - we use whatever address the OS gives us. Only then could we end up with such an address.
- The class space is rarely filled so high that the highest bit of a narrowKlass ID is `1`.

----

Reproduce:


java -XX:CompressedClassSpaceBaseAddress=0x1Fc000000 -XX:CompressedClassSpaceSize=80m -Xlog:metaspace* -Xshare:off


it does not reproduce an error, but causes the JVM to start in XOR mode with an encoding base of `0x1fc000000` and a narrow Klass ID range of `[0 .. 0x5000000)` - bit 26 interleaves. 

The fixed JVM, with this patch, will correctly refuse to use XOR mode, and since MOVK is also not an option, will assert. Note that this is correct behavior: the encoding base `0x1Fc000000` had not been valid to begin with. There is a patch in work by Coleen that will change this assert to an early exit-due-to-invalid-input (https://bugs.openjdk.org/browse/JDK-8340212).

-------------

Commit messages:
 - Merge branch 'openjdk:master' into JDK-8343699-aarch64-Bug-in-MacroAssembler-klass_decode_mode
 - Merge branch 'openjdk:master' into JDK-8343699-aarch64-Bug-in-MacroAssembler-klass_decode_mode
 - JDK-8343699-aarch64-Bug-in-MacroAssembler-klass_decode_mode

Changes: https://git.openjdk.org/jdk/pull/21932/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=21932&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8343699
  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/21932.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/21932/head:pull/21932

PR: https://git.openjdk.org/jdk/pull/21932