RFR: JDK-8314163: os::print_hex_dump prints incorrectly for big endian platforms and unit sizes larger than 1 [v2]

[Thomas Stuefe]

On Tue, 15 Aug 2023 10:51:17 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Comment changes by Aleksey
>>   
>>   Co-authored-by: Aleksey Shipilëv <shipilev at amazon.de>
>
> src/hotspot/share/runtime/os.cpp line 972:
> 
>> 970:     const int bitoffset =
>> 971:       LITTLE_ENDIAN_ONLY(offset * BitsPerByte)
>> 972:       BIG_ENDIAN_ONLY((int)((sizeof(intptr_t) - unitsize - offset) * BitsPerByte));
> 
> Um, my head hurts a bit thinking about this. So for 64-bit `sizeof(intptr_t)` = 8 and unitsize=8, `bitoffset` is negative? Which we then feed into `bitfield`, which does the shift by the negative value, which is undefined behavior?

offset is based on the print address, and that one is aligned to unitsize:

https://github.com/openjdk/jdk/blob/caef3d5ac2ba34610b2fcd65045d9c9185c5de06/src/hotspot/share/runtime/os.cpp#L995

which I now also assert at the entry of print_hex_location.

For unitsize = 8, on 64-bit, print address should equal the address of the intptr_t, and the offset should be 0.

Hence also the align_as in the test: I want to make sure the address range is always aligned in a way that does not conflict with unitsize alignment.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/15256#discussion_r1294484874