RFR: 8206890: jlink --endian XXX generates unusable image if endian-ness does not match architecture

[Alan Bateman]

On Wed, 11 Jan 2023 14:30:46 GMT, Jaikiran Pai <jpai at openjdk.org> wrote:

>> Can I please get a review for this change which proposes to fix the issue reported in https://bugs.openjdk.org/browse/JDK-8206890?
>> 
>> The `jlink` command allows a `--endian` option to specify the byte order in the generated image. Before this change, when such a image was being launched, the code would assume the byte order in the image to be the native order of the host where the image is being launched. That would result in failure to launch java, as noted in the linked issue.
>> 
>> The commit in this PR, changes relevant places to not assume native order and instead determine the byte order by reading the magic bytes in the image file's header content.
>> 
>> A new jtreg test has been added which reproduces the issue and verifies the fix.
>
> While running tier testing, one existing test showed that I hadn't taken into account SecurityManager checks in the new code that I had added. I have now fixed that part and pushed an update to the PR.
> 
> There's one additional unrelated test failing (`TestMaxCachedBufferSize`) in a odd way and I'll investigate it separately.

@jaikiran I've been mulling over this a bit further. When jlink is cross linking, as in creating a run-time image for a different target platform, then I think we should strive to have it create the jimage file in the native endian for the target platform. Right now this means the user needs to help out by providing the --endian option. For example, if the user is on linux-x64 but generating a run-time image for solaris-sparcv9 or aix-ppc, then it requires specifying jlink --endian=big to ensure that the jimage file is in the endianness needed at runtime. I don't disagree that having the 2-arg ImageFileReader::open retry with the opposite of the endian parameter will work but I don't think this is the right thing to do. Instead I think we need to consider determine the endian from the target java.base module. Right now the ModuleTarget class file attribute is not enough but a short term fix might be for jlink to have a simple mapping of known ModuleTarget values to endianness. Overal
 l, it's probably low priority as the most likely users of this will be environments building for embedded platforms.

-------------

PR: https://git.openjdk.org/jdk/pull/11943