RFR: 8206890: jlink --endian XXX generates unusable image if endian-ness does not match architecture

[Jim Laskey]

On Wed, 11 Jan 2023 13:19:32 GMT, Jaikiran Pai <jpai at openjdk.org> wrote:

> Can I please get a review for this change which proposes to fix the issue reported in https://bugs.openjdk.org/browse/JDK-8206890?
> 
> The `jlink` command allows a `--endian` option to specify the byte order in the generated image. Before this change, when such a image was being launched, the code would assume the byte order in the image to be the native order of the host where the image is being launched. That would result in failure to launch java, as noted in the linked issue.
> 
> The commit in this PR, changes relevant places to not assume native order and instead determine the byte order by reading the magic bytes in the image file's header content.
> 
> A new jtreg test has been added which reproduces the issue and verifies the fix.

Changes requested by jlaskey (Reviewer).

src/java.base/share/classes/jdk/internal/jimage/ImageHeader.java line 222:

> 220:         if (bb.getInt() == MAGIC) {
> 221:             return bb.order();
> 222:         }

I would have just added `public static final int REVERSE_MAGIC = 0xDADAFECA;` and add an else clause. to compare to REVERSE_MAGIC. Much simpler.

src/java.base/share/classes/jdk/internal/jimage/ImageHeader.java line 226:

> 224:         ByteOrder altByteOrder = bb.order() == ByteOrder.BIG_ENDIAN
> 225:                 ? ByteOrder.LITTLE_ENDIAN : ByteOrder.BIG_ENDIAN;
> 226:         bb.flip();

Don't you just want to rewind here?

src/java.base/share/native/libjimage/imageFile.cpp line 284:

> 282:     if (reader == NULL || !reader->open()) {
> 283:         // Failed to open.
> 284:         if (reader != NULL && reader->_invalid_magic) {

Similar here. Flip the byte order if matches IMAGE_REVERSE_MAGIC.

-------------

PR: https://git.openjdk.org/jdk/pull/11943