Does CDS archive generation work for crossbuilds?

[Ioi Lam]

On 11/5/21 9:38 AM, Magnus Ihse Bursie wrote:
> On 2021-10-29 21:35, Anton Kozlov wrote:
>> QEMU user space emulation [1] works pretty fast. On my linux-x86 
>> laptop native
>> java -Xshare:dump completes in 0.4 sec, and with qemu-aarch64 it's 
>> about 2.3
>> sec. GHA now provides a runnable instruction on how to create the 
>> sysroot [2].
> But that time excludes booting up the virtual machine, I presume.
>

I think user space emulation doesn't spin up a new virtual machine. It 
runs on the same kernel as the build host. QEMU does the translation 
between the the user program and the OS (system calls, signals, etc). So 
the start-up overhead is pretty minimal.

> Nevertheless, if someone is interested in going down this road to get 
> CDS generation for cross-compilation, I would not object to putting it 
> in the build system.

I filed https://bugs.openjdk.java.net/browse/JDK-8276791 in case there 
are any volunteers :-)

Thanks
- Ioi


>
> /Magnus
>>
>> Regarding problems, different page size for linux-aarch64 first comes 
>> to mind,
>> but it should be fixed by JDK-8268396: "CDS archive with 4K alignment 
>> unusable
>> on machines with 64k pages"
>>
>> Thanks,
>> Anton
>>
>> [1] https://www.qemu.org/docs/master/user/main.html
>> [2] 
>> https://github.com/openjdk/jdk/blob/master/.github/workflows/submit.yml#L514
>>
>>
>>
>> On 10/29/21 14:35, Magnus Ihse Bursie wrote:
>>> On 2021-10-28 22:56, Ioi Lam wrote:
>>>> How reliable would it be to use qemu to run the cross-compiled 
>>>> binaries? Has anyone tried that recently?
>>> I experimented with qemu and the JDK build some years ago (so not 
>>> really recently). As I remember it, the takeaway was that it 
>>> probably was reliable, but it was slow as h*ll, up to the point of 
>>> being practically unusable.
>>>
>>> Add to this the fact that you need to prepare an entire OS image, 
>>> complete with all tools needed to build the JDK... I set up a few 
>>> such images for my own use (I think it was emulating linux-aarch64 
>>> on x64) to run testing. I had an idea of describing how I did to 
>>> share the knowledge, but in the end it was just too complicated and 
>>> slow to even consider recommending.
>>>
>>> /Magnus
>>>>
>>>>
>>>> On 10/23/21 5:48 AM, Thomas Stüfe wrote:
>>>>> Hi Alan,
>>>>>
>>>>> On Sat, Oct 23, 2021 at 9:58 AM Alan Bateman 
>>>>> <Alan.Bateman at oracle.com>
>>>>> wrote:
>>>>>
>>>>>>
>>>>>> On 23/10/2021 07:57, Thomas Stüfe wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> when I crossbuild (for linux aarch64, using a devkit, building 
>>>>>>> on linux
>>>>>>> x64), for some reason I don't
>>>>>>> get the classes.jsa generated inside the images directory.
>>>>>>>
>>>>>>> My configure options:
>>>>>>>
>>>>>>>
>>>>>> --with-devkit=/shared/projects/openjdk/devkits/x86_64-linux-gnu-to-aarch64-linux-gnu 
>>>>>>
>>>>>>> --openjdk-target=aarch64-linux-gnu
>>>>>>> --with-boot-jdk=/shared/projects/openjdk/jdks/sapmachine17
>>>>>>>
>>>>>> --with-build-jdk=/shared/projects/openjdk/jdk-jdk/output-release/images/jdk 
>>>>>>
>>>>>>> --with-gtest=/shared/projects/openjdk/gtest/googletest
>>>>>>> --with-debug-level=fastdebug
>>>>>>>
>>>>>>> The build jdk is a freshly build x64 release VM from the same 
>>>>>>> source
>>>>>> tree.
>>>>>>> Am I missing something obvious? Is CDS archive generation even 
>>>>>>> supported
>>>>>>> for crossbuilds?
>>>>>> It needs the generate run-time to execute "java -Xshare:dump" so 
>>>>>> I don't
>>>>>> expect so. hotspot-runtime-dev is probably the place to discuss the
>>>>>> details. BTW: this came up recently in the context of the jlink 
>>>>>> plugin
>>>>>> that generates the CDS archive. The plugin needed a check to 
>>>>>> ensure that
>>>>>> the target platform matched the current platform as it could 
>>>>>> launch the
>>>>>> target VM to create the dump.
>>>>>>
>>>>>>
>>>>> Thinking for a second, probably it cannot work since we copy binary
>>>>> structures verbatim to the archive; I guess the chance that they 
>>>>> are binary
>>>>> compatible between platforms is very small. But it should be easily
>>>>> rectified by calling Xshare:dump on the target platform.
>>>>>
>>>>> Thank you!
>>>>>
>>>>> ..Thomas
>>>>>
>>>>>
>>>>>> -Alan
>>>>>>
>>>>
>>>
>