RFR: 8348028: Unable to run gtests with CDS enabled

[David Holmes]

On Tue, 25 Feb 2025 00:51:55 GMT, Calvin Cheung <ccheung at openjdk.org> wrote:

> A simple fix in `os::jvm_path()` so that gtest can be run with CDS (`-Xshare:on`). The fix is just to change the directory name from `hotspot` to `server`.
> Note that the bug doesn't exist on macOS and thus no change is required for `os_bsd.cpp`.
> 
> Testing:
> 
> - run gtest with -Xshare:on on linux-x64
> - tier1

So basic problem is that `os::jvm_path` was meant to be used in a very specific way, not as a general "give me the filesystem path to the jvm" - even though it usually does just that. So CDS use of this method reflects an incomplete understanding of what it does. It works fine until the altjvm property is set but then gives this unexpected lib/hotspot/libjvm.so path.

IIUC when running gtests the gtestlauncher is co-located in the same directory as its custom libjvm.so file. So the launcher knows how to load that custom libjvm. The launcher is pointed to a real JDK so that everything else works as expected. The gtest launcher sets the altjvm property but I don't understand why it does that - we are using a custom libjvm.so but we are also using a custom launcher, so this is not the scenario for which `-XXaltjvm=xxx` was designed.

AFAICS the is_altjvm property is only used by this `os::jvm_path` logic, so perhaps the solution is to simply not set that property for the gtestlauncher?

Otherwise for CDS to work it needs to locate the classes.jsa in the real JDK that the gtestlauncher was told to use. So maybe a solution there is for the gtestlauncher to explicitly set the path to the CDS archive using the value of the JDK it is given. It will still have to know about the jvm-type (server, client etc).

-------------

PR Comment: https://git.openjdk.org/jdk/pull/23758#issuecomment-2683612195