From ahmedmuhsin at microsoft.com Fri Jul 1 20:56:14 2022 From: ahmedmuhsin at microsoft.com (Ahmed Muhsin) Date: Fri, 1 Jul 2022 20:56:14 +0000 Subject: State of Windows AARCH64 port after 8284161: Implementation of Virtual Threads (Preview) In-Reply-To: References: Message-ID: Hey everyone, We recently performed an analysis of Loom's impact on the Win/ARM64 port and found only one test that can be attributed to the VirtualThreads commit: jdk/jshell/ToolBasicTest.java (langtools: tier2). I've included more details about our testing below. There are currently no blockers for the JDK 19 release. Please let us know what you think. Thanks, Ahmed Muhsin Test procedure and results: Tests: * JDK: tier1, tier2, tier3, jdk_loom * Hotspot: tier1, tier2, tier3, hotspot_loom * Langtools: tier1, tier2, tier3 Comparison 1: JDK tip (commit: e0382c5) - VritualThreads on vs off * Result: Turning VirtualThreads on led to 5 new jtreg test failures: * java/lang/Thread/virtual/PreviewFeaturesNotEnabled.java * java/lang/invoke/defineHiddenClass/PreviewHiddenClass.java * jdk/modules/etc/DefaultModules.java * tools/javac/launcher/GetResourceTest.java * tools/javac/launcher/SourceLauncherTest.java * Analysis: The new test failures are not concerning for these reasons: * The failures are not due to any VirtualThreads related changes. All the tests failed because the test logic does not account for the possibility that the test JVM might run with the flag --enable-preview . The tests can be split into two groups: * Tests that check the status of the flag --enable-preview and always assume that the test JVM does not use it. * java/lang/Thread/virtual/PreviewFeaturesNotEnabled.java * java/lang/invoke/defineHiddenClass/PreviewHiddenClass.java * Tests that compile code but since they did not account for the case where the test JVM has preview features turned on, they run into this error: error: --enable-preview must be used with --source * jdk/modules/etc/DefaultModules.java * tools/javac/launcher/GetResourceTest.java * tools/javac/launcher/SourceLauncherTest.java * All the failing tests pass when --enable-preview is not used. Comparison 2: JDK tip right before VirtualThreads (commit: 521253) vs right after (commit: 9583e36 + ac274c4) * Results: There was only one new test failure that wasn't fixed in subsequent commits: * jdk/jshell/ToolBasicTest.java * Analysis: The new test failure pops up when VirtualThreads is both on and off. * Test output: https://pastebin.com/Zj7BYzSx * A similar failure was reported in the past (before VritualThreads was committed) on another platform: https://bugs.openjdk.org/browse/JDK-8272744 Conclusion * There is only 1 new test failure that can be attributed to the VirtualThreads commit that hasn't been already fixed is: * jdk/jshell/ToolBasicTest.java -------------- next part -------------- An HTML attachment was scrubbed... URL: From richard.reingruber at sap.com Mon Jul 11 12:07:50 2022 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Mon, 11 Jul 2022 12:07:50 +0000 Subject: State of the ppc64le port of JEP 425: Virtual Threads (Preview) In-Reply-To: References: Message-ID: Hi, the port passes now the basic continuation tests jdk/jdk/internal/vm/Continuation/Basic.java with UseContinuationFastPath disabled. Actually all tests in hotspot_loom and jdk_loom succeed except for 2 of them where the held monitor count is wrong (potentially caused by [1]) and another one with a method that is not compileable. The current version can again be found here: https://github.com/reinrich/loom/commits/ppc_port Richard. [1] https://bugs.openjdk.org/browse/JDK-8286957 From: Reingruber, Richard Date: Thursday, 2. June 2022 at 13:38 To: jdk-dev , porters-dev at openjdk.java.net , loom-dev at openjdk.java.net Subject: State of the ppc64le port of JEP 425: Virtual Threads (Preview) Hi, I learned today that preview features _must_ be implemented by a port in an OpenJDK release [1]. Unfortunately I have to inform you that I don't think the ppc64le port I'm currently working on will be ready in the JDK19 time frame. When I started the work (Jan. or Dec. I think) I expected to finish it before summer. Even after the last status update [2] I thought I could make it. But with the difficulties I still experience and being 6-8 weeks out of office in summer it is now rather unlikely. And until this morning myself (and actually also my colleagues) assumed this would only be a minor issue. Current Status of the Port: * UseContinuationFastPath is disabled * Basic tests where sequences of interpreted and compiled frames with quite some variations are frozen and thawed succeed. * GC with stack chunks on the java heap succeed. * Basic exception handling tests succeed. * Basic tests exercising compiled java calls with stack arguments succeed but need to be revisited because there are issues. [3] is a selection of test cases that I use in development. [4] is the most recent version of the ppc64le port Main Technical Problems * Shared code makes use of the 'unextended sp' of java frames. This breaks the platform abstraction as it makes assumptions on where to find, e.g., stack arguments relative to the unextended sp. * There are non-obvious interdependencies in the code which make it difficult to fix an issue. In an attempt to fix a problem I often have regressions because I missed adaptations of dependent parts. And then it it is extremely tedious to find the cause of the regression running tests and analyzing very long trace output. * Currently I see that the handling of stack arguments of compiled java methods works in quite some cases (see [3]) but there are cases where it doesn't. Trying alternative approaches means going through the tedious and time consuming process described above. * Lack of documentation. Heavily templatized implementation. These problems (except the last) could not be foreseen. From a high level the port simply needs to copy frames between stack and heap and provide some assembler glue code. As I know now it is actually a high effort to get the deatils tuned right. Thanks, Richard. [1] Ports _must_ implement preview features in thread "What should the relationship between ports and developers of large projects be?" https://mail.openjdk.java.net/pipermail/jdk-dev/2022-May/006635.html [2] State of the ppc64le loom port as of April 14 https://mail.openjdk.java.net/pipermail/loom-dev/2022-April/004197.html [3] BasicExp.java tests driving development of the port https://github.com/reinrich/loom/blob/3286bc8b72401dbccac59c994919fc425a51cb52/test/jdk/jdk/internal/vm/Continuation/BasicExp.java [4] Most recent version of the ppc64le loom port https://github.com/reinrich/loom/commits/ppc_port -------------- next part -------------- An HTML attachment was scrubbed... URL: From richard.reingruber at sap.com Fri Jul 22 14:41:54 2022 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Fri, 22 Jul 2022 14:41:54 +0000 Subject: State of the ppc64le port of JEP 425: Virtual Threads (Preview) In-Reply-To: References: Message-ID: Hi, UseContinuationFastPath is now working and enabled by default. The current verion should be close to final one modulo cleanups. Testing: ============================== Test summary ============================== TEST TOTAL PASS FAIL ERROR jtreg:test/hotspot/jtreg:hotspot_loom 63 63 0 0 >> jtreg:test/jdk:jdk_loom 213 209 4 0 << ============================== TEST FAILURE Failures are because of wrong monitor count ([1]?) except one where the method Fuzz.com_pin(int,int) is not compilable. The rather important test jdk/jdk/internal/vm/Continuation/Basic.java succeeds. Next will be cleanup, rebasing, testing. Also I want to split off some shared code changes to reduce the size of the port. There is already a really small one out for review https://bugs.openjdk.org/browse/JDK-8289925 (ping?) The majority of shared code changes are required because on ppc the metadata for call linkage (return address, stack pointer) is stored at the top of the caller frame. So it is external to the callee frame as the stack arguments are. The size of that metadata has to be accounted for, very much like the size of the stack arguments. On x86 the metadata is stored in the callee frame. It is not part of the caller-callee overlap. I hope to start the review of the port in September after summer holidays. The current version can again be found here: https://github.com/reinrich/loom/commits/ppc_port Richard. [1] https://bugs.openjdk.org/browse/JDK-8286957 __________________________________________________________________ From: Reingruber, Richard Sent: Monday, July 11, 2022 14:07 To: hotspot-dev at openjdk.java.net ; porters-dev at openjdk.java.net Subject: Re: State of the ppc64le port of JEP 425: Virtual Threads (Preview) ? Hi, ? the port passes now the basic continuation tests jdk/jdk/internal/vm/Continuation/Basic.java with UseContinuationFastPath disabled. ? Actually all tests in hotspot_loom and jdk_loom succeed except for 2 of them where the held monitor count is wrong (potentially caused by [1]) and another one with a method that is not compileable. ? The current version can again be found here: https://github.com/reinrich/loom/commits/ppc_port ? Richard. ? [1] https://bugs.openjdk.org/browse/JDK-8286957 ? __________________________________________________________________ From: Reingruber, Richard Date: Thursday, 2. June 2022 at 13:38 To: jdk-dev , porters-dev at openjdk.java.net , loom-dev at openjdk.java.net Subject: State of the ppc64le port of JEP 425: Virtual Threads (Preview) Hi, I learned today that preview features _must_ be implemented by a port in an OpenJDK release [1]. Unfortunately I have to inform you that I don't think the ppc64le port I'm currently working on will be ready in the JDK19 time frame. When I started the work (Jan. or Dec. I think) I expected to finish it before summer. Even after the last status update [2] I thought I could make it. But with the difficulties I still experience and being 6-8 weeks out of office in summer it is now rather unlikely. And until this morning myself (and actually also my colleagues) assumed this would only be a minor issue. Current Status of the Port: * UseContinuationFastPath is disabled * Basic tests where sequences of interpreted and compiled frames with quite some ? variations are frozen and thawed succeed. ? * GC with stack chunks on the java heap succeed. * Basic exception handling tests succeed. * Basic tests exercising compiled java calls with stack arguments succeed but ? need to be revisited because there are issues. [3] is a selection of test cases that I use in development. [4] is the most recent version of the ppc64le port Main Technical Problems * Shared code makes use of the 'unextended sp' of java frames. This breaks the ? platform abstraction as it makes assumptions on where to find, e.g., stack ? arguments relative to the unextended sp. ? * There are non-obvious interdependencies in the code which make it difficult to ? fix an issue. In an attempt to fix a problem I often have regressions because ? I missed adaptations of dependent parts. And then it it is extremely tedious ? to find the cause of the regression running tests and analyzing very long ? trace output. * Currently I see that the handling of stack arguments of compiled java methods ? works in quite some cases (see [3]) but there are cases where it ? doesn't. Trying alternative approaches means going through the tedious and ? time consuming process described above. ? * Lack of documentation. Heavily templatized implementation. These problems (except the last) could not be foreseen. From a high level the port simply needs to copy frames between stack and heap and provide some assembler glue code. As I know now it is actually a high effort to get the deatils tuned right. Thanks, Richard. [1] Ports _must_ implement preview features in thread "What should the relationship between ports and developers of large projects be?" ??? https://mail.openjdk.java.net/pipermail/jdk-dev/2022-May/006635.html [2] State of the ppc64le loom port as of April 14 ??? https://mail.openjdk.java.net/pipermail/loom-dev/2022-April/004197.html [3] BasicExp.java tests driving development of the port ??? https://github.com/reinrich/loom/blob/3286bc8b72401dbccac59c994919fc425a51cb52/test/jdk/jdk/internal/vm/Continuation/BasicExp.java [4] Most recent version of the ppc64le loom port ??? https://github.com/reinrich/loom/commits/ppc_port