From richard.reingruber at sap.com Thu Jun 2 11:38:19 2022 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Thu, 2 Jun 2022 11:38:19 +0000 Subject: State of the ppc64le port of JEP 425: Virtual Threads (Preview) Message-ID: Hi, I learned today that preview features _must_ be implemented by a port in an OpenJDK release [1]. Unfortunately I have to inform you that I don't think the ppc64le port I'm currently working on will be ready in the JDK19 time frame. When I started the work (Jan. or Dec. I think) I expected to finish it before summer. Even after the last status update [2] I thought I could make it. But with the difficulties I still experience and being 6-8 weeks out of office in summer it is now rather unlikely. And until this morning myself (and actually also my colleagues) assumed this would only be a minor issue. Current Status of the Port: * UseContinuationFastPath is disabled * Basic tests where sequences of interpreted and compiled frames with quite some variations are frozen and thawed succeed. * GC with stack chunks on the java heap succeed. * Basic exception handling tests succeed. * Basic tests exercising compiled java calls with stack arguments succeed but need to be revisited because there are issues. [3] is a selection of test cases that I use in development. [4] is the most recent version of the ppc64le port Main Technical Problems * Shared code makes use of the 'unextended sp' of java frames. This breaks the platform abstraction as it makes assumptions on where to find, e.g., stack arguments relative to the unextended sp. * There are non-obvious interdependencies in the code which make it difficult to fix an issue. In an attempt to fix a problem I often have regressions because I missed adaptations of dependent parts. And then it it is extremely tedious to find the cause of the regression running tests and analyzing very long trace output. * Currently I see that the handling of stack arguments of compiled java methods works in quite some cases (see [3]) but there are cases where it doesn't. Trying alternative approaches means going through the tedious and time consuming process described above. * Lack of documentation. Heavily templatized implementation. These problems (except the last) could not be foreseen. From a high level the port simply needs to copy frames between stack and heap and provide some assembler glue code. As I know now it is actually a high effort to get the deatils tuned right. Thanks, Richard. [1] Ports _must_ implement preview features in thread "What should the relationship between ports and developers of large projects be?" https://mail.openjdk.java.net/pipermail/jdk-dev/2022-May/006635.html [2] State of the ppc64le loom port as of April 14 https://mail.openjdk.java.net/pipermail/loom-dev/2022-April/004197.html [3] BasicExp.java tests driving development of the port https://github.com/reinrich/loom/blob/3286bc8b72401dbccac59c994919fc425a51cb52/test/jdk/jdk/internal/vm/Continuation/BasicExp.java [4] Most recent version of the ppc64le loom port https://github.com/reinrich/loom/commits/ppc_port From richard.reingruber at sap.com Thu Jun 2 17:09:53 2022 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Thu, 2 Jun 2022 17:09:53 +0000 Subject: State of the ppc64le port of JEP 425: Virtual Threads (Preview) In-Reply-To: <1657907744.901877.1654188604109.JavaMail.zimbra@u-pem.fr> References: <1657907744.901877.1654188604109.JavaMail.zimbra@u-pem.fr> Message-ID: Hi, > are you aware of JDK-8287496 ? > https://bugs.openjdk.java.net/browse/JDK-8287496 Yes I noticed it. Alan recommended to test it with jdk_loom and hotspot_loom tests. I have done that just now on a ppc64le box. The tests in jdk_loom succeed. There are test failures in hotspot_loom but all in serviceability/jvmti/(events|vthread). I understand that this is to be expected and will be handled in a follow-up by adding "@requires vm.continuations" to these tests. So Alan's fallback seems to work on ppc64le which is good to know :) Thanks, Richard. From: Remi Forax Date: Thursday, 2. June 2022 at 18:50 To: Reingruber, Richard Cc: jdk-dev , porters-dev at openjdk.java.net , loom-dev Subject: Re: State of the ppc64le port of JEP 425: Virtual Threads (Preview) Hi Richard, are you aware of JDK-8287496 ? https://bugs.openjdk.java.net/browse/JDK-8287496 It should help you to get a working first implementation for jdk 19. regards, R?mi ----- Original Message ----- > From: "Reingruber, Richard" > To: "jdk-dev" , porters-dev at openjdk.java.net, "loom-dev" > Sent: Thursday, June 2, 2022 1:38:19 PM > Subject: State of the ppc64le port of JEP 425: Virtual Threads (Preview) > Hi, > > I learned today that preview features _must_ be implemented by a port in an > OpenJDK release [1]. > > Unfortunately I have to inform you that I don't think the ppc64le port I'm > currently working on will be ready in the JDK19 time frame. > > When I started the work (Jan. or Dec. I think) I expected to finish it before > summer. Even after the last status update [2] I thought I could make it. But > with the difficulties I still experience and being 6-8 weeks out of office in > summer it is now rather unlikely. And until this morning myself (and actually > also my colleagues) assumed this would only be a minor issue. > > Current Status of the Port: > > * UseContinuationFastPath is disabled > > * Basic tests where sequences of interpreted and compiled frames with quite some > variations are frozen and thawed succeed. > > * GC with stack chunks on the java heap succeed. > > * Basic exception handling tests succeed. > > * Basic tests exercising compiled java calls with stack arguments succeed but > need to be revisited because there are issues. > > [3] is a selection of test cases that I use in development. > [4] is the most recent version of the ppc64le port > > Main Technical Problems > > * Shared code makes use of the 'unextended sp' of java frames. This breaks the > platform abstraction as it makes assumptions on where to find, e.g., stack > arguments relative to the unextended sp. > > * There are non-obvious interdependencies in the code which make it difficult to > fix an issue. In an attempt to fix a problem I often have regressions because > I missed adaptations of dependent parts. And then it it is extremely tedious > to find the cause of the regression running tests and analyzing very long > trace output. > > * Currently I see that the handling of stack arguments of compiled java methods > works in quite some cases (see [3]) but there are cases where it > doesn't. Trying alternative approaches means going through the tedious and > time consuming process described above. > > * Lack of documentation. Heavily templatized implementation. > > These problems (except the last) could not be foreseen. From a high level the > port simply needs to copy frames between stack and heap and provide some > assembler glue code. As I know now it is actually a high effort to get the > deatils tuned right. > > Thanks, Richard. > > [1] Ports _must_ implement preview features in thread "What should the > relationship between ports and developers of large projects be?" > https://mail.openjdk.java.net/pipermail/jdk-dev/2022-May/006635.html > [2] State of the ppc64le loom port as of April 14 > https://mail.openjdk.java.net/pipermail/loom-dev/2022-April/004197.html > [3] BasicExp.java tests driving development of the port > https://github.com/reinrich/loom/blob/3286bc8b72401dbccac59c994919fc425a51cb52/test/jdk/jdk/internal/vm/Continuation/BasicExp.java > [4] Most recent version of the ppc64le loom port > https://github.com/reinrich/loom/commits/ppc_port -------------- next part -------------- An HTML attachment was scrubbed... URL: From shade at redhat.com Wed Jun 15 16:54:19 2022 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 15 Jun 2022 18:54:19 +0200 Subject: State of x86_32 port of JEP 425: Virtual Threads (Preview) Message-ID: Hi, Adding to the pile of port reports for Loom. x86_32 port work is tracked under: https://bugs.openjdk.org/browse/JDK-8286642 ...and relevant changes are in this tree: https://github.com/shipilev/jdk/tree/JDK-8286642-loom-x86-32-port Current status for that port is: - Interpreter and C1 paths are working fine; hotspot_loom and jdk_loom pass reliably; - C2 seems to corrupt the stack somehow, I suspect during the deopt, after running for quite a while; the failures are all over the place, but they are only with C2; - New C2 intrinsics are known to break tests, disabling them makes more tests pass. (I *suspect* that somewhere we stash the "current" thread onto stack, and then remount to another thread, which makes "current" thread obsolete. Assuming current thread does not change is quite pervasive, and saving "current thread" on architectures without dedicated thread register is rather common.) Anyhow, the choice between "ship x86_32 without VM continuations" or "ship x86_32 without C2" is obvious: VM continuations are out. So, barring some debugging miracle, I don't expect x86_32 port to be fully ready for JDK 19. The problems I saw during this work go beyond the x86_32 port work, and should be discussed in separate thread, I'll just mention that the existing implementation is hairy and full of subtle details. Alas, the code and comments do not make it clear what those details are, and which are part of the design, which are clever hacks, and which are the concessions to particular architectures. In fact, it is not even clear to me if the "shared" code is actually fully architecture-independent, or just happens to work on x86_64 and AArch64. The current code needs to have significant cleanup, refactoring and (re-)documentation for sane porting/debugging work. This, I think, should be #1 priority, and before that is done and ports are able to catch up, it would be unwise to pile more Loom stuff on. -- Thanks, -Aleksey From shade at redhat.com Thu Jun 16 09:09:18 2022 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 16 Jun 2022 11:09:18 +0200 Subject: State of x86_32 port of JEP 425: Virtual Threads (Preview) In-Reply-To: <9EFE40DD-2B40-41FB-92F5-080C8C0B582C@oracle.com> References: <9EFE40DD-2B40-41FB-92F5-080C8C0B582C@oracle.com> Message-ID: On 6/15/22 20:26, Ron Pressler wrote: > Am I right to understand that to mean that the x86-32 port does fully implement the virtual > threads spec, albeit with the ?fake virtual thread? implementation rather than VM continuations? Yes. Although it is still not clear to me if that implementation even passes JCK. > The current implementation focused on x64 and aarch64, and it is certainly possible that some > assumptions about those architectures crept in. Over time, I expect that contributions from > maintainers of other ports would gradually guide refactoring of the code into a more platform > independent shape. Of course, any port maintainer who wishes to join the project and participate > more actively is more than welcome. Sure, but let me talk about a huge elephant-in-the-room caveat. I firmly believe port maintainers should not be put into position of deciphering the current Loom implementation, guessing the implementation intents, and only then figuring out where those intents are broken for a particular port. The porters job is the second part: looking at clean implementation, having their "Aha, this code/comment says something that does not hold for my arch" moments, and working those out. The first part is the responsibility of Loom developers: readability, documentation clarity, reasonable pre-/post-conditions, extensive verification, etc. I am sure many non-Loom developers would lend a helping hand in that. I, for one, contributed a few rewrites already. But, someone who wrote the original Loom code should take a lead in improving the VM code for much easier understanding and clarity. I suspect a lot of finer details about VM code implementation are still in the heads of original Loom developers, and such details need to be incorporated in the well-structured, well-documented, well-tested code. IMO, that needs to happen before any more serious work is done in VM parts, if you want ports to ever catch up, or indeed the whole project to succeed. The fact that even the x86_32 implementation -- that shares a lot of already implemented code with x86_64! -- proves hard to implement and debug at the moment, must be quite alarming for any other ports work... It worries me a lot, that's for sure. -- Thanks, -Aleksey From shade at redhat.com Thu Jun 16 09:40:54 2022 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 16 Jun 2022 11:40:54 +0200 Subject: State of x86_32 port of JEP 425: Virtual Threads (Preview) In-Reply-To: <03a66ad0-4801-d2b8-35b0-758bc931f1b8@oracle.com> References: <03a66ad0-4801-d2b8-35b0-758bc931f1b8@oracle.com> Message-ID: On 6/16/22 10:38, dean.long at oracle.com wrote: > > > On 6/15/22 9:54 AM, Aleksey Shipilev wrote: >> (I *suspect* that somewhere we stash the "current" thread onto stack, >> and then remount to another thread, which makes "current" thread >> obsolete. Assuming current thread does not change is quite pervasive, >> and saving "current thread" on architectures without dedicated thread >> register is rather common.) > > That does sound like a problem. I noticed that > SharedRuntime::generate_native_wrapper() does this: > > 1554 // We use rdi as a thread pointer because it is callee save and > 1555 // if we load it once it is usable thru the entire wrapper > 1556 const Register thread = rdi; > > This comment seems wrong for loom. Yes, good spot! In fact, most of the uses for thread registers on x86_32 seem to assume that thread register would be clobbered by calls already, *except* when it is in callee-saved register. Luckily, many of those paths are explicitly asserted, see for example MacroAssembler::call_VM_base: // restore the thread (cannot use the pushed argument since arguments // may be overwritten by C code generated by an optimizing compiler); // however can use the register value directly if it is callee saved. if (LP64_ONLY(true ||) java_thread == rdi || java_thread == rsi) { // rdi & rsi (also r15) are callee saved -> nothing to do #ifdef ASSERT guarantee(java_thread != rax, "change this code"); push(rax); { Label L; get_thread(rax); cmpptr(java_thread, rax); jcc(Assembler::equal, L); STOP("MacroAssembler::call_VM_base: rdi not callee saved?"); bind(L); } pop(rax); #endif } else { get_thread(java_thread); } I am going to sprinkle more thread verification everywhere... -- Thanks, -Aleksey From Alan.Bateman at oracle.com Thu Jun 16 11:04:07 2022 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 16 Jun 2022 12:04:07 +0100 Subject: State of x86_32 port of JEP 425: Virtual Threads (Preview) In-Reply-To: References: <9EFE40DD-2B40-41FB-92F5-080C8C0B582C@oracle.com> Message-ID: On 16/06/2022 10:09, Aleksey Shipilev wrote: > On 6/15/22 20:26, Ron Pressler wrote: >> Am I right to understand that to mean that the x86-32 port does fully >> implement the virtual >> threads spec, albeit with the ?fake virtual thread? implementation >> rather than VM continuations? > Yes. Although it is still not clear to me if that implementation even > passes JCK. The intention is that it does. The jtreg tests that exercise the API surface re-run with -XX:-VMContinuations to ensure that the alternative implementation is tested. The only surprising part may be that it lends on the optionality of JVM TI. > > Sure, but let me talk about a huge elephant-in-the-room caveat. > > I firmly believe port maintainers should not be put into position of > deciphering the current Loom implementation, guessing the > implementation intents, and only then figuring out where those intents > are broken for a particular port. The porters job is the second part: > looking at clean implementation, having their "Aha, this code/comment > says something that does not hold for my arch" moments, and working > those out. > > The first part is the responsibility of Loom developers: readability, > documentation clarity, reasonable pre-/post-conditions, extensive > verification, etc. I am sure many non-Loom developers would lend a > helping hand in that. I, for one, contributed a few rewrites already. We haven't had JVMLS or OpenJDK Committers Workshops for 2 years and those are venues where there may have been opportunities to have walk through and discuss the porting challenges. A few recordings of TOIs might have helped to get more people up to speed and maybe more help on cleanup and re-writing the areas of the hotspot code that need attention. I'm not so sure about the "first part" vs. "second part", at least not for the the first one of two ports after x64/aarch64 as there will be issues, leading to improvements for other ports that follow. -Alan From ron.pressler at oracle.com Wed Jun 15 18:26:38 2022 From: ron.pressler at oracle.com (Ron Pressler) Date: Wed, 15 Jun 2022 18:26:38 +0000 Subject: State of x86_32 port of JEP 425: Virtual Threads (Preview) In-Reply-To: References: Message-ID: <9EFE40DD-2B40-41FB-92F5-080C8C0B582C@oracle.com> > On 15 Jun 2022, at 17:54, Aleksey Shipilev wrote: > > Hi, > > Adding to the pile of port reports for Loom. > Hi Aleksey, and thank you for that great work! > > Anyhow, the choice between "ship x86_32 without VM continuations" or "ship x86_32 without C2" is obvious: VM continuations are out. So, barring some debugging miracle, I don't expect x86_32 port to be fully ready for JDK 19. Am I right to understand that to mean that the x86-32 port does fully implement the virtual threads spec, albeit with the ?fake virtual thread? implementation rather than VM continuations? > > In fact, it is not even clear to me if the "shared" code is actually fully architecture-independent, or just happens to work on x86_64 and AArch64. The current code needs to have significant cleanup, refactoring and (re-)documentation for sane porting/debugging work. This, I think, should be #1 priority, and before that is done and ports are able to catch up, it would be unwise to pile more Loom stuff on. > The current implementation focused on x64 and aarch64, and it is certainly possible that some assumptions about those architectures crept in. Over time, I expect that contributions from maintainers of other ports would gradually guide refactoring of the code into a more platform independent shape. Of course, any port maintainer who wishes to join the project and participate more actively is more than welcome. ? Ron From dean.long at oracle.com Thu Jun 16 08:38:25 2022 From: dean.long at oracle.com (dean.long at oracle.com) Date: Thu, 16 Jun 2022 01:38:25 -0700 Subject: State of x86_32 port of JEP 425: Virtual Threads (Preview) In-Reply-To: References: Message-ID: <03a66ad0-4801-d2b8-35b0-758bc931f1b8@oracle.com> On 6/15/22 9:54 AM, Aleksey Shipilev wrote: > (I *suspect* that somewhere we stash the "current" thread onto stack, > and then remount to another thread, which makes "current" thread > obsolete. Assuming current thread does not change is quite pervasive, > and saving "current thread" on architectures without dedicated thread > register is rather common.) That does sound like a problem. I noticed that SharedRuntime::generate_native_wrapper() does this: 1554 // We use rdi as a thread pointer because it is callee save and 1555 // if we load it once it is usable thru the entire wrapper 1556 const Register thread = rdi; This comment seems wrong for loom. dl From ahmedmuhsin at microsoft.com Wed Jun 22 19:45:44 2022 From: ahmedmuhsin at microsoft.com (Ahmed Muhsin) Date: Wed, 22 Jun 2022 19:45:44 +0000 Subject: State of Windows AARCH64 port after 8284161: Implementation of Virtual Threads (Preview) In-Reply-To: References: Message-ID: Hey everyone, We performed an analysis of Loom's impact on Win/ARM64 and found only one test that can be attributed to the VirtualThreads preview commit. That test is, jdk/jshell/ToolBasicTest.java (langtools: tier2). I've included more details about our testing below. There are currently no blockers for the JDK 19 release. Please let us know what you think. Thanks, Ahmed Muhsin Test procedure and results: Tests: * JDK: tier1, tier2, tier3, jdk_loom * Hotspot: tier1, tier2, tier3, hotspot_loom * Langtools: tier1, tier2, tier3 Comparison 1: JDK tip (commit: e0382c5) - VritualThreads on vs off * Result: Turning VirtualThreads on led to 5 new jtreg test failures: * java/lang/Thread/virtual/PreviewFeaturesNotEnabled.java * java/lang/invoke/defineHiddenClass/PreviewHiddenClass.java * jdk/modules/etc/DefaultModules.java * tools/javac/launcher/GetResourceTest.java * tools/javac/launcher/SourceLauncherTest.java * Analysis: The new test failures are not concerning for these reasons: * The failures are not due to any VirtualThreads related changes. All the tests failed because the test logic does not account for the possibility that the test JVM might run with the flag --enable-preview . The tests can be split into two groups: * Tests that check the status of the flag --enable-preview and always assume that the test JVM does not use it. * java/lang/Thread/virtual/PreviewFeaturesNotEnabled.java * java/lang/invoke/defineHiddenClass/PreviewHiddenClass.java * Tests that compile code but since they did not account for the case where the test JVM has preview features turned on, they run into this error: error: --enable-preview must be used with --source * jdk/modules/etc/DefaultModules.java * tools/javac/launcher/GetResourceTest.java * tools/javac/launcher/SourceLauncherTest.java * All the failing tests pass when --enable-preview is not used. Comparison 2: JDK tip right before VirtualThreads (commit: 521253) vs right after (commit: 9583e36 + ac274c4) * Results: There was only one new test failure that wasn't fixed in subsequent commits: * jdk/jshell/ToolBasicTest.java * Analysis: The new test failure pops up when VirtualThreads is both on and off. * Test output: https://urldefense.com/v3/__https://pastebin.com/Zj7BYzSx__;!!ACWV5N9M2RV99hQ!Om_5WP1WqudVvl8iV8DDs9YrDhbHLmbrCPBW_EI-oiS4DDIRfmEBpkqQtwES26qtikN5DFtyXWVQYc7zYy-tQ3E2Z5Q$ * A similar failure was reported in the past (before VritualThreads was committed) on another platform: https://bugs.openjdk.org/browse/JDK-8272744 Conclusion * There is only 1 new test failure that can be attributed to the VirtualThreads commit that hasn't been already fixed is: * jdk/jshell/ToolBasicTest.java -------------- next part -------------- An HTML attachment was scrubbed... URL: