From aph at redhat.com Tue Oct 1 15:19:41 2019 From: aph at redhat.com (Andrew Haley) Date: Tue, 1 Oct 2019 16:19:41 +0100 Subject: Scoped values: API design Message-ID: <0166427e-1f57-e94c-f165-20ac9e8b59dc@redhat.com> I've been experimenting with alternatives to try to design a reasonable API, with a view to making it as versatile as necessary and as small as possible. After a lot of failed attempts I've come up with this. A class ScopedBinder, which is a tuple of (Scoped name, T value). A class ScopedCollection, which is an immutable collection of ScopedBinder. ScopedCollection implements run(Runnable) and call(Callable). A static Scoped method is(T value), which produces an (immutable) instance of ScopedBinder. A static scoped method collect(ScopedBinder ... values) which produces an (immutable) instance of ScopedCollection. Putting all this together, if you want to bind some scoped values and call some code, you'll: Scoped.collect(myVar.is(999), myInt.is(-1)).run( () -> System.out.println(myVar.get() + ", " + myInt.get()) ); A framework which needs some scoped values to be set can do something like this: ScopedCollection getBindings() { return Scoped.collect(myVar.is(999), myInt.is(-1)); } and the caller would do: myFramework.getBindings().run( () -> myFramework.start(someArgs) ); It makes sense also to have a shorthand for the single-argument form: myVar.is(999).run( () -> System.out.println(myVar.get()) ); We could keep the bind() methods from my earlier proposal, but I'm not sure that they offer anything much. With regard to the earlier AutoCloseable variants of the API we could also do something like try (var x = Scoped.collect(myVar.is(999), myInt.is(-1)).bind()) { System.out.println(myVar.get() + ", " + myInt.get()); } but again I'm not sure it offers very much over the Runnable and Callable versions and it's far more open to abuse because programmers aren't forced to use it in a well-structured way. Comments needed! -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From ron.pressler at oracle.com Tue Oct 1 20:55:00 2019 From: ron.pressler at oracle.com (Ron Pressler) Date: Tue, 1 Oct 2019 21:55:00 +0100 Subject: Scoped values: API design In-Reply-To: <0166427e-1f57-e94c-f165-20ac9e8b59dc@redhat.com> References: <0166427e-1f57-e94c-f165-20ac9e8b59dc@redhat.com> Message-ID: ? My initial, superficial impression is that the proposal has two prominent features: 1. Allow library code to set up multiple bindings. 2. Support both a lambda-based and TWR-based scopes. I think that the latter feature makes the API too rich, and if we can get a good TWR API, wrapping it with a lambda-based one is trivial, but not the other way around. Moreover, AutoCloseables can be easily composed (to close multiple bindings) in a way that makes a public ScopedCollection class redundant. Perhaps the right way is taking it one step at a time, and focus on a TWR approach first. Ron On 1 October 2019 at 16:21:07, Andrew Haley (aph at redhat.com(mailto:aph at redhat.com)) wrote: > I've been experimenting with alternatives to try to design a > reasonable API, with a view to making it as versatile as necessary and > as small as possible. After a lot of failed attempts I've come up with > this. > > > A class ScopedBinder, which is a tuple of (Scoped name, T value). > > A class ScopedCollection, which is an immutable collection of > ScopedBinder. ScopedCollection implements run(Runnable) and > call(Callable). > > A static Scoped method is(T value), which produces an (immutable) > instance of ScopedBinder. > > A static scoped method collect(ScopedBinder ... values) which > produces an (immutable) instance of ScopedCollection. > > > Putting all this together, if you want to bind some scoped values > and call some code, you'll: > > Scoped.collect(myVar.is(999), myInt.is(-1)).run( > () -> System.out.println(myVar.get() + ", " + myInt.get()) > ); > > A framework which needs some scoped values to be set can do something > like this: > > ScopedCollection getBindings() { > return Scoped.collect(myVar.is(999), myInt.is(-1)); > } > > and the caller would do: > > myFramework.getBindings().run( > () -> myFramework.start(someArgs) > ); > > It makes sense also to have a shorthand for the single-argument form: > > myVar.is(999).run( > () -> System.out.println(myVar.get()) > ); > > > We could keep the bind() methods from my earlier proposal, but I'm not > sure that they offer anything much. > > With regard to the earlier AutoCloseable variants of the API we could > also do something like > > try (var x = Scoped.collect(myVar.is(999), myInt.is(-1)).bind()) { > System.out.println(myVar.get() + ", " + myInt.get()); > } > > but again I'm not sure it offers very much over the Runnable and > Callable versions and it's far more open to abuse because programmers > aren't forced to use it in a well-structured way. > > Comments needed! > > -- > Andrew Haley (he/him) > Java Platform Lead Engineer > Red Hat UK Ltd. > https://keybase.io/andrewhaley > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From john.r.rose at oracle.com Tue Oct 1 21:02:46 2019 From: john.r.rose at oracle.com (John Rose) Date: Tue, 1 Oct 2019 14:02:46 -0700 Subject: Scoped values: API design In-Reply-To: References: <0166427e-1f57-e94c-f165-20ac9e8b59dc@redhat.com> Message-ID: <39C873D5-64F0-479B-BB31-1AE5A3D93A49@oracle.com> On Oct 1, 2019, at 1:55 PM, Ron Pressler wrote: > > 2. Support both a lambda-based and TWR-based scopes. > > I think that the latter feature makes the API too rich, and if we can get a good TWR API, > wrapping it with a lambda-based one is trivial, but not the other way around. I agree that it?s better to start with one or the other of TWR or lambdas. From an optimization point of view (I know that?s in the future) the lambda based form will be easier to optimize because its use cases are more constrained. A problem with TWR APIs is that they can be used outside of TWR, and the library has to cope with that somehow, even if it?s just error detection (which by itself is a hard problem). From aph at redhat.com Wed Oct 2 08:12:13 2019 From: aph at redhat.com (Andrew Haley) Date: Wed, 2 Oct 2019 09:12:13 +0100 Subject: Scoped values: API design In-Reply-To: <39C873D5-64F0-479B-BB31-1AE5A3D93A49@oracle.com> References: <0166427e-1f57-e94c-f165-20ac9e8b59dc@redhat.com> <39C873D5-64F0-479B-BB31-1AE5A3D93A49@oracle.com> Message-ID: On 10/1/19 10:02 PM, John Rose wrote: > On Oct 1, 2019, at 1:55 PM, Ron Pressler > wrote: >> >> 2. Support both a lambda-based and TWR-based scopes. >> >> I think that the latter feature makes the API too rich, and if we >> can get a good TWR API, wrapping it with a lambda-based one is >> trivial, but not the other way around. That's true. > I agree that it?s better to start with one or the other of TWR or > lambdas. From an optimization point of view (I know that?s in the > future) the lambda based form will be easier to optimize because its > use cases are more constrained. Yep. > A problem with TWR APIs is that they can be used outside of TWR, and > the library has to cope with that somehow, even if it?s just error > detection (which by itself is a hard problem). The latter is my primary concern. The try-with-resources version exposes the possibility of non-nested use, which is hard to cope with. It is also harder to specify without exposing internals. I'd like not to have to think about it. But I'm not going to go to the stake on this one: we can use TWR for now and defer the final decision, even though I'm sure which way it'll go in the end. ;-) -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From ron.pressler at oracle.com Wed Oct 2 19:58:05 2019 From: ron.pressler at oracle.com (Ron Pressler) Date: Wed, 2 Oct 2019 20:58:05 +0100 Subject: Result: New Loom Committer: Andrew Haley Message-ID: Voting for Andrew Haley [1] is now closed.? Yes: 6 Veto: 0 Abstain: 0 According to the Bylaws definition of Lazy Consensus, this is ? sufficient to approve the nomination. ? Ron ? [1] https://mail.openjdk.java.net/pipermail/loom-dev/2019-September/000714.html From jigga at jigga.pl Sat Oct 5 18:57:56 2019 From: jigga at jigga.pl (=?UTF-8?Q?Arkadiusz_Gasi=C5=84ski?=) Date: Sat, 5 Oct 2019 21:57:56 +0300 Subject: Submitting bugs in loom branch In-Reply-To: References: <20190917100224.GB23956@rbackman> Message-ID: Hi, Has anyone had a chance to look into that? I do realize that the setup required to reproduce the issue is a bit burdensome and I'll try to prepare something simpler in the coming days, but essentially at the moment the VM in which the load-tested service runs crashes every time when the request rate is around 10 per second and above. Interestingly, with the latest changes in the fibers branch, the current thread in the generated hs_err_pid* files is now *fiber-carrier-thread-X* rather than a GC thread. Thanks, Arek On Sat, Sep 21, 2019 at 10:46 PM Arkadiusz Gasi?ski wrote: > Hi, > > A little update from my side on this. I made some tweaks in the code, the > most important, I guess, is increasing the number of executor's threads > > (I was unable to reproduce the issue with single-threaded executor). > > Moreover, I updated tomcat's SocketProcessorBase run method implementation > as > the previous one was rather naive. > > With all these changes, the workshop-service VM crashes almost every time > it is hit with the attack script. > > I added some more hs_err* > > files to the repo. > > Arek > > > On Tue, Sep 17, 2019 at 8:29 PM Arkadiusz Gasi?ski wrote: > >> Well, ok then, will try to describe here how to reproduce the issue. >> >> To give you some context, I wanted to test how Fiber friendly Tomcat is >> (actually spring boot app running with embedded Tomcat). Turned out it is >> not that friendly when it comes to Fibers as all HTTP request processing is >> done within the synchronized block. There's this >> NioEndpoint$SocketProcessor >> >> class that extends from SocketProcessorBase >> >> (Runnable), whose whole run method is within synchronized block, which, if >> I understand correctly, pins http threads. >> >> So I forked tomcat and made the following change >> >> in the SocketProcessorBase class, built it locally and it appears that it >> did the trick - Tomcat became Fiber friendly :) >> >> So how did I tested it... >> >> I have this sample github project >> I created some time ago to experiment a bit with Fibers. It consists of 3 >> simple microservices: discovery-service, echo-service and workshop-service. >> The idea is to have more realistic setup than just an endpoint that sleeps >> the current request serving thread for some time, so in this case you call >> the workshop-service (either http://localhost:8080/loom/test - request >> is processed in Fiber or http://localhost:8081/loom/test - request is >> processed in Thread), which in turn uses discovery-service to look up >> echo-service (it starts on some random port) and then calls its >> /delay/{timeout} endpoint, where timeout is between 0 and 1000 ms. The >> echo-service echoes back the request with the given delay. >> >> Before you actually checkout this project, you first need to clone my >> Tomcat's fork and build it locally, by calling "*ant deploy*" and then "*ant >> embed*". You then need to copy the following jars from *output/embed* dir >> to your local maven repo. You also need to append the version number >> (9.0.27) to these jars as the build does not do it. >> >> - tomcat-embed-core.jar, rename to *tomcat-embed-core-9.0.27.jar* and >> copy to ~/.m2/repository/org/apache/tomcat/embed/tomcat-embed-core/9.0.27 >> (or wherever your local maven repo is) >> - tomcat-embed-el.jar, rename to *tomcat-embed-el-9.0.27.jar* and >> copy to ~/.m2/repository/org/apache/tomcat/embed/tomcat-embed-el/9.0.27 >> - tomcat-embed-websocket.jar, rename to >> *tomcat-embed-websocket-9.0.27.jar* and copy >> to ~/.m2/repository/org/apache/tomcat/embed/tomcat-embed-websocket/9.0.27 >> >> This sample project of mine is a gradle project, so you also need to make >> sure that gradle is configured to use loom build, I have the following in >> my cat ~/.gradle/gradle.properties: >> >> >> org.gradle.java.home=/Library/Java/JavaVirtualMachines/jdk-14.jdk/Contents/Home >> >> where obviously jdk-14.jdk is a loom build. >> >> You should be good to checkout the project now, and once it's done, you >> should start all 3 services, preferably in separate terminals like so: >> >> ./gradlew discovery-service:bootRun >> ./gradlew echo-service:bootRun >> ./gradlew workshop-service:bootRun >> >> Once started, you can then throw some load at workshop-service using >> provided attack.sh script (for it to work you actually need to have >> Vegeta installed locally). You call >> the script with either fibers or threads argument. As already mentioned >> above, this service has 2 connectors, namely: >> >> - 8080 - request processed in Fiber >> - 8081 - request processed in Thread >> >> Both connectors are configured with single threaded executors ( >> SimpleExecutor >> >> class). >> >> The single-threaded fiber connector works with 100% success rate with the >> rate up to about 375 requests per second on my machine. The thread based >> connector is obviously waaay less performant. >> >> jiggbook-1:loom-academy jigga$ ./attack.sh fibers >>> Requests [total, rate] 45000, 375.00 >>> Duration [total, attack, wait] 2m0.863797902s, 2m0.000299s, >>> 863.498902ms >>> Latencies [mean, 50, 95, 99, max] 505.184513ms, 503.86826ms, >>> 957.762176ms, 996.938898ms, 1.056817859s >>> Bytes In [total, mean] 10165188, 225.89 >>> Bytes Out [total, mean] 0, 0.00 >>> Success [ratio] 100.00% >>> Status Codes [code:count] 200:45000 >>> Error Set: >> >> >> So the issues I'm actually facing are that the VM in which >> workshop-service is running occasionally crashes. It does not happen on >> every run, but happened several times today. I included two hs_err* files >> in >> the repo. I believe that this may be GC-related issue as the Current thread >> is always GCTaskThread. >> >> I updated and built the jdk loom on the 2nd of September, have not >> rebuilt since then. >> >> Hope that you'll be able to reproduce it given the above info. >> >> If you need additional info, just let me know. >> >> Thanks, >> Arek >> >> On Tue, Sep 17, 2019 at 12:03 PM Rickard B?ckman < >> rickard.backman at oracle.com> wrote: >> >>> Feel free to report them here right now. >>> If you can provide a reproducer it's great! >>> >>> Thank you! >>> >>> /R >>> >>> On 09/17, Arkadiusz Gasi?ski wrote: >>> > Hi, >>> > >>> > Where do I submit bug reports related to loom-JVM crashes? >>> > >>> > The JVM prints this when it crashes: >>> > >>> > # >>> > # If you would like to submit a bug report, please visit: >>> > # http://bugreport.java.com/bugreport/crash.jsp >>> > # >>> > >>> > But I just wonder if it's not too early for loom crashes to be reported >>> > there? >>> > >>> > Thanks, >>> > Arek >>> >> From ron.pressler at oracle.com Sat Oct 5 19:11:14 2019 From: ron.pressler at oracle.com (Ron Pressler) Date: Sat, 5 Oct 2019 20:11:14 +0100 Subject: Submitting bugs in loom branch In-Reply-To: References: <20190917100224.GB23956@rbackman> Message-ID: Hi. We appreciate these bug reports, and would encourage you to submit more if you find them (and simple reproductions are especially appreciated), but the implementation is undergoing? some major changes, so we may prioritise some new work over looking into bugs. So while we are collecting the reports, they may not be addressed in a timely manner. Thank you again ? for reporting issues as well as understanding. Ron On 5 October 2019 at 19:58:42, Arkadiusz Gasi?ski (jigga at jigga.pl) wrote: Hi, Has anyone had a chance to look into that? I do realize that the setup required to reproduce the issue is a bit burdensome and I'll try to prepare something simpler in the coming days, but essentially at the moment the VM in which the load-tested service runs crashes every time when the request rate is around 10 per second and above. Interestingly, with the latest changes in the fibers branch, the current thread in the generated hs_err_pid* files is now *fiber-carrier-thread-X* rather than a GC thread. Thanks, Arek On Sat, Sep 21, 2019 at 10:46 PM Arkadiusz Gasi?ski wrote: > Hi, > > A little update from my side on this. I made some tweaks in the code, the > most important, I guess, is increasing the number of executor's threads > > (I was unable to reproduce the issue with single-threaded executor). > > Moreover, I updated tomcat's SocketProcessorBase run method implementation > as > the previous one was rather naive. > > With all these changes, the workshop-service VM crashes almost every time > it is hit with the attack script. > > I added some more hs_err* > > files to the repo. > > Arek > > > On Tue, Sep 17, 2019 at 8:29 PM Arkadiusz Gasi?ski wrote: > >> Well, ok then, will try to describe here how to reproduce the issue. >> >> To give you some context, I wanted to test how Fiber friendly Tomcat is >> (actually spring boot app running with embedded Tomcat). Turned out it is >> not that friendly when it comes to Fibers as all HTTP request processing is >> done within the synchronized block. There's this >> NioEndpoint$SocketProcessor >> >> class that extends from SocketProcessorBase >> >> (Runnable), whose whole run method is within synchronized block, which, if >> I understand correctly, pins http threads. >> >> So I forked tomcat and made the following change >> >> in the SocketProcessorBase class, built it locally and it appears that it >> did the trick - Tomcat became Fiber friendly :) >> >> So how did I tested it... >> >> I have this sample github project >> I created some time ago to experiment a bit with Fibers. It consists of 3 >> simple microservices: discovery-service, echo-service and workshop-service. >> The idea is to have more realistic setup than just an endpoint that sleeps >> the current request serving thread for some time, so in this case you call >> the workshop-service (either http://localhost:8080/loom/test - request >> is processed in Fiber or http://localhost:8081/loom/test - request is >> processed in Thread), which in turn uses discovery-service to look up >> echo-service (it starts on some random port) and then calls its >> /delay/{timeout} endpoint, where timeout is between 0 and 1000 ms. The >> echo-service echoes back the request with the given delay. >> >> Before you actually checkout this project, you first need to clone my >> Tomcat's fork and build it locally, by calling "*ant deploy*" and then "*ant >> embed*". You then need to copy the following jars from *output/embed* dir >> to your local maven repo. You also need to append the version number >> (9.0.27) to these jars as the build does not do it. >> >> - tomcat-embed-core.jar, rename to *tomcat-embed-core-9.0.27.jar* and >> copy to ~/.m2/repository/org/apache/tomcat/embed/tomcat-embed-core/9.0.27 >> (or wherever your local maven repo is) >> - tomcat-embed-el.jar, rename to *tomcat-embed-el-9.0.27.jar* and >> copy to ~/.m2/repository/org/apache/tomcat/embed/tomcat-embed-el/9.0.27 >> - tomcat-embed-websocket.jar, rename to >> *tomcat-embed-websocket-9.0.27.jar* and copy >> to ~/.m2/repository/org/apache/tomcat/embed/tomcat-embed-websocket/9.0.27 >> >> This sample project of mine is a gradle project, so you also need to make >> sure that gradle is configured to use loom build, I have the following in >> my cat ~/.gradle/gradle.properties: >> >> >> org.gradle.java.home=/Library/Java/JavaVirtualMachines/jdk-14.jdk/Contents/Home >> >> where obviously jdk-14.jdk is a loom build. >> >> You should be good to checkout the project now, and once it's done, you >> should start all 3 services, preferably in separate terminals like so: >> >> ./gradlew discovery-service:bootRun >> ./gradlew echo-service:bootRun >> ./gradlew workshop-service:bootRun >> >> Once started, you can then throw some load at workshop-service using >> provided attack.sh script (for it to work you actually need to have >> Vegeta installed locally). You call >> the script with either fibers or threads argument. As already mentioned >> above, this service has 2 connectors, namely: >> >> - 8080 - request processed in Fiber >> - 8081 - request processed in Thread >> >> Both connectors are configured with single threaded executors ( >> SimpleExecutor >> >> class). >> >> The single-threaded fiber connector works with 100% success rate with the >> rate up to about 375 requests per second on my machine. The thread based >> connector is obviously waaay less performant. >> >> jiggbook-1:loom-academy jigga$ ./attack.sh fibers >>> Requests [total, rate] 45000, 375.00 >>> Duration [total, attack, wait] 2m0.863797902s, 2m0.000299s, >>> 863.498902ms >>> Latencies [mean, 50, 95, 99, max] 505.184513ms, 503.86826ms, >>> 957.762176ms, 996.938898ms, 1.056817859s >>> Bytes In [total, mean] 10165188, 225.89 >>> Bytes Out [total, mean] 0, 0.00 >>> Success [ratio] 100.00% >>> Status Codes [code:count] 200:45000 >>> Error Set: >> >> >> So the issues I'm actually facing are that the VM in which >> workshop-service is running occasionally crashes. It does not happen on >> every run, but happened several times today. I included two hs_err* files >> in >> the repo. I believe that this may be GC-related issue as the Current thread >> is always GCTaskThread. >> >> I updated and built the jdk loom on the 2nd of September, have not >> rebuilt since then. >> >> Hope that you'll be able to reproduce it given the above info. >> >> If you need additional info, just let me know. >> >> Thanks, >> Arek >> >> On Tue, Sep 17, 2019 at 12:03 PM Rickard B?ckman < >> rickard.backman at oracle.com> wrote: >> >>> Feel free to report them here right now. >>> If you can provide a reproducer it's great! >>> >>> Thank you! >>> >>> /R >>> >>> On 09/17, Arkadiusz Gasi?ski wrote: >>> > Hi, >>> > >>> > Where do I submit bug reports related to loom-JVM crashes? >>> > >>> > The JVM prints this when it crashes: >>> > >>> > # >>> > # If you would like to submit a bug report, please visit: >>> > # http://bugreport.java.com/bugreport/crash.jsp >>> > # >>> > >>> > But I just wonder if it's not too early for loom crashes to be reported >>> > there? >>> > >>> > Thanks, >>> > Arek >>> >> From jigga at jigga.pl Sat Oct 5 19:18:42 2019 From: jigga at jigga.pl (=?UTF-8?Q?Arkadiusz_Gasi=C5=84ski?=) Date: Sat, 5 Oct 2019 22:18:42 +0300 Subject: Submitting bugs in loom branch In-Reply-To: References: <20190917100224.GB23956@rbackman> Message-ID: Hi, Sure, thanks for the info. Will wait for these changes and once they?re in (whenever that will be), I?ll let you know if the issue is still there. Arek On Sat, 5 Oct 2019 at 22:11, Ron Pressler wrote: > Hi. > > We appreciate these bug reports, and would encourage you to submit more if > you find them > (and simple reproductions are especially appreciated), but the > implementation is undergoing > some major changes, so we may prioritise some new work over looking into > bugs. > > So while we are collecting the reports, they may not be addressed in a > timely manner. > > Thank you again ? for reporting issues as well as understanding. > > Ron > > > On 5 October 2019 at 19:58:42, Arkadiusz Gasi?ski (jigga at jigga.pl) wrote: > > Hi, > > Has anyone had a chance to look into that? I do realize that the setup > required to reproduce the issue is a bit burdensome and I'll try to > prepare > something simpler in the coming days, but essentially at the moment the VM > in which the load-tested service runs crashes every time when the request > rate is around 10 per second and above. Interestingly, with the latest > changes in the fibers branch, the current thread in the generated > hs_err_pid* files is now *fiber-carrier-thread-X* rather than a GC thread. > > Thanks, > Arek > > On Sat, Sep 21, 2019 at 10:46 PM Arkadiusz Gasi?ski > wrote: > > > Hi, > > > > A little update from my side on this. I made some tweaks in the code, > the > > most important, I guess, is increasing the number of executor's threads > > < > https://github.com/jigga/loom-academy/blob/master/workshop-service/src/main/java/loom/workshop/SimpleExecutor.java#L25> > > > (I was unable to reproduce the issue with single-threaded executor). > > > > Moreover, I updated tomcat's SocketProcessorBase run method > implementation > > < > https://github.com/jigga/tomcat/blob/fibers/java/org/apache/tomcat/util/net/SocketProcessorBase.java> > as > > the previous one was rather naive. > > > > With all these changes, the workshop-service VM crashes almost every > time > > it is hit with the attack script. > > > > I added some more hs_err* > > > > files to the repo. > > > > Arek > > > > > > On Tue, Sep 17, 2019 at 8:29 PM Arkadiusz Gasi?ski > wrote: > > > >> Well, ok then, will try to describe here how to reproduce the issue. > >> > >> To give you some context, I wanted to test how Fiber friendly Tomcat is > >> (actually spring boot app running with embedded Tomcat). Turned out it > is > >> not that friendly when it comes to Fibers as all HTTP request > processing is > >> done within the synchronized block. There's this > >> NioEndpoint$SocketProcessor > >> < > https://github.com/apache/tomcat/blob/master/java/org/apache/tomcat/util/net/NioEndpoint.java> > > >> class that extends from SocketProcessorBase > >> < > https://github.com/apache/tomcat/blob/master/java/org/apache/tomcat/util/net/SocketProcessorBase.java> > > >> (Runnable), whose whole run method is within synchronized block, which, > if > >> I understand correctly, pins http threads. > >> > >> So I forked tomcat and made the following change > >> < > https://github.com/jigga/tomcat/commit/3bc1becdf50f20580fdf3333181e318f5baed59b> > > >> in the SocketProcessorBase class, built it locally and it appears that > it > >> did the trick - Tomcat became Fiber friendly :) > >> > >> So how did I tested it... > >> > >> I have this sample github project < > https://github.com/jigga/loom-academy> > >> I created some time ago to experiment a bit with Fibers. It consists of > 3 > >> simple microservices: discovery-service, echo-service and > workshop-service. > >> The idea is to have more realistic setup than just an endpoint that > sleeps > >> the current request serving thread for some time, so in this case you > call > >> the workshop-service (either http://localhost:8080/loom/test - request > >> is processed in Fiber or http://localhost:8081/loom/test - request is > >> processed in Thread), which in turn uses discovery-service to look up > >> echo-service (it starts on some random port) and then calls its > >> /delay/{timeout} endpoint, where timeout is between 0 and 1000 ms. The > >> echo-service echoes back the request with the given delay. > >> > >> Before you actually checkout this project, you first need to clone my > >> Tomcat's fork and build it locally, by calling "*ant deploy*" and then > "*ant > >> embed*". You then need to copy the following jars from *output/embed* > dir > >> to your local maven repo. You also need to append the version number > >> (9.0.27) to these jars as the build does not do it. > >> > >> - tomcat-embed-core.jar, rename to *tomcat-embed-core-9.0.27.jar* and > >> copy to > ~/.m2/repository/org/apache/tomcat/embed/tomcat-embed-core/9.0.27 > >> (or wherever your local maven repo is) > >> - tomcat-embed-el.jar, rename to *tomcat-embed-el-9.0.27.jar* and > >> copy to ~/.m2/repository/org/apache/tomcat/embed/tomcat-embed-el/9.0.27 > >> - tomcat-embed-websocket.jar, rename to > >> *tomcat-embed-websocket-9.0.27.jar* and copy > >> to > ~/.m2/repository/org/apache/tomcat/embed/tomcat-embed-websocket/9.0.27 > >> > >> This sample project of mine is a gradle project, so you also need to > make > >> sure that gradle is configured to use loom build, I have the following > in > >> my cat ~/.gradle/gradle.properties: > >> > >> > >> > org.gradle.java.home=/Library/Java/JavaVirtualMachines/jdk-14.jdk/Contents/Home > > >> > >> where obviously jdk-14.jdk is a loom build. > >> > >> You should be good to checkout the project now, and once it's done, you > >> should start all 3 services, preferably in separate terminals like so: > >> > >> ./gradlew discovery-service:bootRun > >> ./gradlew echo-service:bootRun > >> ./gradlew workshop-service:bootRun > >> > >> Once started, you can then throw some load at workshop-service using > >> provided attack.sh script (for it to work you actually need to have > >> Vegeta installed locally). You > call > >> the script with either fibers or threads argument. As already mentioned > >> above, this service has 2 connectors, namely: > >> > >> - 8080 - request processed in Fiber > >> - 8081 - request processed in Thread > >> > >> Both connectors are configured with single threaded executors ( > >> SimpleExecutor > >> < > https://github.com/jigga/loom-academy/blob/master/workshop-service/src/main/java/loom/workshop/SimpleExecutor.java> > > >> class). > >> > >> The single-threaded fiber connector works with 100% success rate with > the > >> rate up to about 375 requests per second on my machine. The thread > based > >> connector is obviously waaay less performant. > >> > >> jiggbook-1:loom-academy jigga$ ./attack.sh fibers > >>> Requests [total, rate] 45000, 375.00 > >>> Duration [total, attack, wait] 2m0.863797902s, 2m0.000299s, > >>> 863.498902ms > >>> Latencies [mean, 50, 95, 99, max] 505.184513ms, 503.86826ms, > >>> 957.762176ms, 996.938898ms, 1.056817859s > >>> Bytes In [total, mean] 10165188, 225.89 > >>> Bytes Out [total, mean] 0, 0.00 > >>> Success [ratio] 100.00% > >>> Status Codes [code:count] 200:45000 > >>> Error Set: > >> > >> > >> So the issues I'm actually facing are that the VM in which > >> workshop-service is running occasionally crashes. It does not happen on > >> every run, but happened several times today. I included two hs_err* > files > >> > in > > >> the repo. I believe that this may be GC-related issue as the Current > thread > >> is always GCTaskThread. > >> > >> I updated and built the jdk loom on the 2nd of September, have not > >> rebuilt since then. > >> > >> Hope that you'll be able to reproduce it given the above info. > >> > >> If you need additional info, just let me know. > >> > >> Thanks, > >> Arek > >> > >> On Tue, Sep 17, 2019 at 12:03 PM Rickard B?ckman < > >> rickard.backman at oracle.com> wrote: > >> > >>> Feel free to report them here right now. > >>> If you can provide a reproducer it's great! > >>> > >>> Thank you! > >>> > >>> /R > >>> > >>> On 09/17, Arkadiusz Gasi?ski wrote: > >>> > Hi, > >>> > > >>> > Where do I submit bug reports related to loom-JVM crashes? > >>> > > >>> > The JVM prints this when it crashes: > >>> > > >>> > # > >>> > # If you would like to submit a bug report, please visit: > >>> > # http://bugreport.java.com/bugreport/crash.jsp > >>> > # > >>> > > >>> > But I just wonder if it's not too early for loom crashes to be > reported > >>> > there? > >>> > > >>> > Thanks, > >>> > Arek > >>> > >> > > From aph at redhat.com Mon Oct 7 16:15:56 2019 From: aph at redhat.com (Andrew Haley) Date: Mon, 7 Oct 2019 17:15:56 +0100 Subject: Baffled asks: what does post_call_nop() do? Message-ID: <45e00a87-ad1b-a734-f0f2-8a1703da201e@redhat.com> I'm seeing this in one of my Scoped benchmarks: 5.30% 0x00007f0f811f0080: mov rsi,QWORD PTR [rsp] 0x00007f0f811f0084: nop 0x00007f0f811f0085: nop 0x00007f0f811f0086: nop 0x00007f0f811f0087: call 0x00007f0f8111f520 ; ImmutableOopMap {[0]=Oop [64]=Oop [72]=Oop [80]=Oop [88]=Oop } ;*invokevirtual consume {reexecute=0 rethrow=0 return_oop=0} ; - org.sample.generated.ThreadLocalTest_getSC_jmhTest::getSC_avgt_jmhStub at 22 (line 239) ; {optimized virtual_call} 71.87% 0x00007f0f811f008c: nop DWORD PTR [rax+rax*1+0x0] ;*invokevirtual consume {reexecute=0 rethrow=0 return_oop=0} ; - org.sample.generated.ThreadLocalTest_getSC_jmhTest::getSC_avgt_jmhStub at 22 (line 239) ;; B16: # out( B11 B17 ) <- in( B15 ) Freq: 64789.4 0x00007f0f811f0094: mov r10,QWORD PTR [rsp+0x40] 0x00007f0f811f0099: movzx r10d,BYTE PTR [r10+0x94] ;*invokestatic scopedCache {reexecute=0 rethrow=0 return_oop=0} ; - java.lang.Scoped::getObject at 6 (line 76) ; - java.lang.MyVariable_1::get at 5 So, about 72% of my total runtime is being accounted against post_call_nop() ! Is there any reason why this isn't just a bogus result? I'm not actually using Fibers or anything like them. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From rickard.backman at oracle.com Mon Oct 7 16:24:36 2019 From: rickard.backman at oracle.com (=?utf-8?Q?Rickard_B=C3=A4ckman?=) Date: Mon, 7 Oct 2019 18:24:36 +0200 Subject: Baffled asks: what does post_call_nop() do? In-Reply-To: <45e00a87-ad1b-a734-f0f2-8a1703da201e@redhat.com> References: <45e00a87-ad1b-a734-f0f2-8a1703da201e@redhat.com> Message-ID: We use it to put metadata about the nmethod in there. Like offset to nmethod header for faster lookup. I?m wondering if those numbers are really attributed to the right instruction. Change it to emit nothing and see if you get a 71% speedup. I?m guessing that % is really the call / return instruction and if post_call_nop wasn?t there it would just be added to the next instruction. > 7 okt. 2019 kl. 18:15 skrev Andrew Haley : > > I'm seeing this in one of my Scoped benchmarks: > > 5.30% 0x00007f0f811f0080: mov rsi,QWORD PTR [rsp] > 0x00007f0f811f0084: nop > 0x00007f0f811f0085: nop > 0x00007f0f811f0086: nop > 0x00007f0f811f0087: call 0x00007f0f8111f520 ; ImmutableOopMap {[0]=Oop [64]=Oop [72]=Oop [80]=Oop [88]=Oop } > ;*invokevirtual consume {reexecute=0 rethrow=0 return_oop=0} > ; - org.sample.generated.ThreadLocalTest_getSC_jmhTest::getSC_avgt_jmhStub at 22 (line 239) > ; {optimized virtual_call} > 71.87% 0x00007f0f811f008c: nop DWORD PTR [rax+rax*1+0x0] ;*invokevirtual consume {reexecute=0 rethrow=0 return_oop=0} > ; - org.sample.generated.ThreadLocalTest_getSC_jmhTest::getSC_avgt_jmhStub at 22 (line 239) > ;; B16: # out( B11 B17 ) <- in( B15 ) Freq: 64789.4 > 0x00007f0f811f0094: mov r10,QWORD PTR [rsp+0x40] > 0x00007f0f811f0099: movzx r10d,BYTE PTR [r10+0x94] ;*invokestatic scopedCache {reexecute=0 rethrow=0 return_oop=0} > ; - java.lang.Scoped::getObject at 6 (line 76) > ; - java.lang.MyVariable_1::get at 5 > > So, about 72% of my total runtime is being accounted against post_call_nop() ! > > Is there any reason why this isn't just a bogus result? I'm not > actually using Fibers or anything like them. > > -- > Andrew Haley (he/him) > Java Platform Lead Engineer > Red Hat UK Ltd. > https://keybase.io/andrewhaley > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From aph at redhat.com Tue Oct 8 21:49:33 2019 From: aph at redhat.com (Andrew Haley) Date: Tue, 8 Oct 2019 22:49:33 +0100 Subject: Baffled asks: what does post_call_nop() do? In-Reply-To: References: <45e00a87-ad1b-a734-f0f2-8a1703da201e@redhat.com> Message-ID: <09828239-cdc7-2a50-6ba5-caaebcd9f433@redhat.com> On 10/7/19 5:24 PM, Rickard B?ckman wrote: > We use it to put metadata about the nmethod in there. Like offset to nmethod header for faster lookup. Aha! I would not have guessed. > I?m wondering if those numbers are really attributed to the right instruction. > Change it to emit nothing and see if you get a 71% speedup. I?m > guessing that % is really the call / return instruction and if > post_call_nop wasn?t there it would just be added to the next > instruction. Mkay, thanks. I think you're right: the delay seems to be caused by a horrendous pipeline stall in JMH's Blackhole::consume, nothing to do with Loom. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From jigga at jigga.pl Wed Oct 9 08:51:11 2019 From: jigga at jigga.pl (=?UTF-8?Q?Arkadiusz_Gasi=C5=84ski?=) Date: Wed, 9 Oct 2019 11:51:11 +0300 Subject: Did Java on Solaris once had a Fiber-like threads? Message-ID: Hi, I recently found the "*JDK 1.1 for Solaris Developer's Guide*" article ( https://docs.oracle.com/cd/E19455-01/806-3461/6jck06gqe/index.html), which says, in the Many-to-Many Model section, that "*a program can have as many threads as are appropriate without making the process too heavy or burdensome. In this model, a user-level threads library provides sophisticated scheduling of user-level threads above kernel threads. The kernel needs to manage only the threads that are currently active. A many-to-many implementation at the user level reduces programming effort as it lifts restrictions on the number of threads that can be effectively used in an application.*" That's just the definition of the many-to-many model, but later in the same section, it's written that "*The Java on Solaris operating environment is the first many-to-many commercial implementation of Java on an MT operating system*". If that's the case, it looks like there already was a version of Java that had Fiber-like threads. Was it only some proprietary implementation just for Solaris? What happened to it? BTW, if I understand correctly, the current versions of Open/Oracle JDK employ the one-to-one multithreading models, correct? Thanks, Arek From dalibor.topic at oracle.com Wed Oct 9 09:28:35 2019 From: dalibor.topic at oracle.com (Dalibor Topic) Date: Wed, 9 Oct 2019 11:28:35 +0200 Subject: Did Java on Solaris once had a Fiber-like threads? In-Reply-To: References: Message-ID: <1ed14052-8c8a-678b-acf1-a084eb4b3c8d@oracle.com> You're probably looking for https://en.wikipedia.org/wiki/Green_threads . ;) cheers, dalibor topic On 09.10.2019 10:51, Arkadiusz Gasi?ski wrote: > Hi, > > I recently found the "*JDK 1.1 for Solaris Developer's Guide*" article ( > https://docs.oracle.com/cd/E19455-01/806-3461/6jck06gqe/index.html), which > says, in the Many-to-Many Model section, that "*a program can have as many > threads as are appropriate without making the process too heavy or > burdensome. In this model, a user-level threads library provides > sophisticated scheduling of user-level threads above kernel threads. The > kernel needs to manage only the threads that are currently active. A > many-to-many implementation at the user level reduces programming effort as > it lifts restrictions on the number of threads that can be effectively used > in an application.*" > > That's just the definition of the many-to-many model, but later in the same > section, it's written that "*The Java on Solaris operating environment is > the first many-to-many commercial implementation of Java on an MT operating > system*". If that's the case, it looks like there already was a version of > Java that had Fiber-like threads. Was it only some proprietary > implementation just for Solaris? What happened to it? > > BTW, if I understand correctly, the current versions of Open/Oracle JDK > employ the one-to-one multithreading models, correct? > > Thanks, > Arek > -- Dalibor Topic | Consulting Product Manager Phone: +494089091214 | Mobile: +491737185961 | Video: dalibor.topic at oracle.com Oracle Global Services Germany GmbH Hauptverwaltung: Riesstr. 25, D-80992 M?nchen Registergericht: Amtsgericht M?nchen, HRB 246209 Gesch?ftsf?hrer: Ralf Herrmann Oracle is committed to developing practices and products that help protect the environment From david.holmes at oracle.com Wed Oct 9 10:01:32 2019 From: david.holmes at oracle.com (David Holmes) Date: Wed, 9 Oct 2019 20:01:32 +1000 Subject: Did Java on Solaris once had a Fiber-like threads? In-Reply-To: References: Message-ID: <98e016ea-2ed6-2dc4-6316-e383b423ca27@oracle.com> On 9/10/2019 6:51 pm, Arkadiusz Gasi?ski wrote: > Hi, > > I recently found the "*JDK 1.1 for Solaris Developer's Guide*" article ( > https://docs.oracle.com/cd/E19455-01/806-3461/6jck06gqe/index.html), which > says, in the Many-to-Many Model section, that "*a program can have as many > threads as are appropriate without making the process too heavy or > burdensome. In this model, a user-level threads library provides > sophisticated scheduling of user-level threads above kernel threads. The > kernel needs to manage only the threads that are currently active. A > many-to-many implementation at the user level reduces programming effort as > it lifts restrictions on the number of threads that can be effectively used > in an application.*" > > That's just the definition of the many-to-many model, but later in the same > section, it's written that "*The Java on Solaris operating environment is > the first many-to-many commercial implementation of Java on an MT operating > system*". If that's the case, it looks like there already was a version of > Java that had Fiber-like threads. Was it only some proprietary > implementation just for Solaris? What happened to it? > > BTW, if I understand correctly, the current versions of Open/Oracle JDK > employ the one-to-one multithreading models, correct? Java post "Green Threads" employs a 1:1 model with operating system threads. However Solaris also had a M:N model between user-level threads and kernel threads (LWPs). So Java was not actively using anything fiber-like, just plain java.lang.Thread that mapped 1:1 to a Solaris UI Thread but it could get some of the benefits that we're ascribing to Fibers, when running on Solaris (with the appropriate configuration options). https://docs.oracle.com/cd/E19455-01/806-5257/6je9h0349/index.html David > Thanks, > Arek > From david.holmes at oracle.com Wed Oct 9 10:06:54 2019 From: david.holmes at oracle.com (David Holmes) Date: Wed, 9 Oct 2019 20:06:54 +1000 Subject: Did Java on Solaris once had a Fiber-like threads? In-Reply-To: <1ed14052-8c8a-678b-acf1-a084eb4b3c8d@oracle.com> References: <1ed14052-8c8a-678b-acf1-a084eb4b3c8d@oracle.com> Message-ID: On 9/10/2019 7:28 pm, Dalibor Topic wrote: > You're probably looking for https://en.wikipedia.org/wiki/Green_threads > . ;) Hmmmm, not quite how I remember Green Threads in Java. My recollection was that Green Thread were for the early Windows and perhaps the Solaris reference implementations. While the Solaris Production VM (a.k.a ExactVM) used Solaris native threading. Hotspot used native threading on all platforms. Cheers, David > cheers, > dalibor topic > > On 09.10.2019 10:51, Arkadiusz Gasi?ski wrote: >> Hi, >> >> I recently found the "*JDK 1.1 for Solaris Developer's Guide*" article ( >> https://docs.oracle.com/cd/E19455-01/806-3461/6jck06gqe/index.html), >> which >> says, in the Many-to-Many Model section, that "*a program can have as >> many >> threads as are appropriate without making the process too heavy or >> burdensome. In this model, a user-level threads library provides >> sophisticated scheduling of user-level threads above kernel threads. The >> kernel needs to manage only the threads that are currently active. A >> many-to-many implementation at the user level reduces programming >> effort as >> it lifts restrictions on the number of threads that can be effectively >> used >> in an application.*" >> >> That's just the definition of the many-to-many model, but later in the >> same >> section, it's written that "*The Java on Solaris operating environment is >> the first many-to-many commercial implementation of Java on an MT >> operating >> system*". If that's the case, it looks like there already was a >> version of >> Java that had Fiber-like threads. Was it only some proprietary >> implementation just for Solaris? What happened to it? >> >> BTW, if I understand correctly, the current versions of Open/Oracle JDK >> employ the one-to-one multithreading models, correct? >> >> Thanks, >> Arek >> > From ChrisPhi at LGonQn.Org Wed Oct 9 17:22:16 2019 From: ChrisPhi at LGonQn.Org ("Chris Phillips"@T O) Date: Wed, 9 Oct 2019 13:22:16 -0400 Subject: Did Java on Solaris once had a Fiber-like threads? In-Reply-To: References: <1ed14052-8c8a-678b-acf1-a084eb4b3c8d@oracle.com> Message-ID: <72b81d6a-e5af-95c2-e590-bef3c4b568ed@LGonQn.Org> Hi On 09/10/19 06:06 AM, David Holmes wrote: > On 9/10/2019 7:28 pm, Dalibor Topic wrote: >> You're probably looking for >> https://en.wikipedia.org/wiki/Green_threads . ;) > > Hmmmm, not quite how I remember Green Threads in Java. My recollection > was that Green Thread were for the early Windows and perhaps the > Solaris reference implementations. While the Solaris Production VM > (a.k.a ExactVM) used Solaris native threading. Hotspot used native > threading on all platforms. > > Cheers, > David As a former member of the JTG group that did the early work on Solaris threads, this is what I remember: 0) "Classic" JVM with "Green" threads. 1) A port of the "classic" JVM to the Solaris threads model [JDK1.1]. It was only m:n at that point. Lack of control of threads and thread scheduling was an issue. 2) ExactVM, which was JDK 1.2.x in Solaris was written with the Solaris threads model, but took Posix threads into account. Solaris introduced the direct model [1:1] library and you could use either model. 3 HotSpot (less clear here because it happened mostly in California) which was Posix threads. Cheers! Chris > >> cheers, >> dalibor topic >> >> On 09.10.2019 10:51, Arkadiusz Gasi?ski wrote: >>> Hi, >>> >>> I recently found the "*JDK 1.1 for Solaris Developer's Guide*" >>> article ( >>> https://docs.oracle.com/cd/E19455-01/806-3461/6jck06gqe/index.html), >>> which >>> says, in the Many-to-Many Model section, that "*a program can have >>> as many >>> threads as are appropriate without making the process too heavy or >>> burdensome. In this model, a user-level threads library provides >>> sophisticated scheduling of user-level threads above kernel threads. >>> The >>> kernel needs to manage only the threads that are currently active. A >>> many-to-many implementation at the user level reduces programming >>> effort as >>> it lifts restrictions on the number of threads that can be >>> effectively used >>> in an application.*" >>> >>> That's just the definition of the many-to-many model, but later in >>> the same >>> section, it's written that "*The Java on Solaris operating >>> environment is >>> the first many-to-many commercial implementation of Java on an MT >>> operating >>> system*". If that's the case, it looks like there already was a >>> version of >>> Java that had Fiber-like threads. Was it only some proprietary >>> implementation just for Solaris? What happened to it? >>> >>> BTW, if I understand correctly, the current versions of Open/Oracle JDK >>> employ the one-to-one multithreading models, correct? >>> >>> Thanks, >>> Arek >>> >> > > > From jigga at jigga.pl Sat Oct 12 20:03:31 2019 From: jigga at jigga.pl (=?UTF-8?Q?Arkadiusz_Gasi=C5=84ski?=) Date: Sat, 12 Oct 2019 22:03:31 +0200 Subject: Did Java on Solaris once had a Fiber-like threads? In-Reply-To: <98e016ea-2ed6-2dc4-6316-e383b423ca27@oracle.com> References: <98e016ea-2ed6-2dc4-6316-e383b423ca27@oracle.com> Message-ID: Hi, Thanks for clarifying. I got only your mail directly, but noticed in the loom-dev digests that other replied as well, thanks! One follow up question I have now is: do fibers actually employ the M:N model? A week ago I would have said without any doubt that they do, but I guess I've read too much about multithreading models recently and now I'm confused. Thanks, Arek On Wed, Oct 9, 2019 at 12:01 PM David Holmes wrote: > On 9/10/2019 6:51 pm, Arkadiusz Gasi?ski wrote: > > Hi, > > > > I recently found the "*JDK 1.1 for Solaris Developer's Guide*" article ( > > https://docs.oracle.com/cd/E19455-01/806-3461/6jck06gqe/index.html), > which > > says, in the Many-to-Many Model section, that "*a program can have as > many > > threads as are appropriate without making the process too heavy or > > burdensome. In this model, a user-level threads library provides > > sophisticated scheduling of user-level threads above kernel threads. The > > kernel needs to manage only the threads that are currently active. A > > many-to-many implementation at the user level reduces programming effort > as > > it lifts restrictions on the number of threads that can be effectively > used > > in an application.*" > > > > That's just the definition of the many-to-many model, but later in the > same > > section, it's written that "*The Java on Solaris operating environment is > > the first many-to-many commercial implementation of Java on an MT > operating > > system*". If that's the case, it looks like there already was a version > of > > Java that had Fiber-like threads. Was it only some proprietary > > implementation just for Solaris? What happened to it? > > > > BTW, if I understand correctly, the current versions of Open/Oracle JDK > > employ the one-to-one multithreading models, correct? > > Java post "Green Threads" employs a 1:1 model with operating system > threads. However Solaris also had a M:N model between user-level threads > and kernel threads (LWPs). So Java was not actively using anything > fiber-like, just plain java.lang.Thread that mapped 1:1 to a Solaris UI > Thread but it could get some of the benefits that we're ascribing to > Fibers, when running on Solaris (with the appropriate configuration > options). > > https://docs.oracle.com/cd/E19455-01/806-5257/6je9h0349/index.html > > David > > > Thanks, > > Arek > > > From david.holmes at oracle.com Sun Oct 13 02:37:16 2019 From: david.holmes at oracle.com (David Holmes) Date: Sun, 13 Oct 2019 12:37:16 +1000 Subject: Did Java on Solaris once had a Fiber-like threads? In-Reply-To: References: <98e016ea-2ed6-2dc4-6316-e383b423ca27@oracle.com> Message-ID: On 13/10/2019 6:03 am, Arkadiusz Gasi?ski wrote: > Hi, > > Thanks for clarifying. I got only your mail directly, but noticed in the > loom-dev digests that other replied as well, thanks! > > One follow up question I have now is: do fibers actually employ the M:N > model? A week ago I would have said without any doubt that they do, but > I guess I've read too much about multithreading models recently and now > I'm confused. I believe the current Fiber threading model is M:N in general, but at time a Fiber may have to be pinned to a specific thread. David > Thanks, > Arek > > > > On Wed, Oct 9, 2019 at 12:01 PM David Holmes > wrote: > > On 9/10/2019 6:51 pm, Arkadiusz Gasi?ski wrote: > > Hi, > > > > I recently found the "*JDK 1.1 for Solaris Developer's Guide*" > article ( > > > https://docs.oracle.com/cd/E19455-01/806-3461/6jck06gqe/index.html), > which > > says, in the Many-to-Many Model section, that "*a program can > have as many > > threads as are appropriate without making the process too heavy or > > burdensome. In this model, a user-level threads library provides > > sophisticated scheduling of user-level threads above kernel > threads. The > > kernel needs to manage only the threads that are currently active. A > > many-to-many implementation at the user level reduces programming > effort as > > it lifts restrictions on the number of threads that can be > effectively used > > in an application.*" > > > > That's just the definition of the many-to-many model, but later > in the same > > section, it's written that "*The Java on Solaris operating > environment is > > the first many-to-many commercial implementation of Java on an MT > operating > > system*". If that's the case, it looks like there already was a > version of > > Java that had Fiber-like threads. Was it only some proprietary > > implementation just for Solaris? What happened to it? > > > > BTW, if I understand correctly, the current versions of > Open/Oracle JDK > > employ the one-to-one multithreading models, correct? > > Java post "Green Threads" employs a 1:1 model with operating system > threads. However Solaris also had a M:N model between user-level > threads > and kernel threads (LWPs). So Java was not actively using anything > fiber-like, just plain java.lang.Thread that mapped 1:1 to a Solaris UI > Thread but it could get some of the benefits that we're ascribing to > Fibers, when running on Solaris (with the appropriate configuration > options). > > https://docs.oracle.com/cd/E19455-01/806-5257/6je9h0349/index.html > > David > > > Thanks, > > Arek > > > From lxsameer at pm.me Tue Oct 15 14:37:53 2019 From: lxsameer at pm.me (lxsameer) Date: Tue, 15 Oct 2019 14:37:53 +0000 Subject: Using blocking IO in fibers Message-ID: Hi folks, I'm interested to learn about the implementation of fibers, specially about how do you park a fiber when it's gonna block. Do you have a way to find out about external blocking IO on a fiber ? Would you mind giving me really quick and brief overview of the implementation or point me to the code please ? From Alan.Bateman at oracle.com Tue Oct 15 15:39:40 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 15 Oct 2019 16:39:40 +0100 Subject: Using blocking IO in fibers In-Reply-To: References: Message-ID: On 15/10/2019 15:37, lxsameer wrote: > Hi folks, > I'm interested to learn about the implementation of fibers, specially about how do you park a fiber when it's gonna block. Do you have a way to find out about external blocking IO on a fiber ? Would you mind giving me really quick and brief overview of the implementation or point me to the code please ? > The Project Loom update from JVMLS 2019 [1] might be useful. About 7mins in, we used ServerSocket::accept as an example and you'll see the code path (in the form of a stack trace) to see how the yielding when there are no connections to accept. We follow that with the unpark/continue when a connection is ready to be accepted. -Alan [1] https://www.youtube.com/watch?v=NV46KFV1m-4 From jigga at jigga.pl Wed Oct 16 20:18:04 2019 From: jigga at jigga.pl (=?UTF-8?Q?Arkadiusz_Gasi=C5=84ski?=) Date: Wed, 16 Oct 2019 22:18:04 +0200 Subject: What triggers Fiber.unpark? Message-ID: Hi, As in the subject line, namely what triggers Fiber.unpark? Is it the schedulers job to do it? If so, how does it know when to call unpark? Say my Fiber was parked waiting for I/O to finish, the I/O completes... and then what? Thanks, Arek From jigga at jigga.pl Wed Oct 16 20:58:19 2019 From: jigga at jigga.pl (=?UTF-8?Q?Arkadiusz_Gasi=C5=84ski?=) Date: Wed, 16 Oct 2019 22:58:19 +0200 Subject: What triggers Fiber.unpark? In-Reply-To: References: Message-ID: Just noticed that the previous question in the mailing list was pretty much the same (what a coincidence) and actually checked your talk from JVMLS, @Alan. I?m still wondering what?s calling this InnocuousThread that in turn unpark Fiber? Also, is this thread used for all Fiber unparking or only in case of I/O? Arek On Wed, 16 Oct 2019 at 22:18, Arkadiusz Gasi?ski wrote: > Hi, > > As in the subject line, namely what triggers Fiber.unpark? Is it the > schedulers job to do it? If so, how does it know when to call unpark? Say > my Fiber was parked waiting for I/O to finish, the I/O completes... and > then what? > > Thanks, > Arek > From Alan.Bateman at oracle.com Thu Oct 17 07:53:21 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 17 Oct 2019 08:53:21 +0100 Subject: What triggers Fiber.unpark? In-Reply-To: References: Message-ID: On 16/10/2019 21:58, Arkadiusz Gasi?ski wrote: > Just noticed that the previous question in the mailing list was pretty much > the same (what a coincidence) and actually checked your talk from JVMLS, > @Alan. I?m still wondering what?s calling this InnocuousThread that in turn > unpark Fiber? Also, is this thread used for all Fiber unparking or only in > case of I/O? > sun.nio.ch.Poller is probably what you want. The `polled` method is invoked when a file descriptor is ready and it unparks the fiber that is parked waiting on that file descriptor. -Alan From chris.plummer at oracle.com Thu Oct 17 19:30:58 2019 From: chris.plummer at oracle.com (Chris Plummer) Date: Thu, 17 Oct 2019 12:30:58 -0700 Subject: What triggers Fiber.unpark? In-Reply-To: References: Message-ID: On 10/17/19 12:53 AM, Alan Bateman wrote: > On 16/10/2019 21:58, Arkadiusz Gasi?ski wrote: >> Just noticed that the previous question in the mailing list was >> pretty much >> the same (what a coincidence) and actually checked your talk from JVMLS, >> @Alan. I?m still wondering what?s calling this InnocuousThread that >> in turn >> unpark Fiber? Also, is this thread used for all Fiber unparking or >> only in >> case of I/O? >> > sun.nio.ch.Poller is probably what you want. The `polled` method is > invoked when a file descriptor is ready and it unparks the fiber that > is parked waiting on that file descriptor. > > -Alan This part of the video shows the stack when the poll completes and the fiber is unparked: https://youtu.be/NV46KFV1m-4?t=509 Maybe Alan can copy-n-paste the stack trace from the presentation here for the record. Note this is on macos, which uses KQueuePoller. On linux is uses EPollPoller. Poller.startPollerThread() is where the InnocuousThread is created: ??????????? Thread t = JLA.executeOnCarrierThread(() -> ??????????????????? InnocuousThread.newSystemThread(name, poller)); And the above code is triggered from the Poller static initializer. Chris From jigga at jigga.pl Thu Oct 17 19:45:25 2019 From: jigga at jigga.pl (=?UTF-8?Q?Arkadiusz_Gasi=C5=84ski?=) Date: Thu, 17 Oct 2019 21:45:25 +0200 Subject: What triggers Fiber.unpark? In-Reply-To: References: Message-ID: Awesome, thanks! I think that this YT video link is enough of a record (no pun intended on the records JEP ;)) Also, I assume that this is only for I/O, right? If a Fiber is waiting for a lock, it's a different entity that triggers unparking? Thanks, Arek On Thu, Oct 17, 2019, 21:33 Chris Plummer wrote: > On 10/17/19 12:53 AM, Alan Bateman wrote: > > On 16/10/2019 21:58, Arkadiusz Gasi?ski wrote: > >> Just noticed that the previous question in the mailing list was > >> pretty much > >> the same (what a coincidence) and actually checked your talk from JVMLS, > >> @Alan. I?m still wondering what?s calling this InnocuousThread that > >> in turn > >> unpark Fiber? Also, is this thread used for all Fiber unparking or > >> only in > >> case of I/O? > >> > > sun.nio.ch.Poller is probably what you want. The `polled` method is > > invoked when a file descriptor is ready and it unparks the fiber that > > is parked waiting on that file descriptor. > > > > -Alan > This part of the video shows the stack when the poll completes and the > fiber is unparked: > > https://youtu.be/NV46KFV1m-4?t=509 > > Maybe Alan can copy-n-paste the stack trace from the presentation here > for the record. > > Note this is on macos, which uses KQueuePoller. On linux is uses > EPollPoller. Poller.startPollerThread() is where the InnocuousThread is > created: > > Thread t = JLA.executeOnCarrierThread(() -> > InnocuousThread.newSystemThread(name, poller)); > > And the above code is triggered from the Poller static initializer. > > Chris > > From chris.plummer at oracle.com Thu Oct 17 20:10:35 2019 From: chris.plummer at oracle.com (Chris Plummer) Date: Thu, 17 Oct 2019 13:10:35 -0700 Subject: What triggers Fiber.unpark? In-Reply-To: References: Message-ID: <36d101f3-77d3-52f2-f17b-02057445f87b@oracle.com> Hi Arek, Look at the 7m mark of the video for an example of what happens when java.util.concurrent.locks.ReentrantLock.unlock() is called, triggering the unpark of a Fiber waiting on that lock. It's the holder of the lock that triggers the unpark (and this case it was another fiber holding the lock). So yes, totally different than how the I/O case is handled. Chris On 10/17/19 12:45 PM, Arkadiusz Gasi?ski wrote: > Awesome, thanks! > > I think that this YT video link is enough of a record (no pun intended > on the records JEP ;)) > > Also, I assume that this is only for I/O, right? If a Fiber is waiting > for a lock, it's a different entity that triggers unparking? > > Thanks, > Arek > > On Thu, Oct 17, 2019, 21:33 Chris Plummer > wrote: > > On 10/17/19 12:53 AM, Alan Bateman wrote: > > On 16/10/2019 21:58, Arkadiusz Gasi?ski wrote: > >> Just noticed that the previous question in the mailing list was > >> pretty much > >> the same (what a coincidence) and actually checked your talk > from JVMLS, > >> @Alan. I?m still wondering what?s calling this InnocuousThread > that > >> in turn > >> unpark Fiber? Also, is this thread used for all Fiber unparking or > >> only in > >> case of I/O? > >> > > sun.nio.ch.Poller is probably what you want. The `polled` method is > > invoked when a file descriptor is ready and it unparks the fiber > that > > is parked waiting on that file descriptor. > > > > -Alan > This part of the video shows the stack when the poll completes and > the > fiber is unparked: > > https://youtu.be/NV46KFV1m-4?t=509 > > Maybe Alan can copy-n-paste the stack trace from the presentation > here > for the record. > > Note this is on macos, which uses KQueuePoller. On linux is uses > EPollPoller. Poller.startPollerThread() is where the > InnocuousThread is > created: > > ???????????? Thread t = JLA.executeOnCarrierThread(() -> > ???????????????????? InnocuousThread.newSystemThread(name, poller)); > > And the above code is triggered from the Poller static initializer. > > Chris > From jigga at jigga.pl Thu Oct 17 21:04:29 2019 From: jigga at jigga.pl (=?UTF-8?Q?Arkadiusz_Gasi=C5=84ski?=) Date: Thu, 17 Oct 2019 23:04:29 +0200 Subject: What triggers Fiber.unpark? In-Reply-To: <36d101f3-77d3-52f2-f17b-02057445f87b@oracle.com> References: <36d101f3-77d3-52f2-f17b-02057445f87b@oracle.com> Message-ID: Hey Chris, Thanks again! I just had to stare long enough at this stack trace from the 7m mark of Alan?s presentation to get that it?s that fiber/thread that unlocks the lock that also re-submits the waiting fiber/thread for execution. Arek On Thu, 17 Oct 2019 at 22:12, Chris Plummer wrote: > Hi Arek, > > Look at the 7m mark of the video for an example of what happens when > java.util.concurrent.locks.ReentrantLock.unlock() is called, triggering > the unpark of a Fiber waiting on that lock. It's the holder of the lock > that triggers the unpark (and this case it was another fiber holding the > lock). So yes, totally different than how the I/O case is handled. > > Chris > > On 10/17/19 12:45 PM, Arkadiusz Gasi?ski wrote: > > Awesome, thanks! > > > > I think that this YT video link is enough of a record (no pun intended > > on the records JEP ;)) > > > > Also, I assume that this is only for I/O, right? If a Fiber is waiting > > for a lock, it's a different entity that triggers unparking? > > > > Thanks, > > Arek > > > > On Thu, Oct 17, 2019, 21:33 Chris Plummer > > wrote: > > > > On 10/17/19 12:53 AM, Alan Bateman wrote: > > > On 16/10/2019 21:58, Arkadiusz Gasi?ski wrote: > > >> Just noticed that the previous question in the mailing list was > > >> pretty much > > >> the same (what a coincidence) and actually checked your talk > > from JVMLS, > > >> @Alan. I?m still wondering what?s calling this InnocuousThread > > that > > >> in turn > > >> unpark Fiber? Also, is this thread used for all Fiber unparking or > > >> only in > > >> case of I/O? > > >> > > > sun.nio.ch.Poller is probably what you want. The `polled` method is > > > invoked when a file descriptor is ready and it unparks the fiber > > that > > > is parked waiting on that file descriptor. > > > > > > -Alan > > This part of the video shows the stack when the poll completes and > > the > > fiber is unparked: > > > > https://youtu.be/NV46KFV1m-4?t=509 > > > > Maybe Alan can copy-n-paste the stack trace from the presentation > > here > > for the record. > > > > Note this is on macos, which uses KQueuePoller. On linux is uses > > EPollPoller. Poller.startPollerThread() is where the > > InnocuousThread is > > created: > > > > Thread t = JLA.executeOnCarrierThread(() -> > > InnocuousThread.newSystemThread(name, poller)); > > > > And the above code is triggered from the Poller static initializer. > > > > Chris > > > > > From Alan.Bateman at oracle.com Tue Oct 22 13:59:18 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 22 Oct 2019 14:59:18 +0100 Subject: A lightweight thread is a Thread Message-ID: This project has been trying to figure out the API to expose for lightweight threads, and its relationship with the Thread API, for some time. There has been many prototypes (lightweight thread = Thread, Fiber <: Thread, Fiber and Thread <: Strand, disjoint Fiber and Thread types). Thread.currentThread() and thread locals featured in all prototypes as it's critical that the current Thread cannot change value in a thread of execution. I think we are at the point where we are concluding that a lightweight thread should be represented by a Thread. No Fiber or other sub-class exposed in the API. We'll use the term "lightweight thread" rather than "fiber". The approach maintains the mental model of a thread of execution that we are familiar with here, and the mental model that developers coming to the platform should be able to figure out. The Thread API does come with baggage (ThreadGroup, TCCL, ...) but it may not be too bad when you consider that most developers don't use the Thread API directly. In addition, we can degrade and/or eliminate at least some of this baggage over time, starting with the deprecated features. Moving to Thread as the API for lightweight threads means we need to restart some of the API work. In particular the support for structured concurrency and cancellation will need to move aside temporarily as we rebuild from the bottom. I will push a change to the "fibers" branch soon with the first steps in that direction. The first steps include 4 static factory methods on Thread to create lightweight threads (unnamed/named * default or BYO scheduler). There are several discussion points around what lightweight threads support and don't support. For now, the default is support thread locals and not inherit thread locals but it can overridden by specifying characteristics to the factory methods. The API is primitive for now and will likely change many times. API aside, one of the benefits of representing a lightweight thread as a Thread is that it makes it possible to make progress in the serviceability areas that we've had to ignore to date. We can finally start to figure how the existing monitoring and management APIs, JFR, and diagnostic features might work with lightweight threads. Good progress has been made on JVM TI and the debugger support in the last year and it shouldn't require too much re-work to get the debugger support re-aligned. One final thing for now is footprint.? There will be some refactoring of Thread required to reduce its footprint. The initial changes will move some of fields that aren't performance critical to a sidecar/helper object. Doug Lea has changes to the FJ code in the works that should avoid the need for padding/@Contended on the fields used by ThreadLocalRandom. The object size will, at least initially, be a bit larger that I had hoped but I'm sure we will improve this in time. -Alan. From org.openjdk at io7m.com Tue Oct 22 14:22:45 2019 From: org.openjdk at io7m.com (Mark Raynsford) Date: Tue, 22 Oct 2019 15:22:45 +0100 Subject: A lightweight thread is a Thread In-Reply-To: References: Message-ID: <20191022152245.16ea1f30@almond.int.arc7.info> On 2019-10-22T14:59:18 +0100 Alan Bateman wrote: > > I think we are at the point where we are concluding that a lightweight > thread should be represented by a Thread. Will this not have some nasty consequences when talking to native code? For example, external graphics APIs such as OpenGL and Vulkan have strict restrictions on which threads can make calls into the APIs at any given time. If I'm calling native code, and I specifically create several individual threads for the purposes of doing so... How can I be sure that each thread I've created corresponds to exactly one operating system thread? I feel like having a distinction at the type level between a Fiber and a Thread makes the difference in cost model explicit, even if there's no other API difference. I'm reminded of FreeBSD, for example, which used to have an m:n model for its own pthreads library, but eventually switched back to a 1:1 model for performance reasons. I believe one of the issues was that over a decade of software had grown up around the idea that a thread had a particular cost model (in other words, was a heavyweight scheduling primitive) and that by changing the cost model to an m:n model ended up making most software perform poorly. -- Mark Raynsford | http://www.io7m.com From ron.pressler at oracle.com Tue Oct 22 15:07:14 2019 From: ron.pressler at oracle.com (Ron Pressler) Date: Tue, 22 Oct 2019 16:07:14 +0100 Subject: A lightweight thread is a Thread In-Reply-To: <20191022152245.16ea1f30@almond.int.arc7.info> References: <20191022152245.16ea1f30@almond.int.arc7.info> Message-ID: ? On 22 October 2019 at 15:23:09, Mark Raynsford (org.openjdk at io7m.com(mailto:org.openjdk at io7m.com)) wrote: > On 2019-10-22T14:59:18 +0100 > Alan Bateman wrote: > > > > I think we are at the point where we are concluding that a lightweight > > thread should be represented by a Thread. > > Will this not have some nasty consequences when talking to native code? > > For example, external graphics APIs such as OpenGL and Vulkan have > strict restrictions on which threads can make calls into the APIs at > any given time. If I'm calling native code, and I specifically create > several individual threads for the purposes of doing so... How can I be > sure that each thread I've created corresponds to exactly one operating > system thread? By making it a heavyweight thread. When you create a Thread you choose whether it is scheduled by the OS (heavyweight) or by the JDK (lightweight). > > I feel like having a distinction at the type level between a Fiber and > a Thread makes the difference in cost model explicit, even if there's > no other API difference. > > I'm reminded of FreeBSD, for example, which used to have an m:n model > for its own pthreads library, but eventually switched back to a 1:1 > model for performance reasons. I believe one of the issues was that > over a decade of software had grown up around the idea that a thread > had a particular cost model (in other words, was a heavyweight > scheduling primitive) and that by changing the cost model to an m:n > model ended up making most software perform poorly. People hardly use the Thread API directly *today*, relying instead on? j.u.c.Executor and the like.?This will likely remain the same with lightweight? threads: People will use the structured-concurrency constructs to manage them. Where Thread is used directly is in calls to Thread.currentThread (and, to a Lesser degree, the interruption mechanism, but that?s a separate discussion). This use is so pervasive that breaking assumptions about it would mean little existing code could run in lightweight threads. Now we get to have our cake and eat it too. Users will mostly interact with a new API, but those places where Thread is used extensively will continue working without change. Ron From Alan.Bateman at oracle.com Tue Oct 22 15:19:15 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 22 Oct 2019 16:19:15 +0100 Subject: A lightweight thread is a Thread In-Reply-To: <20191022152245.16ea1f30@almond.int.arc7.info> References: <20191022152245.16ea1f30@almond.int.arc7.info> Message-ID: <1360aec6-5d70-69fc-0f9d-3d7a1a0bc859@oracle.com> On 22/10/2019 15:22, Mark Raynsford wrote: > : > Will this not have some nasty consequences when talking to native code? > > For example, external graphics APIs such as OpenGL and Vulkan have > strict restrictions on which threads can make calls into the APIs at > any given time. If I'm calling native code, and I specifically create > several individual threads for the purposes of doing so... How can I be > sure that each thread I've created corresponds to exactly one operating > system thread? The existing constructors work as before and this project might also introduce static factory methods to create heavyweight threads. But maybe you mean something else? If you mean there may be lightweight threads wanting to do OpenGL calls then you'll probably have your own scheduler over a pool of heavyweight threads that are allowed to do the graphics ops. When this project is further along then we might have to add to JNI (we're staying away from that just now because Project Panama is making progress and that may be more important). -Alan From duke at openjdk.java.net Tue Oct 22 15:20:22 2019 From: duke at openjdk.java.net (duke) Date: Tue, 22 Oct 2019 15:20:22 GMT Subject: git: openjdk/loom: fibers: Migrate lightweight threads to Thread API Message-ID: <01801716-85d5-4dc7-823b-a776fd559849@openjdk.java.net> Changeset: 82fb1e3c Author: Alan Bateman Date: 2019-10-22 11:55:51 +0000 URL: https://git.openjdk.java.net/loom/commit/82fb1e3c Migrate lightweight threads to Thread API ! src/hotspot/share/classfile/javaClasses.cpp ! src/hotspot/share/classfile/javaClasses.hpp ! src/hotspot/share/classfile/systemDictionary.hpp ! src/hotspot/share/classfile/vmSymbols.hpp ! src/hotspot/share/runtime/thread.cpp ! src/hotspot/share/runtime/vmStructs.cpp ! src/java.base/linux/classes/sun/nio/ch/EPollPoller.java ! src/java.base/linux/classes/sun/nio/ch/PollerProvider.java ! src/java.base/macosx/classes/sun/nio/ch/KQueuePoller.java ! src/java.base/macosx/classes/sun/nio/ch/PollerProvider.java ! src/java.base/share/classes/java/lang/Fiber.java - src/java.base/share/classes/java/lang/FiberScope.java - src/java.base/share/classes/java/lang/InheritableThreadContext.java ! src/java.base/share/classes/java/lang/InheritableThreadLocal.java ! src/java.base/share/classes/java/lang/Object.java - src/java.base/share/classes/java/lang/ShadowThread.java ! src/java.base/share/classes/java/lang/StringCoding.java ! src/java.base/share/classes/java/lang/System.java ! src/java.base/share/classes/java/lang/Thread.java ! src/java.base/share/classes/java/lang/ThreadGroup.java ! src/java.base/share/classes/java/lang/ThreadLocal.java ! src/java.base/share/classes/java/net/InetAddress.java ! src/java.base/share/classes/java/util/concurrent/ConcurrentHashMap.java ! src/java.base/share/classes/java/util/concurrent/ForkJoinPool.java ! src/java.base/share/classes/java/util/concurrent/ForkJoinWorkerThread.java ! src/java.base/share/classes/java/util/concurrent/SynchronousQueue.java ! src/java.base/share/classes/java/util/concurrent/locks/AbstractOwnableSynchronizer.java ! src/java.base/share/classes/java/util/concurrent/locks/AbstractQueuedLongSynchronizer.java ! src/java.base/share/classes/java/util/concurrent/locks/AbstractQueuedSynchronizer.java ! src/java.base/share/classes/java/util/concurrent/locks/LockSupport.java ! src/java.base/share/classes/java/util/concurrent/locks/ReentrantLock.java ! src/java.base/share/classes/jdk/internal/access/JavaLangAccess.java ! src/java.base/share/classes/jdk/internal/math/FloatingDecimal.java ! src/java.base/share/classes/jdk/internal/math/FormattedFloatingDecimal.java ! src/java.base/share/classes/jdk/internal/misc/Blocker.java ! src/java.base/share/classes/jdk/internal/misc/InnocuousThread.java + src/java.base/share/classes/jdk/internal/misc/LightweightThreads.java - src/java.base/share/classes/jdk/internal/misc/Strands.java ! src/java.base/share/classes/jdk/internal/misc/TerminatingThreadLocal.java ! src/java.base/share/classes/jdk/internal/reflect/NativeConstructorAccessorImpl.java ! src/java.base/share/classes/jdk/internal/reflect/NativeMethodAccessorImpl.java ! src/java.base/share/classes/sun/nio/ch/DatagramChannelImpl.java ! src/java.base/share/classes/sun/nio/ch/IOVecWrapper.java ! src/java.base/share/classes/sun/nio/ch/NioSocketImpl.java ! src/java.base/share/classes/sun/nio/ch/Poller.java ! src/java.base/share/classes/sun/nio/ch/SelChImpl.java ! src/java.base/share/classes/sun/nio/ch/SelectorImpl.java ! src/java.base/share/classes/sun/nio/ch/ServerSocketChannelImpl.java ! src/java.base/share/classes/sun/nio/ch/SocketChannelImpl.java ! src/java.base/share/classes/sun/nio/ch/Util.java ! src/java.base/share/classes/sun/nio/cs/ThreadLocalCoders.java ! src/java.base/share/classes/sun/nio/fs/NativeBuffers.java ! src/java.base/share/native/libjava/Thread.c ! src/java.base/unix/classes/sun/nio/ch/NativeThread.java ! src/java.base/unix/classes/sun/nio/ch/SinkChannelImpl.java ! src/java.base/unix/classes/sun/nio/ch/SourceChannelImpl.java ! src/java.base/unix/classes/sun/nio/fs/UnixPath.java ! src/java.base/windows/classes/sun/nio/ch/NativeThread.java ! src/java.base/windows/classes/sun/nio/ch/PollerProvider.java ! test/hotspot/jtreg/serviceability/jvmti/DoContinueSingleStepTest/DoContinueSingleStepTest.java ! test/hotspot/jtreg/serviceability/jvmti/FiberTest/MyPackage/FiberTest.java - test/jdk/java/lang/Fiber/Basic.java - test/jdk/java/lang/Fiber/NetSockets.java - test/jdk/java/lang/Fiber/NioChannels.java - test/jdk/java/lang/Fiber/Scopes.java + test/jdk/java/lang/Thread/lightweight/Collectable.java + test/jdk/java/lang/Thread/lightweight/Locking.java + test/jdk/java/lang/Thread/lightweight/NetSockets.java + test/jdk/java/lang/Thread/lightweight/NioChannels.java + test/jdk/java/lang/Thread/lightweight/Parking.java + test/jdk/java/lang/Thread/lightweight/TestHelper.java + test/jdk/java/lang/Thread/lightweight/ThreadAPI.java + test/jdk/java/lang/Thread/lightweight/ThreadLocals.java + test/jdk/java/lang/Thread/lightweight/WaitNotify.java From org.openjdk at io7m.com Tue Oct 22 15:36:10 2019 From: org.openjdk at io7m.com (Mark Raynsford) Date: Tue, 22 Oct 2019 16:36:10 +0100 Subject: A lightweight thread is a Thread In-Reply-To: References: <20191022152245.16ea1f30@almond.int.arc7.info> Message-ID: <20191022163610.0f0058bb@almond.int.arc7.info> On 2019-10-22T16:07:14 +0100 Ron Pressler wrote: > > On 22 October 2019 at 15:23:09, Mark Raynsford (org.openjdk at io7m.com(mailto:org.openjdk at io7m.com)) wrote: > > How can I be > > sure that each thread I've created corresponds to exactly one operating > > system thread? > > By making it a heavyweight thread. When you create a Thread you choose whether > it is scheduled by the OS (heavyweight) or by the JDK (lightweight). Ok, good. > > I feel like having a distinction at the type level between a Fiber and > > a Thread makes the difference in cost model explicit, even if there's > > no other API difference. > > > > I'm reminded of FreeBSD, for example, which used to have an m:n model > > for its own pthreads library, but eventually switched back to a 1:1 > > model for performance reasons. I believe one of the issues was that > > over a decade of software had grown up around the idea that a thread > > had a particular cost model (in other words, was a heavyweight > > scheduling primitive) and that by changing the cost model to an m:n > > model ended up making most software perform poorly. > > People hardly use the Thread API directly *today*, relying instead on? > j.u.c.Executor and the like.?This will likely remain the same with lightweight? > threads: People will use the structured-concurrency constructs to manage them. > > Where Thread is used directly is in calls to Thread.currentThread (and, to a > Lesser degree, the interruption mechanism, but that?s a separate discussion). > This use is so pervasive that breaking assumptions about it would mean little > existing code could run in lightweight threads. > > Now we get to have our cake and eat it too. Users will mostly interact with > a new API, but those places where Thread is used extensively will continue > working without change. Ok, right. I too fall into the camp of pretty much only using the Executor/ExecutorService constructs - except where native code forces me to do otherwise. I was concerned when I saw Alan's original email that the ability to have any say in whether something was a Thread or a Fiber would be taken away. -- Mark Raynsford | http://www.io7m.com From org.openjdk at io7m.com Tue Oct 22 15:43:00 2019 From: org.openjdk at io7m.com (Mark Raynsford) Date: Tue, 22 Oct 2019 16:43:00 +0100 Subject: A lightweight thread is a Thread In-Reply-To: <1360aec6-5d70-69fc-0f9d-3d7a1a0bc859@oracle.com> References: <20191022152245.16ea1f30@almond.int.arc7.info> <1360aec6-5d70-69fc-0f9d-3d7a1a0bc859@oracle.com> Message-ID: <20191022164300.24adda99@almond.int.arc7.info> On 2019-10-22T16:19:15 +0100 Alan Bateman wrote: > On 22/10/2019 15:22, Mark Raynsford wrote: > > : > > Will this not have some nasty consequences when talking to native code? > > > > For example, external graphics APIs such as OpenGL and Vulkan have > > strict restrictions on which threads can make calls into the APIs at > > any given time. If I'm calling native code, and I specifically create > > several individual threads for the purposes of doing so... How can I be > > sure that each thread I've created corresponds to exactly one operating > > system thread? > The existing constructors work as before and this project might also > introduce static factory methods to create heavyweight threads. But > maybe you mean something else? If you mean there may be lightweight > threads wanting to do OpenGL calls then you'll probably have your own > scheduler over a pool of heavyweight threads that are allowed to do the > graphics ops. When this project is further along then we might have to > add to JNI (we're staying away from that just now because Project Panama > is making progress and that may be more important). Nope, your first reading was pretty much it. For example, I might create a pair of threads (T0, T1) to manage a pair of OpenGL contexts (C0, C1). It's critical that I only make calls to C0 on T0, and calls to C1 on T1. OpenGL drivers tend to react spectacularly if you get this wrong. Additionally, it's important for performance reasons that T0 and T1 are distinct kernel threads because, if I'm using multiple contexts, I'm almost certainly doing it so that I can run long-running asynchronous operations on one of them (such as uploading large textures), and running rendering commands on the other. You can imagine the fun that could result if T0 and T1 turned out to be lightweight threads both being scheduled on a single kernel thread... "Why are my texture uploads holding up my rendering commands?!". Ron's reply indicated that I will still be able to get the necessary control for the above. -- Mark Raynsford | http://www.io7m.com From elias at vasylenko.uk Tue Oct 22 16:12:23 2019 From: elias at vasylenko.uk (Elias N Vasylenko) Date: Tue, 22 Oct 2019 17:12:23 +0100 Subject: A lightweight thread is a Thread In-Reply-To: <1360aec6-5d70-69fc-0f9d-3d7a1a0bc859@oracle.com> References: <20191022152245.16ea1f30@almond.int.arc7.info> <1360aec6-5d70-69fc-0f9d-3d7a1a0bc859@oracle.com> Message-ID: <91a77048c7f1de6e6aadeb62cbfb2ee07bc63385.camel@vasylenko.uk> This strikes me as similar to the trouble in the Valhalla project in finding where inline types fit into the class hierarchy. IIRC they went through a similar permutation of prototypes, and arrived at a similar conclusion. Just as how lightweight and heavyweight threads should both be Threads, they hope that both inline types and reference types can simply be Objects. This choice also comes with similar drawbacks. Just as inline types don't want to be burdened with all the state and behaviour baggage of the Object class, so it is for fibers wrt the Thread class. Does this seem like a coherent comparison? Well, my point is that one of the solutions which they are exploring is to make Object an abstract class, so that there can be ReferenceObject and InlineObject types can provide their own implementations independently. https://mail.openjdk.java.net/pipermail/valhalla-spec-experts/2019-April/000980.html As described in that thread, this means that existing instances of "new Object" and "extends Object" will have to be transparently rewritten, but this isn't necessarily an impossible goal. Of course it's possible that this will all just be special-cased by the compiler and VM, or that they will end up going down a different road altogether... But if some more general mechanism were provided to evolve API in this way (i.e. to make an existing concrete class abstract without breaking backwards compatibility) could this apply to you down the line? // declares HeavyweightThread as the canonical // concrete implementation so that "new Thread" // and "extends Thread" defer to it. sealed abstract class Thread {} class LightweightThread extends Thread {} class HeavyweightThread extends Thread {} LightweightThread and HeavyweightThread don't need to be exposed in the API of course, they may just an implementation detail. Anyway it's just a bit of curiosity and a thought for the future. Of course I'm not suggesting making any design decisions based on something so speculative. On Tue, 2019-10-22 at 16:19 +0100, Alan Bateman wrote: > On 22/10/2019 15:22, Mark Raynsford wrote: > > : > > Will this not have some nasty consequences when talking to native > > code? > > > > For example, external graphics APIs such as OpenGL and Vulkan have > > strict restrictions on which threads can make calls into the APIs > > at > > any given time. If I'm calling native code, and I specifically > > create > > several individual threads for the purposes of doing so... How can > > I be > > sure that each thread I've created corresponds to exactly one > > operating > > system thread? > The existing constructors work as before and this project might also > introduce static factory methods to create heavyweight threads. But > maybe you mean something else? If you mean there may be lightweight > threads wanting to do OpenGL calls then you'll probably have your > own > scheduler over a pool of heavyweight threads that are allowed to do > the > graphics ops. When this project is further along then we might have > to > add to JNI (we're staying away from that just now because Project > Panama > is making progress and that may be more important). > > -Alan From david.lloyd at redhat.com Tue Oct 22 16:24:59 2019 From: david.lloyd at redhat.com (David Lloyd) Date: Tue, 22 Oct 2019 11:24:59 -0500 Subject: A lightweight thread is a Thread In-Reply-To: References: Message-ID: I like this idea! It feels a lot less risky... On Tue, Oct 22, 2019 at 9:00 AM Alan Bateman wrote: > > This project has been trying to figure out the API to expose for > lightweight threads, and its relationship with the Thread API, for some > time. There has been many prototypes (lightweight thread = Thread, Fiber > <: Thread, Fiber and Thread <: Strand, disjoint Fiber and Thread types). > Thread.currentThread() and thread locals featured in all prototypes as > it's critical that the current Thread cannot change value in a thread of > execution. > > I think we are at the point where we are concluding that a lightweight > thread should be represented by a Thread. No Fiber or other sub-class > exposed in the API. We'll use the term "lightweight thread" rather than > "fiber". The approach maintains the mental model of a thread of > execution that we are familiar with here, and the mental model that > developers coming to the platform should be able to figure out. The > Thread API does come with baggage (ThreadGroup, TCCL, ...) but it may > not be too bad when you consider that most developers don't use the > Thread API directly. In addition, we can degrade and/or eliminate at > least some of this baggage over time, starting with the deprecated features. > > Moving to Thread as the API for lightweight threads means we need to > restart some of the API work. In particular the support for structured > concurrency and cancellation will need to move aside temporarily as we > rebuild from the bottom. I will push a change to the "fibers" branch > soon with the first steps in that direction. The first steps include 4 > static factory methods on Thread to create lightweight threads > (unnamed/named * default or BYO scheduler). There are several discussion > points around what lightweight threads support and don't support. For > now, the default is support thread locals and not inherit thread locals > but it can overridden by specifying characteristics to the factory > methods. The API is primitive for now and will likely change many times. > > API aside, one of the benefits of representing a lightweight thread as a > Thread is that it makes it possible to make progress in the > serviceability areas that we've had to ignore to date. We can finally > start to figure how the existing monitoring and management APIs, JFR, > and diagnostic features might work with lightweight threads. Good > progress has been made on JVM TI and the debugger support in the last > year and it shouldn't require too much re-work to get the debugger > support re-aligned. > > One final thing for now is footprint. There will be some refactoring of > Thread required to reduce its footprint. The initial changes will move > some of fields that aren't performance critical to a sidecar/helper > object. Doug Lea has changes to the FJ code in the works that should > avoid the need for padding/@Contended on the fields used by > ThreadLocalRandom. The object size will, at least initially, be a bit > larger that I had hoped but I'm sure we will improve this in time. > > -Alan. -- - DML From david.lloyd at redhat.com Tue Oct 22 16:27:40 2019 From: david.lloyd at redhat.com (David Lloyd) Date: Tue, 22 Oct 2019 11:27:40 -0500 Subject: A lightweight thread is a Thread In-Reply-To: <1360aec6-5d70-69fc-0f9d-3d7a1a0bc859@oracle.com> References: <20191022152245.16ea1f30@almond.int.arc7.info> <1360aec6-5d70-69fc-0f9d-3d7a1a0bc859@oracle.com> Message-ID: On Tue, Oct 22, 2019 at 10:19 AM Alan Bateman wrote: > > On 22/10/2019 15:22, Mark Raynsford wrote: > > : > > Will this not have some nasty consequences when talking to native code? > > > > For example, external graphics APIs such as OpenGL and Vulkan have > > strict restrictions on which threads can make calls into the APIs at > > any given time. If I'm calling native code, and I specifically create > > several individual threads for the purposes of doing so... How can I be > > sure that each thread I've created corresponds to exactly one operating > > system thread? > The existing constructors work as before and this project might also > introduce static factory methods to create heavyweight threads. But > maybe you mean something else? If you mean there may be lightweight > threads wanting to do OpenGL calls then you'll probably have your own > scheduler over a pool of heavyweight threads that are allowed to do the > graphics ops. When this project is further along then we might have to > add to JNI (we're staying away from that just now because Project Panama > is making progress and that may be more important). It might be a good time to introduce a builder API for threads akin to ProcessBuilder. The constructors are already a bit out of hand! One would still have to allow subclassing of threads though IMO, even in the lightweight case. This is enormously useful in application server and framework environments (which is to say, a substantial percentage of existing user bases). -- - DML From seth.lytle at gmail.com Tue Oct 22 17:04:05 2019 From: seth.lytle at gmail.com (seth lytle) Date: Tue, 22 Oct 2019 13:04:05 -0400 Subject: A lightweight thread is a Thread In-Reply-To: References: <20191022152245.16ea1f30@almond.int.arc7.info> Message-ID: Do you have any data to back up this assertion ? Ron Pressler wrote: > People hardly use the Thread API directly *today*, relying instead on > j.u.c.Executor and the like. On Tue, Oct 22, 2019 at 11:07 AM Ron Pressler wrote: > > > > > On 22 October 2019 at 15:23:09, Mark Raynsford (org.openjdk at io7m.com(mailto:org.openjdk at io7m.com)) wrote: > > > On 2019-10-22T14:59:18 +0100 > > Alan Bateman wrote: > > > > > > I think we are at the point where we are concluding that a lightweight > > > thread should be represented by a Thread. > > > > Will this not have some nasty consequences when talking to native code? > > > > For example, external graphics APIs such as OpenGL and Vulkan have > > strict restrictions on which threads can make calls into the APIs at > > any given time. If I'm calling native code, and I specifically create > > several individual threads for the purposes of doing so... How can I be > > sure that each thread I've created corresponds to exactly one operating > > system thread? > > By making it a heavyweight thread. When you create a Thread you choose whether > it is scheduled by the OS (heavyweight) or by the JDK (lightweight). > > > > > > I feel like having a distinction at the type level between a Fiber and > > a Thread makes the difference in cost model explicit, even if there's > > no other API difference. > > > > I'm reminded of FreeBSD, for example, which used to have an m:n model > > for its own pthreads library, but eventually switched back to a 1:1 > > model for performance reasons. I believe one of the issues was that > > over a decade of software had grown up around the idea that a thread > > had a particular cost model (in other words, was a heavyweight > > scheduling primitive) and that by changing the cost model to an m:n > > model ended up making most software perform poorly. > > > People hardly use the Thread API directly *today*, relying instead on > j.u.c.Executor and the like. This will likely remain the same with lightweight > threads: People will use the structured-concurrency constructs to manage them. > > Where Thread is used directly is in calls to Thread.currentThread (and, to a > Lesser degree, the interruption mechanism, but that?s a separate discussion). > This use is so pervasive that breaking assumptions about it would mean little > existing code could run in lightweight threads. > > Now we get to have our cake and eat it too. Users will mostly interact with > a new API, but those places where Thread is used extensively will continue > working without change. > > > Ron From brian.goetz at oracle.com Wed Oct 23 01:47:02 2019 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 22 Oct 2019 21:47:02 -0400 Subject: A lightweight thread is a Thread In-Reply-To: References: <20191022152245.16ea1f30@almond.int.arc7.info> Message-ID: Pretty much only the JUC implementation should be using the Thread API directly. So if the assertion is wrong, we should still proceed as if it is right. > On Oct 22, 2019, at 1:04 PM, seth lytle wrote: > > Do you have any data to back up this assertion ? > > Ron Pressler wrote: >> People hardly use the Thread API directly *today*, relying instead on >> j.u.c.Executor and the like. > > > > On Tue, Oct 22, 2019 at 11:07 AM Ron Pressler wrote: >> >> >> >> >> On 22 October 2019 at 15:23:09, Mark Raynsford (org.openjdk at io7m.com(mailto:org.openjdk at io7m.com)) wrote: >> >>> On 2019-10-22T14:59:18 +0100 >>> Alan Bateman wrote: >>>> >>>> I think we are at the point where we are concluding that a lightweight >>>> thread should be represented by a Thread. >>> >>> Will this not have some nasty consequences when talking to native code? >>> >>> For example, external graphics APIs such as OpenGL and Vulkan have >>> strict restrictions on which threads can make calls into the APIs at >>> any given time. If I'm calling native code, and I specifically create >>> several individual threads for the purposes of doing so... How can I be >>> sure that each thread I've created corresponds to exactly one operating >>> system thread? >> >> By making it a heavyweight thread. When you create a Thread you choose whether >> it is scheduled by the OS (heavyweight) or by the JDK (lightweight). >> >> >>> >>> I feel like having a distinction at the type level between a Fiber and >>> a Thread makes the difference in cost model explicit, even if there's >>> no other API difference. >>> >>> I'm reminded of FreeBSD, for example, which used to have an m:n model >>> for its own pthreads library, but eventually switched back to a 1:1 >>> model for performance reasons. I believe one of the issues was that >>> over a decade of software had grown up around the idea that a thread >>> had a particular cost model (in other words, was a heavyweight >>> scheduling primitive) and that by changing the cost model to an m:n >>> model ended up making most software perform poorly. >> >> >> People hardly use the Thread API directly *today*, relying instead on >> j.u.c.Executor and the like. This will likely remain the same with lightweight >> threads: People will use the structured-concurrency constructs to manage them. >> >> Where Thread is used directly is in calls to Thread.currentThread (and, to a >> Lesser degree, the interruption mechanism, but that?s a separate discussion). >> This use is so pervasive that breaking assumptions about it would mean little >> existing code could run in lightweight threads. >> >> Now we get to have our cake and eat it too. Users will mostly interact with >> a new API, but those places where Thread is used extensively will continue >> working without change. >> >> >> Ron From forax at univ-mlv.fr Wed Oct 23 06:51:56 2019 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 23 Oct 2019 08:51:56 +0200 (CEST) Subject: A lightweight thread is a Thread In-Reply-To: References: <20191022152245.16ea1f30@almond.int.arc7.info> Message-ID: <1114963774.911635.1571813516946.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "seth lytle" > ?: "Ron Pressler" > Cc: "loom-dev" > Envoy?: Mardi 22 Octobre 2019 19:04:05 > Objet: Re: A lightweight thread is a Thread > Do you have any data to back up this assertion ? The numerous jars that implement a reactive and/or async and/or worker API are using a j.u.c.Executor instead of a plain old j.l.Thread on Maven Central ? R?mi [1] https://mvnrepository.com/repos/central > > Ron Pressler wrote: >> People hardly use the Thread API directly *today*, relying instead on >> j.u.c.Executor and the like. > > > > On Tue, Oct 22, 2019 at 11:07 AM Ron Pressler wrote: >> >> >> >> >> On 22 October 2019 at 15:23:09, Mark Raynsford >> (org.openjdk at io7m.com(mailto:org.openjdk at io7m.com)) wrote: >> >> > On 2019-10-22T14:59:18 +0100 >> > Alan Bateman wrote: >> > > >> > > I think we are at the point where we are concluding that a lightweight >> > > thread should be represented by a Thread. >> > >> > Will this not have some nasty consequences when talking to native code? >> > >> > For example, external graphics APIs such as OpenGL and Vulkan have >> > strict restrictions on which threads can make calls into the APIs at >> > any given time. If I'm calling native code, and I specifically create >> > several individual threads for the purposes of doing so... How can I be >> > sure that each thread I've created corresponds to exactly one operating >> > system thread? >> >> By making it a heavyweight thread. When you create a Thread you choose whether >> it is scheduled by the OS (heavyweight) or by the JDK (lightweight). >> >> >> > >> > I feel like having a distinction at the type level between a Fiber and >> > a Thread makes the difference in cost model explicit, even if there's >> > no other API difference. >> > >> > I'm reminded of FreeBSD, for example, which used to have an m:n model >> > for its own pthreads library, but eventually switched back to a 1:1 >> > model for performance reasons. I believe one of the issues was that >> > over a decade of software had grown up around the idea that a thread >> > had a particular cost model (in other words, was a heavyweight >> > scheduling primitive) and that by changing the cost model to an m:n >> > model ended up making most software perform poorly. >> >> >> People hardly use the Thread API directly *today*, relying instead on >> j.u.c.Executor and the like. This will likely remain the same with lightweight >> threads: People will use the structured-concurrency constructs to manage them. >> >> Where Thread is used directly is in calls to Thread.currentThread (and, to a >> Lesser degree, the interruption mechanism, but that?s a separate discussion). >> This use is so pervasive that breaking assumptions about it would mean little >> existing code could run in lightweight threads. >> >> Now we get to have our cake and eat it too. Users will mostly interact with >> a new API, but those places where Thread is used extensively will continue >> working without change. >> >> > > Ron From bsideup at gmail.com Wed Oct 23 06:55:25 2019 From: bsideup at gmail.com (Sergei Egorov) Date: Wed, 23 Oct 2019 08:55:25 +0200 Subject: A lightweight thread is a Thread In-Reply-To: <1114963774.911635.1571813516946.JavaMail.zimbra@u-pem.fr> References: <20191022152245.16ea1f30@almond.int.arc7.info> <1114963774.911635.1571813516946.JavaMail.zimbra@u-pem.fr> Message-ID: > The numerous jars that implement a reactive and/or async and/or worker API are using a j.u.c.Executor instead of a plain old j.l.Thread on Maven Central ? FTR Project Reactor uses ExecutorService/ScheduledExecutorService, even for Schedulers.single() [1] [1] https://github.com/reactor/reactor-core/blob/9e03cfbdfe4e91aaae5a513fd2bf96a6ad7701de/reactor-core/src/main/java/reactor/core/scheduler/SingleScheduler.java#L69 On Wed, Oct 23, 2019 at 8:52 AM Remi Forax wrote: > ----- Mail original ----- > > De: "seth lytle" > > ?: "Ron Pressler" > > Cc: "loom-dev" > > Envoy?: Mardi 22 Octobre 2019 19:04:05 > > Objet: Re: A lightweight thread is a Thread > > > Do you have any data to back up this assertion ? > > The numerous jars that implement a reactive and/or async and/or worker API > are using a j.u.c.Executor instead of a plain old j.l.Thread on Maven > Central ? > > R?mi > > [1] https://mvnrepository.com/repos/central > > > > > Ron Pressler wrote: > >> People hardly use the Thread API directly *today*, relying instead on > >> j.u.c.Executor and the like. > > > > > > > > On Tue, Oct 22, 2019 at 11:07 AM Ron Pressler > wrote: > >> > >> > >> > >> > >> On 22 October 2019 at 15:23:09, Mark Raynsford > >> (org.openjdk at io7m.com(mailto:org.openjdk at io7m.com)) wrote: > >> > >> > On 2019-10-22T14:59:18 +0100 > >> > Alan Bateman wrote: > >> > > > >> > > I think we are at the point where we are concluding that a > lightweight > >> > > thread should be represented by a Thread. > >> > > >> > Will this not have some nasty consequences when talking to native > code? > >> > > >> > For example, external graphics APIs such as OpenGL and Vulkan have > >> > strict restrictions on which threads can make calls into the APIs at > >> > any given time. If I'm calling native code, and I specifically create > >> > several individual threads for the purposes of doing so... How can I > be > >> > sure that each thread I've created corresponds to exactly one > operating > >> > system thread? > >> > >> By making it a heavyweight thread. When you create a Thread you choose > whether > >> it is scheduled by the OS (heavyweight) or by the JDK (lightweight). > >> > >> > >> > > >> > I feel like having a distinction at the type level between a Fiber and > >> > a Thread makes the difference in cost model explicit, even if there's > >> > no other API difference. > >> > > >> > I'm reminded of FreeBSD, for example, which used to have an m:n model > >> > for its own pthreads library, but eventually switched back to a 1:1 > >> > model for performance reasons. I believe one of the issues was that > >> > over a decade of software had grown up around the idea that a thread > >> > had a particular cost model (in other words, was a heavyweight > >> > scheduling primitive) and that by changing the cost model to an m:n > >> > model ended up making most software perform poorly. > >> > >> > >> People hardly use the Thread API directly *today*, relying instead on > >> j.u.c.Executor and the like. This will likely remain the same with > lightweight > >> threads: People will use the structured-concurrency constructs to > manage them. > >> > >> Where Thread is used directly is in calls to Thread.currentThread (and, > to a > >> Lesser degree, the interruption mechanism, but that?s a separate > discussion). > >> This use is so pervasive that breaking assumptions about it would mean > little > >> existing code could run in lightweight threads. > >> > >> Now we get to have our cake and eat it too. Users will mostly interact > with > >> a new API, but those places where Thread is used extensively will > continue > >> working without change. > >> > >> > > > Ron > -- Best regards, Sergei Egorov From aph at redhat.com Wed Oct 23 08:59:48 2019 From: aph at redhat.com (Andrew Haley) Date: Wed, 23 Oct 2019 09:59:48 +0100 Subject: A lightweight thread is a Thread In-Reply-To: References: <20191022152245.16ea1f30@almond.int.arc7.info> <1360aec6-5d70-69fc-0f9d-3d7a1a0bc859@oracle.com> Message-ID: <93d42320-b574-b989-94d2-7762bbdb655e@redhat.com> On 10/22/19 5:27 PM, David Lloyd wrote: > One would still have to allow subclassing of threads though IMO, even > in the lightweight case. This is enormously useful What is it about subclassing Thread that is enormously useful? I can't immediately think of any case where a scoped local couldn't achieve the same thing. Instead of (MyFoo)Thread.current(); you'd have CurrentFoo.get(); ... which would work in lightweight and heavyweight Threads. Is there something that definitely wouldn't work with this mechanism? -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From forax at univ-mlv.fr Wed Oct 23 09:38:16 2019 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 23 Oct 2019 11:38:16 +0200 (CEST) Subject: A lightweight thread is a Thread In-Reply-To: <93d42320-b574-b989-94d2-7762bbdb655e@redhat.com> References: <20191022152245.16ea1f30@almond.int.arc7.info> <1360aec6-5d70-69fc-0f9d-3d7a1a0bc859@oracle.com> <93d42320-b574-b989-94d2-7762bbdb655e@redhat.com> Message-ID: <524249269.1026916.1571823496761.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Andrew Haley" > ?: "David Lloyd" , "Alan Bateman" > Cc: "loom-dev" > Envoy?: Mercredi 23 Octobre 2019 10:59:48 > Objet: Re: A lightweight thread is a Thread > On 10/22/19 5:27 PM, David Lloyd wrote: >> One would still have to allow subclassing of threads though IMO, even >> in the lightweight case. This is enormously useful > > What is it about subclassing Thread that is enormously useful? I can't > immediately think of any case where a scoped local couldn't achieve the > same thing. Instead of > > (MyFoo)Thread.current(); > > you'd have > > CurrentFoo.get(); > > ... which would work in lightweight and heavyweight Threads. Is there > something that definitely wouldn't work with this mechanism ? all codes that already exist :) being able to just do a super(..., /*lightweight*/ true) to use a lightweight thread instead of an heavyweight one without having to rewrite all the client code of that subclass of Thread that already exist. > > -- > Andrew Haley (he/him) > Java Platform Lead Engineer > Red Hat UK Ltd. > https://keybase.io/andrewhaley > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 R?mi From volkan.yazici at gmail.com Wed Oct 23 10:45:39 2019 From: volkan.yazici at gmail.com (=?UTF-8?B?Vm9sa2FuIFlhesSxY8Sx?=) Date: Wed, 23 Oct 2019 12:45:39 +0200 Subject: A lightweight thread is a Thread In-Reply-To: References: <20191022152245.16ea1f30@almond.int.arc7.info> <1114963774.911635.1571813516946.JavaMail.zimbra@u-pem.fr> Message-ID: I have my doubts about that. For BlockHound (great tool BTW) one needs to extend Thread's from reactor.core.scheduler.NonBlocking, hence yet another use case for "new Thread". AFAIC, many libraries leverage similar Thread labeling techniques to implement business logic based on the type of the carrier Thread. Could one achieve a similar functionality without using "new Thread"? On Wed, Oct 23, 2019 at 8:57 AM Sergei Egorov wrote: > > The numerous jars that implement a reactive and/or async and/or worker > API are using a j.u.c.Executor instead of a plain old j.l.Thread on Maven > Central ? > > FTR Project Reactor uses ExecutorService/ScheduledExecutorService, even for > Schedulers.single() [1] > > > [1] > > https://github.com/reactor/reactor-core/blob/9e03cfbdfe4e91aaae5a513fd2bf96a6ad7701de/reactor-core/src/main/java/reactor/core/scheduler/SingleScheduler.java#L69 > > On Wed, Oct 23, 2019 at 8:52 AM Remi Forax wrote: > > > ----- Mail original ----- > > > De: "seth lytle" > > > ?: "Ron Pressler" > > > Cc: "loom-dev" > > > Envoy?: Mardi 22 Octobre 2019 19:04:05 > > > Objet: Re: A lightweight thread is a Thread > > > > > Do you have any data to back up this assertion ? > > > > The numerous jars that implement a reactive and/or async and/or worker > API > > are using a j.u.c.Executor instead of a plain old j.l.Thread on Maven > > Central ? > > > > R?mi > > > > [1] https://mvnrepository.com/repos/central > > > > > > > > Ron Pressler wrote: > > >> People hardly use the Thread API directly *today*, relying instead on > > >> j.u.c.Executor and the like. > > > > > > > > > > > > On Tue, Oct 22, 2019 at 11:07 AM Ron Pressler > > > wrote: > > >> > > >> > > >> > > >> > > >> On 22 October 2019 at 15:23:09, Mark Raynsford > > >> (org.openjdk at io7m.com(mailto:org.openjdk at io7m.com)) wrote: > > >> > > >> > On 2019-10-22T14:59:18 +0100 > > >> > Alan Bateman wrote: > > >> > > > > >> > > I think we are at the point where we are concluding that a > > lightweight > > >> > > thread should be represented by a Thread. > > >> > > > >> > Will this not have some nasty consequences when talking to native > > code? > > >> > > > >> > For example, external graphics APIs such as OpenGL and Vulkan have > > >> > strict restrictions on which threads can make calls into the APIs at > > >> > any given time. If I'm calling native code, and I specifically > create > > >> > several individual threads for the purposes of doing so... How can I > > be > > >> > sure that each thread I've created corresponds to exactly one > > operating > > >> > system thread? > > >> > > >> By making it a heavyweight thread. When you create a Thread you choose > > whether > > >> it is scheduled by the OS (heavyweight) or by the JDK (lightweight). > > >> > > >> > > >> > > > >> > I feel like having a distinction at the type level between a Fiber > and > > >> > a Thread makes the difference in cost model explicit, even if > there's > > >> > no other API difference. > > >> > > > >> > I'm reminded of FreeBSD, for example, which used to have an m:n > model > > >> > for its own pthreads library, but eventually switched back to a 1:1 > > >> > model for performance reasons. I believe one of the issues was that > > >> > over a decade of software had grown up around the idea that a thread > > >> > had a particular cost model (in other words, was a heavyweight > > >> > scheduling primitive) and that by changing the cost model to an m:n > > >> > model ended up making most software perform poorly. > > >> > > >> > > >> People hardly use the Thread API directly *today*, relying instead on > > >> j.u.c.Executor and the like. This will likely remain the same with > > lightweight > > >> threads: People will use the structured-concurrency constructs to > > manage them. > > >> > > >> Where Thread is used directly is in calls to Thread.currentThread > (and, > > to a > > >> Lesser degree, the interruption mechanism, but that?s a separate > > discussion). > > >> This use is so pervasive that breaking assumptions about it would mean > > little > > >> existing code could run in lightweight threads. > > >> > > >> Now we get to have our cake and eat it too. Users will mostly interact > > with > > >> a new API, but those places where Thread is used extensively will > > continue > > >> working without change. > > >> > > >> > > > > Ron > > > > > -- > Best regards, > Sergei Egorov > From volkan.yazici at gmail.com Wed Oct 23 10:55:40 2019 From: volkan.yazici at gmail.com (=?UTF-8?B?Vm9sa2FuIFlhesSxY8Sx?=) Date: Wed, 23 Oct 2019 12:55:40 +0200 Subject: A lightweight thread is a Thread In-Reply-To: References: <20191022152245.16ea1f30@almond.int.arc7.info> Message-ID: If this (purposing j.l.Thread only within JSL) is the direction Java is aiming for, why not deprecating it over time? If there are still cases where such a deprecation would be too harsh and/or one might still need to reach for them, doesn't this constitute the sufficient justification for the explicit accessibility of this API? The bottom line of my comment is, if they are gonna be exposed, let's embrace this as a fact and make it explicit and accessible as a first-class citizen; otherwise, let them decay over time and communicate this intent in the API too. On Wed, Oct 23, 2019 at 3:47 AM Brian Goetz wrote: > Pretty much only the JUC implementation should be using the Thread API > directly. So if the assertion is wrong, we should still proceed as if it > is right. > > > On Oct 22, 2019, at 1:04 PM, seth lytle wrote: > > > > Do you have any data to back up this assertion ? > > > > Ron Pressler wrote: > >> People hardly use the Thread API directly *today*, relying instead on > >> j.u.c.Executor and the like. > > > > > > > > On Tue, Oct 22, 2019 at 11:07 AM Ron Pressler > wrote: > >> > >> > >> > >> > >> On 22 October 2019 at 15:23:09, Mark Raynsford (org.openjdk at io7m.com > (mailto:org.openjdk at io7m.com)) wrote: > >> > >>> On 2019-10-22T14:59:18 +0100 > >>> Alan Bateman wrote: > >>>> > >>>> I think we are at the point where we are concluding that a lightweight > >>>> thread should be represented by a Thread. > >>> > >>> Will this not have some nasty consequences when talking to native code? > >>> > >>> For example, external graphics APIs such as OpenGL and Vulkan have > >>> strict restrictions on which threads can make calls into the APIs at > >>> any given time. If I'm calling native code, and I specifically create > >>> several individual threads for the purposes of doing so... How can I be > >>> sure that each thread I've created corresponds to exactly one operating > >>> system thread? > >> > >> By making it a heavyweight thread. When you create a Thread you choose > whether > >> it is scheduled by the OS (heavyweight) or by the JDK (lightweight). > >> > >> > >>> > >>> I feel like having a distinction at the type level between a Fiber and > >>> a Thread makes the difference in cost model explicit, even if there's > >>> no other API difference. > >>> > >>> I'm reminded of FreeBSD, for example, which used to have an m:n model > >>> for its own pthreads library, but eventually switched back to a 1:1 > >>> model for performance reasons. I believe one of the issues was that > >>> over a decade of software had grown up around the idea that a thread > >>> had a particular cost model (in other words, was a heavyweight > >>> scheduling primitive) and that by changing the cost model to an m:n > >>> model ended up making most software perform poorly. > >> > >> > >> People hardly use the Thread API directly *today*, relying instead on > >> j.u.c.Executor and the like. This will likely remain the same with > lightweight > >> threads: People will use the structured-concurrency constructs to > manage them. > >> > >> Where Thread is used directly is in calls to Thread.currentThread (and, > to a > >> Lesser degree, the interruption mechanism, but that?s a separate > discussion). > >> This use is so pervasive that breaking assumptions about it would mean > little > >> existing code could run in lightweight threads. > >> > >> Now we get to have our cake and eat it too. Users will mostly interact > with > >> a new API, but those places where Thread is used extensively will > continue > >> working without change. > >> > >> > >> Ron > > From volkan.yazici at gmail.com Wed Oct 23 10:57:37 2019 From: volkan.yazici at gmail.com (=?UTF-8?B?Vm9sa2FuIFlhesSxY8Sx?=) Date: Wed, 23 Oct 2019 12:57:37 +0200 Subject: A lightweight thread is a Thread In-Reply-To: References: Message-ID: I am assuming the distinction will still be accessible by the user code, right? That is, one could think of cases which necessitate "if (currentThreadIsFiber) { doSth(); } else { doSthElse(); }". On Tue, Oct 22, 2019 at 3:59 PM Alan Bateman wrote: > This project has been trying to figure out the API to expose for > lightweight threads, and its relationship with the Thread API, for some > time. There has been many prototypes (lightweight thread = Thread, Fiber > <: Thread, Fiber and Thread <: Strand, disjoint Fiber and Thread types). > Thread.currentThread() and thread locals featured in all prototypes as > it's critical that the current Thread cannot change value in a thread of > execution. > > I think we are at the point where we are concluding that a lightweight > thread should be represented by a Thread. No Fiber or other sub-class > exposed in the API. We'll use the term "lightweight thread" rather than > "fiber". The approach maintains the mental model of a thread of > execution that we are familiar with here, and the mental model that > developers coming to the platform should be able to figure out. The > Thread API does come with baggage (ThreadGroup, TCCL, ...) but it may > not be too bad when you consider that most developers don't use the > Thread API directly. In addition, we can degrade and/or eliminate at > least some of this baggage over time, starting with the deprecated > features. > > Moving to Thread as the API for lightweight threads means we need to > restart some of the API work. In particular the support for structured > concurrency and cancellation will need to move aside temporarily as we > rebuild from the bottom. I will push a change to the "fibers" branch > soon with the first steps in that direction. The first steps include 4 > static factory methods on Thread to create lightweight threads > (unnamed/named * default or BYO scheduler). There are several discussion > points around what lightweight threads support and don't support. For > now, the default is support thread locals and not inherit thread locals > but it can overridden by specifying characteristics to the factory > methods. The API is primitive for now and will likely change many times. > > API aside, one of the benefits of representing a lightweight thread as a > Thread is that it makes it possible to make progress in the > serviceability areas that we've had to ignore to date. We can finally > start to figure how the existing monitoring and management APIs, JFR, > and diagnostic features might work with lightweight threads. Good > progress has been made on JVM TI and the debugger support in the last > year and it shouldn't require too much re-work to get the debugger > support re-aligned. > > One final thing for now is footprint. There will be some refactoring of > Thread required to reduce its footprint. The initial changes will move > some of fields that aren't performance critical to a sidecar/helper > object. Doug Lea has changes to the FJ code in the works that should > avoid the need for padding/@Contended on the fields used by > ThreadLocalRandom. The object size will, at least initially, be a bit > larger that I had hoped but I'm sure we will improve this in time. > > -Alan. > From Alan.Bateman at oracle.com Wed Oct 23 11:08:23 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 23 Oct 2019 12:08:23 +0100 Subject: A lightweight thread is a Thread In-Reply-To: References: Message-ID: <28dfe7ef-e311-c184-7836-71d44ea67f62@oracle.com> On 23/10/2019 11:57, Volkan Yaz?c? wrote: > I am assuming the distinction will still be accessible by the user > code, right? That is, one could think of cases which necessitate "if > (currentThreadIsFiber) { doSth(); } else { doSthElse(); }". > Yes, its Thread::isLightweight in the current prototype. -Alan. From bsideup at gmail.com Wed Oct 23 11:37:09 2019 From: bsideup at gmail.com (Sergei Egorov) Date: Wed, 23 Oct 2019 13:37:09 +0200 Subject: A lightweight thread is a Thread In-Reply-To: References: <20191022152245.16ea1f30@almond.int.arc7.info> <1114963774.911635.1571813516946.JavaMail.zimbra@u-pem.fr> Message-ID: There is no such requirement in BlockHound, to extend/implement anything. One can provide a Thread instance predicate to mark it blocking/non-blocking. Soon we will support dynamic predicates too, so that the flag is not cached per thread. On Wed, 23 Oct 2019 at 12:45, Volkan Yaz?c? wrote: > I have my doubts about that. For BlockHound (great tool BTW) one needs to > extend Thread's from reactor.core.scheduler.NonBlocking, hence yet another > use case for "new Thread". AFAIC, many libraries leverage similar Thread > labeling techniques to implement business logic based on the type of the > carrier Thread. Could one achieve a similar functionality without using > "new Thread"? > > On Wed, Oct 23, 2019 at 8:57 AM Sergei Egorov wrote: > >> > The numerous jars that implement a reactive and/or async and/or worker >> API are using a j.u.c.Executor instead of a plain old j.l.Thread on Maven >> Central ? >> >> FTR Project Reactor uses ExecutorService/ScheduledExecutorService, even >> for >> Schedulers.single() [1] >> >> >> [1] >> >> https://github.com/reactor/reactor-core/blob/9e03cfbdfe4e91aaae5a513fd2bf96a6ad7701de/reactor-core/src/main/java/reactor/core/scheduler/SingleScheduler.java#L69 >> >> On Wed, Oct 23, 2019 at 8:52 AM Remi Forax wrote: >> >> > ----- Mail original ----- >> > > De: "seth lytle" >> > > ?: "Ron Pressler" >> > > Cc: "loom-dev" >> > > Envoy?: Mardi 22 Octobre 2019 19:04:05 >> > > Objet: Re: A lightweight thread is a Thread >> > >> > > Do you have any data to back up this assertion ? >> > >> > The numerous jars that implement a reactive and/or async and/or worker >> API >> > are using a j.u.c.Executor instead of a plain old j.l.Thread on Maven >> > Central ? >> > >> > R?mi >> > >> > [1] https://mvnrepository.com/repos/central >> > >> > > >> > > Ron Pressler wrote: >> > >> People hardly use the Thread API directly *today*, relying instead on >> > >> j.u.c.Executor and the like. >> > > >> > > >> > > >> > > On Tue, Oct 22, 2019 at 11:07 AM Ron Pressler < >> ron.pressler at oracle.com> >> > wrote: >> > >> >> > >> >> > >> >> > >> >> > >> On 22 October 2019 at 15:23:09, Mark Raynsford >> > >> (org.openjdk at io7m.com(mailto:org.openjdk at io7m.com)) wrote: >> > >> >> > >> > On 2019-10-22T14:59:18 +0100 >> > >> > Alan Bateman wrote: >> > >> > > >> > >> > > I think we are at the point where we are concluding that a >> > lightweight >> > >> > > thread should be represented by a Thread. >> > >> > >> > >> > Will this not have some nasty consequences when talking to native >> > code? >> > >> > >> > >> > For example, external graphics APIs such as OpenGL and Vulkan have >> > >> > strict restrictions on which threads can make calls into the APIs >> at >> > >> > any given time. If I'm calling native code, and I specifically >> create >> > >> > several individual threads for the purposes of doing so... How can >> I >> > be >> > >> > sure that each thread I've created corresponds to exactly one >> > operating >> > >> > system thread? >> > >> >> > >> By making it a heavyweight thread. When you create a Thread you >> choose >> > whether >> > >> it is scheduled by the OS (heavyweight) or by the JDK (lightweight). >> > >> >> > >> >> > >> > >> > >> > I feel like having a distinction at the type level between a Fiber >> and >> > >> > a Thread makes the difference in cost model explicit, even if >> there's >> > >> > no other API difference. >> > >> > >> > >> > I'm reminded of FreeBSD, for example, which used to have an m:n >> model >> > >> > for its own pthreads library, but eventually switched back to a 1:1 >> > >> > model for performance reasons. I believe one of the issues was that >> > >> > over a decade of software had grown up around the idea that a >> thread >> > >> > had a particular cost model (in other words, was a heavyweight >> > >> > scheduling primitive) and that by changing the cost model to an m:n >> > >> > model ended up making most software perform poorly. >> > >> >> > >> >> > >> People hardly use the Thread API directly *today*, relying instead on >> > >> j.u.c.Executor and the like. This will likely remain the same with >> > lightweight >> > >> threads: People will use the structured-concurrency constructs to >> > manage them. >> > >> >> > >> Where Thread is used directly is in calls to Thread.currentThread >> (and, >> > to a >> > >> Lesser degree, the interruption mechanism, but that?s a separate >> > discussion). >> > >> This use is so pervasive that breaking assumptions about it would >> mean >> > little >> > >> existing code could run in lightweight threads. >> > >> >> > >> Now we get to have our cake and eat it too. Users will mostly >> interact >> > with >> > >> a new API, but those places where Thread is used extensively will >> > continue >> > >> working without change. >> > >> >> > >> >> > > > Ron >> > >> >> >> -- >> Best regards, >> Sergei Egorov >> > -- Best regards, Sergei Egorov From david.lloyd at redhat.com Wed Oct 23 12:49:46 2019 From: david.lloyd at redhat.com (David Lloyd) Date: Wed, 23 Oct 2019 07:49:46 -0500 Subject: A lightweight thread is a Thread In-Reply-To: <93d42320-b574-b989-94d2-7762bbdb655e@redhat.com> References: <20191022152245.16ea1f30@almond.int.arc7.info> <1360aec6-5d70-69fc-0f9d-3d7a1a0bc859@oracle.com> <93d42320-b574-b989-94d2-7762bbdb655e@redhat.com> Message-ID: On Wed, Oct 23, 2019 at 3:59 AM Andrew Haley wrote: > > On 10/22/19 5:27 PM, David Lloyd wrote: > > One would still have to allow subclassing of threads though IMO, even > > in the lightweight case. This is enormously useful > > What is it about subclassing Thread that is enormously useful? I can't > immediately think of any case where a scoped local couldn't achieve the > same thing. Instead of > > (MyFoo)Thread.current(); > > you'd have > > CurrentFoo.get(); > > ... which would work in lightweight and heavyweight Threads. Is there > something that definitely wouldn't work with this mechanism? Apart from being the fastest possible way to have thread local values in Java, there are a couple things we can only do by subclassing Thread. In JBoss EAP and WildFly, we support custom interrupt handlers (this is something that is in the JDK but is not public, thus subclassing Thread is the only way to do it) and interrupt deferral (only Thread subclasses can do this). Vert.x/Netty uses a custom thread subclass to support "faster" thread locals (though looking at the code I suspect these "faster" thread locals might be heavier than the JDK's thread local support). Sometimes you want to be able to do `ThreadSubclass foo = notCurrentThread(); foo.something();`. From what I understand, one can't easily get a scoped local for some other thread. We also use it for user-supplied thread shutdown hooks, but of course that can also be done by wrapping the Runnable (via ThreadFactory for instance). I did a search in my IDE's Quarkus code base and in the JDK and supporting projects, there are presently 374 Thread subclasses. I'm not sure what percentage of those really *needs* to be an actual subclass though. Most of the JDK cases (based on a random sampling) appear to just override the `run()` method and probably carry over (at least stylewise) from a very old JDK. A project called "zkclient" is also handling thread interruption, so I guess we're not the only ones. Maybe it's worth making that API be public. -- - DML From aph at redhat.com Wed Oct 23 17:38:28 2019 From: aph at redhat.com (Andrew Haley) Date: Wed, 23 Oct 2019 18:38:28 +0100 Subject: A lightweight thread is a Thread In-Reply-To: References: <20191022152245.16ea1f30@almond.int.arc7.info> <1360aec6-5d70-69fc-0f9d-3d7a1a0bc859@oracle.com> <93d42320-b574-b989-94d2-7762bbdb655e@redhat.com> Message-ID: On 10/23/19 1:49 PM, David Lloyd wrote: > On Wed, Oct 23, 2019 at 3:59 AM Andrew Haley wrote: >> >> On 10/22/19 5:27 PM, David Lloyd wrote: >>> One would still have to allow subclassing of threads though IMO, even >>> in the lightweight case. This is enormously useful >> >> What is it about subclassing Thread that is enormously useful? I can't >> immediately think of any case where a scoped local couldn't achieve the >> same thing. Instead of >> >> (MyFoo)Thread.current(); >> >> you'd have >> >> CurrentFoo.get(); >> >> ... which would work in lightweight and heavyweight Threads. Is there >> something that definitely wouldn't work with this mechanism? > > Apart from being the fastest possible way to have thread local values > in Java, I don't think you'd notice any difference: Benchmark Mode Cnt Score Error Units ThreadLocalTest.getFieldFromLocal avgt 3 1.728 ? 0.018 ns/op ThreadLocalTest.getFieldFromThread avgt 3 1.439 ? 0.003 ns/op @Benchmark public void getFieldFromLocal(MyState state) { Sink.getSink().drain(BenchmarkState.myScoped7.get()); } @Benchmark public void getFieldFromThread(MyState state) { Sink.getSink().drain(((MyThread)Thread.currentThread()).userObjectField); } That's 6 clock cycles (@ 3.5GHz) for the scoped value, 5 for the field in Thread. Darn! :-) In contrast, ThreadLocal: @Benchmark public void getFieldFromThreadLocal(MyState state) { Sink.getSink().drain(BenchmarkState.t1.get()); } Benchmark Mode Cnt Score Error Units ThreadLocalTest.getFieldFromThreadLocal avgt 3 4.922 ? 2.382 ns/op (17 clocks.) The usual caveats: C2 heuristics can completely mess this up, so YMMV. Benchmarking such extremely narrow intervals of time is very tricky, even with JMH. This is a Threadripper 2950X and I think its L1 cache latency is 5 clocks. > there are a couple things we can only do by subclassing Thread. In > JBoss EAP and WildFly, we support custom interrupt handlers (this is > something that is in the JDK but is not public, thus subclassing > Thread is the only way to do it) and interrupt deferral (only Thread > subclasses can do this). I don't know how much of this will be supported by lightweight threads. I could guess but I'll leave that to the people involved. > Vert.x/Netty uses a custom thread subclass to support "faster" > thread locals (though looking at the code I suspect these "faster" > thread locals might be heavier than the JDK's thread local support). @Benchmark public void getFieldFromFastThreadLocal(MyState state) { Sink.getSink().drain(BenchmarkState.f1.get()); } Benchmark Mode Cnt Score Error Units ThreadLocalTest.getFieldFromFastThreadLocal avgt 3 2.287 ? 0.063 ns/op 10 clocks. That's actually pretty damned good considering they didn't touch the JVM. There is the trade-off, though, that every thread that uses this has a fixed-size array of thread locals at least as large as the highest-numbered FastThreadLocal it uses. It's a really sweet idea, but I'm not sure I could get away with it in a general-purpose facility because of scaling issues. Food for thought... > Sometimes you want to be able to do `ThreadSubclass foo = > notCurrentThread(); foo.something();`. From what I understand, one > can't easily get a scoped local for some other thread. True. Apart from any other issues I don't do anything thread-safe. TBH I'm not sure if this is safe for ThreadLocal either, unless they are very careful about concurrent access to the pool of ThreadLocals. > We also use it for user-supplied thread shutdown hooks, but of course > that can also be done by wrapping the Runnable (via ThreadFactory for > instance). > > I did a search in my IDE's Quarkus code base and in the JDK and > supporting projects, there are presently 374 Thread subclasses. I'm > not sure what percentage of those really *needs* to be an actual > subclass though. Most of the JDK cases (based on a random sampling) > appear to just override the `run()` method and probably carry over (at > least stylewise) from a very old JDK. Yes, I'd guess so. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From aph at redhat.com Wed Oct 23 17:41:29 2019 From: aph at redhat.com (Andrew Haley) Date: Wed, 23 Oct 2019 18:41:29 +0100 Subject: A lightweight thread is a Thread In-Reply-To: <524249269.1026916.1571823496761.JavaMail.zimbra@u-pem.fr> References: <20191022152245.16ea1f30@almond.int.arc7.info> <1360aec6-5d70-69fc-0f9d-3d7a1a0bc859@oracle.com> <93d42320-b574-b989-94d2-7762bbdb655e@redhat.com> <524249269.1026916.1571823496761.JavaMail.zimbra@u-pem.fr> Message-ID: <4c86dd4e-7c71-bed9-d6ed-50431cd4c93c@redhat.com> On 10/23/19 10:38 AM, Remi Forax wrote: > >> De: "Andrew Haley" >> On 10/22/19 5:27 PM, David Lloyd wrote: >>> One would still have to allow subclassing of threads though IMO, even >>> in the lightweight case. This is enormously useful >> >> What is it about subclassing Thread that is enormously useful? I can't >> immediately think of any case where a scoped local couldn't achieve the >> same thing. Instead of >> >> (MyFoo)Thread.current(); >> >> you'd have >> >> CurrentFoo.get(); >> >> ... which would work in lightweight and heavyweight Threads. Is there >> something that definitely wouldn't work with this mechanism ? > > all codes that already exist :) > > being able to just do a super(..., /*lightweight*/ true) to use a lightweight thread instead of an heavyweight one without having to rewrite all the client code of that subclass of Thread that already exist. I should have perhaps said "except legacy". If we're going to support all of the current properties of Thread we might as well cancel this project now. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From aph at redhat.com Wed Oct 23 17:45:34 2019 From: aph at redhat.com (Andrew Haley) Date: Wed, 23 Oct 2019 18:45:34 +0100 Subject: A lightweight thread is a Thread In-Reply-To: References: <20191022152245.16ea1f30@almond.int.arc7.info> <1360aec6-5d70-69fc-0f9d-3d7a1a0bc859@oracle.com> <93d42320-b574-b989-94d2-7762bbdb655e@redhat.com> Message-ID: <6d553175-57c3-8d5a-1196-122fa64fc438@redhat.com> On 10/23/19 6:38 PM, Andrew Haley wrote: > Benchmark Mode Cnt Score Error Units > ThreadLocalTest.getFieldFromFastThreadLocal avgt 3 2.287 ? 0.063 ns/op > > 10 clocks. Sorry, 8 clocks. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From forax at univ-mlv.fr Wed Oct 23 17:55:26 2019 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Wed, 23 Oct 2019 19:55:26 +0200 (CEST) Subject: A lightweight thread is a Thread In-Reply-To: <4c86dd4e-7c71-bed9-d6ed-50431cd4c93c@redhat.com> References: <20191022152245.16ea1f30@almond.int.arc7.info> <1360aec6-5d70-69fc-0f9d-3d7a1a0bc859@oracle.com> <93d42320-b574-b989-94d2-7762bbdb655e@redhat.com> <524249269.1026916.1571823496761.JavaMail.zimbra@u-pem.fr> <4c86dd4e-7c71-bed9-d6ed-50431cd4c93c@redhat.com> Message-ID: <271720059.1237021.1571853326500.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Andrew Haley" > ?: "Remi Forax" > Cc: "David Lloyd" , "Alan Bateman" , "loom-dev" > > Envoy?: Mercredi 23 Octobre 2019 19:41:29 > Objet: Re: A lightweight thread is a Thread > On 10/23/19 10:38 AM, Remi Forax wrote: >> >>> De: "Andrew Haley" > >>> On 10/22/19 5:27 PM, David Lloyd wrote: >>>> One would still have to allow subclassing of threads though IMO, even >>>> in the lightweight case. This is enormously useful >>> >>> What is it about subclassing Thread that is enormously useful? I can't >>> immediately think of any case where a scoped local couldn't achieve the >>> same thing. Instead of >>> >>> (MyFoo)Thread.current(); >>> >>> you'd have >>> >>> CurrentFoo.get(); >>> >>> ... which would work in lightweight and heavyweight Threads. Is there >>> something that definitely wouldn't work with this mechanism ? >> >> all codes that already exist :) >> >> being able to just do a super(..., /*lightweight*/ true) to use a lightweight >> thread instead of an heavyweight one without having to rewrite all the client >> code of that subclass of Thread that already exist. > > I should have perhaps said "except legacy". If we're going to support all of > the current properties of Thread we might as well cancel this project now. While i agree that "all" is a dangerous word because those two kind of threads have not the same semantics so "all" is not something we can reach. At the same, time every differences between a fat thread and a lightweight thread may result in a lot of head scratching for our users. Retrofitting is at the same time a dark art and how you grow a language. > > -- > Andrew Haley (he/him) > Java Platform Lead Engineer > Red Hat UK Ltd. > https://keybase.io/andrewhaley > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 R?mi From david.lloyd at redhat.com Wed Oct 23 19:24:26 2019 From: david.lloyd at redhat.com (David Lloyd) Date: Wed, 23 Oct 2019 14:24:26 -0500 Subject: A lightweight thread is a Thread In-Reply-To: References: <20191022152245.16ea1f30@almond.int.arc7.info> <1360aec6-5d70-69fc-0f9d-3d7a1a0bc859@oracle.com> <93d42320-b574-b989-94d2-7762bbdb655e@redhat.com> Message-ID: On Wed, Oct 23, 2019 at 12:38 PM Andrew Haley wrote: > On 10/23/19 1:49 PM, David Lloyd wrote: > > On Wed, Oct 23, 2019 at 3:59 AM Andrew Haley wrote: > >> > >> On 10/22/19 5:27 PM, David Lloyd wrote: > >>> One would still have to allow subclassing of threads though IMO, even > >>> in the lightweight case. This is enormously useful > >> > >> What is it about subclassing Thread that is enormously useful? I can't > >> immediately think of any case where a scoped local couldn't achieve the > >> same thing. Instead of > >> > >> (MyFoo)Thread.current(); > >> > >> you'd have > >> > >> CurrentFoo.get(); > >> > >> ... which would work in lightweight and heavyweight Threads. Is there > >> something that definitely wouldn't work with this mechanism? > > > > Apart from being the fastest possible way to have thread local values > > in Java, > > I don't think you'd notice any difference: > [...] > That's 6 clock cycles (@ 3.5GHz) for the scoped value, 5 for the field > in Thread. Darn! :-) Nice! That's very cool. > > Vert.x/Netty uses a custom thread subclass to support "faster" > > thread locals (though looking at the code I suspect these "faster" > > thread locals might be heavier than the JDK's thread local support). > > @Benchmark > public void getFieldFromFastThreadLocal(MyState state) { > Sink.getSink().drain(BenchmarkState.f1.get()); > } > > Benchmark Mode Cnt Score Error Units > ThreadLocalTest.getFieldFromFastThreadLocal avgt 3 2.287 ? 0.063 ns/op > > [8] clocks. That's actually pretty damned good considering they > didn't touch the JVM. There is the trade-off, though, that every > thread that uses this has a fixed-size array of thread locals at least > as large as the highest-numbered FastThreadLocal it uses. > > It's a really sweet idea, but I'm not sure I could get away with it > in a general-purpose facility because of scaling issues. Food for > thought... Nice, that's better than I expected given my initial impression of the complexity of the classes involved. > > Sometimes you want to be able to do `ThreadSubclass foo = > > notCurrentThread(); foo.something();`. From what I understand, one > > can't easily get a scoped local for some other thread. > > True. Apart from any other issues I don't do anything thread-safe. TBH > I'm not sure if this is safe for ThreadLocal either, unless they are > very careful about concurrent access to the pool of ThreadLocals. Right of course much (perhaps most) of the time that thread local values are used, the values are explicitly and deliberately not thread safe. So you wouldn't normally use that facility for this kind of thread attachment. But maybe a generalized thread attachment mechanism is called for. OTOH perhaps allowing access to the Runnable via a simple getter on Thread would accomplish the exact same thing in a very similar way to what subclassing Thread would do? -- - DML From david.lloyd at redhat.com Wed Oct 23 19:29:02 2019 From: david.lloyd at redhat.com (David Lloyd) Date: Wed, 23 Oct 2019 14:29:02 -0500 Subject: A lightweight thread is a Thread In-Reply-To: References: <20191022152245.16ea1f30@almond.int.arc7.info> <1360aec6-5d70-69fc-0f9d-3d7a1a0bc859@oracle.com> <93d42320-b574-b989-94d2-7762bbdb655e@redhat.com> Message-ID: On Wed, Oct 23, 2019 at 12:38 PM Andrew Haley wrote: > On 10/23/19 1:49 PM, David Lloyd wrote: > > there are a couple things we can only do by subclassing Thread. In > > JBoss EAP and WildFly, we support custom interrupt handlers (this is > > something that is in the JDK but is not public, thus subclassing > > Thread is the only way to do it) and interrupt deferral (only Thread > > subclasses can do this). > > I don't know how much of this will be supported by lightweight > threads. I could guess but I'll leave that to the people involved. Alan, did you have any thoughts about how thread interruption might work with lightweight threads? I imagine that it wouldn't have to be too terribly different in a lightweight thread compared to "normal" threads, but I'm thinking that there is definitely a scheduling effect as interrupt tends to imply a wakeup/unpark in most if not all cases, and there is also a question of whether or how an LWT (to attempt to coin an abbreviation) interrupt would propagate to its scheduling thread (and in whatever way that relationship is defined to work). -- - DML From Alan.Bateman at oracle.com Wed Oct 23 20:12:51 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 23 Oct 2019 21:12:51 +0100 Subject: A lightweight thread is a Thread In-Reply-To: References: <20191022152245.16ea1f30@almond.int.arc7.info> <1360aec6-5d70-69fc-0f9d-3d7a1a0bc859@oracle.com> <93d42320-b574-b989-94d2-7762bbdb655e@redhat.com> Message-ID: On 23/10/2019 20:29, David Lloyd wrote: > Alan, did you have any thoughts about how thread interruption might > work with lightweight threads? I imagine that it wouldn't have to be > too terribly different in a lightweight thread compared to "normal" > threads, but I'm thinking that there is definitely a scheduling effect > as interrupt tends to imply a wakeup/unpark in most if not all cases, > and there is also a question of whether or how an LWT (to attempt to > coin an abbreviation) interrupt would propagate to its scheduling > thread (and in whatever way that relationship is defined to work). They have an interrupt status, interrupt will set the status and unpark the thread. So locking, I/O operations, Object::wait, and everything else that reacts to interrupt should work. I saw one of your mails where you mention overriding the interrupt method. At things stand, propagation of the interrupt status to the carrier thread has to bypass the override, otherwise there is potential to run arbitrary code at sensitive times (e.g. mounting). -Alan From ron.pressler at oracle.com Wed Oct 23 21:16:42 2019 From: ron.pressler at oracle.com (Ron Pressler) Date: Wed, 23 Oct 2019 22:16:42 +0100 Subject: A lightweight thread is a Thread In-Reply-To: References: Message-ID: I'd like to briefly explain the reasoning that has led us to this approach. As is usually the case when introducing a change to an established platform, we are confronted by two, often opposing, forces: 1. The desire to have as much existing code be "forward compatible" and run with the new construct -- in this case, lightweight threads. 2. The desire to correct past decisions which were either mistakes or made in an environment that has since changed, and the wish to shield newcomers from possibly outdated APIs -- in this case the baggage of the Thread class -- by starting afresh. Two observations informed our current position: 1. The use of Thread.currentThread() is pervasive. If we were to follow the previous approach of lazily creating an adapter Thread instance ("shadow Thread") that wraps a Fiber, we would see this adapter instantiated in a very large percentage of cases, and it still wouldn't be entirely transparent, as the adapter Thread instance would not have the same identity as the corresponding Fiber instance; this not only negates any footprint gains in those case but could also lead to subtle errors. 2. Ever since the introduction of java.util.concurrent package we've discouraged people from using the Thread API directly (mostly of the start and join methods) anyway. These observations mean that the new approach -- a lightweight thread is a Thread (and a heavyweight thread is also a Thread) -- can nicely balance the two forces, or so we hope. A lot of existing code could run in lightweight threads, yet most of the interaction with threads would circumvent the Thread API, at least the parts of it we like less, and would be done through APIs that express our current thinking about the "right way" of working with threads. The understanding that Thread's footprint could be drastically reduced without harming the performance of existing code removes that obstacle as well. - Ron On 22 October 2019 at 14:59:32, Alan Bateman (alan.bateman at oracle.com(mailto:alan.bateman at oracle.com)) wrote: > This project has been trying to figure out the API to expose for > lightweight threads, and its relationship with the Thread API, for some > time. There has been many prototypes (lightweight thread = Thread, Fiber > <: Thread, Fiber and Thread <: Strand, disjoint Fiber and Thread types). > Thread.currentThread() and thread locals featured in all prototypes as > it's critical that the current Thread cannot change value in a thread of > execution. > > I think we are at the point where we are concluding that a lightweight > thread should be represented by a Thread. No Fiber or other sub-class > exposed in the API. We'll use the term "lightweight thread" rather than > "fiber". The approach maintains the mental model of a thread of > execution that we are familiar with here, and the mental model that > developers coming to the platform should be able to figure out. The > Thread API does come with baggage (ThreadGroup, TCCL, ...) but it may > not be too bad when you consider that most developers don't use the > Thread API directly. In addition, we can degrade and/or eliminate at > least some of this baggage over time, starting with the deprecated features. > > Moving to Thread as the API for lightweight threads means we need to > restart some of the API work. In particular the support for structured > concurrency and cancellation will need to move aside temporarily as we > rebuild from the bottom. I will push a change to the "fibers" branch > soon with the first steps in that direction. The first steps include 4 > static factory methods on Thread to create lightweight threads > (unnamed/named * default or BYO scheduler). There are several discussion > points around what lightweight threads support and don't support. For > now, the default is support thread locals and not inherit thread locals > but it can overridden by specifying characteristics to the factory > methods. The API is primitive for now and will likely change many times. > > API aside, one of the benefits of representing a lightweight thread as a > Thread is that it makes it possible to make progress in the > serviceability areas that we've had to ignore to date. We can finally > start to figure how the existing monitoring and management APIs, JFR, > and diagnostic features might work with lightweight threads. Good > progress has been made on JVM TI and the debugger support in the last > year and it shouldn't require too much re-work to get the debugger > support re-aligned. > > One final thing for now is footprint. There will be some refactoring of > Thread required to reduce its footprint. The initial changes will move > some of fields that aren't performance critical to a sidecar/helper > object. Doug Lea has changes to the FJ code in the works that should > avoid the need for padding/@Contended on the fields used by > ThreadLocalRandom. The object size will, at least initially, be a bit > larger that I had hoped but I'm sure we will improve this in time. > > -Alan. From john at neggie.net Thu Oct 24 01:01:53 2019 From: john at neggie.net (John Belmonte) Date: Thu, 24 Oct 2019 10:01:53 +0900 Subject: clarifications in structured concurrency overview Message-ID: https://wiki.openjdk.java.net/display/loom/Structured+Concurrency The Cancellation section doesn't mention what the default option is-- would someone add a clarification? Also, there is a typo in one of the code samples: PROPAGTE_CANCEL --> PROPAGATE_CANCEL Thank you (Not being able to edit, comment, or file a bug against documentation without an OpenJDK account seems prohibitive-- it took me a while to track down this list.) From Alan.Bateman at oracle.com Thu Oct 24 11:55:35 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 24 Oct 2019 12:55:35 +0100 Subject: clarifications in structured concurrency overview In-Reply-To: References: Message-ID: On 24/10/2019 02:01, John Belmonte wrote: > https://wiki.openjdk.java.net/display/loom/Structured+Concurrency > > The Cancellation section doesn't mention what the default option is-- would > someone add a clarification? > > Also, there is a typo in one of the code samples: PROPAGTE_CANCEL > --> PROPAGATE_CANCEL > That prototype has been put aside for the moment but the default did not propagation cancellation (propagation of cancellation requires tracking the threads, the default was simplify to just wait until all threads scheduled in the scope terminated). I have it on my list to update the wiki to align with where we. -Alan. From duke at openjdk.java.net Mon Oct 28 09:55:22 2019 From: duke at openjdk.java.net (duke) Date: Mon, 28 Oct 2019 09:55:22 GMT Subject: git: openjdk/loom: fibers: fixed two issues in JVMTI support for fibers; added one more check into FiberTest Message-ID: Changeset: 1011c862 Author: Serguei Spitsyn Date: 2019-10-28 09:53:47 +0000 URL: https://git.openjdk.java.net/loom/commit/1011c862 fixed two issues in JVMTI support for fibers; added one more check into FiberTest ! src/hotspot/share/prims/jvmtiEnvBase.cpp ! src/hotspot/share/prims/jvmtiExport.cpp ! test/hotspot/jtreg/serviceability/jvmti/FiberTest/libFiberTest.c From tmay at clearwateranalytics.com Mon Oct 28 22:35:40 2019 From: tmay at clearwateranalytics.com (Thomas May) Date: Mon, 28 Oct 2019 22:35:40 +0000 Subject: Lightweight thread and synchronized Message-ID: It was my understanding that one issue with Fibers was that Object#monitor and the synchronized keyword would block the carrier thread. With the transition to Lightweight Threads, does that mean this is no longer an issue? How shielded from the carrier threads will Lightweight threads be? ________________________________ NOTICE: This e-mail message, together with any attachments, contains information of Clearwater Analytics and/or its affiliates that may be confidential, proprietary copyrighted and/or legally privileged, and is intended solely for the use of the individual or entity named on this message. If you are not the intended recipient, and have received this message in error, please immediately delete it. The information we provide is from sources Clearwater Analytics considers reliable, but Clearwater Analytics provides no warranties regarding the accuracy of the information. Further, nothing in the email should be construed as legal, financial, or tax advice, and any questions regarding the intended recipient's individual circumstances should be addressed to that recipient's lawyer and/or accountant. Clearwater Analytics, 777 W. Main St, Boise, ID 83702 If you prefer not to receive emails from Clearwater Analytics you may unsubscribe. From Alan.Bateman at oracle.com Tue Oct 29 07:05:58 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 29 Oct 2019 07:05:58 +0000 Subject: Lightweight thread and synchronized In-Reply-To: References: Message-ID: <172b1481-b40a-ea9f-707a-eab9d85a88da@oracle.com> On 28/10/2019 22:35, Thomas May wrote: > It was my understanding that one issue with Fibers was that Object#monitor and the synchronized keyword would block the carrier thread. > > With the transition to Lightweight Threads, does that mean this is no longer an issue? How shielded from the carrier threads will Lightweight threads be? > This hasn't changed. If a lightweight thread blocks on monitorenter or Object::wait then it will pin the underlying carrier thread. We want that limitation to go away eventually, it's just not something that will happen in the short term. -Alan From rahman.usta.88 at gmail.com Thu Oct 31 09:01:52 2019 From: rahman.usta.88 at gmail.com (Rahman USTA) Date: Thu, 31 Oct 2019 10:01:52 +0100 Subject: Building Project Loom Message-ID: Hi, I tried to build project loom with given instructions in Wiki page. However, the build exit with error. I attached logs, could you please help? (MacOS Catalina 10.15) Thank you -- Rahman USTA Istanbul JUG https://github.com/rahmanusta From volkan.yazici at gmail.com Thu Oct 31 09:35:31 2019 From: volkan.yazici at gmail.com (=?UTF-8?B?Vm9sa2FuIFlhesSxY8Sx?=) Date: Thu, 31 Oct 2019 10:35:31 +0100 Subject: Building Project Loom In-Reply-To: References: Message-ID: Hello Rahman, Attachments get truncated in the mailing list. You better put them to a publicly accessible medium (gist?) and share the link. Best. On Thu, Oct 31, 2019 at 10:02 AM Rahman USTA wrote: > Hi, > > I tried to build project loom with given instructions in Wiki page. > However, the build exit with error. I attached logs, could you please help? > (MacOS Catalina 10.15) > > Thank you > > -- > Rahman USTA > Istanbul JUG > https://github.com/rahmanusta > From rahman.usta.88 at gmail.com Thu Oct 31 09:45:25 2019 From: rahman.usta.88 at gmail.com (Rahman USTA) Date: Thu, 31 Oct 2019 10:45:25 +0100 Subject: Building Project Loom In-Reply-To: References: Message-ID: For sure Volkan, Here you go https://gist.github.com/rahmanusta/db03a9e4a6963204e5688b6bae81a796 Rahman Volkan Yaz?c? , 31 Eki 2019 Per, 10:35 tarihinde ?unu yazd?: > Hello Rahman, > > Attachments get truncated in the mailing list. You better put them to a > publicly accessible medium (gist?) and share the link. > > Best. > > On Thu, Oct 31, 2019 at 10:02 AM Rahman USTA > wrote: > >> Hi, >> >> I tried to build project loom with given instructions in Wiki page. >> However, the build exit with error. I attached logs, could you please >> help? >> (MacOS Catalina 10.15) >> >> Thank you >> >> -- >> Rahman USTA >> Istanbul JUG >> https://github.com/rahmanusta >> > -- Rahman USTA Istanbul JUG https://github.com/rahmanusta From Alan.Bateman at oracle.com Thu Oct 31 09:51:00 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 31 Oct 2019 09:51:00 +0000 Subject: Building Project Loom In-Reply-To: References: Message-ID: <3bcef152-527d-bc70-5028-84e4f8bc0d4a@oracle.com> On 31/10/2019 09:01, Rahman USTA wrote: > Hi, > > I tried to build project loom with given instructions in Wiki page. > However, the build exit with error. I attached logs, could you please help? > (MacOS Catalina 10.15) Catalina is brand new, I don't know off-hand which Xcode version to use or whether there are other issues. I build on 10.13 and 10.14 without issues. Can you bring this to build-dev as I assume building issues aren't specific to the loom repo, you'll have the same thing with the jdk repo. -Alan