My experience using loom for a load generator

[Alan Bateman]

On 22/05/2020 15:39, Eric Ackermann wrote:
> Good afternoon from Germany,
>
> in [3] was mentioned that feedback on the experience using Loom was appreciated, so I would like to give some feedback.
> I am developing a load generator for web applications in Kotlin for my bachelor’s thesis. It aims at being able to simulate as many users per second as possible. To reach this goal, currently users are represented by Kotlin coroutines. To send requests, asynchronous http via asynchttpclient [1] is used.
> I wanted to try using Loom’s virtual threads instead of Kotlin coroutines and see if there was any improvement, and there was.
> I used the Windows Early Access binary from [2] (15-loom+7-141, specifically). My test setup consisted of a quad-core/8 GB RAM “client” system  running Windows 10 and an eight-core/ 16 GB RAM “server” system also running Windows 10 with IIS 8.3 serving a static website. I ran both my current implementation based on Kotlin coroutines and a changed version that used virtual threads created using Thread.startVirtualThread(). I chose not to use an ExecutorService since I also do not pool the Kotlin coroutines.
> In my test case, 100.000 users send one request each to the target system concurrently. I measured the number of requests that the load generator can send per second as a metric of how efficiently it is running.
> The Kotlin implementation reached up to 10058 requests per second on average, while the loom-based implementation reached up to 10101 requests per second on average (over 3 runs). Also, the time to create the 100.000 lightweight threads using loom was considerably (approx. 2 seconds) shorter than using Kotlin.
> Finally, I would like to add that the development experience was really smooth – migration from Kotlin coroutines to Loom took just under 2 person-hours for this 9300 lines of code project, and all of my tests were green immediately. Also, IntelliJ IDEA recognised the new functions and offered autocompletion etc.
>
Thank you for the nice mail. Would it possible to say a bit more on what 
these 100,00 threads actually do? I see async-http-client is based on 
the Netty library and it looks like threads might block on the 
equivalent of Future::get waiting for the HTTP response. Is this what 
the threads in the load generator do?

-Alan