Interesting Benchmarks

[Eric Kolotyluk]

Okay, more learning...

Using JMH for what I am doing is completely wrong �� I started using it
without understanding it. I am not sure if it can do what I want? When I
started using AtomicLong based on Ignaz suggestion, I discovered that JMH
was calling my code many more times than I thought, which was wrong. I will
have to try to understand JMH better...

Mostly with this project, I am learning how wrong my assumptions were, but
I am learning to correct them.

* Prime numbers                   to 1,000             to 10,000
     to 10,000,000
*                                 tasks op/ms  ratio   tasks  op/ms
ratio   tasks    op/ms   ratio
* virtualCachedThreadPool          499  6.329  3.224   3711  34.722
1.625   3103283 115.452  0.834
* virtualThreadPerTaskExecutor     499  8.772  2.599   4906  65.789
3.894   3454111 138.765 31.616
* platformCachedThreadPool         256  1.953  0.310    828  21.367
0.615      3690 138.378  1.198
* platformThreadPerTaskExecutor    149  3.356  0.385    140   4.766
0.072       186   4.389  0.032

Here are my latest benchmarks

   1. My prime numbers simulated a network using Thread.sleep()
   2. Initially I tested with .newFixedThreadPool() and
   .newSingleThreadExecutor, but did not find the results interesting, just
   predictably slow
   3. I used AtomicLong to keep track of the number and maximum number of
   tasks spawned by each ExecutorService
   4. platformCachedThreadPool and platformThreadPerTaskExecutor are very
   good at restricting the number of tasks spawned
   5. When designing an application, it's important to benchmark
   CachedThreadPool vs. ThreadPerTaskExecutor
   6. The ratios compare Virtual Threads with Platform Threads - in one
   case, Platform Threads did better???

Will keep working to understand better...

Cheers, Eric



On Wed, Nov 10, 2021 at 12:16 PM Ignaz Birnstingl <ignazb at gmail.com> wrote:

> Hi Eric,
>
> here are my thoughts:
>
> Creating platform threads is an expensive operation. Your benchmarks show
> this very well. I would guess that starting a single platform thread
> costs
> somewhere between 10µs to 1ms. So spinning up platform threads that run
> for ~30ms before terminating means that at most 30000 platform threads
> would run in parallel. You could easily check that by using two
> AtomicIntegers in your isPrime() method: One keeps a count of currently
> running tasks. You increment it in the beginning of isPrime() and
> decrement it in the end of isPrime(). The other one would hold the
> maximum
> of the first one after incrementation. I would guess that the maximum of
> platform threads running in parallel is much lower than for virtual
> threads.
>
> If you want to compare the throughput of the solutions it would probably
> make sense to pass in the ExecutorService you want to use to your
> futurePrimes22() method instead of creating it in there.
> Then you could compare Executors.newCachedThreadPool() vs.
> Executors.newFixedThreadPool() (with different numbers) and
> Executors.newVirtualThreadExecutor().
>
> Ignaz
>