throughput vs latency, what to look at?

[Kirk Pepperdine]

Hi Maneesh,

I don’t know why the throughput vs latency fallacy still exists today. There is direct causality of throughput and latency simply because we don’t live in infinite compute environments and everything queues. That means that all latency will contain measures of active and dead times. The dead times will be related to time in queues. The more queuing the more dead time == higher latencies. More queuing implies you’ve hit some throughput limit on a shared resource (and you may even see a regression in throughput given the nature of sharinig). Throughput can be predicted by Little’s law. It states that throughput = 1 / service time. Given you’ve added work I would anticipate your service times will increase meaning throughput will drop and latencies will be inflated (see Markov for more details). There are also queue management activities happening which will also likely add to your grief. Controlling the effects of all this is why one typically throttles input into the UoT.

HTHs...

Kind regards,
Kirk Pepperdine


> On Apr 15, 2021, at 8:05 AM, Maneesh Bhunwal <maneesh.bhunwal at gmail.com> wrote:
> 
> Hi Team,
> 
> I have a generic question and a specific question. can you please help me
> with this?
> 
> JMH reports 2 gives out metrics throughput (ops/ms) and latency (ms/ops).
> When we are looking at results, which metric should we focus on? Is there
> any general guideline?
> 
> i have a scenario, where methodA calls methodB. When I try to benchmark
> both methodA and methodB(these are done as part of the same run on a
> standalone vm, so no possibility of outside interference and IMO results
> should be comparable.), there is a huge difference in throughput (1800
> vs 2700 ops/ms) but marginal difference between latencies (19ms vs 18ms).
> There is no locking or contention involved.
> I am not able to understand how to interpret the results. IMO if there are
> no locks and similar latency are there, throughput should be the same. So I
> am confused.
> 
> 
> Regards
> Maneesh Bhunwal