Unexpected exceptions during benchmark

[John Auden]

Good evening,

Each thread updates only one bucket, and the key is equal to thread ID. 

Long id = Thread.currentThread().getId(); <— get thread ID
map.put(id, 0L);                          <— initialize a bucket with key == thread ID
for (long i = 0; i < 100_000; i++) {
     map.put(id, map.get(id) + i);        <— Update the bucket

Exception can only happen if there is a context switch in the middle of the method and different thread restarts execution from the middle of the method, otherwise it should be protected by the program order.
On top of that it works fine when number of threads is 1..12, and only starts to fail when concurrency is set to 13+

I am only setting number of threads, I do not set number of warmup threads.

Regards,
Alex Dubrouski


> On Sep 22, 2023, at 7:45 PM, Henri Tremblay <henri.tremblay at gmail.com> wrote:
> 
> Hi,
> 
> I don't get it.
> You are putting items in a non-concurrent hashmap from multiple threads.
> I have no doubt it will eventually fail.
> 
> Reading quickly, the second error seems to be thrown (and could be more explanatory) when you want more threads then the number of warmup threads.
> So you want more threads than the warmup could prime and you get this error.
> 
> Anything I'm missing?
> 
> Regards,
> Henri  
> 
> 
> On Fri, 22 Sept 2023 at 22:05, John Auden <lovedthefaioli at gmail.com <mailto:lovedthefaioli at gmail.com>> wrote:
>> Hi,
>> 
>> I was trying to create a concurrency benchmark, but noticed that it fails in a very basic logic block. 
>> I know that it is a bad idea to run benchmarks inside a VM, but I do it rather for quick check and because I have debug JDK releases + hsdis there. This email is not to submit a bug, but rather just for your information.
>> Host: 16 cores Azure VM (Intel(R) Xeon(R) CPU E5-2673 v4 @ 2.30GHz)
>> $ lscpu
>> Architecture:          x86_64
>> CPU op-mode(s):        32-bit, 64-bit
>> Byte Order:            Little Endian
>> CPU(s):                16
>> On-line CPU(s) list:   0-15
>> 
>> OS: CentOS Linux release 7.9.2009
>> JDK: 17.0.5
>> @Threads(Threads.MAX)
>> public class HMBenchmark {
>>   private final static Map<Long, Long> map = new HashMap<>(Runtime.getRuntime().availableProcessors());
>> 
>>   @Benchmark
>>   public void testHM() {
>>     Long id = Thread.currentThread().getId();
>>     map.put(id, 0L);
>>     for (long i = 0; i < 100_000; i++) {
>>       map.put(id, map.get(id) + i);
>>     }
>>   }
>> }
>> If the number of threads is set above 12 (up to Threads.MAX) this benchmark starts to fail. WIth 13, 14, 15 it fails with NPE (even though program order should theoretically protect code against NPE):
>> java.lang.NullPointerException: Cannot invoke "java.lang.Long.longValue()" because the return value of "java.util.Map.get(Object)" is null
>> at org.ad.HMBenchmark.testHM(HMBenchmark.java:33)
>> at org.ad.jmh_generated.HMBenchmark_testHM_jmhTest.testHM_Throughput(HMBenchmark_testHM_jmhTest.java:78)
>> 
>> WIth MAX threads (or 16) there is also this interesting exception:
>> java.lang.IllegalStateException: More threads than expected
>> at org.openjdk.jmh.runner.InfraControlL2.announceWarmupReady(InfraControl.java:222)
>> at org.openjdk.jmh.runner.InfraControl.announceWarmupReady(InfraControl.java:40)
>> at org.ad.jmh_generated.HMBenchmark_testHM_jmhTest.testHM_Throughput(HMBenchmark_testHM_jmhTest.java:76)
>> 
>> Regards,
>> Alex Dubrouski
>> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/jmh-dev/attachments/20230922/63452fa4/attachment-0001.htm>