RFR: 8257074 Update the ByteBuffers micro benchmark
The ByteBuffers micro benchmark seems to be a little dated. It should be a useful resource to leverage when analysing the performance impact of any potential implementation changes in the byte buffer classes. More specifically, the impact of such changes on the performance of sharp memory access operations. This issue proposes to update the benchmark in the following ways to meet the aforementioned use-case: 1. Remove allocation from the individual benchmarks - it just creates noise. 2. Consolidate per-thread shared heap and direct buffers. 3. All scenarios now use absolute memory access operations - so no state of the shared buffers is considered. 4. Provide more reasonable default fork, warmup, etc, out-of-the-box. 5. There seems to have been an existing bug in the test where the size parameter was not always considered - this is now fixed. ------------- Commit messages: - Initial changes Changes: https://git.openjdk.java.net/jdk/pull/1430/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1430&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8257074 Stats: 69 lines in 1 file changed: 22 ins; 14 del; 33 mod Patch: https://git.openjdk.java.net/jdk/pull/1430.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1430/head:pull/1430 PR: https://git.openjdk.java.net/jdk/pull/1430
On Wed, 25 Nov 2020 12:41:40 GMT, Chris Hegarty <chegar@openjdk.org> wrote:
The ByteBuffers micro benchmark seems to be a little dated.
It should be a useful resource to leverage when analysing the performance impact of any potential implementation changes in the byte buffer classes. More specifically, the impact of such changes on the performance of sharp memory access operations.
This issue proposes to update the benchmark in the following ways to meet the aforementioned use-case:
1. Remove allocation from the individual benchmarks - it just creates noise. 2. Consolidate per-thread shared heap and direct buffers. 3. All scenarios now use absolute memory access operations - so no state of the shared buffers is considered. 4. Provide more reasonable default fork, warmup, etc, out-of-the-box. 5. There seems to have been an existing bug in the test where the size parameter was not always considered - this is now fixed.
Overall a reasonable clean-up. I do wonder if there's some value to at least some of these noisy, allocating variants, though. Could be interesting to zoom in on code where we allocate transient, small buffers, e.g. when encoding/decoding strings. But perhaps that needs to be designed differently and focus on a specific use case. ------------- Marked as reviewed by redestad (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1430
On Wed, 25 Nov 2020 13:12:34 GMT, Claes Redestad <redestad@openjdk.org> wrote:
I do wonder if there's some value to at least some of these noisy, allocating variants, though. Could be interesting to zoom in on code where we allocate transient, small buffers, e.g. when encoding/decoding strings. But perhaps that needs to be designed differently and focus on a specific use case.
Yes, that is an interest case, but largely orthogonal to the primary use-case for this particular micro benchmark (or maybe; the primary use-case that I want to use this benchmark for ;-) ). It appears to me that the primary purpose is to evaluate the memory access operations, so I want to keep buffer allocation out of the picture. Of course, we could decide to just create a new benchmark, but then I'm not sure what we'd use the old one for. ------------- PR: https://git.openjdk.java.net/jdk/pull/1430
On Wed, 25 Nov 2020 12:41:40 GMT, Chris Hegarty <chegar@openjdk.org> wrote:
The ByteBuffers micro benchmark seems to be a little dated.
It should be a useful resource to leverage when analysing the performance impact of any potential implementation changes in the byte buffer classes. More specifically, the impact of such changes on the performance of sharp memory access operations.
This issue proposes to update the benchmark in the following ways to meet the aforementioned use-case:
1. Remove allocation from the individual benchmarks - it just creates noise. 2. Consolidate per-thread shared heap and direct buffers. 3. All scenarios now use absolute memory access operations - so no state of the shared buffers is considered. 4. Provide more reasonable default fork, warmup, etc, out-of-the-box. 5. There seems to have been an existing bug in the test where the size parameter was not always considered - this is now fixed.
I am not an expert in JMH microbenchmarks but the proposed changes look reasonable to me. ------------- Marked as reviewed by dfuchs (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1430
On Wed, 25 Nov 2020 12:41:40 GMT, Chris Hegarty <chegar@openjdk.org> wrote:
The ByteBuffers micro benchmark seems to be a little dated.
It should be a useful resource to leverage when analysing the performance impact of any potential implementation changes in the byte buffer classes. More specifically, the impact of such changes on the performance of sharp memory access operations.
This issue proposes to update the benchmark in the following ways to meet the aforementioned use-case:
1. Remove allocation from the individual benchmarks - it just creates noise. 2. Consolidate per-thread shared heap and direct buffers. 3. All scenarios now use absolute memory access operations - so no state of the shared buffers is considered. 4. Provide more reasonable default fork, warmup, etc, out-of-the-box. 5. There seems to have been an existing bug in the test where the size parameter was not always considered - this is now fixed.
Marked as reviewed by jvernee (Committer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1430
On Wed, 25 Nov 2020 12:41:40 GMT, Chris Hegarty <chegar@openjdk.org> wrote:
The ByteBuffers micro benchmark seems to be a little dated.
It should be a useful resource to leverage when analysing the performance impact of any potential implementation changes in the byte buffer classes. More specifically, the impact of such changes on the performance of sharp memory access operations.
This issue proposes to update the benchmark in the following ways to meet the aforementioned use-case:
1. Remove allocation from the individual benchmarks - it just creates noise. 2. Consolidate per-thread shared heap and direct buffers. 3. All scenarios now use absolute memory access operations - so no state of the shared buffers is considered. 4. Provide more reasonable default fork, warmup, etc, out-of-the-box. 5. There seems to have been an existing bug in the test where the size parameter was not always considered - this is now fixed.
Looks like a good cleanup. I agree that mixing access and instantiation is not good from a benchmark perspective - although, given that direct buffers have a rather convoluted allocation process, I guess it would be useful, in the long term, to track performances of that too (this can be done by coupling direct buffer allocation with calls to Unsafe::invokeCleaner, so that the benchmark is not affected by the GC activity). Also, given a carrier e.g. `float` there are at least two ways to get a float from a byte array: * ByteBuffer::getFloat (both direct/heap) * FloatBuffer::get() (both direct/heap) The benchmark here focuses on the first bullet, but I think at some point we should also cover the remaining axis, so that, for each carrier, we really do at least 4 tests. Of course if you take into account swappiness for non-byte carriers, you get an extra variant for direct buffers as well, as you wanna test native endian vs. non-native endian. So many possibilities :-) ------------- Marked as reviewed by mcimadamore (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/1430
On Wed, 25 Nov 2020 13:37:48 GMT, Maurizio Cimadamore <mcimadamore@openjdk.org> wrote:
Looks like a good cleanup. I agree that mixing access and instantiation is not good from a benchmark perspective - although, given that direct buffers have a rather convoluted allocation process, I guess it would be useful, in the long term, to track performances of that too (this can be done by coupling direct buffer allocation with calls to Unsafe::invokeCleaner, so that the benchmark is not affected by the GC activity).
I'll see if I can write a test for this.
Also, given a carrier e.g. `float` there are at least two ways to get a float from a byte array:
* ByteBuffer::getFloat (both direct/heap) * FloatBuffer::get() (both direct/heap)
The benchmark here focuses on the first bullet, but I think at some point we should also cover the remaining axis, so that, for each carrier, we really do at least 4 tests. Of course if you take into account swappiness for non-byte carriers, you get an extra variant for direct buffers as well, as you wanna test native endian vs. non-native endian.
Yes, good observation - I had similar thoughts. Let me check if we could easily integrate some of these variants. What I'm trying to end up with is a comprehensive(ish) set of test scenarios that can be used to analyse performance ( of which views and endianness are important). It may be reasonable for this particular benchmark to include views, etc, with appropriately named benchmark methods, so that one can easily run a subset, e.g. "org.openjdk.bench.java.nio.ByteBuffers.test(Direct)?(BulkPut|SinglePutByte)", when trying to "zoom in" on a specific code path. ------------- PR: https://git.openjdk.java.net/jdk/pull/1430
The ByteBuffers micro benchmark seems to be a little dated.
It should be a useful resource to leverage when analysing the performance impact of any potential implementation changes in the byte buffer classes. More specifically, the impact of such changes on the performance of sharp memory access operations.
This issue proposes to update the benchmark in the following ways to meet the aforementioned use-case:
1. Remove allocation from the individual benchmarks - it just creates noise. 2. Consolidate per-thread shared heap and direct buffers. 3. All scenarios now use absolute memory access operations - so no state of the shared buffers is considered. 4. Provide more reasonable default fork, warmup, etc, out-of-the-box. 5. There seems to have been an existing bug in the test where the size parameter was not always considered - this is now fixed.
Chris Hegarty has updated the pull request incrementally with one additional commit since the last revision: Add additional carrier views and endianness variants. A large number of variants are now covered. The individual benchmarks conform to the following convention: test(Direct|Heap)(Bulk|Single)(Get|Put)(Byte|Char|Short|Int|Long|Float|Double)(View)?(Swap)? This allows to easily run a subset of particular interest. For example: Direct only :- "org.openjdk.bench.java.nio.ByteBuffers.testDirect.*" Char only :- "org.openjdk.bench.java.nio.ByteBuffers.test.*Char.*" Bulk only :- "org.openjdk.bench.java.nio.ByteBuffers.test.*Bulk.*" Put with Int or Long carrier :- test(Direct|Heap)(Single)(Put)(Int|Long)(View)?(Swap)?" Running all variants together is likely not all that useful, since there will be a lot of data. The param sizes are changed so as to better allow for wider carrier views. There are a lot of individual benchmark methods, but their implementation is trivial, and largely mechanical. Question: where do folk stand on having a `main` method in a benchmark - as a standalone-run sanity? I found this useful to assert the validity of the benchmark code. It can be commented out if it could somehow affect the benchmark runs. ( I omitted read-only views, since they less interesting, and we already have a lot of variants ) ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1430/files - new: https://git.openjdk.java.net/jdk/pull/1430/files/5e91e63e..84dabc30 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1430&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1430&range=00-01 Stats: 1022 lines in 1 file changed: 995 ins; 1 del; 26 mod Patch: https://git.openjdk.java.net/jdk/pull/1430.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1430/head:pull/1430 PR: https://git.openjdk.java.net/jdk/pull/1430
The ByteBuffers micro benchmark seems to be a little dated.
It should be a useful resource to leverage when analysing the performance impact of any potential implementation changes in the byte buffer classes. More specifically, the impact of such changes on the performance of sharp memory access operations.
This issue proposes to update the benchmark in the following ways to meet the aforementioned use-case:
1. Remove allocation from the individual benchmarks - it just creates noise. 2. Consolidate per-thread shared heap and direct buffers. 3. All scenarios now use absolute memory access operations - so no state of the shared buffers is considered. 4. Provide more reasonable default fork, warmup, etc, out-of-the-box. 5. There seems to have been an existing bug in the test where the size parameter was not always considered - this is now fixed.
Chris Hegarty has updated the pull request incrementally with one additional commit since the last revision: whitespace ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1430/files - new: https://git.openjdk.java.net/jdk/pull/1430/files/84dabc30..814e1819 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1430&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1430&range=01-02 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/1430.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1430/head:pull/1430 PR: https://git.openjdk.java.net/jdk/pull/1430
On Thu, 26 Nov 2020 14:38:17 GMT, Chris Hegarty <chegar@openjdk.org> wrote:
The ByteBuffers micro benchmark seems to be a little dated.
It should be a useful resource to leverage when analysing the performance impact of any potential implementation changes in the byte buffer classes. More specifically, the impact of such changes on the performance of sharp memory access operations.
This issue proposes to update the benchmark in the following ways to meet the aforementioned use-case:
1. Remove allocation from the individual benchmarks - it just creates noise. 2. Consolidate per-thread shared heap and direct buffers. 3. All scenarios now use absolute memory access operations - so no state of the shared buffers is considered. 4. Provide more reasonable default fork, warmup, etc, out-of-the-box. 5. There seems to have been an existing bug in the test where the size parameter was not always considered - this is now fixed.
Chris Hegarty has updated the pull request incrementally with one additional commit since the last revision:
whitespace
Looks a great improvements. Two comments: * the names always mention the "Single" word, when in fact all benchmark involve some kind of a loop. I'd suggest making that more explicit, both in the benchmark method and in the helpers. * I see that you added support for views - but note that, for heap buffers there are two different kind of views (!!). E.g. I believe that: ByteBuffer.allocate(size * 4).asFloatBuffer() returns a buffer of a different class than: FloatBuffer.allocate(size); I think you are only testing the former? ------------- PR: https://git.openjdk.java.net/jdk/pull/1430
Maurizio,
On 27 Nov 2020, at 18:04, Maurizio Cimadamore <mcimadamore@openjdk.java.net> wrote:
... Looks a great improvements. Two comments:
* the names always mention the "Single" word, when in fact all benchmark involve some kind of a loop. I'd suggest making that more explicit, both in the benchmark method and in the helpers.
Yeah, I dislike (the preexisting) “Single” too. How about “Loop” - so we have [Bulk|Loop] ?
* I see that you added support for views - but note that, for heap buffers there are two different kind of views (!!). E.g. I believe that:
ByteBuffer.allocate(size * 4).asFloatBuffer() returns a buffer of a different class than:
FloatBuffer.allocate(size); I think you are only testing the former?
Correct. I did consider the latter, but then thought that this micro had enough scenarios. Let me take another look and see how much more scenarios it adds. -Chris.
The ByteBuffers micro benchmark seems to be a little dated.
It should be a useful resource to leverage when analysing the performance impact of any potential implementation changes in the byte buffer classes. More specifically, the impact of such changes on the performance of sharp memory access operations.
This issue proposes to update the benchmark in the following ways to meet the aforementioned use-case:
1. Remove allocation from the individual benchmarks - it just creates noise. 2. Consolidate per-thread shared heap and direct buffers. 3. All scenarios now use absolute memory access operations - so no state of the shared buffers is considered. 4. Provide more reasonable default fork, warmup, etc, out-of-the-box. 5. There seems to have been an existing bug in the test where the size parameter was not always considered - this is now fixed.
Chris Hegarty has updated the pull request incrementally with two additional commits since the last revision: - Add explicitly allocated heap carrier buffer tests - Replace Single with Loop ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1430/files - new: https://git.openjdk.java.net/jdk/pull/1430/files/814e1819..5ee5d8bf Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1430&range=03 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1430&range=02-03 Stats: 407 lines in 1 file changed: 151 ins; 0 del; 256 mod Patch: https://git.openjdk.java.net/jdk/pull/1430.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1430/head:pull/1430 PR: https://git.openjdk.java.net/jdk/pull/1430
On Mon, 30 Nov 2020 13:39:12 GMT, Chris Hegarty <chegar@openjdk.org> wrote:
The ByteBuffers micro benchmark seems to be a little dated.
It should be a useful resource to leverage when analysing the performance impact of any potential implementation changes in the byte buffer classes. More specifically, the impact of such changes on the performance of sharp memory access operations.
This issue proposes to update the benchmark in the following ways to meet the aforementioned use-case:
1. Remove allocation from the individual benchmarks - it just creates noise. 2. Consolidate per-thread shared heap and direct buffers. 3. All scenarios now use absolute memory access operations - so no state of the shared buffers is considered. 4. Provide more reasonable default fork, warmup, etc, out-of-the-box. 5. There seems to have been an existing bug in the test where the size parameter was not always considered - this is now fixed.
Chris Hegarty has updated the pull request incrementally with two additional commits since the last revision:
- Add explicitly allocated heap carrier buffer tests - Replace Single with Loop
Marked as reviewed by bpb (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/1430
On Mon, 30 Nov 2020 20:54:09 GMT, Brian Burkhalter <bpb@openjdk.org> wrote:
Chris Hegarty has updated the pull request incrementally with two additional commits since the last revision:
- Add explicitly allocated heap carrier buffer tests - Replace Single with Loop
Marked as reviewed by bpb (Reviewer).
While the updated set of scenarios covered by this benchmark is reasonably (and vastly improves coverage), I find myself reluctant to add the last remaining buffer property - read-only views. It's time to template the generation of this code! ------------- PR: https://git.openjdk.java.net/jdk/pull/1430
On Thu, 10 Dec 2020 09:46:07 GMT, Chris Hegarty <chegar@openjdk.org> wrote:
Marked as reviewed by bpb (Reviewer).
While the updated set of scenarios covered by this benchmark is reasonably (and vastly improves coverage), I find myself reluctant to add the last remaining buffer property - read-only views. It's time to template the generation of this code!
If the cases get to be too many, you might also want to consider splitting the benchmark class into several classes that cover different cases (we did this for the memory access benchmarks as well). That would also make it easier to run a subset of cases in isolation. ------------- PR: https://git.openjdk.java.net/jdk/pull/1430
On Thu, 10 Dec 2020 14:01:56 GMT, Jorn Vernee <jvernee@openjdk.org> wrote:
While the updated set of scenarios covered by this benchmark is reasonably (and vastly improves coverage), I find myself reluctant to add the last remaining buffer property - read-only views. It's time to template the generation of this code!
If the cases get to be too many, you might also want to consider splitting the benchmark class into several classes that cover different cases (we did this for the memory access benchmarks as well). That would also make it easier to run a subset of cases in isolation.
All comments to date have been addressed. Unless there are any further comments, I would like to integrate this change. ------------- PR: https://git.openjdk.java.net/jdk/pull/1430
The ByteBuffers micro benchmark seems to be a little dated.
It should be a useful resource to leverage when analysing the performance impact of any potential implementation changes in the byte buffer classes. More specifically, the impact of such changes on the performance of sharp memory access operations.
This issue proposes to update the benchmark in the following ways to meet the aforementioned use-case:
1. Remove allocation from the individual benchmarks - it just creates noise. 2. Consolidate per-thread shared heap and direct buffers. 3. All scenarios now use absolute memory access operations - so no state of the shared buffers is considered. 4. Provide more reasonable default fork, warmup, etc, out-of-the-box. 5. There seems to have been an existing bug in the test where the size parameter was not always considered - this is now fixed.
Chris Hegarty has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains seven additional commits since the last revision: - Template generation of carrier specific micro-benchmarks. * Benchmarks are now split out into carrier specific source files * All variants and views are covered * Test scenario naming is idiomatic Even with the split by carrier, regex expressions can be used to easily run subsets the benchmarks. E.g. test all memory operations related to Short with read-only buffers: $ java -jar benchmarks.jar "org.openjdk.bench.java.nio..*Buffers.testDirect.*Short.*" -l Benchmarks: org.openjdk.bench.java.nio.ByteBuffers.testDirectLoopGetShortRO org.openjdk.bench.java.nio.ByteBuffers.testDirectLoopGetShortSwapRO org.openjdk.bench.java.nio.ShortBuffers.testDirectBulkGetShortViewRO org.openjdk.bench.java.nio.ShortBuffers.testDirectBulkGetShortViewSwapRO org.openjdk.bench.java.nio.ShortBuffers.testDirectLoopGetShortViewRO org.openjdk.bench.java.nio.ShortBuffers.testDirectLoopGetShortViewSwapRO - Merge branch 'master' into bb-bench - Add explicitly allocated heap carrier buffer tests - Replace Single with Loop - whitespace - Add additional carrier views and endianness variants. A large number of variants are now covered. The individual benchmarks conform to the following convention: test(Direct|Heap)(Bulk|Single)(Get|Put)(Byte|Char|Short|Int|Long|Float|Double)(View)?(Swap)? This allows to easily run a subset of particular interest. For example: Direct only :- "org.openjdk.bench.java.nio.ByteBuffers.testDirect.*" Char only :- "org.openjdk.bench.java.nio.ByteBuffers.test.*Char.*" Bulk only :- "org.openjdk.bench.java.nio.ByteBuffers.test.*Bulk.*" Put with Int or Long carrier :- test(Direct|Heap)(Single)(Put)(Int|Long)(View)?(Swap)?" Running all variants together is likely not all that useful, since there will be a lot of data. The param sizes are changed so as to better allow for wider carrier views. There are a lot of individual benchmark methods, but their implementation is trivial, and largely mechanical. Question: where do folk stand on having a `main` method in a benchmark - as a standalone-run sanity? I found this useful to assert the validity of the benchmark code. It can be commented out if it could somehow affect the benchmark runs. ( I omitted read-only views, since they less interesting, and we already have a lot of variants ) - Initial changes ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1430/files - new: https://git.openjdk.java.net/jdk/pull/1430/files/5ee5d8bf..a8e81a84 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1430&range=04 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1430&range=03-04 Stats: 46605 lines in 988 files changed: 29018 ins; 8180 del; 9407 mod Patch: https://git.openjdk.java.net/jdk/pull/1430.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1430/head:pull/1430 PR: https://git.openjdk.java.net/jdk/pull/1430
The ByteBuffers micro benchmark seems to be a little dated.
It should be a useful resource to leverage when analysing the performance impact of any potential implementation changes in the byte buffer classes. More specifically, the impact of such changes on the performance of sharp memory access operations.
This issue proposes to update the benchmark in the following ways to meet the aforementioned use-case:
1. Remove allocation from the individual benchmarks - it just creates noise. 2. Consolidate per-thread shared heap and direct buffers. 3. All scenarios now use absolute memory access operations - so no state of the shared buffers is considered. 4. Provide more reasonable default fork, warmup, etc, out-of-the-box. 5. There seems to have been an existing bug in the test where the size parameter was not always considered - this is now fixed.
Chris Hegarty has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains ten additional commits since the last revision: - Update copyright year range and tailing whitespace - Merge branch 'master' into bb-bench - Template generation of carrier specific micro-benchmarks. * Benchmarks are now split out into carrier specific source files * All variants and views are covered * Test scenario naming is idiomatic Even with the split by carrier, regex expressions can be used to easily run subsets the benchmarks. E.g. test all memory operations related to Short with read-only buffers: $ java -jar benchmarks.jar "org.openjdk.bench.java.nio..*Buffers.testDirect.*Short.*" -l Benchmarks: org.openjdk.bench.java.nio.ByteBuffers.testDirectLoopGetShortRO org.openjdk.bench.java.nio.ByteBuffers.testDirectLoopGetShortSwapRO org.openjdk.bench.java.nio.ShortBuffers.testDirectBulkGetShortViewRO org.openjdk.bench.java.nio.ShortBuffers.testDirectBulkGetShortViewSwapRO org.openjdk.bench.java.nio.ShortBuffers.testDirectLoopGetShortViewRO org.openjdk.bench.java.nio.ShortBuffers.testDirectLoopGetShortViewSwapRO - Merge branch 'master' into bb-bench - Add explicitly allocated heap carrier buffer tests - Replace Single with Loop - whitespace - Add additional carrier views and endianness variants. A large number of variants are now covered. The individual benchmarks conform to the following convention: test(Direct|Heap)(Bulk|Single)(Get|Put)(Byte|Char|Short|Int|Long|Float|Double)(View)?(Swap)? This allows to easily run a subset of particular interest. For example: Direct only :- "org.openjdk.bench.java.nio.ByteBuffers.testDirect.*" Char only :- "org.openjdk.bench.java.nio.ByteBuffers.test.*Char.*" Bulk only :- "org.openjdk.bench.java.nio.ByteBuffers.test.*Bulk.*" Put with Int or Long carrier :- test(Direct|Heap)(Single)(Put)(Int|Long)(View)?(Swap)?" Running all variants together is likely not all that useful, since there will be a lot of data. The param sizes are changed so as to better allow for wider carrier views. There are a lot of individual benchmark methods, but their implementation is trivial, and largely mechanical. Question: where do folk stand on having a `main` method in a benchmark - as a standalone-run sanity? I found this useful to assert the validity of the benchmark code. It can be commented out if it could somehow affect the benchmark runs. ( I omitted read-only views, since they less interesting, and we already have a lot of variants ) - Initial changes ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1430/files - new: https://git.openjdk.java.net/jdk/pull/1430/files/a8e81a84..9e8014cf Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1430&range=05 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1430&range=04-05 Stats: 123643 lines in 3239 files changed: 61246 ins; 39707 del; 22690 mod Patch: https://git.openjdk.java.net/jdk/pull/1430.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1430/head:pull/1430 PR: https://git.openjdk.java.net/jdk/pull/1430
On Wed, 25 Nov 2020 12:41:40 GMT, Chris Hegarty <chegar@openjdk.org> wrote:
The ByteBuffers micro benchmark seems to be a little dated.
It should be a useful resource to leverage when analysing the performance impact of any potential implementation changes in the byte buffer classes. More specifically, the impact of such changes on the performance of sharp memory access operations.
This issue proposes to update the benchmark in the following ways to meet the aforementioned use-case:
1. Remove allocation from the individual benchmarks - it just creates noise. 2. Consolidate per-thread shared heap and direct buffers. 3. All scenarios now use absolute memory access operations - so no state of the shared buffers is considered. 4. Provide more reasonable default fork, warmup, etc, out-of-the-box. 5. There seems to have been an existing bug in the test where the size parameter was not always considered - this is now fixed.
This pull request has now been integrated. Changeset: ac276bb3 Author: Chris Hegarty <chegar@openjdk.org> URL: https://git.openjdk.java.net/jdk/commit/ac276bb3 Stats: 2771 lines in 11 files changed: 2612 ins; 9 del; 150 mod 8257074: Update the ByteBuffers micro benchmark Reviewed-by: redestad, dfuchs, jvernee, bpb ------------- PR: https://git.openjdk.java.net/jdk/pull/1430
participants (7)
-
Brian Burkhalter
-
Chris Hegarty
-
Chris Hegarty
-
Claes Redestad
-
Daniel Fuchs
-
Jorn Vernee
-
Maurizio Cimadamore