<div dir="ltr">Thank you, Maurizio, for delving into the benchmark. I will follow this thread closely!<br><div><br></div><div>Best,</div><div>-Antoine</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Mar 20, 2024 at 12:47 PM Maurizio Cimadamore <<a href="mailto:maurizio.cimadamore@oracle.com">maurizio.cimadamore@oracle.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><u></u>
<div>
<div>
<p style="margin:0px 0px 1.2em">Hi,<br>
I did some more analysis of your benchmark. Most of the
non-unrolled, non-vectorized benchmarks perform similarly.
Looking at the generated assembly, it seems that bound checks
are correctly hoisted outside the loop. So the suggestion I made
earlier is not going to make any difference - in fact the
scalarSegmentSegment version is already competitive with
scalarUnsafeUnsafe.</p>
<p style="margin:0px 0px 1.2em">But there is indeed a
big difference between scalarUnsafeUnsafe and everything else.
The assembly indicates that this version is the only one that
uses vectorized instructions (e.g. addpd instead of addsd). This
probably indicates that <em>some</em> optimization is failing
(across the board, except in that specific case), but I’m not a
C2 expert, so I can’t point to the underlying cause.</p>
<p style="margin:0px 0px 1.2em">I’m adding Vlad and
Roland, as they might have a better idea of what’s going on.</p>
<p style="margin:0px 0px 1.2em">To make things easier
to test, I’ve put together a jdk branch which includes a bunch
of relevant benchmarks, including a subset of your AddBenchmark:</p>
<p style="margin:0px 0px 1.2em"><a href="https://github.com/openjdk/jdk/compare/master...mcimadamore:jdk:AddBenchmark?expand=1" target="_blank">https://github.com/openjdk/jdk/compare/master...mcimadamore:jdk:AddBenchmark?expand=1</a></p>
<p style="margin:0px 0px 1.2em">I think a good place
to start would be to explain the difference between
scalarUnsafeArray and scalarUnsafeUnsafe. The other benchmark
can be looked at later, as (after looking at the assembly) I
don’t think this is an issue that is specific to FFM.</p>
<p style="margin:0px 0px 1.2em">For the records, here
are the results I get:</p>
<pre style="font-family:Consolas,Inconsolata,Courier,monospace;font-size:1em;line-height:1.2em;margin:1.2em 0px"><code style="font-size:0.85em;font-family:Consolas,Inconsolata,Courier,monospace;margin:0px 0.15em;background-color:rgb(248,248,248);white-space:pre-wrap;overflow:auto;border-radius:3px;border:1px solid rgb(204,204,204);padding:0.5em 0.7em;display:block">Benchmark Mode Cnt Score Error Units
AddBenchmark.scalarArrayArray avgt 30 94.475 ± 0.554 ns/op
AddBenchmark.scalarArrayArrayLongStride avgt 30 481.030 ± 4.477 ns/op
AddBenchmark.scalarBufferArray avgt 30 339.244 ± 4.546 ns/op
AddBenchmark.scalarBufferBuffer avgt 30 329.813 ± 2.504 ns/op
AddBenchmark.scalarSegmentArray avgt 30 376.254 ± 5.192 ns/op
AddBenchmark.scalarSegmentSegment avgt 30 302.793 ± 4.767 ns/op
AddBenchmark.scalarSegmentSegmentLongStride avgt 30 305.078 ± 4.252 ns/op
AddBenchmark.scalarUnsafeArray avgt 30 95.765 ± 1.295 ns/op
AddBenchmark.scalarUnsafeUnsafe avgt 30 358.060 ± 4.868 ns/op
</code></pre>
<p style="margin:0px 0px 1.2em">Cheers<br>
Maurizio</p>
<p style="margin:0px 0px 1.2em">On 20/03/2024 10:26,
Maurizio Cimadamore wrote:</p>
<p style="margin:0px 0px 1.2em"></p>
<div>
<p></p>
<blockquote type="cite">Hi
Antoine,
<br>
thanks for the benchmark. From the numbers you are getting in
the AddBenchmark, my gut feeling is that, for memory segments,
bound checks are not being hoisted outside the loop. That
would cause the kind of degradation you are seeing here. I'm
also surprised to see, for that benchmark, that Unsafe is >
2x faster than using plain arrays, after all the size of the
array is a loop invariant, and no check should occur there. On
top of my head, I recall a similar issue with a benchmark in
our repository [1] (you will probably recognize the shape
there, as it's very similar to yours). In that case, to get to
optimal performance, some extra casts to `long` needed to be
added as C2 cannot yet optimize loops with that particular
shape. Note that all the bound check analysis on memory
segments is built on longs (unlike arrays and byte buffers)
and we rely on C2 to optimize common cases where accessed
offset is clearly a "small long". In some cases this check
doesn't work (yet), and some "manual help" is needed. From my
note with a conversation with Roland (who did most of the
optimization work here):
<br>
<br>
<blockquote type="cite">The expectation is that the loop
variable and the exit test operate on a single type
<br>
</blockquote>
At the time, we had bigger fishes to fry, but if this turns
out to be the reason behind the numbers you are seeing, then
it might be time to look again and try to fix this.
<br>
<br>
Cheers
<br>
Maurizio
<br>
<br>
[1] -
<a href="https://github.com/openjdk/jdk/blob/master/test/micro/org/openjdk/bench/java/lang/foreign/UnrolledAccess.java" target="_blank">https://github.com/openjdk/jdk/blob/master/test/micro/org/openjdk/bench/java/lang/foreign/UnrolledAccess.java</a><br>
<br>
On 20/03/2024 09:00, Antoine Chambille wrote:
<br>
<blockquote type="cite">Hi everyone,
<br>
<br>
I'm looking at two array arithmetic benchmarks with Panama.
<br>
<a href="https://github.com/chamb/panama-benchmarks" target="_blank">https://github.com/chamb/panama-benchmarks</a>
<br>
<br>
AddBenchmark: benchmark the element-wise addition of two
arrays of numbers. We test over standard Java arrays and
(off-heap) native memory, via array access, Unsafe and
MemorySegment. Using and not using the vector API.
<br>
<br>
SumBenchmark: Sum all the elements in an array of numbers.
We benchmark over standard Java arrays and (off-heap) native
memory, via array access, Unsafe and MemorySegment. Using
and not using the vector API.
<br>
<br>
I'm building openjdk from the master at
<a href="https://github.com/openjdk/panama-vector" target="_blank">https://github.com/openjdk/panama-vector</a>
<br>
Windows laptop with Intel core i9-11950H.
<br>
<br>
Impressive to perform SIMD on native memory in pure Java!
And I hope it's possible to optimize it further.
<br>
<br>
AddBenchmark
<br>
.scalarArrayArray 4741341.171 ops/s
<br>
.scalarArrayArrayLongStride 973926.689 ops/s
<br>
.scalarSegmentArray 1809480.000 ops/s
<br>
.scalarSegmentSegment 1231606.029 ops/s
<br>
.scalarUnsafeArray 10972240.434 ops/s
<br>
.scalarUnsafeUnsafe 1246565.503 ops/s
<br>
.unrolledArrayArray 1236491.068 ops/s
<br>
.unrolledSegmentArray 1787171.351 ops/s
<br>
.unrolledUnsafeArray 5700087.751 ops/s
<br>
.unrolledUnsafeUnsafe 1236456.434 ops/s
<br>
.vectorArrayArray 7252565.080 ops/s
<br>
.vectorArraySegment 6938948.826 ops/s
<br>
.vectorSegmentArray 4953042.042 ops/s
<br>
.vectorSegmentSegment 4606278.152 ops/s
<br>
<br>
Loops over arrays seem automatically optimized, but not when
the loop has a 'long' stride.
<br>
Reading from Segment seems to defeat loop optimisations
and/or add overhead. It gets worse when writing to Segment.
<br>
Manual unrolling makes things worse in all cases.
<br>
The performance of 'scalarUnsafeArray' (read with Unsafe,
write with array) is twice faster than almost anything else.
<br>
The vector API is fast and consistent, but maybe not at its
full potential, and the use of Segment degrades performance.
<br>
<br>
<br>
SumBenchmark
<br>
.scalarArray 671030.727 ops/s
<br>
.scalarUnsafe 669296.228 ops/s
<br>
.unrolledArray 2600591.019 ops/s
<br>
.unrolledUnsafe 2448826.428 ops/s
<br>
.vectorArrayV1 7313657.874 ops/s
<br>
.vectorArrayV2 2239302.424 ops/s
<br>
.vectorSegmentV1 7470192.252 ops/s
<br>
.vectorSegmentV2 2183291.818 ops/s
<br>
<br>
This is more in line. Manual unrolling seems to enable some
level of optimization, and then the vector API gives the
best performance.
<br>
<br>
<br>
Best,
<br>
-Antoine
<br>
</blockquote>
</blockquote>
<p></p>
</div>
<p style="margin:0px 0px 1.2em"></p>
<div title="MDH:PHA+SGksPGJyPkkgZGlkIHNvbWUgbW9yZSBhbmFseXNpcyBvZiB5b3VyIGJlbmNobWFyay4gTW9z
dCBvZiB0aGUgbm9uLXVucm9sbGVkLCBub24tdmVjdG9yaXplZCBiZW5jaG1hcmtzIHBlcmZvcm0g
c2ltaWxhcmx5LiBMb29raW5nIGF0IHRoZSBnZW5lcmF0ZWQgYXNzZW1ibHksIGl0IHNlZW1zIHRo
YXQgYm91bmQgY2hlY2tzIGFyZSBjb3JyZWN0bHkgaG9pc3RlZCBvdXRzaWRlIHRoZSBsb29wLiBT
byB0aGUgc3VnZ2VzdGlvbiBJIG1hZGUgZWFybGllciBpcyBub3QgZ29pbmcgdG8gbWFrZSBhbnkg
ZGlmZmVyZW5jZSAtIGluIGZhY3QgdGhlIHNjYWxhclNlZ21lbnRTZWdtZW50IHZlcnNpb24gaXMg
YWxyZWFkeSBjb21wZXRpdGl2ZSB3aXRoIHNjYWxhclVuc2FmZVVuc2FmZS48YnI+PC9wPjxwPkJ1
dCB0aGVyZSBpcyBpbmRlZWQgYSBiaWcgZGlmZmVyZW5jZSBiZXR3ZWVuIHNjYWxhclVuc2FmZVVu
c2FmZSBhbmQgZXZlcnl0aGluZyBlbHNlLiBUaGUgYXNzZW1ibHkgaW5kaWNhdGVzIHRoYXQgdGhp
cyB2ZXJzaW9uIGlzIHRoZSBvbmx5IG9uZSB0aGF0IHVzZXMgdmVjdG9yaXplZCBpbnN0cnVjdGlv
bnMgKGUuZy4gYWRkcGQgaW5zdGVhZCBvZiBhZGRzZCkuIFRoaXMgcHJvYmFibHkgaW5kaWNhdGVz
IHRoYXQgX3NvbWVfIG9wdGltaXphdGlvbiBpcyBmYWlsaW5nIChhY3Jvc3MgdGhlIGJvYXJkLCBl
eGNlcHQgaW4gdGhhdCBzcGVjaWZpYyBjYXNlKSwgYnV0IEknbSBub3QgYSBDMiBleHBlcnQsIHNv
IEkgY2FuJ3QgcG9pbnQgdG8gdGhlIHVuZGVybHlpbmcgY2F1c2UuPC9wPjxwPkknbSBhZGRpbmcg
VmxhZCBhbmQgUm9sYW5kLCBhcyB0aGV5IG1pZ2h0IGhhdmUgYSBiZXR0ZXIgaWRlYSBvZiB3aGF0
J3MgZ29pbmcgb24uPGJyPjwvcD48cD5UbyBtYWtlIHRoaW5ncyBlYXNpZXIgdG8gdGVzdCwgSSd2
ZSBwdXQgdG9nZXRoZXIgYSBqZGsgYnJhbmNoIHdoaWNoIGluY2x1ZGVzIGEgYnVuY2ggb2YgcmVs
ZXZhbnQgYmVuY2htYXJrcywgaW5jbHVkaW5nIGEgc3Vic2V0IG9mIHlvdXIgQWRkQmVuY2htYXJr
OjwvcD48cD5odHRwczovL2dpdGh1Yi5jb20vb3Blbmpkay9qZGsvY29tcGFyZS9tYXN0ZXIuLi5t
Y2ltYWRhbW9yZTpqZGs6QWRkQmVuY2htYXJrP2V4cGFuZD0xPC9wPjxwPkkgdGhpbmsgYSBnb29k
IHBsYWNlIHRvIHN0YXJ0IHdvdWxkIGJlIHRvIGV4cGxhaW4gdGhlIGRpZmZlcmVuY2UgYmV0d2Vl
biBzY2FsYXJVbnNhZmVBcnJheSBhbmQgc2NhbGFyVW5zYWZlVW5zYWZlLiBUaGUgb3RoZXIgYmVu
Y2htYXJrIGNhbiBiZSBsb29rZWQgYXQgbGF0ZXIsIGFzIChhZnRlciBsb29raW5nIGF0IHRoZSBh
c3NlbWJseSkgSSBkb24ndCB0aGluayB0aGlzIGlzIGFuIGlzc3VlIHRoYXQgaXMgc3BlY2lmaWMg
dG8gRkZNLjwvcD48cD5Gb3IgdGhlIHJlY29yZHMsIGhlcmUncyB0aGUgcmVzdWx0cyBJIGdldDo8
L3A+PHA+YGBgPGJyPkJlbmNobWFyayZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNw
OyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZu
YnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNw
OyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZu
YnNwOyBNb2RlJm5ic3A7IENudCZuYnNwOyZuYnNwOyZuYnNwOyBTY29yZSZuYnNwOyZuYnNwOyBF
cnJvciZuYnNwOyBVbml0czxicj5BZGRCZW5jaG1hcmsuc2NhbGFyQXJyYXlBcnJheSZuYnNwOyZu
YnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNw
OyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyBhdmd0Jm5ic3A7Jm5ic3A7IDMwJm5ic3A7Jm5ic3A7
IDk0LjQ3NSDCsSAwLjU1NCZuYnNwOyBucy9vcDxicj5BZGRCZW5jaG1hcmsuc2NhbGFyQXJyYXlB
cnJheUxvbmdTdHJpZGUmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsgYXZndCZuYnNwOyZu
YnNwOyAzMCZuYnNwOyA0ODEuMDMwIMKxIDQuNDc3Jm5ic3A7IG5zL29wPGJyPkFkZEJlbmNobWFy
ay5zY2FsYXJCdWZmZXJBcnJheSZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZu
YnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyBhdmd0Jm5ic3A7
Jm5ic3A7IDMwJm5ic3A7IDMzOS4yNDQgwrEgNC41NDYmbmJzcDsgbnMvb3A8YnI+QWRkQmVuY2ht
YXJrLnNjYWxhckJ1ZmZlckJ1ZmZlciZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNw
OyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyBhdmd0Jm5ic3A7Jm5i
c3A7IDMwJm5ic3A7IDMyOS44MTMgwrEgMi41MDQmbmJzcDsgbnMvb3A8YnI+QWRkQmVuY2htYXJr
LnNjYWxhclNlZ21lbnRBcnJheSZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZu
YnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyBhdmd0Jm5ic3A7Jm5ic3A7
IDMwJm5ic3A7IDM3Ni4yNTQgwrEgNS4xOTImbmJzcDsgbnMvb3A8YnI+QWRkQmVuY2htYXJrLnNj
YWxhclNlZ21lbnRTZWdtZW50Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5i
c3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7IGF2Z3QmbmJzcDsmbmJzcDsgMzAmbmJzcDsgMzAy
Ljc5MyDCsSA0Ljc2NyZuYnNwOyBucy9vcDxicj5BZGRCZW5jaG1hcmsuc2NhbGFyU2VnbWVudFNl
Z21lbnRMb25nU3RyaWRlJm5ic3A7IGF2Z3QmbmJzcDsmbmJzcDsgMzAmbmJzcDsgMzA1LjA3OCDC
sSA0LjI1MiZuYnNwOyBucy9vcDxicj5BZGRCZW5jaG1hcmsuc2NhbGFyVW5zYWZlQXJyYXkmbmJz
cDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsm
bmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsgYXZndCZuYnNwOyZuYnNwOyAzMCZuYnNwOyZuYnNwOyA5
NS43NjUgwrEgMS4yOTUmbmJzcDsgbnMvb3A8YnI+QWRkQmVuY2htYXJrLnNjYWxhclVuc2FmZVVu
c2FmZSZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNw
OyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyBhdmd0Jm5ic3A7Jm5ic3A7IDMwJm5ic3A7IDM1OC4w
NjAgwrEgNC44NjgmbmJzcDsgbnMvb3A8YnI+YGBgPC9wPjxwPkNoZWVyczxicj5NYXVyaXppbzxi
cj48L3A+PHA+PGJyPjwvcD48ZGl2IGNsYXNzPSJtb3otY2l0ZS1wcmVmaXgiPk9uIDIwLzAzLzIw
MjQgMTA6MjYsIE1hdXJpemlvIENpbWFkYW1vcmUgd3JvdGU6PGJyPjwvZGl2PjxibG9ja3F1b3Rl
IHR5cGU9ImNpdGUiIGNpdGU9Im1pZDo5YzkwYjdmYi02NGMwLTQ4MzQtYTliOS1mMmI4YTEzZDll
N2VAb3JhY2xlLmNvbSI+SGkgQW50b2luZSwKPGJyPnRoYW5rcyBmb3IgdGhlIGJlbmNobWFyay4g
RnJvbSB0aGUgbnVtYmVycyB5b3UgYXJlIGdldHRpbmcgaW4gdGhlIApBZGRCZW5jaG1hcmssIG15
IGd1dCBmZWVsaW5nIGlzIHRoYXQsIGZvciBtZW1vcnkgc2VnbWVudHMsIGJvdW5kIGNoZWNrcyAK
YXJlIG5vdCBiZWluZyBob2lzdGVkIG91dHNpZGUgdGhlIGxvb3AuIFRoYXQgd291bGQgY2F1c2Ug
dGhlIGtpbmQgb2YgCmRlZ3JhZGF0aW9uIHlvdSBhcmUgc2VlaW5nIGhlcmUuIEknbSBhbHNvIHN1
cnByaXNlZCB0byBzZWUsIGZvciB0aGF0IApiZW5jaG1hcmssIHRoYXQgVW5zYWZlIGlzICZndDsg
MnggZmFzdGVyIHRoYW4gdXNpbmcgcGxhaW4gYXJyYXlzLCBhZnRlciBhbGwgCnRoZSBzaXplIG9m
IHRoZSBhcnJheSBpcyBhIGxvb3AgaW52YXJpYW50LCBhbmQgbm8gY2hlY2sgc2hvdWxkIG9jY3Vy
IAp0aGVyZS4gT24gdG9wIG9mIG15IGhlYWQsIEkgcmVjYWxsIGEgc2ltaWxhciBpc3N1ZSB3aXRo
IGEgYmVuY2htYXJrIGluIApvdXIgcmVwb3NpdG9yeSBbMV0gKHlvdSB3aWxsIHByb2JhYmx5IHJl
Y29nbml6ZSB0aGUgc2hhcGUgdGhlcmUsIGFzIGl0J3MgCnZlcnkgc2ltaWxhciB0byB5b3Vycyku
IEluIHRoYXQgY2FzZSwgdG8gZ2V0IHRvIG9wdGltYWwgcGVyZm9ybWFuY2UsIApzb21lIGV4dHJh
IGNhc3RzIHRvIGBsb25nYCBuZWVkZWQgdG8gYmUgYWRkZWQgYXMgQzIgY2Fubm90IHlldCBvcHRp
bWl6ZSAKbG9vcHMgd2l0aCB0aGF0IHBhcnRpY3VsYXIgc2hhcGUuIE5vdGUgdGhhdCBhbGwgdGhl
IGJvdW5kIGNoZWNrIGFuYWx5c2lzIApvbiBtZW1vcnkgc2VnbWVudHMgaXMgYnVpbHQgb24gbG9u
Z3MgKHVubGlrZSBhcnJheXMgYW5kIGJ5dGUgYnVmZmVycykgCmFuZCB3ZSByZWx5IG9uIEMyIHRv
IG9wdGltaXplIGNvbW1vbiBjYXNlcyB3aGVyZSBhY2Nlc3NlZCBvZmZzZXQgaXMgCmNsZWFybHkg
YSAic21hbGwgbG9uZyIuIEluIHNvbWUgY2FzZXMgdGhpcyBjaGVjayBkb2Vzbid0IHdvcmsgKHll
dCksIGFuZCAKc29tZSAibWFudWFsIGhlbHAiIGlzIG5lZWRlZC4gRnJvbSBteSBub3RlIHdpdGgg
YSBjb252ZXJzYXRpb24gd2l0aCAKUm9sYW5kICh3aG8gZGlkIG1vc3Qgb2YgdGhlIG9wdGltaXph
dGlvbiB3b3JrIGhlcmUpOgo8YnI+Cjxicj48YmxvY2txdW90ZSB0eXBlPSJjaXRlIj5UaGUgZXhw
ZWN0YXRpb24gaXMgdGhhdCB0aGUgbG9vcCB2YXJpYWJsZSBhbmQgdGhlIGV4aXQgdGVzdCBvcGVy
YXRlIG9uIGEgc2luZ2xlIHR5cGUKPGJyPjwvYmxvY2txdW90ZT5BdCB0aGUgdGltZSwgd2UgaGFk
IGJpZ2dlciBmaXNoZXMgdG8gZnJ5LCBidXQgaWYgdGhpcyB0dXJucyBvdXQgdG8gYmUgCnRoZSBy
ZWFzb24gYmVoaW5kIHRoZSBudW1iZXJzIHlvdSBhcmUgc2VlaW5nLCB0aGVuIGl0IG1pZ2h0IGJl
IHRpbWUgdG8gCmxvb2sgYWdhaW4gYW5kIHRyeSB0byBmaXggdGhpcy4KPGJyPgo8YnI+Q2hlZXJz
Cjxicj5NYXVyaXppbwo8YnI+Cjxicj5bMV0gLSAKaHR0cHM6Ly9naXRodWIuY29tL29wZW5qZGsv
amRrL2Jsb2IvbWFzdGVyL3Rlc3QvbWljcm8vb3JnL29wZW5qZGsvYmVuY2gvamF2YS9sYW5nL2Zv
cmVpZ24vVW5yb2xsZWRBY2Nlc3MuamF2YQo8YnI+Cjxicj5PbiAyMC8wMy8yMDI0IDA5OjAwLCBB
bnRvaW5lIENoYW1iaWxsZSB3cm90ZToKPGJyPjxibG9ja3F1b3RlIHR5cGU9ImNpdGUiPkhpIGV2
ZXJ5b25lLAo8YnI+Cjxicj5JJ20gbG9va2luZyBhdCB0d28gYXJyYXkgYXJpdGhtZXRpYyBiZW5j
aG1hcmtzIHdpdGggUGFuYW1hLgo8YnI+aHR0cHM6Ly9naXRodWIuY29tL2NoYW1iL3BhbmFtYS1i
ZW5jaG1hcmtzCjxicj4KPGJyPkFkZEJlbmNobWFyazogYmVuY2htYXJrIHRoZSBlbGVtZW50LXdp
c2UgYWRkaXRpb24gb2YgdHdvIGFycmF5cyBvZiAKbnVtYmVycy4gV2UgdGVzdCBvdmVyIHN0YW5k
YXJkIEphdmEgYXJyYXlzIGFuZCAob2ZmLWhlYXApIG5hdGl2ZSAKbWVtb3J5LCB2aWEgYXJyYXkg
YWNjZXNzLCBVbnNhZmUgYW5kIE1lbW9yeVNlZ21lbnQuIFVzaW5nIGFuZCBub3QgCnVzaW5nIHRo
ZSB2ZWN0b3IgQVBJLgo8YnI+Cjxicj5TdW1CZW5jaG1hcms6IFN1bSBhbGwgdGhlIGVsZW1lbnRz
IGluIGFuIGFycmF5IG9mIG51bWJlcnMuIFdlIApiZW5jaG1hcmsgb3ZlciBzdGFuZGFyZCBKYXZh
IGFycmF5cyBhbmQgKG9mZi1oZWFwKSBuYXRpdmUgbWVtb3J5LCB2aWEgCmFycmF5IGFjY2Vzcywg
VW5zYWZlIGFuZCBNZW1vcnlTZWdtZW50LiBVc2luZyBhbmQgbm90IHVzaW5nIHRoZSB2ZWN0b3Ig
CkFQSS4KPGJyPgo8YnI+SSdtIGJ1aWxkaW5nIG9wZW5qZGsgZnJvbSB0aGUgbWFzdGVyIGF0IApo
dHRwczovL2dpdGh1Yi5jb20vb3Blbmpkay9wYW5hbWEtdmVjdG9yCjxicj5XaW5kb3dzIGxhcHRv
cCB3aXRoIEludGVsIGNvcmUgaTktMTE5NTBILgo8YnI+Cjxicj5JbXByZXNzaXZlIHRvIHBlcmZv
cm0gU0lNRCBvbiBuYXRpdmUgbWVtb3J5IGluIHB1cmUgSmF2YSEgQW5kIEkgaG9wZSAKaXQncyBw
b3NzaWJsZSZuYnNwO3RvIG9wdGltaXplIGl0IGZ1cnRoZXIuCjxicj4KPGJyPkFkZEJlbmNobWFy
awo8YnI+Jm5ic3A7LnNjYWxhckFycmF5QXJyYXkgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7
ICZuYnNwOyAmbmJzcDsgNDc0MTM0MS4xNzEgb3BzL3MKPGJyPiZuYnNwOy5zY2FsYXJBcnJheUFy
cmF5TG9uZ1N0cmlkZSAmbmJzcDsgJm5ic3A7OTczOTI2LjY4OSBvcHMvcwo8YnI+Jm5ic3A7LnNj
YWxhclNlZ21lbnRBcnJheSAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7IDE4MDk0
ODAuMDAwIG9wcy9zCjxicj4mbmJzcDsuc2NhbGFyU2VnbWVudFNlZ21lbnQgJm5ic3A7ICZuYnNw
OyAmbmJzcDsgJm5ic3A7IDEyMzE2MDYuMDI5IG9wcy9zCjxicj4mbmJzcDsuc2NhbGFyVW5zYWZl
QXJyYXkgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOyAxMDk3MjI0MC40MzQgb3Bz
L3MKPGJyPiZuYnNwOy5zY2FsYXJVbnNhZmVVbnNhZmUgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5i
c3A7ICZuYnNwOyAxMjQ2NTY1LjUwMyBvcHMvcwo8YnI+Jm5ic3A7LnVucm9sbGVkQXJyYXlBcnJh
eSAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7IDEyMzY0OTEuMDY4IG9wcy9zCjxi
cj4mbmJzcDsudW5yb2xsZWRTZWdtZW50QXJyYXkgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7
IDE3ODcxNzEuMzUxIG9wcy9zCjxicj4mbmJzcDsudW5yb2xsZWRVbnNhZmVBcnJheSAmbmJzcDsg
Jm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7NTcwMDA4Ny43NTEgb3BzL3MKPGJyPiZuYnNwOy51
bnJvbGxlZFVuc2FmZVVuc2FmZSAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgMTIzNjQ1Ni40
MzQgb3BzL3MKPGJyPiZuYnNwOy52ZWN0b3JBcnJheUFycmF5ICZuYnNwOyAmbmJzcDsgJm5ic3A7
ICZuYnNwOyAmbmJzcDsgJm5ic3A7IDcyNTI1NjUuMDgwIG9wcy9zCjxicj4mbmJzcDsudmVjdG9y
QXJyYXlTZWdtZW50ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgNjkzODk0OC44
MjYgb3BzL3MKPGJyPiZuYnNwOy52ZWN0b3JTZWdtZW50QXJyYXkgJm5ic3A7ICZuYnNwOyAmbmJz
cDsgJm5ic3A7ICZuYnNwOyA0OTUzMDQyLjA0MiBvcHMvcwo8YnI+Jm5ic3A7LnZlY3RvclNlZ21l
bnRTZWdtZW50ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOyA0NjA2Mjc4LjE1MiBvcHMvcwo8
YnI+Cjxicj5Mb29wcyBvdmVyIGFycmF5cyBzZWVtIGF1dG9tYXRpY2FsbHkgb3B0aW1pemVkLCBi
dXQgbm90IHdoZW4gdGhlIGxvb3AgCmhhcyBhICdsb25nJyBzdHJpZGUuCjxicj5SZWFkaW5nIGZy
b20gU2VnbWVudCBzZWVtcyB0byBkZWZlYXQgbG9vcCBvcHRpbWlzYXRpb25zIGFuZC9vciBhZGQg
Cm92ZXJoZWFkLiBJdCBnZXRzIHdvcnNlIHdoZW4gd3JpdGluZyB0byBTZWdtZW50Lgo8YnI+TWFu
dWFsIHVucm9sbGluZyBtYWtlcyB0aGluZ3Mgd29yc2UgaW4gYWxsIGNhc2VzLgo8YnI+VGhlIHBl
cmZvcm1hbmNlIG9mICdzY2FsYXJVbnNhZmVBcnJheScgKHJlYWQgd2l0aCBVbnNhZmUsIHdyaXRl
IHdpdGggCmFycmF5KSBpcyB0d2ljZSBmYXN0ZXIgdGhhbiBhbG1vc3QgYW55dGhpbmcgZWxzZS4K
PGJyPlRoZSB2ZWN0b3IgQVBJIGlzIGZhc3QgYW5kIGNvbnNpc3RlbnQsIGJ1dCBtYXliZSBub3Qg
YXQgaXRzIGZ1bGwgCnBvdGVudGlhbCwgYW5kIHRoZSB1c2Ugb2YgU2VnbWVudCBkZWdyYWRlcyBw
ZXJmb3JtYW5jZS4KPGJyPgo8YnI+Cjxicj5TdW1CZW5jaG1hcmsKPGJyPiZuYnNwOy5zY2FsYXJB
cnJheSAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5i
c3A7ICZuYnNwOyA2NzEwMzAuNzI3IG9wcy9zCjxicj4mbmJzcDsuc2NhbGFyVW5zYWZlICZuYnNw
OyAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7
NjY5Mjk2LjIyOCBvcHMvcwo8YnI+Jm5ic3A7LnVucm9sbGVkQXJyYXkgJm5ic3A7ICZuYnNwOyAm
bmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOzI2MDA1OTEuMDE5IG9wcy9z
Cjxicj4mbmJzcDsudW5yb2xsZWRVbnNhZmUgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZu
YnNwOyAmbmJzcDsgJm5ic3A7IDI0NDg4MjYuNDI4IG9wcy9zCjxicj4mbmJzcDsudmVjdG9yQXJy
YXlWMSAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5i
c3A7NzMxMzY1Ny44NzQgb3BzL3MKPGJyPiZuYnNwOy52ZWN0b3JBcnJheVYyICZuYnNwOyAmbmJz
cDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsyMjM5MzAyLjQyNCBv
cHMvcwo8YnI+Jm5ic3A7LnZlY3RvclNlZ21lbnRWMSAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJz
cDsgJm5ic3A7ICZuYnNwOyAmbmJzcDs3NDcwMTkyLjI1MiBvcHMvcwo8YnI+Jm5ic3A7LnZlY3Rv
clNlZ21lbnRWMiAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJz
cDsyMTgzMjkxLjgxOCBvcHMvcwo8YnI+Cjxicj5UaGlzIGlzIG1vcmUgaW4gbGluZS4gTWFudWFs
IHVucm9sbGluZyBzZWVtcyB0byBlbmFibGUgc29tZSBsZXZlbCBvZiAKb3B0aW1pemF0aW9uLCBh
bmQgdGhlbiB0aGUgdmVjdG9yIEFQSSBnaXZlcyB0aGUgYmVzdCBwZXJmb3JtYW5jZS4KPGJyPgo8
YnI+Cjxicj5CZXN0LAo8YnI+LUFudG9pbmUKPGJyPjwvYmxvY2txdW90ZT48L2Jsb2NrcXVvdGU+" style="height:0px;width:0px;max-height:0px;max-width:0px;overflow:hidden;font-size:0em;padding:0px;margin:0px"></div>
</div>
</div>
</blockquote></div>