ClassValue perf?

Thu May 19 12:32:32 UTC 2016

Hi Peter,

thank you. As some background info, the machine I'm running the benchmarks on is a 32-core Xeon.

Results of running your latest benchmarks with unmodified 9-dev and your two patches (02, 03) are attached. Overall, it seems your solution 03 is a bit slower than 02, and 02 shines especially in the expunging benchmark, but also for random access with large numbers of classes and class values. It appears to be somewhat slower than the unmodified case, though.

It may well be that running the bechmark so few times does not deliver a stable enough result. I'd like Aleksey to comment on this: is adopting Peter's code worthwhile given it improves on footprint and reduces code size and complexity?

I agree regarding whether there's a point in optimising for single-value storage whilst maintaining full flexibility. In a scenario where it is known that only one value will be associated with a class, it's better to use static fields.

Best,

Michael

> Am 08.05.2016 um 17:18 schrieb Peter Levart <peter.levart at gmail.com>:
> 
> Hi Michael,
> 
> On 05/06/2016 04:48 PM, Michael Haupt wrote:
>> Hi Peter,
>> 
>> thank you. I've run the full benchmark in my setup and uploaded the updated cumulative results to  <http://cr.openjdk.java.net/%7Emhaupt/8031043/>http://cr.openjdk.java.net/~mhaupt/8031043/ <http://cr.openjdk.java.net/~mhaupt/8031043/>.
>> 
>> The benchmark indeed shows that this latest addition to the group slows down random and sequential access, especially for small numbers of values and classes. The OpenJDK tests are fine; I'm running a batch of internal tests as well.
>> 
>> Given that one concern with this issue, next to reducing footprint, was to optimise for the single-value case, I'm still a bit hesitant even though the sheer amount of code reduction is impressive. I'll evaluate further.
> 
> Interesting. I observed quite the opposite on my machine (i7-4771, 8 MiB cache) . For sequential access pattern or for random access with small number of CV(s) and Class(es) the results are comparable. Only for 256 CV(s) x 1024 Class(es) and with random access pattern, I observed about 20% drop of performance which I attributed to the difference in design of CHM vs. the 'cache' of JDK 9 ClassValue (worse CPU cache locality for CHM):
> 
>     http://cr.openjdk.java.net/~plevart/misc/ClassValue.Alternative2/ClassValueBench.java <http://cr.openjdk.java.net/~plevart/misc/ClassValue.Alternative2/ClassValueBench.java>
> 
> I doubt that single value per Class instance is something that is beneficial to optimize. Such optimization would be very fragile so it would not be something to rely on. Typical or even worst case performance is more important in my opinion.
> 
> The fast-path lookup performance is the most important performance aspect of ClassValue, but it is not the only one that can be observed. Footprint and consequential GC / expunging overhead is also something to consider. The implementation presented in my webrev.02 maintains a linked list of weakly-referenced ClassValueMap(s). For each stale dequeued key, it probes each map and removes such key from any live map(s) containing it. This works optimally when the matrix of (ClassValue, Class) pairs is not sparse. I did an experiment with alternative expunging design where I maintain an array of weakly-referenced ClassValueMap(s) on each key that is inserted in them. This has approx. 10% additiona footprint overhead compared to original expunging design (but still just half the footprint overhead of jdk 9 ClassValue design):
> 
>     http://cr.openjdk.java.net/~plevart/misc/ClassValue.Alternative2/webrev.03/ <http://cr.openjdk.java.net/~plevart/misc/ClassValue.Alternative2/webrev.03/>
> 
> The situation I envisioned was when a single JVM hosts multiple (say N) isolated applications (in an app server for example) and when one such application is re-deployed.
> 
> In original design (webrev.02) each dequeued ClassValue.key is probed against all class maps that remain and belong to the other N-1 applications. In the alternative expunging design (webrev.03) the dequeued key just scans the array of weakly-referenced maps that the key was inserted into.
> 
> I created a benchmark to exercise such situation(s):
> 
>     http://cr.openjdk.java.net/~plevart/misc/ClassValue.Alternative2/ClassValueExpungeBench.java <http://cr.openjdk.java.net/~plevart/misc/ClassValue.Alternative2/ClassValueExpungeBench.java>
> 
> It measures the time of a hypothetical redeployment of one application in an app server where there are 16 such running applications. The measurement includes class-loading, GC time and initialization of ClassValue(s). Results show that alternative expunging design (webrev.03) doesn't bring any improvements (or that original supposedly sub-optimal expunging design (webrev.02) doesn't show any weaknesses) for the range of parameters exercised in the benchmark.
> 
> What this benchmark shows too is that original jdk 9 ClassValue has at least 2x overhead with cleanup compared to my designs (note that benchmark includes classloading time too).
> 
>  Regards, Peter
> 
>> 
>> Best,
>> 
>> Michael
>> 
>>> Am 05.05.2016 um 17:21 schrieb Peter Levart <peter.levart at gmail.com <mailto:peter.levart at gmail.com>>:
>>> 
>>> Hi Michael,
>>> 
>>> On 05/04/2016 06:02 PM, Michael Haupt wrote:
>>>> Hi Peter,
>>>> 
>>>> thank you for chiming in again! :-) I'll look at this in depth on Friday.
>>> 
>>> Good. Because I found bugs in expunging logic and a discrepancy of behavior when a value is installed concurrently by some other thread and then later removed while the 1st thread is still calculating the value. Current ClassValue re-tries the computation until it can make sure there were no concurrent changes to the entry during its computation. I fixed both things and verified that the behavior is now the same:
>>> 
>>>      <http://cr.openjdk.java.net/%7Eplevart/misc/ClassValue.Alternative2/webrev.02/>http://cr.openjdk.java.net/~plevart/misc/ClassValue.Alternative2/webrev.02/ <http://cr.openjdk.java.net/~plevart/misc/ClassValue.Alternative2/webrev.02/>
>>> 
>>> Regards, Peter
>> 
>> 
>> -- 
>> 
>>  <http://www.oracle.com/>
>> Dr. Michael Haupt | Principal Member of Technical Staff
>> Phone: +49 331 200 7277 | Fax: +49 331 200 7561
>> Oracle Java Platform Group | LangTools Team | Nashorn
>> Oracle Deutschland B.V. & Co. KG | Schiffbauergasse 14 | 14467 Potsdam, Germany
>> 
>> ORACLE Deutschland B.V. & Co. KG | Hauptverwaltung: Riesstraße 25, D-80992 München
>> Registergericht: Amtsgericht München, HRA 95603
>> 
>> Komplementärin: ORACLE Deutschland Verwaltung B.V. | Hertogswetering 163/167, 3543 AS Utrecht, Niederlande
>> Handelsregister der Handelskammer Midden-Nederland, Nr. 30143697
>> Geschäftsführer: Alexander van der Ven, Jan Schultheiss, Val Maher
>>  <http://www.oracle.com/commitment>	Oracle is committed to developing practices and products that help protect the environment
>> 
>> 
>> 
>> _______________________________________________
>> mlvm-dev mailing list
>> mlvm-dev at openjdk.java.net <mailto:mlvm-dev at openjdk.java.net>
>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev <http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev>
> 

-- 

 <http://www.oracle.com/>
Dr. Michael Haupt | Principal Member of Technical Staff
Phone: +49 331 200 7277 | Fax: +49 331 200 7561
Oracle Java Platform Group | LangTools Team | Nashorn
Oracle Deutschland B.V. & Co. KG | Schiffbauergasse 14 | 14467 Potsdam, Germany

ORACLE Deutschland B.V. & Co. KG | Hauptverwaltung: Riesstraße 25, D-80992 München
Registergericht: Amtsgericht München, HRA 95603

Komplementärin: ORACLE Deutschland Verwaltung B.V. | Hertogswetering 163/167, 3543 AS Utrecht, Niederlande
Handelsregister der Handelskammer Midden-Nederland, Nr. 30143697
Geschäftsführer: Alexander van der Ven, Jan Schultheiss, Val Maher
 <http://www.oracle.com/commitment>	Oracle is committed to developing practices and products that help protect the environment

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/mlvm-dev/attachments/20160519/6f708e0d/attachment-0004.html>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: result-plain.txt
URL: <http://mail.openjdk.java.net/pipermail/mlvm-dev/attachments/20160519/6f708e0d/result-plain-0001.txt>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/mlvm-dev/attachments/20160519/6f708e0d/attachment-0005.html>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: result-plevart-02.txt
URL: <http://mail.openjdk.java.net/pipermail/mlvm-dev/attachments/20160519/6f708e0d/result-plevart-02-0001.txt>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/mlvm-dev/attachments/20160519/6f708e0d/attachment-0006.html>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: result-plevart-03.txt
URL: <http://mail.openjdk.java.net/pipermail/mlvm-dev/attachments/20160519/6f708e0d/result-plevart-03-0001.txt>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/mlvm-dev/attachments/20160519/6f708e0d/attachment-0007.html>