Contribution: Lock Contention Profiler for HotSpot

Tue Nov 8 07:32:14 UTC 2016

Fair enough.  We sample instead of getting detailed information, but we are
generally trying to profile an application running on lots of JVMs, so we
have some scale that not everyone does.

We wouldn't have been able to live with 7.8% overhead either, though.

I bet someone could come up with a reasonable strategy to make
MonitorContendedExit work if they were motivated. :)

Jeremy

On Mon, Nov 7, 2016 at 11:35 AM, Peter Hofer <peter.hofer at jku.at> wrote:

> Hi Jeremy,
>
> On 2016-11-04 19:24, Jeremy Manson wrote:
>
>> Why aren't these extensions to JVMTI, which already has
>> MonitorContendedEnter and MonitorContendedEntered events? You could
>> just add a MonitorContendedRelease event to cover what you want.
>> Then the bulk of the tracking work can be done in JVMTI.
>>
>
>  One of our main goals was to make profiling very lightweight so that
> there is a chance that the profiler can be used on production systems. In
> the HotSpot code, we can record events and maintain state very efficiently.
>
> I agree that a profiler that uses only JVMTI and extension methods would
> be more modular. We actually tried to implement a comparable profiler using
> JVMTI. It performs very frequent state transitions to the agent and back,
> requires wrapping all references and data structures, and needs tagging to
> associate state with objects. Moreover, it cannot efficiently cache stack
> traces without always resolving inlined methods from the compiler's debug
> information (which makes a lot of difference in our HotSpot-internal
> profiler). The JVMTI-based profiler turned out to be rather inefficient,
> which is why we didn't pursue this approach further.
>
> As Alex pointed out, there used to be a MonitorContendedExit event in
> early versions of JVMTI. It was eliminated because the context of a monitor
> exit is not really safe for invoking a JVMTI callback, which is another
> issue that would need to be addressed first.
>
> Cheers,
>  Peter
>
> At Google, we've built on these JVMTI primitives quite successfully.
>> The only internal enhancements we've had to make is to make them support
>> j.u.c locks.
>>
>> (We've also done the hotspot lock contention work, but it has been less
>> directly useful.)
>>
>> Jeremy
>>
>> On Fri, Nov 4, 2016 at 4:26 AM, David Gnedt <David.Gnedt at jku.at
>> <mailto:David.Gnedt at jku.at>> wrote:
>>
>>     Hello,
>>
>>     I am one of the authors of this work and I gladly support this
>>     contribution.
>>
>>     Best regards,
>>     David Gnedt
>>
>>     >>> Peter Hofer <peter.hofer at jku.at <mailto:peter.hofer at jku.at>>
>>
>>     04.11.16 11.01 Uhr >>>
>>     Hello everyone,
>>
>>     we are researchers at the University of Linz and have worked on a lock
>>     contention profiler that is built into HotSpot. We would like to
>>     contribute this work to the OpenJDK community.
>>
>>     Our profiler records an event when a thread fails to acquire a
>> contended
>>
>>     lock and also when a thread releases a contended lock. It further
>>     efficiently records the stack traces where these events occur. We
>>     devised a versatile visualization tool that analyzes the recorded
>> events
>>
>>     and determines when and where threads _cause_ contention by holding a
>>     contended lock. The visualization tool can show the contention by
>> stack
>>     trace, by lock, by lock class, by thread, and by any combination of
>>     those aspects.
>>
>>     We described our profiler in more detail in a research paper at ICPE
>>     2016. [1] In our evaluation, we found that the overhead is typically
>>     below 10% for common multi-threaded Java benchmarks. Please find a
>> free
>>     download of the paper on our website:
>>     > http://mevss.jku.at/lct/
>>
>>     I contribute this work on behalf of Dynatrace Austria (the sponsor of
>>     this research), my colleagues David Gnedt and Andreas Schoergenhumer,
>>     and myself. The necessary OCAs have already been submitted.
>>
>>     We provide two patches:
>>
>>     Patch 1. A patch for OpenJDK 8u102-b14 with the profiler that we
>>     described and evaluated in our paper, plus minor improvements. It
>>     records events for Java intrinsic locks (monitors) and for
>>     java.util.concurrent locks (ReentrantLock and ReentrantReadWriteLock).
>>     We support only Linux on 64-bit x86 hardware.
>>
>>     >
>>     http://cr.openjdk.java.net/~tschatzl/phofer/webrev_hotspot_
>> jdk8u102b14/
>>     <http://cr.openjdk.java.net/~tschatzl/phofer/webrev_hotspot_
>> jdk8u102b14/>
>>     >
>>     http://cr.openjdk.java.net/~tschatzl/phofer/webrev_jdk_jdk8u102b14/
>>     <http://cr.openjdk.java.net/~tschatzl/phofer/webrev_jdk_jdk8u102b14/>
>>
>>     Patch 2. A patch for OpenJDK 9+140 with a profiler for VM-internal
>>     native locks only. We consider this to be useful for HotSpot
>> developers
>>     to find locking bottlenecks in HotSpot itself. We tested this patch
>> only
>>
>>     on Linux on 64-bit x86 hardware, but it should require few changes for
>>     other platforms.
>>
>>     >
>>     http://cr.openjdk.java.net/~tschatzl/phofer/webrev_nativeloc
>> ksonly_hotspot_jdk9%2b140/
>>     <http://cr.openjdk.java.net/~tschatzl/phofer/webrev_nativelo
>> cksonly_hotspot_jdk9%2b140/>
>>     >
>>     http://cr.openjdk.java.net/~tschatzl/phofer/webrev_nativeloc
>> ksonly_jdk_jdk-9%2b140/
>>     <http://cr.openjdk.java.net/~tschatzl/phofer/webrev_nativelo
>> cksonly_jdk_jdk-9%2b140/>
>>
>>     With both patches, the profiler is enabled with
>> -XX:+EnableEventTracing.
>>
>>     By default, an uncompressed event trace is written to file
>> "output.trc".
>>
>>     More detailed usage information and a download of the corresponding
>>     visualization tool is available on our website,
>>     http://mevss.jku.at/lct/.
>>
>>     Kind regards,
>>       Peter Hofer
>>
>>
>>     --
>>     Peter Hofer
>>     Christian Doppler Laboratory on Monitoring and Evolution of
>>     Very-Large-Scale Software Systems / Institute for System Software
>>     University of Linz
>>
>>
>>     [1] Peter Hofer, David Gnedt, Andreas Schoergenhumer, Hanspeter
>>     Moessenboeck. Efficient Tracing and Versatile Analysis of Lock
>>     Contention in Java Applications on the Virtual Machine Level.
>>     Proceedings of the 7th ACM/SPEC International Conference on
>> Performance
>>     Engineering (ICPE’16), Delft, Netherlands, 2016.
>>
>>
>>
>>