RFR (S): JEP-142: Reduce Cache Contention on Specified Fields

[Kirk Pepperdine]

> On 11/25/12 16:22, Kirk Pepperdine wrote:
> 
>> I'm completely in agreement that there are some applications where this
>> optimization is necessary. That said, I've more fundamental concerns with
>> respect to this API. Traditionally the JVM has steered away from having
>> developers hint at optimization as these optimizations have typically been
>> ill thought out. My mantra has always been, measure, don't guess. WIthout
>> denigrating any group of developers I would say that very very few know how
>> to get the measures to even start to understand how to apply this
>> "optimization". I would suggest that it would be better to leave the
>> application of this optimization to HotSpot and the JIT compiler.
> 
> This is a common reaction, but ...
> 
> As witnessed for example by C11/C++11 standards (that support
> better ways of obtaining alignment and padding), memory contention
> issues are increasingly part of core library and lower-level programming,
> in any language.

And it's a problem that naturally is just going to get worse....

> Naively, one might hope that HW memory controllers,
> OSes, and/or VMs could sense problems and take evasive action, but
> no practical techniques for doing so are known. If JDK/Java takes
> the position that it is best to hope for miracles, developers
> who need this functionality to create libraries and frameworks
> will eventually choose to abandon the platform. Which might be
> OK if your goal is to hasten the COBOLization of Java. But
> most people seem to want Java to remain a broad-spectrum language,
> in which case, it cannot adopt policies that apply to only 99.9%
> of the spectrum.


I think to suggest that we do nothing and rely on "magic" wasn't the message that I was trying to send. My concerns with this JEP come to me at many different levels. I might add that I tried to raise a flag w.r.t. autoboxing and as successful as it was (my editorial in JDJ remains one of the most popular to date) it was far too little and way too late. What I see here is a good idea, it helps address a problem that I believe damages performance far more than people realize. That said, I'm not sure that this is the "Java" direction. Once more, what generics has taught us that once in, these things become very very difficult to make right and impossible to remove or replace. More over, what you've taught me is that we need to be ultra-conservative when it comes to how we implement and push to production. IMHO, this annotation changes the a very fundamental assumption that developers should write code and the compilers/JIT should look after everything they can. It's also breaks another very fundamental direction where the JVM has been able to adapt to different environments and even to changing conditions with-in an environment. Correct me if I'm wrong but I feel that these annotations assumes that developers can predict deployment environments, the problems that they will encounter within those environments. While that maybe true in some cases I might also suggest that the JVM could be able to pick up on them and make the necessary adjustments.

At the next level of concerns it's an optimization that very few people know how to measure for. Even so, measuring in the context of a micro-benchmark is unlikely to lead to a measure that will allow one to take reasonable corrective action that holds in macro-environments. In other words, and as I know you are well aware, there are a number of competing concerns that render these micro-optimizations invisible to the end user. This not to say that we shouldn't reach for these optimization, it's just saying that to the vast majority of applications it is very likely that the results will be an unnecessary introduction of low level (and potentially fragile) detail into the code without receiving any benefit. Where as applying these techniques using a runtime profiler will keep the low level details out of the code (reduce fragility of the optimization) and while providing meaningful benefit (as has been our experience with HotSpot/JIT).

My last concern is about the general level of knowledge and the mix of these hardcore features into this blend. We don't have any durable (or even fragile) measures that are accessible to the every day developer meaning.. lets guess that we have a problem and then apply this fix.... this isn't a direction that I would subscribe to.

I'd like to weave in Richard's response. Full disclosure, Richard is one of my subcontractors and we had him working on tooling that looks for these conditions by programming the event counters on Intel chips (machine specific registers). It's a piece of work that has unfortunately been pre-empted due to some higher priority items in the hopper so there is still much to be done here. What we are looking for are groups counter values that would be an indication that the software/hardware aren't cooperating as well as they should be. We believe that one of the key indicators would be a decrease in the rate at which instructions are retired. Using this plus other factors we are working to correlate drops in these key indicators (such as retirement rates) to activity in the code. From there we'd hope to be able to at worst, identify the underlying problem or at best, suggest a way to make the software cooperate better with the hardware. More over, using hardware provided counters helps cheapen the cost of detecting these problems. My work in building customized production system ready profilers has taught me that the cost of running targeted diagnostics can be accurate, quick and cheap. I believe that to do this right, one will need a combination of all of these techniques, a trigger, an analytic along with a diagnostic.

To address David, as you can see I'm not suggesting we do nothing. That said, there is a third alternative that has 

sent. Richard responded in another post and I'd like to argument that by saying we know how to get the measurements using machine specific registers to understand when there is a low level problem. We've been looking at ways to correlate that to application behaviour and I think it's only going to be a matter of time before we'll be able to tell a developer exactly what needs to be done to reduce the problem. That said, I do recognize that it's not been fully explored and that the path isn't completely clear but.... I'd much rather say ok, at least we know a JVM based solution is impossible before forcing developers to muck up code.


> 
> (BTW, stay tuned for other VM support proposals along these lines.)

Any hints would be very much appreciated :-)

Regards,
Kirk