RFR: 8255471: ZGC: Rework root iterators and closures

[Erik Österlund]

On Thu, 29 Oct 2020 11:01:05 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

> Today we have the ZRootsIteratorClosure type to cater for the different type of roots and their containers (oop storage, threads, nmethods, runtime oop data structures). This closure is then passed down and applied to the different containers. We split the root sets into different sets: concurrent strong, concurrent weak, and non-concurrent weak. Not all of those sets contain all types of containers. This forces some of the code to implement, or at least consider, parts of the ZRootsIteratorClosure interface that it doesn't care about. It also moves some of the closure logic into shared code, even when it's only used by one of the closures.
> 
> I propose that we turn this inside out, and use direct closures that match the visited containers (NMethodClosure for nmethods, et.c.). The intention is to make it a bit easier to see from the call sites what code is run and, hopefully, making localized changes stay more local in the code.
> 
> Note: I started this code with a version building upon https://bugs.openjdk.java.net/browse/JDK-8212879, but I don't want to wait for that work to finish before sending this out, so the current patch contains handling of non-conurrent weak roots. As soon as that gets integrated, we can get rid of the ZWeakSerial and the associated non-concurrent weak roots handling.
> 
> I've tested this on tier1-7 on linux x64.

In the patch, the heap iterator uses a ThreadToOopClosure, which implicitly uses a CodeBlobToOop closure for on-stack nmethods. I don't know how I feel about this. I think it might be desirable to use a ZThreadToOopClosure instead, that you pass in the explicitly passed in NMethodClosure to instead. Or something along those lines, the important thing being to pass in the NMethodClosure passed to the iterator, and not use a CodeBlobToOopClosure instead. Otherwise, it is rather confusing if you explicitly pass in an NMethodClosure to the strong root iterator, telling what should be done to strong nmethods, but then it is not applied to nmethods encountered on the stack. Instead, it is only applied to the code cache nmethods, when ClassUnloading is disabled. However, there does not seem to be any reason for distinguishing the two cases, AFAICT. The NMethodClosure describes what should be done to a strong nmethod, and how we found the strong nmethod (stack or code cache depending on Clas
 sUnloading), does not seem to make a difference. Differenting the two does however conflict with the goal of making the code more straight and understandable. If you pass in an NMethodClosure to the strong root iterator, I would expect that NMethodClosure to be applied to all strong nmethods found, and not have to deal with what happens with and without class unloading as a special case, that does not look like it would have to be special.

On a different yet related note, I see that ZHeapIteratorRootOopClosure now always performs some kind of NativeAccess::oop_load on all oops, including misaligned nmethod oops. I'm not sure how I feel about that. I think it's fine, but let me elaborate my thoughts. I suppose the contract for how it is used is that the oops have already been fixed by nmethod barriers (under the per-nmethod lock) to be good, before loading potentially misaligned oops. That should imply that the oops are never self healed, which is a requirement to not cause undefined behaviour on the HW level, performing atomics on misaligned data. So I suppose ensuring the nmethod barrier has completed before reading is crucial here. When reading the NMethodClosure, it is clear how that is the case. We apply the nmethod barrier and then walk the oops. But using the CodeBlobToOopClosure, now with parallel iteration as of lately, does seem a bit scary. I guess it still is okay, because Thread::oops_do() calls finish_proc
 essing(), which ensures all on-stack nmethods enter their barriers before applying the OopClosure. So I guess this all works correctly, but again would read a bit more intuitively for me at least, if we had a ZThreadToOopClosure that you pass in the root iterator supplied NMethodClosure to. I think it works with and without that, but would be easier to understand why it works if the NMethodClosure was passed in to the [Z?]ThreadToOopClosure. Again, it might just be me thinking that  would be easier to understand, but I would at least like to point that out, so I can hear your thoughts about it.

Apart from that, this looks great, and I think it is much more readable now that you can just tell the iterators what to do with various types of roots, without getting random callbacks that may or may not make sense. Great job!

-------------

Changes requested by eosterlund (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/928