[jdk11u-dev] RFR: 8250597: G1: Improve inlining around trim_queue

[Dmitry Chuyko]

On Fri, 11 Jun 2021 13:53:36 GMT, Dmitry Chuyko <dchuyko at openjdk.org> wrote:

> This change improves latency with G1 in some cases and makes trim_queue() code structure closer to current mainline.
> 
> The original patch does not apply cleanly in particular because of https://bugs.openjdk.java.net/browse/JDK-8200545 . However the refactoring is rather simple because it is a rearrangement of code between functions with compiler hints. It has been reproduced (short description in the next comment).
> 
> The change is isolated and is a set of code moves so the risk is low.
> 
> Testing: tier1, tier2 on aarch64 and x86_64 in development, the change has been included in Liberica JDK 11u EA for some time.
> 
> As an example of performance improvement, Critical-jOPS on Graviton2 raised +1%:+6.2% for 8GB heap.

G1ParScanThreadState::trim_queue(): 
Moved from g1ParScanThreadState.cpp to g1ParScanThreadState.inline.hpp, 'do-while !is_empty()' eleiminated.

G1ParScanThreadState::do_oop_evac():
Moved from g1ParScanThreadState.inline.hpp to g1ParScanThreadState.cpp, copy_to_survivor_space() -> do_copy_to_survivor_space()

[G1ParScanThreadState::do_partial_array()]
G1ParScanThreadState::do_oop_partial_array():
Moved from g1ParScanThreadState.inline.hpp to g1ParScanThreadState.cpp, made not 'inline'.

[G1ParScanThreadState::dispatch_task()]
G1ParScanThreadState::dispatch_reference():
Moved from g1ParScanThreadState.inline.hpp to g1ParScanThreadState.cpp, made not 'inline'.

G1ParScanThreadState::needs_partial_trimming():
Made not 'inline' in g1ParScanThreadState.inline.hpp.

G1ParScanThreadState::trim_queue_to_threshold()
Declaration moved from g1ParScanThreadState.inline.hpp to g1ParScanThreadState.cpp, 'inline' changed to 'ATTRIBUTE_FLATTEN NOINLINE' in g1ParScanThreadState.cpp, got 'do-while !overflow_empty()', made not 'inline' in g1ParScanThreadState.hpp.

G1ParScanThreadState::steal_and_trim_queue():
Declaration moved from g1ParScanThreadState.inline.hpp to g1ParScanThreadState.cpp,  with 'ATTRIBUTE_FLATTEN', comments, made not 'inline' in g1ParScanThreadState.hpp.

G1ParScanThreadState::allocate_copy_slow():
Added in g1ParScanThread_surviving_young_words[young_index] += word_szState.cpp as 'NOINLINE', extracted from copy_to_survivor_space/if (obj_ptr == NULL) except the case when handle_evacuation_failure_par() is needed, part before tracer is wrapped in if (!(dest_attr->is_old() && _old_gen_is_full)), tracing is wrapped in if (obj_ptr != NULL), declaration added in g1ParScanThreadState.hpp.

G1ParScanThreadState::undo_allocation():
Added in g1ParScanThreadState.cpp as 'NOINLINE', extracted from copy_to_survivor_space/_plab_allocator->undo_allocation, declaration added in g1ParScanThreadState.hpp.

G1ParScanThreadState::copy_to_survivor_space():
Renamed to do_copy_to_survivor_space() in g1ParScanThreadState.cpp + comment, copy_to_survivor_space() added as a wrapper in g1ParScanThreadState.cpp, do_copy_to_survivor_space() declaration added and copy_to_survivor_space() declaration corrected in g1ParScanThreadState.hpp.
assert(is_in_cset()) has been added.
if(_old_gen_is_full && dest_state.is_old()) block moved.
_surviving_young_words[young_index] += word_sz moved into new block around young_index >  0 asserts

G1ParScanThreadState::handle_evacuation_failure_par():
'NOINLINE' in g1ParScanThreadState.cpp.

G1ParScanThreadState::is_partially_trimmed():
Manually inlined into trim_queue_partially().

G1ParScanThreadState::trim_queue_partially():
'inline' in g1ParScanThreadState.hpp instead of g1ParScanThreadState.inline.hpp.
do-while moved (in the patch it is replaced by asserts).

G1ParScanThreadState::trim_queue():
Made 'inline' in g1ParScanThreadState.hpp.

ATTRIBUTE_FLATTEN
Defined in globalDefinitions.hpp, globalDefinitions_gcc.hpp.

Manually inline deal_with_reference()-s into G1ParScanThreadState::dispatch_reference().

-------------

PR: https://git.openjdk.java.net/jdk11u-dev/pull/26