RFR: JDK-8303154: Investigate and improve instruction cache flushing during compilation [v3]

[Boris Ulasevich]

On Tue, 14 Mar 2023 13:22:20 GMT, Damon Fenacci <duke at openjdk.org> wrote:

>> It was noticed that we flush the instruction cache too much for a single C1 compilation. The same is true for the C2 compilation.
>> There are several places in the code where the instruction cache is called and many of them are very intertwined (see [bug report](https://bugs.openjdk.org/browse/JDK-8303154)).
>> 
>> This PR is meant to be a "minimum" set of changes that improve the situation without introducing excessive extra information to keep track of the origin of the call through call stacks. This is done by avoiding calls to flush the ICache:
>> * at `Compilation::emit_code_epilog`
>> * when calling `CodeCache::commit` as flushing is done anyway when copying from the temporary buffer into the code cache in `CodeBuffer::copy_code_to`. This results in flushing the ICache only once instead of 3 times for a C1 compilation and twice for a C2 compilation. Additionally this halves the number of flushes during adapters generation (lots of calls).
>> * at `SharedRuntime::generate_i2c2i_adapters` as this is called with a temporary buffer and an ICache flush is not needed
>> 
>> This change decreases the number of calls to flush the ICache for a simple _Hello world_ program on Mac OSX aarch64 from 3569 to 2028 on C1 (43.2% improvement) and from  3572 to 1952 on C2 (45.4% improvement).
>> 
>> This fix includes changes for x86_32/64 and aarch64, which I could test thoroughly but also for **arm** and **riscv**, for which I would need some help with testing.
>
> Damon Fenacci has updated the pull request incrementally with one additional commit since the last revision:
> 
>   JDK-8303154: remove flush in SharedRuntime::generate_i2c2i_adapters

HI, tier1-2 tests on ARM32 are OK!

-------------

PR: https://git.openjdk.org/jdk/pull/12877