RFR: JDK-8303154: Investigate and improve instruction cache flushing during compilation

[Vladimir Kozlov]

On Mon, 6 Mar 2023 08:37:50 GMT, Damon Fenacci <duke at openjdk.org> wrote:

> It was noticed that we flush the instruction cache too much for a single C1 compilation. The same is true for the C2 compilation.
> There are several places in the code where the instruction cache is called and many of them are very intertwined (see [bug report](https://bugs.openjdk.org/browse/JDK-8303154)).
> 
> This PR is meant to be a "minimum" set of changes that improve the situation without introducing excessive extra information to keep track of the origin of the call through call stacks. This is done by avoiding calls to flush the ICache at `Compilation::emit_code_epilog` and  when copying from the temporary buffer into the code cache in `CodeBuffer::copy_code_to` if there is a call to `CodeCache::commit` later on. This results in flushing the ICache only once instead of 3 times for a C1 compilation and twice for a C2 compilation respectively.
> 
> This change decreases the number of calls to flush the ICache for a simple _Hello world_ program on Mac OSX aarch64 from 3568 to 3487 on C1 (2.27% improvement) and from  3499 to 3415 on C2 (2.40% improvement).

Did you look on how many times we flush ICache during adapters generation?
It has most numerous cases when I looked on it:
"CodeCache::commit() is also used for adapters. But adapters uses RuntimeBlob which calls CodeBuffer::copy_code_to()."

I thought we would remove flush from CodeCache::commit() and not from copy_code_to().

-------------

PR: https://git.openjdk.org/jdk/pull/12877