RFR: 8221554: aarch64 cross-modifying code

[Andrew Haley]

On Thu, 15 Oct 2020 16:10:22 GMT, Robbin Ehn <rehn at openjdk.org> wrote:

>>> A question, ISB don't flush the I-cache which I thought was needed?
>>> I would have expected something more similar to gcc clear_cache.
>> 
>> (Possible I've missed something in your question...)
>> Any cache flushing would be performed by the thread that modifies the code.
>> The cross_modify_fence is for the other threads after the code has been modified. The only thing they need to do is an
>> isb.
>
>> > A question, ISB don't flush the I-cache which I thought was needed?
>> > I would have expected something more similar to gcc clear_cache.
>> 
>> (Possible I've missed something in your question...)
>> Any cache flushing would be performed by the thread that modifies the code.
>> The cross_modify_fence is for the other threads after the code has been modified. The only thing they need to do is an
>> isb.
> 
> Correct me if I'm wrong:
> We write the updated instructions to D-cache which is not coherent with the I-cache.
> To synchronize the I-cache to what we wrote in the D-cache, we first need make sure the D-cache is committed to memory.
> Then each processor must invalidate it's I-cache, thus fetching the new instructions from memory.
> So yes the thread that modifies the code should make sure the D-cache is consistent with main memory.
> Can the I-cache invalidation be done by the modifying thread in a distributed way ?
> 
> When we do "fix_oop_relocations" we end up calling: MacroAssembler::patch_oop(..) in aarch64 case.
> To me this indicate that we just changed an oop in a code segment, but I see no signs of above.
> 
> Maybe I got something wrong about how this I-cache/D-cache works or missed something in our code?
> 
> I think @theRealAph suggested having a indirect to the oops and thus move them of out code and avoiding I-cache.

On 15/10/2020 17:10, Robbin Ehn wrote:
>
> Correct me if I'm wrong:
> We write the updated instructions to D-cache which is not coherent with the I-cache.
> To synchronize the I-cache to what we wrote in the D-cache, we first need make sure the D-cache is committed to memory.
Yes.

> Then each processor must invalidate it's I-cache, thus fetching the new instructions from memory.
> So yes the thread that modifies the code should make sure the D-cache is consistent with main memory.
> Can the I-cache invalidation be done by the modifying thread in a distributed way ?

Yes, it is. In a multiprocessor system, IC IVAU instruction is broadcast to all cores in the same memory domain.

> When we do "fix_oop_relocations" we end up calling: MacroAssembler::patch_oop(..) in aarch64 case.
> To me this indicate that we just changed an oop in a code segment, but I see no signs of above.

The call stack looks like this:

#0 ICache::invalidate_range  /home/aph/jdk-jdk/src/hotspot/os_cpu/linux_aarch64/icache_linux_aarch64.hpp:40
#1 Relocation::pd_set_data_value  /home/aph/jdk-jdk/src/hotspot/cpu/aarch64/relocInfo_aarch64.cpp:58
#2 DataRelocation::set_value  /home/aph/jdk-jdk/src/hotspot/share/code/relocInfo.hpp:849
#3 DataRelocation::set_value  /home/aph/jdk-jdk/src/hotspot/share/code/relocInfo.hpp:844
#4 oop_Relocation::fix_oop_relocation  /home/aph/jdk-jdk/src/hotspot/share/code/relocInfo.cpp:554
#5 nmethod::fix_oop_relocations  /home/aph/jdk-jdk/src/hotspot/share/code/nmethod.cpp:1051

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671

-------------

PR: https://git.openjdk.java.net/jdk/pull/428