RFR: 8304824: NMT should not use ThreadCritical [v3]

[Robert Toyonaga]

On Thu, 12 Sep 2024 14:37:22 GMT, Robert Toyonaga <duke at openjdk.org> wrote:

>> ### Summary
>> This PR just replaces `ThreadCritical` with a lock specific to NMT.  `ThreadCritical` is a big lock and is unnecessary for the purposes of NMT. I've implemented the new lock with a semaphore so that it can be used early before VM init.  There is also the possibility of adding assertions in places we expect NMT to have synchronization. I haven't added assertions yet in many places because some code paths such as the (NMT tests)  don't lock yet. I think it makes sense to close any gaps in locking in another PR in which I can also add more assertions. 
>> 
>> Testing:
>> - hotspot_nmt
>> - gtest:VirtualSpace
>> - tier1
>
> Robert Toyonaga has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Switch to using a Hotspot Mutex.

After switching to a Hotspot Mutex, it looks like the `windows-x64 / test (hs/tier1 common) GHA` is failing because the test `release_bad_ranges` in test_os.cpp is expecting an assertion and an error message to be printed. However, when this printing happens, `tty_lock` gets acquired out of rank order with the already held `NMT_lock`, causing the lock rank assertion fail. One solution would be to lower the rank of `tty_lock`. I'm not sure that's the best solution because that might cause rank conflicts with other locks (and it makes sense to give locks the highest possible rank to minimize future conflicts).

-------------

PR Comment: https://git.openjdk.org/jdk/pull/20852#issuecomment-2349040682