RFR: 8366154: Validate thread type requirements in debug commands

[Kerem Kat]

On Mon, 1 Sep 2025 21:58:02 GMT, David Holmes <dholmes at openjdk.org> wrote:

>> Prevents segmentation faults during `gdb` sessions. The crashes were caused by the `ResourceMark` constructor being called on a native thread, which is not supported. This happened when invoking debug commands that require a `Thread` or `JavaThread` context from an incorrect thread type.
>> 
>> ### Solution
>> 
>> This change introduces `onThread()` and `onJavaThread()` helper methods to the `Command` class. These methods validate the thread context and ensure `ResourceMark` is only created when on a valid VM thread. All thread-dependent debug commands now use these guards to validate the context, printing a clear error and exiting gracefully upon failure.
>> 
>> ### Testing
>> 
>> Manually verified using `gdb` by calling the modified commands (`ps`, `universe`, `pns`, etc.) from different thread contexts (native, Java, and non-java threads) to ensure they fail gracefully with an error message instead of crashing the debug session.
>
> src/hotspot/share/utilities/debug.cpp line 326:
> 
>> 324:     }
>> 325: 
>> 326:     if (!_has_rm) {
> 
> Is it even possible for this not to be false with correct usage?

No, `onThread` would have to be called at least twice.

> src/hotspot/share/utilities/debug.cpp line 327:
> 
>> 325: 
>> 326:     if (!_has_rm) {
>> 327:       ::new (&_rm) ResourceMark();
> 
> There should be `#include <new>` to use global placement-new.

added, thanks.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/27033#discussion_r2315520918
PR Review Comment: https://git.openjdk.org/jdk/pull/27033#discussion_r2315521505