RFR: 8319875: Add macOS implementation for jcmd System.map [v2]
Thomas Stuefe
stuefe at openjdk.org
Thu Dec 5 14:35:43 UTC 2024
On Mon, 2 Dec 2024 14:52:38 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:
>> Hello, @tstuefe , and thanks for your comments. I'll address a few here while I work on the others.
>> I have changed the os-specific names to lowercase, but I don't think it makes them stand out more. The square brackets were intended to do that. Might I change this back?
>>
>> I think there is only one JAVAHEAP segment because due to an issue with my build[1] there was no CDS archive available.
>>
>> I will look at the META and CLASS entries and see if there are hidden properties that I can surface, or if there's another reason for so many entries.
>>
>> [1] I had given a target architecture to the configure command, which turned on cross-compiles (which disables CDS archive building) even when building on the target platform.
>
> Hi @stooke !
>
>> Hello, @tstuefe , and thanks for your comments. I'll address a few here while I work on the others. I have changed the os-specific names to lowercase, but I don't think it makes them stand out more. The square brackets were intended to do that. Might I change this back?
>
> Sure, if it looks worse. I just wanted to make sure we can cleanly distinguish NMT sections from OS sections.
>
>>
>> I think there is only one JAVAHEAP segment because due to an issue with my build[1] there was no CDS archive available.
>
> Has nothing to do with CDS. The heap consists of committed and reserved areas. Committed areas have backing swap space allocated for them, and are accessible. Reserved areas have not and are generally not. API wise the difference is that Reserved sections set the MAP_NORESERVE flag for mmap, and are generally allocated with PROT_NONE.
>
> So, the heap should show up with several neighboring sections, some committed, some just reserved. Similar how most of the stacks should show up with two entries, one for the writable stack, one for the guard page that is protected.
>
> ---
>
>
> Simple test I did on MacOS with your patch: I reserve 1G of memory at startup, uncommitted (added to os::init_2)
>
>
> if (UseNewCode) {
> char* p = os::reserve_memory(G, false, mtInternal);
> tty->print_cr("Pointer is %p", p);
> }
>
>
>
> vmmap shows:
>
>
> VM_ALLOCATE 10ccb4000-14ccb4000 [ 1.0G 0K 0K 0K] ---/rwx SM=NUL
>
>
> so, looks good. 1GB, with all protection flags cleared. But System.map shows nothing for this address range.
>
>
> Now, I commit the second half of the range:
>
>
>
> if (UseNewCode) {
> char* p = os::reserve_memory(G, false, mtInternal);
> tty->print_cr("Pointer is %p", p);
> bool b = os::commit_memory(p + (512 * M), 512 * M, false);
> assert(b,"???");
> }
>
>
> vmmap shows only the committed part now, omitting the still uncommitted first half. But it gets the protection flags right again (rw now):
>
>
> VM_ALLOCATE (reserved) 148000000-168000000 [512.0M 0K 0K 0K] rw-/rwx SM=NUL reserved VM address space (unallocated)
>
>
> System.map shows nothing.
>
>
> What goes on? Is the OS lying to us? Do we have an error? Both vmmap and System.map seem to struggle, with vmmap being somewhat more correct.
> @tstuefe I've look into your test, and I will modify the PR to display these regions - it was incorrectly identifying them as "free". As to the strange vmmap behaviour, I found that the two sections appeared in different places: the uncommitted spaces appeared in "==== Non-writable regions for process": `VM_ALLOCATE 300000000-320000000 [512.0M 0K 0K 0K] ---/rwx SM=NUL ` and the committed spaces in "==== Writable regions for process": `VM_ALLOCATE (reserved) 320000000-340000000 [512.0M 0K 0K 0K] rw-/rwx SM=NUL reserved VM address space (unallocated) ` I have made a few changes, track reserved and committed memory better, and uploaded an updated sample output. [vm_memory_map_89174.txt](https://github.com/user-attachments/files/18013640/vm_memory_map_89174.txt)
Yes, this is better.
Metaspace sections look like this:
0x000130000000-0x000130010000 65536 rw-/rwx pvt 0 META
0x000130010000-0x000130020000 65536 rw-/rwx pvt 0 META
0x000130020000-0x000130400000 4063232 ---/rwx --- 0x20000 META
0x000130400000-0x000130410000 65536 rw-/rwx pvt 0 META
0x000130410000-0x000134000000 62849024 ---/rwx --- 0x410000 META
A single 64MB space node. First three entries together are the initial 4MB chunk the boot class loader uses. Forth line, together with some space from the fifth line will belong to the next chunk of the next class loader.
Class space is still a bit weird:
0x018001000000-0x018001010000 65536 rw-/rwx pvt 0 CLASS
0x018001010000-0x018001040000 196608 ---/rwx --- 0x1010000 CLASS
0x018001040000-0x018001050000 65536 rw-/rwx pvt 0 CLASS
0x018001050000-0x018008000000 117112832 ---/rwx --- 0x1050000 CLASS
0x018008000000-0x018010000000 134217728 ---/rwx --- 0 CLASS
0x018010000000-0x018018000000 134217728 ---/rwx --- 0 CLASS
0x018018000000-0x018020000000 134217728 ---/rwx --- 0 CLASS
0x018020000000-0x018028000000 134217728 ---/rwx --- 0 CLASS
0x018028000000-0x018030000000 134217728 ---/rwx --- 0 CLASS
0x018030000000-0x018038000000 134217728 ---/rwx --- 0 CLASS
0x018038000000-0x018040000000 134217728 ---/rwx --- 0 CLASS
0x018040000000-0x018041000000 16777216 ---/rwx --- 0 CLASS
Class space by default is 1GB, and that figures (all lines together), and the first two lines are the initial bootloader chunk of 256k. But following that, I would have expected a single line for the remaining uncommitted space, but it seems to report segments at a 128MB-aligned boundary.
I would have expected something like:
0x018001000000-0x018001010000 65536 rw-/rwx pvt 0 CLASS
0x018001010000-0x018001040000 196608 ---/rwx --- 0x1010000 CLASS
0x018001040000-0x018001050000 65536 rw-/rwx pvt 0 CLASS
0x018001050000-0x018041000000 1073414144 ---/rwx --- 0x1050000 CLASS
Same can be seen with the JAVAHEAP:
0x000700000000-0x000708000000 134217728 rw-/rwx --- 0 JAVAHEAP
0x000708000000-0x000710000000 134217728 rw-/rwx pvt 0 JAVAHEAP
0x000710000000-0x000718000000 134217728 ---/rwx --- 0 JAVAHEAP
0x000718000000-0x000720000000 134217728 ---/rwx --- 0 JAVAHEAP
0x000720000000-0x000728000000 134217728 ---/rwx --- 0 JAVAHEAP
0x000728000000-0x000730000000 134217728 ---/rwx --- 0 JAVAHEAP
0x000730000000-0x000738000000 134217728 ---/rwx --- 0 JAVAHEAP
0x000738000000-0x000740000000 134217728 ---/rwx --- 0 JAVAHEAP
...
I first wondered whether we somehow cause this 128MB segmentation thing, but I don't think this is something we do. It is also appearant in malloc segments:
0x000148000000-0x000150000000 134217728 rw-/rwx pvt 0 [malloc_medium]
0x000150000000-0x000158000000 134217728 rw-/rwx pvt 0 [malloc_medium]
0x000158000000-0x000160000000 134217728 rw-/rwx pvt 0 [malloc_medium]
0x000160000000-0x000168000000 134217728 rw-/rwx pvt 0 [malloc_medium]
Do you understand this 128MB segmentation? Would it be possible to fold these into a single line, as we do on Linux? (On Linux, the kernel does it).
Thank you!
-------------
PR Comment: https://git.openjdk.org/jdk/pull/20953#issuecomment-2520488961
More information about the hotspot-runtime-dev
mailing list