RFR: 8234930: Use MAP_JIT when allocating pages for code cache on macOS [v6]
Anton Kozlov
akozlov at openjdk.java.net
Sat Dec 5 10:56:13 UTC 2020
On Sat, 5 Dec 2020 05:34:57 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:
> I found this: https://stackoverflow.com/questions/7718964/how-can-i-force-macos-to-release-madv-freed-pages. One remark recommends MADV_FREE_REUSABLE to deal with the display problem; could that be a solution
I'd found MADV_FREE_REUSABLE as well. One problem is that it's barely documented. The only description from the vendor I could find was
#define MADV_FREE 5 /* pages unneeded, discard contents */
#define MADV_ZERO_WIRED_PAGES 6 /* zero the wired pages that have not been unwired before the entry is deleted */
#define MADV_FREE_REUSABLE 7 /* pages can be reused (by anyone) */
#define MADV_FREE_REUSE 8 /* caller wants to reuse those pages */
The other problem, it cannot substitute mmap completely, see below.
> My only remaining question is: is there really an observable difference between replacing the mapping with mmap and calling madvice(MADV_FREE)? And if there is, does it matter in practice?
Yes, it is. For a sample program after uncommit implemented by different ways, mmap the only way to reduce occupied memory size in Activity Monitor (system GUI application user will likely look to).
* no uncommit
./test noop
do not uncommit
Physical footprint: 512.3M
Physical footprint (peak): 512.3M
VM_ALLOCATE 109951000-111951000 [128.0M 128.0M 128.0M 0K] rw-/rwx SM=COW
VM_ALLOCATE 111951000-119951000 [128.0M 128.0M 128.0M 0K] rw-/rwx SM=COW
VM_ALLOCATE 119951000-121951000 [128.0M 128.0M 128.0M 0K] rw-/rwx SM=COW
VM_ALLOCATE 121951000-129951000 [128.0M 128.0M 128.0M 0K] rw-/rwx SM=COW
VIRTUAL RESIDENT DIRTY SWAPPED VOLATILE NONVOL EMPTY REGION
REGION TYPE SIZE SIZE SIZE SIZE SIZE SIZE SIZE COUNT (non-coalesced)
=========== ======= ======== ===== ======= ======== ====== ===== =======
VM_ALLOCATE 512.0M 512.0M 512.0M 0K 0K 0K 0K 4
* MADV_FREE reduces Dirty size, but does not affect Rss and Physical footprint
./test madv_free
madvise
Physical footprint: 512.3M
Physical footprint (peak): 512.3M
VM_ALLOCATE 108269000-110269000 [128.0M 128.0M 0K 0K] rw-/rwx SM=COW
VM_ALLOCATE 110269000-118269000 [128.0M 128.0M 0K 0K] rw-/rwx SM=COW
VM_ALLOCATE 118269000-120269000 [128.0M 128.0M 0K 0K] rw-/rwx SM=COW
VM_ALLOCATE 120269000-128269000 [128.0M 128.0M 0K 0K] rw-/rwx SM=COW
VIRTUAL RESIDENT DIRTY SWAPPED VOLATILE NONVOL EMPTY REGION
REGION TYPE SIZE SIZE SIZE SIZE SIZE SIZE SIZE COUNT (non-coalesced)
=========== ======= ======== ===== ======= ======== ====== ===== =======
VM_ALLOCATE 512.0M 512.0M 0K 0K 0K 0K 0K 4
* MADV_FREE_REUSABLE reduces Physical footprint (whatever it is)
./test madv_free_reuse
madvise reuse
Physical footprint: 292K
Physical footprint (peak): 512.3M
VM_ALLOCATE 10d568000-115568000 [128.0M 128.0M 0K 0K] rw-/rwx SM=COW
VM_ALLOCATE 115568000-11d568000 [128.0M 128.0M 0K 0K] rw-/rwx SM=COW
VM_ALLOCATE 11d568000-125568000 [128.0M 128.0M 0K 0K] rw-/rwx SM=COW
VM_ALLOCATE 125568000-12d568000 [128.0M 128.0M 0K 0K] rw-/rwx SM=COW
VIRTUAL RESIDENT DIRTY SWAPPED VOLATILE NONVOL EMPTY REGION
REGION TYPE SIZE SIZE SIZE SIZE SIZE SIZE SIZE COUNT (non-coalesced)
=========== ======= ======== ===== ======= ======== ====== ===== =======
VM_ALLOCATE 512.0M 512.0M 0K 0K 0K 0K 0K 4
There is a problem that Activity Monitor apparently looks for Rss and reports no change:
* and mmap reduces Rss as well
./test mmap
new mmap
Physical footprint: 292K
Physical footprint (peak): 512.3M
VM_ALLOCATE 10df01000-12df01000 [512.0M 0K 0K 0K] ---/rwx SM=NUL
VIRTUAL RESIDENT DIRTY SWAPPED VOLATILE NONVOL EMPTY REGION
REGION TYPE SIZE SIZE SIZE SIZE SIZE SIZE SIZE COUNT (non-coalesced)
=========== ======= ======== ===== ======= ======== ====== ===== =======
VM_ALLOCATE 512.0M 0K 0K 0K 0K 0K 0K
Activity Monitor (results are OK now)
So, when possible, we should do new mmap for uncommit.
---
Source code for the test:
#include <unistd.h>
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
#include <sys/mman.h>
int main(int argc, char *argv[]) {
char r;
int pagesize = 4096;
int size = 512 * 1024 * 1024;
char *a = mmap(NULL, size,
PROT_NONE,
MAP_ANON | MAP_PRIVATE, -1, 0);
if (a == MAP_FAILED) {
perror("mmap()");
return 1;
}
if (mprotect(a, size, PROT_READ | PROT_WRITE)) {
perror("mprotect");
return 1;
}
for (int i = 0; i < size; i += 4096) {
a[i] = 1;
}
if (!strcmp(argv[1], "madv_free")) {
printf("madvise\n");
if (mprotect(a, PROT_NONE, size)) {
perror("mprotect");
return 1;
}
if (madvise(a, size, MADV_FREE)) {
perror("madvise");
return 1;
}
} else if (!strcmp(argv[1], "madv_free_reuse")) {
printf("madvise reuse\n");
if (mprotect(a, PROT_NONE, size)) {
perror("mprotect");
return 1;
}
if (madvise(a, size, MADV_FREE_REUSABLE)) {
perror("madvise");
return 1;
}
} else if (!strcmp(argv[1], "mmap")) {
printf("new mmap\n");
if (MAP_FAILED == mmap(a, size, PROT_NONE, MAP_ANON | MAP_PRIVATE| MAP_FIXED, -1, 0)) {
perror("mmap2");
return 1;
}
} else {
printf("do not uncommit\n");
}
fflush(stdout);
char cmd[128];
snprintf(cmd, sizeof(cmd), "vmmap %d | awk '/Phys/ || /VM_ALLOCATE/'", getpid());
system(cmd);
read(0, &r, 1);
return 0;
}
-------------
PR: https://git.openjdk.java.net/jdk/pull/294
More information about the hotspot-dev
mailing list