A bug in C2 that causes a large amount of physical memory to be allocated

[nijiaben]

Hi, All

Recently, I‘m troubleshooting a JVM problem (JDK1.8.0_191-b12), and found that the process is always killed by the OS, which is caused by memory leaks. Finally, it was discovered that OOM is caused by a large amount of memory allocated by C2 thread. This is a bug in C2. The following is the troubleshooting process:

First, through /proc//smaps, I saw a lot of 64MB of memory allocation, and RSS is basically exhausted.
7fd690000000-7fd693f23000 rw-p 00000000 00:00 0  Size:              64652 kB Rss:               64652 kB Pss:               64652 kB Shared_Clean:          0 kB Shared_Dirty:          0 kB Private_Clean:         0 kB Private_Dirty:     64652 kB Referenced:        64652 kB Anonymous:         64652 kB AnonHugePages:         0 kB Swap:                  0 kB KernelPageSize:        4 kB MMUPageSize:           4 kB Locked:                0 kB VmFlags: rd wr mr mw me nr sd  7fd693f23000-7fd694000000 ---p 00000000 00:00 0  Size:                884 kB Rss:                   0 kB Pss:                   0 kB Shared_Clean:          0 kB Shared_Dirty:          0 kB Private_Clean:         0 kB Private_Dirty:         0 kB Referenced:            0 kB Anonymous:             0 kB AnonHugePages:         0 kB Swap:                  0 kB KernelPageSize:        4 kB MMUPageSize:           4 kB Locked:                0 kB VmFlags: mr mw me nr sd 

Then trace the system call through the strace command, combined with the above virtual address, we found the corresponding mmap system call[pid 71] 13:34:41.982589 mmap(0x7fd690000000, 67108864, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0x7fd690000000 <0.000107>The thread that executes mmap is the 71 thread, so the thread is dumped through jstack, and the corresponding thread is found to be C2 CompilerThread0.
"C2 CompilerThread0" #39 daemon prio=9 os_prio=0 tid=0x00007fd8acebb000 nid=0x47 runnable [0x0000000000000000]    java.lang.Thread.State: RUNNABLE

Then grep the output of strace, see a lot of memory were allocated by this thread, probably more than 2G.

C2 is memory-limited under normal circumstances, but why is there a memory consumption greater than 2G? Finally, we found that a bug in the JVM will cause memory leaks. The location of this code is the nmethod::metadata_do method of nmethod.cpp:
void nmethod::metadata_do(void f(Metadata*)) {   address low_boundary = verified_entry_point();   if (is_not_entrant()) {     low_boundary += NativeJump::instruction_size;     // %%% Note:  On SPARC we patch only a 4-byte trap, not a full NativeJump.     // (See comment above.)   }   {     // Visit all immediate references that are embedded in the instruction stream.     RelocIterator iter(this, low_boundary);     while (iter.next()) {       if (iter.type() == relocInfo::metadata_type ) {         metadata_Relocation* r = iter.metadata_reloc();         // In this metadata, we must only follow those metadatas directly embedded in         // the code.  Other metadatas (oop_index>0) are seen as part of         // the metadata section below.         assert(1 == (r->metadata_is_immediate()) +                (r->metadata_addr() >= metadata_begin() && r->metadata_addr() < metadata_end()),                “metadata must be found in exactly one place”);         if (r->metadata_is_immediate() && r->metadata_value() != NULL) {           Metadata* md = r->metadata_value();           if (md != _method) f(md);         }       } else if (iter.type() == relocInfo::virtual_call_type) {         // Check compiledIC holders associated with this nmethod         CompiledIC *ic = CompiledIC_at(&iter);         if (ic->is_icholder_call()) {           CompiledICHolder* cichk = ic->cached_icholder();           f(cichk->holder_metadata());           f(cichk->holder_klass());         } else {           Metadata* ic_oop = ic->cached_metadata();           if (ic_oop != NULL) {             f(ic_oop);           }         }       }     }   }

Because CompiledIC is a ResourceObj, but it is not freed by ResourceMark, so when this method is called multiple times, OOM will appear.

The fix is very simple, just add ResourceMark rm; before CompiledIC *ic = CompiledIC_at(&iter);

And I found out that it has been solved on JDK12.

http://hg.openjdk.java.net/jdk-updates/jdk12u/rev/aa3bfacc912c

Can this patch be entered into JDK8?




Thanks,

nijiaben @ PerfMa