What gc.log lines reliably indicate STW pauses for ZGC?

[Leslie Zhai]

> 2023年9月8日 04:45，Misha Dmitriev <mdmitriev at linkedin.com> 写道：
> 
> Hi Leslie,
> 
> Thank you for your response. I am not sure I understand it, though. GCViewer is not a small tool, which extracts all kinds of information from the GC log. However, our goal for now is very limited: we just need to recognize and parse log lines that indicate STW GC pauses, that's it. We cannot reuse GCViewer code. On the other hand, I was hoping that I could get a more definitive/accurate answer directly from ZGC developers, rather than by reading code of a 3rd party tool. 
> 
> May I ask you to answer at least my second question: what's the meaning of lines like the one below?
> 
> [2023-09-05T18:47:44.003+0000][505898.295s][debug  ][gc,phases        ] GC(76661) Pause Mark Try Complete (ZWorker#2) 0.002ms

It is the Concurrent Mark Subphase, which doesn’t need to STW at all.

Thanks,
Leslie Zhai

> 
> Thank you,
> 
> Misha
> From: Leslie Zhai <zhaixiang at loongson.cn>
> Sent: Wednesday, September 6, 2023 5:35 PM
> To: Misha Dmitriev <mdmitriev at linkedin.com>
> Cc: zgc-dev at openjdk.org <zgc-dev at openjdk.org>
> Subject: Re: What gc.log lines reliably indicate STW pauses for ZGC?
>  Hi Misha,
> 
> Could you try GCViewer?  https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fchewiebug%2FGCViewer&data=05%7C01%7Cmdmitriev%40linkedin.com%7Ce07cc8ab5300462417f408dbaf3a598d%7C72f988bf86f141af91ab2d7cd011db47%7C0%7C0%7C638296437390082029%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=YjtYLdtX%2FyjxNld0k7LcHZVe1YlS2IoDOedJv3JzHsQ%3D&reserved=0
> 
> enhance support for zgc and shenandoah https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fchewiebug%2FGCViewer%2Fpull%2F261&data=05%7C01%7Cmdmitriev%40linkedin.com%7Ce07cc8ab5300462417f408dbaf3a598d%7C72f988bf86f141af91ab2d7cd011db47%7C0%7C0%7C638296437390082029%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=YvfbyGgBl9s5fZ4aNqCkget1AXKGIJr0mYnhSC%2FAAXc%3D&reserved=0
> 
> 
> > 2023年9月7日 03:15，Misha Dmitriev <mdmitriev at linkedin.com> 写道：
> > 
> > Hi ZGC developers,
> > 
> > I work at LinkedIn, where we have internal tooling for end-to-end GC performance monitoring and analysis. To record GC pauses and other interesting events, we parse GC logs as they are generated, in near real time. Since JDK 11, the unified logging format allowed us to extract STW pauses for all GCs in the same straightforward way: we look for  lines such as:
> > 
> > [2020-03-09T19:12:22.542+0000][info][gc             ] GC(62189) Pause Young (Allocation Failure) 14263M->9334M(14745M) 41.359ms
> > 
> > that is, we check every line for the following condition (we don't use pattern matching for speed):
> > 
> > if (line.endsWith("ms") && line.contains("[gc ") && line.contains("Pause ")) {
> >   // Extract pause time in ms from the end of the line
> > 
> > However, looks like ZGC, though it uses the unified logging format, has a different format for STW pause lines. From the logs that we have, I am not 100% sure I understand everything. So far looks like the candidates lines look like
> 
> Please look at the enhance support for ZGC about GCViewer.
> 
> Thanks,
> Leslie Zhai
> 
> > 
> > [2023-09-05T18:47:27.323+0000][505881.615s][info   ][gc,phases        ] GC(76657) Pause Mark Start 0.024ms
> > 
> > i.e. the condition should be
> > 
> > if (line.endsWith("ms") && line.contains("[gc,phases ") && line.contains("Pause ")) {
> >   // Extract pause time in ms from the end of line
> > 
> > I wonder if this is all we need or I am missing something?
> > 
> > Also, what's the meaning of lines like these:
> > 
> > [2023-09-05T18:47:44.003+0000][505898.295s][debug  ][gc,phases        ] GC(76661) Pause Mark Try Complete (ZWorker#2) 0.002ms
> > 
> > Formally they look similar to the previous line, but "debug" and "ZWorker#2" suggests that it may not be a STW pause but pausing just one thread? Please advise.
> > 
> > Thank you,
> > 
> > Misha