gc logging basic question

[Tao Feng]

Just FYI, at linkedin we encountered this issue many times. And one of our
coworker recently wrote a blog posted about this issue:
https://engineering.linkedin.com/blog/2016/02/eliminating-large-jvm-gc-pauses-caused-by-background-io-traffic
.

Thanks,
-Tao

On Wed, Feb 10, 2016 at 1:04 PM, Srinivas Ramakrishna <ysr1729 at gmail.com>
wrote:

> We've run into similar issues at Twitter, and are readying a patch (re #
> (4) of Thomas' mail) for buffering in-memory
> and writing to disk/file out-of-line wrt safepoints or loggers. The patch
> is under test and review internally at this time,
> and we hope to publish it for review on Open JDK once internal test and
> review are complete.
>
> Thomas, does unified logging include GC and will it be done outside of
> safepoints? In that case, we can use the same
> code rather than spinning code specifically for gc logging (as I am doing
> in our patch for this). I must confess I haven't
> looked at the unified logging jep or its state in the JDK 9 repo.
>
> thanks.
> -- ramki
>
> On Wed, Feb 10, 2016 at 9:12 AM, Gustav Åkesson <
> gustav.r.akesson at gmail.com> wrote:
>
>> Hi,
>>
>> Well, AFAIK the loggning IS part of STW cycle. I had troubles with this
>> before, which results in GCs with high systime. Since then I always print
>> the logs to RAM disk, which solves the problem.
>>
>> Best Regards,
>> Gustav Åkesson
>> Den 9 feb 2016 18:46 skrev "Tao Feng" <fengtao04 at gmail.com>:
>>
>>> Hi Jenny, Thomas,
>>>
>>> Thanks a lot for your reply. Indeed we observe high real time in gc
>>> which we correlate with high write syscall time. This makes us doubt
>>> whether GC logging is part of cycle.
>>>
>>> Thanks,
>>> -Tao
>>>
>>> On Tue, Feb 9, 2016 at 2:11 AM, Thomas Schatzl <
>>> thomas.schatzl at oracle.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> On Mon, 2016-02-08 at 22:35 -0800, Yu Zhang wrote:
>>>> > Tao,
>>>> >
>>>> > GC logging is not part of GC STW pause. Though we may see application
>>>> > being stopped due to gc logging, if there are a lot of I/O activities
>>>> > on the system.
>>>> > The gc log file is opened by the following
>>>> > _fd = open(file_name, O_WRONLY | O_CREAT | O_TRUNC, 0666);
>>>> >
>>>> >  Thanks,
>>>> > Jenny
>>>> > On 2/4/2016 11:02 PM, Tao Feng wrote:
>>>> > > Hi,
>>>> > >
>>>> > > I have a basic question out of curiosity: is GC logging part of GC
>>>> > > cycle? If yes, is it a blocking IO or non-blocking IO? And it would
>>>> > > be good if someone could help point out the code base about the GC
>>>> > > logging part as I have difficult to identify the code location.
>>>>
>>>>   I think any logging enabled by -Xlog opens the file in asynchronous
>>>> write mode, output is buffered. There is no deliberate fsync'ing going
>>>> on (again, afaik).
>>>>
>>>> However, lower levels (OS, file system implementations) might impose
>>>> additional synchronization. If a different process saturates I/O to the
>>>> disk you want to write to at the same time, the behavior might not be
>>>> optimal. :)
>>>>
>>>> See e.g. http://mail.openjdk.java.net/pipermail/hotspot-gc-use/2014-Apr
>>>> il/001835.html
>>>> <http://mail.openjdk.java.net/pipermail/hotspot-gc-use/2014-April/001835.html>
>>>> .
>>>>
>>>> If you noticed a problem because of that, there are the following
>>>> options:
>>>>
>>>> 1) use a separate disk for the logging (throwing the "tmpfs" keyword
>>>> into this bucket)
>>>>
>>>> 2) use JFR. Afaik its events are written asynchronously using separate
>>>> threads. That might or might not be a problem if you saturate I/O
>>>> bandwidth of your target disk for a long time (go to option 1 in this
>>>> case)
>>>>
>>>> 3) experiment with different FS implementations that do less blocking.
>>>>
>>>> 4) provide a patch to improve this situation :-)
>>>>
>>>> Thanks,
>>>>   Thomas
>>>>
>>>>
>>>>
>>>
>>> _______________________________________________
>>> hotspot-gc-use mailing list
>>> hotspot-gc-use at openjdk.java.net
>>> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
>>>
>>>
>> _______________________________________________
>> hotspot-gc-use mailing list
>> hotspot-gc-use at openjdk.java.net
>> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/hotspot-gc-use/attachments/20160211/6b01c4ec/attachment.html>