Unified Logging for network

[Thomas Stüfe]

Hi Yasumasa,

similar solutions - buffering UL output and draining it in a separate
thread (regardless the sink) - have been proposed in the past, see e.g.
Amazon's proposal here:
https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-January/037849.html.
In their case, to make gc logging less jittery.

I think the strategy to stop all loggers if the buffer is full does not
really solve anything. A laggy network connection can still screw up the
timing behavior of the VM. Therefore, optionally one should be able to
specify that log output gets discarded rather than loggers be stopped, and
that should be marked in the output of course. Also, maybe the log buffer
should be able to resize itself, and shrink once a log spike is over. And
so on.

Then, we have the problem of synchronization. Ideally, such a log buffer
would be implemented in a lock less fashion. Otherwise log calls cause
synchronization over all threads.

So if done right, this gets a bit complex. And I think it is worth doing
right. Therefore I think, like John and Robbin, that before reinventing
this wheel we should check if there are standard tools which do all this
already.

The fact that this discussion springs up again and again means that there
is a need for a solution. Let's hear what the others think.

Cheers, Thomas

Thanks, Thomas


On Sat, Nov 7, 2020 at 2:26 PM Yasumasa Suenaga <suenaga at oss.nttdata.com>
wrote:

> Hi all,
>
> Thank you for sharing various opinions!
> The most of concerns seems TCP socket handling in UL.
>
> So I described my opinion on Gist. Could you read it?
>
>    https://gist.github.com/YaSuenag/dacb6d94d8684915422232c7a08d5b5d
>
>
> Thanks,
>
> Yasumasa
>
>
> On 2020/11/02 20:20, Yasumasa Suenaga wrote:
> > Hi all,
> >
> > We need to out UL to stdout and/or file. If we can out it to TCP socket,
> I think it is useful.
> >
> > For example, some system gather all logs to document oriented databases
> (e.g. Elasticsearch) and/or cloud monitoring platform (e.g. CloudWatch). If
> HotSpot can out UL to TCP socket, we can send all logs to them via TCP
> input plugin (Fluentd, Logstash).
> >
> > I think it is useful for container platform. What do you think?
> > If it is worth to work, I will add CSR and JBS ticket, and also will
> create patch.
>