Unified Logging for network

[Thomas Stüfe]

Hi Kirk,

On Mon, Nov 2, 2020 at 8:10 PM Kirk Pepperdine <kirk at kodewerk.com> wrote:

> Hi Thomas,
>
> I appreciate Yasumasa’s desire to be able to redirect UL output to
> somewhere other than… I also appreciate that the highly granular nature of
> how UL messages are currently structure can be and indeed are an issue.
> That said, I’d also like the ability to push the data to some where other
> than a file on disk.
>
> To the point of granularity, UL might benefit from some message
> coarsening. This might also help in with other logging related performance
> issues that I’ve noted here and there. Quite frankly dealing with logs in
> containers isn’t a wonderful experience. And while I firmly believe that
> there is more that containers can do to ease this, being able to redirect
> output to something other than a log file does feel like it would be
> helpful. That said, I’m also concerned about the potential performance
> impacts but I think for this things that one would generally log, this
> should be minimal.
>

About this proposal, see my answer to Yasumasa.

About UL granularity: feel free to raise concrete examples in JBS or on the
mailing lists! This is often not that difficult to fix.

Kind Regards, Thomas


>
> Kind regards,
> Kirk Pepperdine
>
>
> > On Nov 2, 2020, at 4:26 AM, Thomas Stüfe <thomas.stuefe at gmail.com>
> wrote:
> >
> > Hi Yasumasa,
> >
> > one problem I see is that this could introduce a surprising amount of lag
> > into log() calls which do look inconspicuous, thereby distorting timing
> > behavior or even create timeout effects. We already have that problem now
> > to some degree when logging to network shares.
> >
> > Another thing, log output can be very fine granular, which would create a
> > lot of network traffic.
> >
> > Such an addition may also open some security questions.
> >
> > From a more philosophical standpoint, I like the "do one thing and do it
> > right" Unix way and this seems more like something an outside tool should
> > be doing. Which could also aggregate log output better. But I admit that
> > argument is weak.
> >
> > Cheers, Thomas
> >
> >
> >
> > On Mon, Nov 2, 2020 at 12:21 PM Yasumasa Suenaga <
> suenaga at oss.nttdata.com>
> > wrote:
> >
> >> Hi all,
> >>
> >> We need to out UL to stdout and/or file. If we can out it to TCP
> socket, I
> >> think it is useful.
> >>
> >> For example, some system gather all logs to document oriented databases
> >> (e.g. Elasticsearch) and/or cloud monitoring platform (e.g.
> CloudWatch). If
> >> HotSpot can out UL to TCP socket, we can send all logs to them via TCP
> >> input plugin (Fluentd, Logstash).
> >>
> >> I think it is useful for container platform. What do you think?
> >> If it is worth to work, I will add CSR and JBS ticket, and also will
> >> create patch.
> >>
>
>