Unified Logging for network

[Kirk Pepperdine]

Hi Thomas,

I’m reading with interest.. and I’m thinking of submitting a small patch which I guess would need to be preceded with a JBS entry.

Kind regards,
Kirk


> On Nov 3, 2020, at 12:12 AM, Thomas Stüfe <thomas.stuefe at gmail.com> wrote:
> 
> Hi Kirk,
> 
> On Mon, Nov 2, 2020 at 8:10 PM Kirk Pepperdine <kirk at kodewerk.com> wrote:
> 
>> Hi Thomas,
>> 
>> I appreciate Yasumasa’s desire to be able to redirect UL output to
>> somewhere other than… I also appreciate that the highly granular nature of
>> how UL messages are currently structure can be and indeed are an issue.
>> That said, I’d also like the ability to push the data to some where other
>> than a file on disk.
>> 
>> To the point of granularity, UL might benefit from some message
>> coarsening. This might also help in with other logging related performance
>> issues that I’ve noted here and there. Quite frankly dealing with logs in
>> containers isn’t a wonderful experience. And while I firmly believe that
>> there is more that containers can do to ease this, being able to redirect
>> output to something other than a log file does feel like it would be
>> helpful. That said, I’m also concerned about the potential performance
>> impacts but I think for this things that one would generally log, this
>> should be minimal.
>> 
> 
> About this proposal, see my answer to Yasumasa.
> 
> About UL granularity: feel free to raise concrete examples in JBS or on the
> mailing lists! This is often not that difficult to fix.
> 
> Kind Regards, Thomas
> 
> 
>> 
>> Kind regards,
>> Kirk Pepperdine
>> 
>> 
>>> On Nov 2, 2020, at 4:26 AM, Thomas Stüfe <thomas.stuefe at gmail.com>
>> wrote:
>>> 
>>> Hi Yasumasa,
>>> 
>>> one problem I see is that this could introduce a surprising amount of lag
>>> into log() calls which do look inconspicuous, thereby distorting timing
>>> behavior or even create timeout effects. We already have that problem now
>>> to some degree when logging to network shares.
>>> 
>>> Another thing, log output can be very fine granular, which would create a
>>> lot of network traffic.
>>> 
>>> Such an addition may also open some security questions.
>>> 
>>> From a more philosophical standpoint, I like the "do one thing and do it
>>> right" Unix way and this seems more like something an outside tool should
>>> be doing. Which could also aggregate log output better. But I admit that
>>> argument is weak.
>>> 
>>> Cheers, Thomas
>>> 
>>> 
>>> 
>>> On Mon, Nov 2, 2020 at 12:21 PM Yasumasa Suenaga <
>> suenaga at oss.nttdata.com>
>>> wrote:
>>> 
>>>> Hi all,
>>>> 
>>>> We need to out UL to stdout and/or file. If we can out it to TCP
>> socket, I
>>>> think it is useful.
>>>> 
>>>> For example, some system gather all logs to document oriented databases
>>>> (e.g. Elasticsearch) and/or cloud monitoring platform (e.g.
>> CloudWatch). If
>>>> HotSpot can out UL to TCP socket, we can send all logs to them via TCP
>>>> input plugin (Fluentd, Logstash).
>>>> 
>>>> I think it is useful for container platform. What do you think?
>>>> If it is worth to work, I will add CSR and JBS ticket, and also will
>>>> create patch.
>>>> 
>> 
>>