Unified Logging for network

[Kirk Pepperdine]

Hi Thomas,

I appreciate Yasumasa’s desire to be able to redirect UL output to somewhere other than… I also appreciate that the highly granular nature of how UL messages are currently structure can be and indeed are an issue. That said, I’d also like the ability to push the data to some where other than a file on disk.

To the point of granularity, UL might benefit from some message coarsening. This might also help in with other logging related performance issues that I’ve noted here and there. Quite frankly dealing with logs in containers isn’t a wonderful experience. And while I firmly believe that there is more that containers can do to ease this, being able to redirect output to something other than a log file does feel like it would be helpful. That said, I’m also concerned about the potential performance impacts but I think for this things that one would generally log, this should be minimal.

Kind regards,
Kirk Pepperdine


> On Nov 2, 2020, at 4:26 AM, Thomas Stüfe <thomas.stuefe at gmail.com> wrote:
> 
> Hi Yasumasa,
> 
> one problem I see is that this could introduce a surprising amount of lag
> into log() calls which do look inconspicuous, thereby distorting timing
> behavior or even create timeout effects. We already have that problem now
> to some degree when logging to network shares.
> 
> Another thing, log output can be very fine granular, which would create a
> lot of network traffic.
> 
> Such an addition may also open some security questions.
> 
> From a more philosophical standpoint, I like the "do one thing and do it
> right" Unix way and this seems more like something an outside tool should
> be doing. Which could also aggregate log output better. But I admit that
> argument is weak.
> 
> Cheers, Thomas
> 
> 
> 
> On Mon, Nov 2, 2020 at 12:21 PM Yasumasa Suenaga <suenaga at oss.nttdata.com>
> wrote:
> 
>> Hi all,
>> 
>> We need to out UL to stdout and/or file. If we can out it to TCP socket, I
>> think it is useful.
>> 
>> For example, some system gather all logs to document oriented databases
>> (e.g. Elasticsearch) and/or cloud monitoring platform (e.g. CloudWatch). If
>> HotSpot can out UL to TCP socket, we can send all logs to them via TCP
>> input plugin (Fluentd, Logstash).
>> 
>> I think it is useful for container platform. What do you think?
>> If it is worth to work, I will add CSR and JBS ticket, and also will
>> create patch.
>>