Feature ITW logging tasks implementation

[Andrew Azores]

On 11/01/2013 10:39 AM, Jiri Vanek wrote:
> Hi all, before to much code is written. I would like to ask you to 
> opinions about suggested approaches:
>
> *Logging to file:*
> Right now, i'm implementing  the logging of C part to file. Th 
> implementation itelf is pretty simple. but from higher point, there 
> are two crucial differences with several different approaches:
>  - few notes notes
>    - file logs are saved at 
> {~/.config,$XDG_CONFIG_DIR}/icedtea-web/log without possibility to 
> change this location... I think this is correct and I'm not going to 
> change it (just going to higlight in ITW settings)
>    - fielname is itw-numberOfMilissecondsSinceEpoch.log - Unless 
> something against, I would like to change it to better name (date, 
> time - sure - but in readable form)
>    - java.stderr, java.stdout are no longer used
>
> logging to file(s) approaches:
>  - C side a and java side are each logging into own file in same 
> directory with as similar name as possible, distinguished via some 
> suffix PLUGIN/NETX
>    - most easy to implement
>    - different logs for C part and Java side (can be both positive and 
> negative)
>    - to have the name really same, name must be passed to java part 
> similarly as next approach
>  - C and java sides are sharing the file and each is writing on its own
>    - also easy to implement, and surprisingly, even most dummy 
> approaches [1] [2] (flushes are important) have  very good results 
> (eg. messages are not mixed - Ihave palyed with similar approaches, 
> process handling quite a lot))
>    - name is passed to java side via parameter during lunching of jvm 
> (as names of pipes are)
>    - same log for C part and Java side (can be both positive and 
> negative)
>  - Combination of above. One shared file, and one file for each side. 
> Probably the best, and now right under construction
>  - C and java sides are sharing the file and only one part is 
> responsible for writing.
>   - this is probably just wrong idea, but there is no danger of 
> conflict during writing into
>     - C part is sending the messages via pipe (new or current one), 
> and java side is logging them
>       - advantage is usage of multithreaded logging on java side
>     - Java part is sending the messages via pipe (new or current one), 
> and C side is logging them
>       - advantage can be in case of syslog(-ng) sytstem logger. The C 
> api is much cleaner

For the "combination" option, does this mean that the combined logfile 
will just contain the same information as in the other two separate 
logfiles, but with the log messages interleaved? This sounds good on one 
hand because it means some issues will be very easy to trace as data 
flows from one side to the other, but on the other hand it seems like 
this also leads to the logfiles' total size being 2x what it would be if 
we didn't have the combined file. What about keeping two separate files, 
so that debugging is easy and clean for issues that only really affect 
one side, but with timestamps on each message so if you are debugging an 
issue that needs both, you can still easily trace the data flow between 
them? This would not be as easy to read through as the combined file 
however, and maybe the extra space taken up by the combined logfile is 
worth it. Anyway I guess I'm saying I like options 1 and 3 here.

> *System logging:*
> To be honest I got into dead end. Both me and pavel have tried several 
> approaches, (abrt, syslog...) but none fit.
> - I would like to log into syslog by default always and everything, 
> and so if again somebody came and complain "it donot work", I will 
> just tell him "look into well_knonw_location".
> - Also for system admin should be logs of all users on some place, so 
> he can look into logs on "well known location"
>    - so logging should not  delay ITW run
>    - multiline loggs must be allowed
>    - logs should be automatically rolled
>    - location should be something like 
> /var/log/icedtea-web/$USER/rolled_logs where directory user should be 
> 700 and fiels inside 600, owned by user
> As from above, only classical syslog was close enough to fill the 
> requirements, but he did not fill the last condition, nor multiline 
> messages. Also the disadvantage of configured logging is.. well 
> annoying, but survivable.
> At the end I finished with syslog-ng. It looks like it is he is 
> fulfilling all above, however it added  other:
>   - it is still to new, nearly none system have it by default. It will 
> never go into rhel<=7 .. not sure even about 8 :(
>   - it is not widely , nor (haha) by default used/known

I agree with all of the design decisions here but I don't know enough 
about system logging implementations to be able to help with choosing 
one or finding an alternative to syslog-ng. If it won't be in RHEL any 
time too soon, and if most other systems won't already have it 
installed, do we really want to make this system logger a new dependency 
for ITW? Even an optional one? Maybe in the future when it's more 
widespread. But again I'm not sure what to use in its place otherwise. 
It seems better to settle on something we can depend on being available 
on our end users' systems right now, even if that means it won't be as 
elegant as syslog-ng.

> - Distors with packages can have syslog-ng as requirement, can start 
> its daemon in post-install and can modify the syslog-ng properties 
> file. (the config to reach above is not to simple). Not sure how to in 
> non-packaging distros.

In non-packaging distros, users would just need to set it up manually, I 
suppose. I don't think that's very ideal. We would then need to at least 
include some readme on how to configure this properly, and I'm sure most 
users would never bother with it at all anyway. Perhaps these users are 
a minority compared to those on packaging distros but I don't think we 
should make something as basic as logging be difficult to set up for 
them. I don't think it would be feasible for us to maintain a 
"setup-logging.sh" ourselves for this purpose either.

> *Logging settings"*
> Last item I would like to bring up before first line of code, is setting.
> Right now the settings are
>  - global
>   - debugging on/off  checkbook in itw-settings( or 
> ICEDTEAPLUGIN_DEBUG or -verbose for javaws)
>   - headers on/of itw settings checkbox
> - which are affecting all  underlying outputs:
>   - streams
>   - file (only java implementation right now)
>   - system (nothing implemented)
>
> I'm thinking about individual settings (verbose and headers) of each 
> stream.
> In ideal world I would like to see:
>   - streams - debug and headers off
>   - file  - debug and headers off, and headers off when debug on
>   - system (nothing implemented) -  debug and headers on
>
> With possibility to switch each separately to on/of for debug, and 
> on/of for headers (so change for checkbox to commbobox in 
> itw-settings, but I have no idea how to deal with -verbose and 
> ICEDTEA_PLUGIN for this granular settings)

Maybe those two options/switches should just add the minimal level of 
extra debug output. If that's still not enough then more verbosity can 
be enabled in settings. Or we can use multiple -verbose switches (and 
maybe add new short -v flag alias) to increase the amount of verbosity? 
One -v for some extra debug output, -vvv for maximum verbosity, 
something like that. I like the idea of having more granular control 
over what gets logged though.

Thanks,

-- 
Andrew A