RFR: 1063: Lower log severity on HTTP errors

Wed Jun 2 23:23:32 UTC 2021

On Wed, 2 Jun 2021 23:12:32 GMT, Erik Joelsson <erikj at openjdk.org> wrote:

> The Skara bots currently log all HTTP errors as SEVERE. With the log configuration we use, this results in admin alarms being sent whenever this happens (with some throttling). Now that we have metrics, we should instead leverage those and rely on alarms from our metrics gathering system instead. Typically HTTP errors aren't severe on their own, but if we get a lot of them, across multiple bot runners, then we need to react.
> 
> This patch does the following:
> 
> 1. Introduces a new specific exception which the RestRequest class throws when a bad HTTP response code is received.
> 2. Changes the logging behavior in the BotRunner when this new exception is caught.
> 3. Improves the logging of bad RestRequests so we actually log the request (instead of just the class name of the RequestBuilder).
> 4. Adds a new label on the skara_http_response counter to track if an error code was handled or not. We have a defined way of handling a bad HTTP response, and if this was used, we shouldn't send alarms as it's expected behavior.
> 5. Applies the http counters in two places where they were currently missing.

Looks good.

-------------

Marked as reviewed by kcr (Reviewer).

PR: https://git.openjdk.java.net/skara/pull/1180