Integrated: 1513: Reduce polling of mailing list archives

[Erik Joelsson]

On Fri, 29 Jul 2022 17:31:23 GMT, Erik Joelsson <erikj at openjdk.org> wrote:

> This patch changes the strategy used by the MailmanListReader for polling the mailman archives. The current implementation relies on the server supporting "etag" in order to trust any cached results. Recent testing has shown that etags aren't supported by mail.openjdk.org, which means no results are ever cached, we just keep spamming the mailman archives for the last 12 months over and over.
> 
> My new implementation assumes that new emails will only ever appear in the current and previous months archives. (If this proves to be wrong, I still think that would be rare enough that it doesn't matter, as the full 12 months will be re-evaluated on bot restart.) So for anything older than the previous month, all successful (200) or non-existent (404) results will be cached and never re-queried.
> 
> The reason mlbridge needs to query emails for up to a year back is that it needs to piece together conversations and trace them back to the original post in order to correctly identify the PR link associating the conversation with a certain PR. (It's possible that this could be made more efficient in a separate change.)
> 
> The change itself is rather small, but in order to test it, I needed to expand functionality in the TestMailmanServer. The existing tests did not verify any calls to archives for months other than the current, so I needed to add support for actually handling that. I also moved the data to in memory storage in HashMaps instead of writing to temp files.
> 
> My only worry here is that I messed up with the test so that it will start failing on certain days of the year.

This pull request has now been integrated.

Changeset: 515f2e14
Author:    Erik Joelsson <erikj at openjdk.org>
URL:       https://git.openjdk.org/skara/commit/515f2e14d99b02494fcfee4ea01203429b2f8e0b
Stats:     126 lines in 3 files changed: 110 ins; 6 del; 10 mod

1513: Reduce polling of mailing list archives

Reviewed-by: kcr

-------------

PR: https://git.openjdk.org/skara/pull/1343