RFR: 2610: Mailman 3 support

[Erik Joelsson]

On Wed, 26 Nov 2025 00:23:43 GMT, Zhao Song <zsong at openjdk.org> wrote:

>> The OpenJDK mail archive will move to Mailman 3 at some point in the future. To prepare for that, Skara needs to be made Mailman 3 compatible, specifically for reading emails from the archive to be able to publish them as comments in PRs. There are two major changes
>> 
>> ### 1. REST API
>> The REST API for reading mbox archives is different. It's a lot better in Mailman3. Instead of having to read in fixed monthly chunks, we can request a custom date interval. The data is also returned gzipped instead of in plain text. To retain backwards compatibility, I chose to implement the Mailman 2 and 3 support as subclasses with some shared implementation. This caused a lot of mechanical changes in tests, just updating method names or signatures since the API for creating a Mailman server object changes. It also forced some cleanup in bots where MailmanServer objects were created unnecessarily. I updated some tests to use the Mailman3 implementation where possible to exercise the new code more. 
>> 
>> The new implementation is somewhat taking advantage of the new capabilities in the API, but it could maybe be done more efficiently when polling for updates. 
>> 
>> ### 2. Mbox format
>> The format of the mail bodies in the returned mbox format is quite different. It's now MIME encoded and can be encoded in a few different ways ("7bit", "8bit" and "quoted-printable" have been observed so far). I've tried to implement support for decoding all the ways I've so far observed.
>> 
>> Some examples of this patch in action can be found in https://github.com/openjdk/playground/pull/246. Note that the earlier comments have problems that eventually got resolved as I ironed out the details. 
>> 
>> An observation I've made is that the new server introduces hard line breaks in emails as they are archived. Those are visible already when browsing the archive. I'm not sure if this is a setting in Mailman itself or not. Those line breaks are not part of the email I received when subscribing to a list, just in the archive, and so also in any comments posted by Skara.
>> 
>> This is a big patch, so sorry in advance.
>
> bots/mlbridge/src/main/java/org/openjdk/skara/bots/mlbridge/MailingListBridgeBotFactory.java line 199:
> 
>> 197:         MailingListServer mailmanServer;
>> 198:         if (archiveType == null || archiveType.equals("mailman2")) {
>> 199:             mailmanServer = MailingListServerFactory.createMailman2Server(listArchive, listSmtp, Duration.ZERO, useEtag);
> 
> Previously, we set the interval in configuration file and the value is not 0, why do we set it to 0 by default now?

You are right, this was a mistake. In the old code, the MailmanServer created in this class still got Duration.ZERO, see line 104, but the bot itself received a `sendInterval` parameter from the configuration. That parameter was in turn used in ArchiveWorkItem to create a new MailmanServer instance which was used to send email. I got rid of that instance creation in ArchiveWorkItem, but that also means we should pick up the configured interval and set that here. Good catch!

> mailinglist/src/main/java/org/openjdk/skara/mailinglist/mailman/MailmanListReader.java line 223:
> 
>> 221:                 var mboxUri = URIBuilder.base(server.getArchiveUri()).appendPath("list/").appendPath(name)
>> 222:                         .appendPath("@").appendPath(server.getArchiveUri().getHost())
>> 223:                         .appendPath("/export/foo.mbox.gz").setQuery(query).build();
> 
> This looks a bit weird, seems like we are downloading it instead of reading it, is it because if we just read it, the files are not in mbox format?

This is the URL I found in the test server we have running internally. There are download links in the UI which I modified to our needs. Not sure what you mean with downloading vs reading. The "foo" part can be anything you like as it will translate to a filename when using the URL in a browser.

-------------

PR Review Comment: https://git.openjdk.org/skara/pull/1743#discussion_r2566591810
PR Review Comment: https://git.openjdk.org/skara/pull/1743#discussion_r2566596724