Just a quick update about the state of the project before I disappear from civilization for this weekend.

Pluggable workflows#

MR @ gitlab

I have now finally got the pluggable workflows branch to test succesfully! With workflows being loaded dynamically, with workflow steps being saved completely, full backward compatibility, also database migrations and REST backward compatibility.

That branch is still missing more tests for diffcov to pass.

https://gitlab.com/J08nY/mailman/pipelines/9645615

^^python 3.4 CI envs are currently broken, since aiosmtpd 1.1 is python 3.5 only, so don’t mind that.

As I note in one of my previous GSoC project updates, the PGP-enabled mailing lists require two new features from Mailman Core, both related to workflows. The first is to be able to inject steps into a subscription workflow to request the users pubkey before moderation checks. The second it to be able to require confirmation on all commands (to prevent replay attacks), although this could be avoided by storing the hashes from all the user sent signed commands. In this post I propose a way of refactoring the workflows into pluggable components.

The first evaluation period is here, I have already sent in my evaluation so I thought I would recap the state of the project.

During the first two weeks or so, I mostly focused on features in Mailman Core that would be necessary for making a plugin like this work, my MRs introduce an easy to use/configure plugin infrastructure. Where plugins can provide components and even custom REST routes in Mailman Core. This with just installing the plugin into the same environment as Mailman Core and some config additions. Also some minor additions of selecting the list style on creation in Postorius and appropriate changes in mailman-client.

This week things moved along with the mailman-pgp plugin. As I finally had a good PGP library to work with as well as some basic plugin API patches outstanding to Mailman Core.

So with that I implemented large parts of the mailman_pgp.pgp package, including tests, as well as a basic signature checking rule. Also, as the package now contains tests, I added a gitlab CI config and started running it.

repo @ neuromancer.sk repo @ gitlab

Since my last post, I’ve been looking for a suitable PGP python library as I realized my original candidate python-pgp just wouldn’t work:

• Its not actively maintained.
• Its tests/docs etc.. are lacking.
• Its security is questionable.

I’ve since found and tried using python-gnupg, gnupg, python-gpgmime and finally PGPy.

As last time I described outstanding PRs to Mailman Core and other Mailman’s components, this time I am going to describe the plugin I am building on top of those components to enable PGP encrypted mailing lists.

In fact there are two plugins to present, with the first one being a general and quick plugin example that showcases how the new plugin API might be used and also provides some nice services for the PGP plugin.

mailman-rest-events#

repo @ gitlab

This plugin subscribes itself to receive events in Mailman Core and sends them to urls configured. It is a variation of what mailman!264 is intending to do, based on the plugin API implemented in my MRs. It currently offers only basic functionality, a URL is specified in config per endpoint along with an API key (very similar to mailman-hyperkitty) and a regex of event class names that will be sent. The data sent currently is only the event class name in a JSON object. Config can have many endpoints specified with the form shown below.

This week has been quite productive. I started laying the groundwork for the encrypted lists plugin by making changes from core_changes to different components of both Mailman Core, Mailman-Client and Postorius. I will describe the MRs and ideas behind them here.

Mailman Core#

Move pipelines to their own package and instantiate dynamically#

mailman!287

This MR makes Pipelines in Mailman Core work in the same dynamic way as Handlers,Rules,Chains and other similar Mailman’s components work.

Since a plugin-like out-of-tree approach is required for implementing encrypted lists into Mailman, a straight forward integration into Postorius and HyperKitty (as first proposed) by making them “aware” of the encrypted lists plugin is not possible.

Thus a new approach for providing their functionality and conforming to the project requirements is necessary. I see three possible pathways forward and a middle-ground between them.

Structure#

• pgpmailman - A Core plugin.

  • styles - Both styles generate a list keypair based on plugin settings on list creation as well as set other attributes for an encrypted mailing list. Such as the custom encrypted chain.

    • EncryptedDefaultStyle
    • EncryptedAnnounceStyle

  • pgp

  • rules

    • EncryptionRule - Decrypts message and enforces per-list encryption requirements.
    • SignatureRule - Checks message signature and enforces per-list signature requirements. Strips signature to msgdata.

Plugin API enhancements in Core#

To cleanly implement encrypted mailing lists as a plugin to Mailman Core I propose several general changes to the plugin api, to allow for cleaner integration of plugins, more flexibility and easier plugin deployment. First I present the current state of pluggability in Mailman core and then the proposed changes.

Current state#

Relevant mailman-developers thread from GSoC 2015

Example plugin

• A plugin creator has many ways of “injecting” his code to run at certain phases of Mailman’s operation, since Mailman looks for its classes and components dynamically, it doesn’t care whether they are from a plugin or originally from core.
  • Implementing IHandler, IChain, IRule, IEmailCommand, ICLISubCommand or IStyle and placing modules containing the classes in the appropriate directories where Mailman finds them and instantiates them.