the sporks space

As a developer, I need to work with mailing lists a lot. Developer-focused mailing lists tend to have certain conventions most usage of email has moved on from, as have the clients. While some people who trawl mailing lists configure some terminal email client, I tend to not like those. Most of the time, I use the stock Apple mail application (Mail.app). It otherwise works well and has platform integration.

However, I do want to avoid a lot of common mailing list faux pas, like top posting and not wrapping lines. Mail.app can write plain text emails, but it doesn’t push you in the right direction for mailing lists. However, Mail.app does provide an extension API. I figured I could write an extension to make my life on mailing lists easier. Turns out it’s possible!

The extension I wrote is called MailTools (tentative until a better name is found). Feel free to build or download it yourself; this article explains the challenges I faced.

Background: MailKit and what came before

In the Before Times, Mail.app was extensible through mail bundles. These plugins loaded code into Mail.app, but carried the heavy caveat that almost everything was undocumented with no guarantees. Swizzling methods you discerned through dumping classes was the name of the game. Plugins could do anything, but it came at costs of even the smallest refractor breaking them. Two such plugins existed for making the mailing list experience in Mail.app better; MailWrap and MailFlow, both from the same author.

So it turns out monkey-patching a complicated Objective-C application that changes over time is a bad idea. On top of the compatibility issues, there’s a ton of risks for stability and security with running third-party code in your process. So, Apple came up with a replacement API called MailKit, using the existing concept of app extensions. The main advantage is it runs everything out of process (via XPC voodoo) with clear points for where you can extend Mail.app. For example, a compose extension can do preflight checks before sending an email or add headers, and is the obvious choice for what I want to do. Running out of process also makes it far easier to debug.

Sidebar: Another interesting implication with MailKit would be running the mail app extensions on iOS. Currently, MailKit only supports macOS. The only macOS dependencies in the API seem to be AppKit view controllers, and it wouldn’t be much work to change NSViewController to UIViewController. However, the iOS mail app is a lot less featureful.

Unfortunately, the downside is you can’t do everything you want, since Mail.app only calls your extension at certain points. There’s no extending the editor or changing the body of the message, like the plugins I mentioned earlier could do. Some of it might be features they haven’t gotten around to it, or because it could be easily abused. Regardless, it means we have to think of how to implement what we want in the framework of MailKit. In my case, the easiest option is to just check before sending and warn the user. Being able to augment the editor experience would be better, but not possible yet. File a Radar, I guess.

Another Sidebar: Other projects have dealt with making similar tradeoffs with extensibility. Famously, Mozilla abandoned XUL addons that could do anything, in favour of WebExtensions. While the loss of functionality hurt a lot of users, XUL addons blocked a lot of improvements. With them gone, Firefox could make performance and security improvements. David Teller’s recounting of the story is worth a read.

Actually developing the extension (and the first hurdle I ran into)

To start with, I first had to make an application hosting the extension inside of it. There’s good reason why Apple wants extensions (be it for Mail.app or anything else) to have applications that contain them. Instead of inscrutable subdirectories under /Library, the user can just drag to or from /Applications to install or uninstall the extension. Unfortunately, if your application extension is doing most of the lifting (rather than providing features for an existing application), this means you have a mostly useless .app hanging around. Of course, that .app can provide instructions on how to enable the extension, or configuration screens.

The next step is to add a target (in Xcode parlance) for the mail extension. You tell it what kind of handlers you want to add, and it’ll dutifully provide sample stubs. One unfortunate issue I had was with the Swift version of the template. The way preflight checks in allowMessageSendForSession work is by giving it an error in the callback if things aren’t OK. (This is not using the async pattern in Swift, but it is an Objective-C API.) Unfortunately, the default template is broken, and in a very subtle way. Below is the sample code that Xcode provides:

    enum ComposeSessionError: LocalizedError {
        case invalidRecipientDomain
        
        var errorDescription: String? {
            switch self {
            case .invalidRecipientDomain:
                return "example.com is not a valid recipient domain"
            }
        }
    }
    
    func allowMessageSendForSession(_ session: MEComposeSession, completion: @escaping (Error?) -> Void) {
        // Before Mail sends a message, your extension can validate the
        // contents of the compose session. If the message is ready to be sent,
        // call the compltion block with nil. If the message isn't ready to be
        // sent, call the completion with an error.
        if session.mailMessage.allRecipientAddresses.contains(where: { $0.rawString.hasSuffix("@example.com")}) {
            completion(ComposeSessionError.invalidRecipientDomain)
        } else {
            completion(nil)
        }
    }

The errors are an enum implementing the LocalizedError protocol for the message. Unfortunately, when you run this, you’ll get no message, instead of the error’s localized description:

What did work for me was shoving an NSError in the callback instead. This is what the Objective-C version of the template does instead. I suspect it might be some inscrutable XPC issue with shoving Swift Error types down the pipe. Either the template or Mail.app should be fixed so this works out of the box. For the Apple employees reading this, look at FB15329798.

Using SwiftUI, and SwiftData

Once I got past this initial confusion, it was smooth sailing actually developing the extension. There’s been a lot of improvements to Swift and SwiftUI since that made it pretty easy to develop for. SwiftUI on Mac in particular has improved a lot. Making a single-window fixed-sized application used to be annoying. You constantly having to bail out to AppKit via delegates to override SwiftUI’s default behaviours. Now it’s a matter of using Window instead of WindowGroup and putting on the right windowResizability modifier.

For persisting data, like mail rules (what rules for which domain/address?), I’m using SwiftData. The developer experience has been surprisingly smooth, and my first attempt worked on the first try. I just had to add an app group so the two different bundles could share resources in the sandbox. (It’s worth noting you must follow its advice about team IDs in the group name, or users are going to get pestered.) SwiftData will automatically use the one you add. SwiftUI also has integration so using it from views in the app and extension are trivial as well.

The nice thing with SwiftData is the editing experience over Core Data. Instead of graphical editors outputting huge XML blobs, it’s just codegen with macros. Even more complicated things like complex migrations just use the type system. Unfortunately, there is some Core Data abstraction leakage. Things Swift can represent that Core Data can’t (like enums with associated values) have to implement Codable. Things like uniqueness or query-based sorting won’t work on those attributes, as they’re just opaque blobs to Core Data.

Unfortunately, taking advantage of this means I have to target fairly new macOS. Specifically, 14, which is only one behind the current version. The developer experience improvements have been worth it though.

The plain text editing experience in Mail.app (and parsing it)

If you look at the raw MIME message of a plain text email while you’re composing it, you’ll notice it’s not plain text – it’s HTML! Mail.app relies on some semantic HTML with classes, and then converts to plain text when sent. This way, it can keep formatting as you switch between modes. This means instead of looking at plain text in our extension, we can rely on the semantic elements instead. Below is a chart of the classes that Mail.app uses, to the best of my knowledge:

Class/ID name	Purpose
ApplePlainTextBody	Used for the body of a plain text email
AppleOriginalContents	Used for quoting the message in a reply
lineBreakAtBeginningOfMessage	Used for the line break before where you type when top-posting in a reply
Apple-interchange-newline	Used for line breaks inside of quotes

To actually parse this, I relied on the MimeParser and SwiftSoup libraries. MimeParser extracted the HTML body of a multipart mail message. Then I throw it into SwiftSoup to build an intermediate representation from the HTML. From IR, I then apply the heuristics I use. The main advantage of building an IR is I can work with both HTML and real plain text. Mail.app doesn’t support “true” plain text, but if it ever does, I can just turn that into IR too.

Conclusions

This project came together pretty easily, except for the strange issues I ran into which had little help available. It matches my previous experience with developing on Apple platforms, but with the newer and nicer Swift-first frameworks. When it works, it’s magical and incredibly productive. The APIs implement a ton of functionality and push you to use good design patterns. When it goes wrong, you’re flying blind and no one has the answers outside of Objective-C wizards. Thankfully, the problems are usually small and can be resolved yourself enough to not be a black mark.

If you are interested in trying MailTools, go for it! Let me know if you run into any issues or have any requests.