Neomutt + isync / mbsync for Office365

My university continues to push Microsoft products and services on us, and I take my small victories where I can find them. Today was one such small victory: being able to create a complete local copy of my Office365 IMAP folders. This allows me to search through my email with regular command line tools, and find things when offline.

This is a brief overview of how to setup the neomutt email client with isync nee mbsync. I will be glossing over details about configuring neomutt and will mainly focus on isync. isync is the tool which synchronises IMAP and Maildir folders in different places, and is separate from mutt but can easily be integrated.

My plan for this setup is to still keep Office365 as a "source of truth" / backup for my emails; that is, I want local copies of my mail for the purpose of being able to quickly search and sort mail without removing the IMAP mail. Maybe if I feel more confident in the future with this setup, I will purge my Microsoft inbox and keep my data to myself.

Surveying the land

The first hurdle to jump over is authentication. Without a doubt the easiest solution to this is the path of least resistance, and of piggy-backing on Mozilla's Thunderbird email client's hard work. That is to say, using XOAUTH2. Here's a very short explanation:

  • Microsoft requires applications that use API access, including to IMAP / SMTP, to be registered in their applications portal. Doing so is a bit of a headache, but this is only to scope permissions; and since all we need are email permissions, we can pretend we're a different email application (like Mozilla's Thunderbird), and access the same scopes.

  • Inspired by this blog post, we use a delicately modified version of Mutt's, which should be included with many distributions of neomutt. I've already taken the liberty of including the client ID and secret from Thunderbird:

    ./ OUTPUT_TOKEN_FILE_NAME --verbose --authorize \
        --client-id='08162f7c-0fd2-4200-a84a-f25a4db0b584' \

    This will spawn a Single-Sign-On verification process that will grant us the OAUTH2 token, and encrypt it with GPG. It's worth reading around in to see what it's doing, and also to make a handful of configuration changes relevant to your personal needs (e.g. which GPG key to use).

  • isync uses the Simple Authenticity and Security Layer to do authentication. That means we need a SASL plugin for doing XOAUTH2, which luckily is simple to get and install.

    From source:

    On Arch linux from the AUR:

    yay -S cyrus-sasl-xoauth2-git

Last, and certainly not least, we need to have isync installed. This is an exercise left to the reader, but it's about as easy as checking your distro's repositories and gleefully finding it is there.

Setting up shop

Most of the hard work is already done. We just need to wire together the configuration to make it work.

isync used to go by mbsync, so it's unsurprising the executable / manpages / configuration file maintain a degree of backwards compatability. Therefore, we edit ~/.mbsyncrc. The configuration files are a little odd until you realise there is an imperative order, going from top to bottom:

# ~/.mbsyncrc

# Global defaults that apply to all channels / accounts / stores

# Create in both places
Create Both
# Do not remove if missing in one place
Remove None
# Permanently delete those messages marked for deletion
Expunge Both
# Synchronise everything
Sync All
SyncState *
CopyArrivalDate yes
MaxMessages 0

# Office365 configuration
# This is tailored for Bristol's University email

IMAPAccount bristol-ac-uk
Port 993
AuthMechs XOAUTH2
# Use Mutt's OAUTH2 token refresh script to decrypt and refresh
PassCmd "~/.cache/mutt/ ~/.cache/mutt/TOKEN"
# Use TLS
SystemCertificates yes
Timeout 15

IMAPStore bristol-ac-uk-remote
Account bristol-ac-uk

MaildirStore bristol-ac-uk-local
SubFolders Verbatim
Path ~/.mail/bristol-ac-uk/
Inbox ~/.mail/bristol-ac-uk/INBOX

# Channel used to sync with `mbsync CHANNEL-NAME`
Channel bristol
Far :bristol-ac-uk-remote:
Near :bristol-ac-uk-local:
Patterns *

Channel bristol-inbox
Far :bristol-ac-uk-remote:
Near :bristol-ac-uk-local:
Patterns INBOX

Note that I have created two channels. The second one is so that I can just sync the inbox without everything else, which can speed up the synchronisation significantly.

Some important things to point out

  • PassCmd is the command used to get the OAUTH2 token. This is specific to my personal setup in terms of the file paths of the script and encrypted token.
  • The MaildirStore is the local email storage location.

To synchronise mailboxes, all that is left is to invoke mbsync with the appropriate channel we wish to sync.

  • To sync everything:

    mbsync bristol
  • To sync only the inbox

    mbsync bristol-inbox

Open for business

The last detail is then the neomutt configuration. For this, we still want to keep the SMTP and IMAP configuration, as this will allow us to still easily send mail.

# ~/.config/mutt/muttrc

# Pipe decrypted password
set smtp_pass = "gpg -dq uni-pass.gpg |"
set imap_pass = "gpg -dq uni-pass.gpg |"

set hostname = ""

set imap_user = ""
set imap_authenticators = "xoauth2"
set imap_oauth_refresh_command = "~/.cache/mutt/ ~/.cache/mutt/TOKEN"

set smtp_url = "smtp://"
set smtp_authenticators = ${imap_authenticators}
set smtp_oauth_refresh_command = ${imap_oauth_refresh_command}

# Ensure TLS always used
set ssl_force_tls = "yes"
set ssl_starttls = "yes"

# Point folder at our Maildir setup
set folder = "~/.cache/mail/bristol-ac-uk/"
set spool_file = "+INBOX"
set postponed = "+Drafts"
set record = "+Sent\ Items"
set trash = "+Deleted\ Items"

# Select a subset of mailboxes to display in Mutt
mailboxes -notify -label "Inbox" =INBOX
mailboxes -nonotify ="Flagged" ="Sent Items" =Drafts ="Deleted Items" +Teaching

Neomutt should now work as before, except that it is using the local Maildir folders instead of the IMAP folders. That means to sync we need to invoke an external command, which is easy to do with a macro

macro index,pager ~~ "<shell-escape>mbsync bristol-inbox<enter>" "Sync inbox"
macro index,pager ~@ "<shell-escape>mbsync bristol<enter>" "Sync all"
# fetch mail periodically when mutt is launched and no input is received
set timeout = 60
timeout-hook 'echo `mbsync bristol-inbox`'

Now, hitting ~~ whilst in the index or pager will fetch just the inbox, whereas ~@ (adjacent on my keyboard) will fetch all mail folders. Furthermore, using one of the hook features of neomutt, the mbsync command is invoked to refresh the inbox every minute whilst mutt is running in the background.