Neomutt + isync / mbsync for Office365

Surveying the land

The first hurdle to jump over is authentication. Without a doubt the easiest solution to this is the path of least resistance, and of piggy-backing on Mozilla's Thunderbird email client's hard work. That is to say, using XOAUTH2. Here's a very short explanation:

• Microsoft requires applications that use API access, including to IMAP / SMTP, to be registered in their applications portal. Doing so is a bit of a headache, but this is only to scope permissions; and since all we need are email permissions, we can pretend we're a different email application (like Mozilla's Thunderbird), and access the same scopes.

• Inspired by this blog post, we use a delicately modified version of Mutt's mutt_oauth2.py, which should be included with many distributions of neomutt. I've already taken the liberty of including the client ID and secret from Thunderbird:

  ./mutt_oauth2.py OUTPUT_TOKEN_FILE_NAME --verbose --authorize \
      --client-id='08162f7c-0fd2-4200-a84a-f25a4db0b584' \
      --client-secret='TxRBilcHdC6WGBee]fs?QR:SJ8nI[g82'

  This will spawn a Single-Sign-On verification process that will grant us the OAUTH2 token, and encrypt it with GPG. It's worth reading around in mutt_oauth2.py to see what it's doing, and also to make a handful of configuration changes relevant to your personal needs (e.g. which GPG key to use).

• isync uses the Simple Authenticity and Security Layer to do authentication. That means we need a SASL plugin for doing XOAUTH2, which luckily is simple to get and install.

  From source: https://github.com/moriyoshi/cyrus-sasl-xoauth2

  On Arch linux from the AUR:

  yay -S cyrus-sasl-xoauth2-git

Last, and certainly not least, we need to have isync installed. This is an exercise left to the reader, but it's about as easy as checking your distro's repositories and gleefully finding it is there.

Setting up shop

Most of the hard work is already done. We just need to wire together the configuration to make it work.

isync used to go by mbsync, so it's unsurprising the executable / manpages / configuration file maintain a degree of backwards compatability. Therefore, we edit ~/.mbsyncrc. The configuration files are a little odd until you realise there is an imperative order, going from top to bottom:

# ~/.mbsyncrc

# Global defaults that apply to all channels / accounts / stores

# Create in both places
Create Both
# Do not remove if missing in one place
Remove None
# Permanently delete those messages marked for deletion
Expunge Both
# Synchronise everything
Sync All
SyncState *
CopyArrivalDate yes
MaxMessages 0

# Office365 configuration
# This is tailored for Bristol's University email

IMAPAccount bristol-ac-uk
Host outlook.office365.com
Port 993
User USERNAME@bristol.ac.uk
AuthMechs XOAUTH2
# Use Mutt's OAUTH2 token refresh script to decrypt and refresh
PassCmd "~/.cache/mutt/mutt_oauth2.py ~/.cache/mutt/TOKEN"
# Use TLS
SSLType IMAPS
SystemCertificates yes
Timeout 15

IMAPStore bristol-ac-uk-remote
Account bristol-ac-uk

MaildirStore bristol-ac-uk-local
SubFolders Verbatim
Path ~/.mail/bristol-ac-uk/
Inbox ~/.mail/bristol-ac-uk/INBOX

# Channel used to sync with `mbsync CHANNEL-NAME`
Channel bristol
Far :bristol-ac-uk-remote:
Near :bristol-ac-uk-local:
Patterns *

Channel bristol-inbox
Far :bristol-ac-uk-remote:
Near :bristol-ac-uk-local:
Patterns INBOX

Note that I have created two channels. The second one is so that I can just sync the inbox without everything else, which can speed up the synchronisation significantly.

Some important things to point out

• PassCmd is the command used to get the OAUTH2 token. This is specific to my personal setup in terms of the file paths of the script and encrypted token.
• The MaildirStore is the local email storage location.

To synchronise mailboxes, all that is left is to invoke mbsync with the appropriate channel we wish to sync.

• To sync everything:

  mbsync bristol

• To sync only the inbox

  mbsync bristol-inbox

Open for business

The last detail is then the neomutt configuration. For this, we still want to keep the SMTP and IMAP configuration, as this will allow us to still easily send mail.

# ~/.config/mutt/muttrc

# Pipe decrypted password
set smtp_pass = "gpg -dq uni-pass.gpg |"
set imap_pass = "gpg -dq uni-pass.gpg |"

set hostname = "bristol.ac.uk"

set imap_user = "USERNAME@bristol.ac.uk"
set imap_authenticators = "xoauth2"
set imap_oauth_refresh_command = "~/.cache/mutt/mutt_oauth2.py ~/.cache/mutt/TOKEN"

set smtp_url = "smtp://USERNAME@bristol.ac.uk@smtp.office365.com:587"
set smtp_authenticators = ${imap_authenticators}
set smtp_oauth_refresh_command = ${imap_oauth_refresh_command}

# Ensure TLS always used
set ssl_force_tls = "yes"
set ssl_starttls = "yes"

# Point folder at our Maildir setup
set folder = "~/.cache/mail/bristol-ac-uk/"
set spool_file = "+INBOX"
set postponed = "+Drafts"
set record = "+Sent\ Items"
set trash = "+Deleted\ Items"

# Select a subset of mailboxes to display in Mutt
mailboxes -notify -label "Inbox" =INBOX
mailboxes -nonotify ="Flagged" ="Sent Items" =Drafts ="Deleted Items" +Teaching

Neomutt should now work as before, except that it is using the local Maildir folders instead of the IMAP folders. That means to sync we need to invoke an external command, which is easy to do with a macro

macro index,pager ~~ "<shell-escape>mbsync bristol-inbox<enter>" "Sync inbox"
macro index,pager ~@ "<shell-escape>mbsync bristol<enter>" "Sync all"
# fetch mail periodically when mutt is launched and no input is received
set timeout = 60
timeout-hook 'echo `mbsync bristol-inbox`'

Now, hitting ~~ whilst in the index or pager will fetch just the inbox, whereas ~@ (adjacent on my keyboard) will fetch all mail folders. Furthermore, using one of the hook features of neomutt, the mbsync command is invoked to refresh the inbox every minute whilst mutt is running in the background.