What are we gonna do?

Packages

I am using the following packages

  • neomutt (MUA)
  • notmuch (tag mails)
  • msmtp (send mails)
  • offlineimap (sync mails)
  • khard (contacts book)
  • vdirsyncer (sync contacts)
  • cron (automate scripts)
  • pass (password storage)
  • dunst (notification deamon)

Part 1: Contact synchronisation

I assume we have a installed and configured pass and have our nextcloud password. First we need a configuration file for vdirsyncer to synchronize the contacts with nextcloud

~/.config/vdirsyncer/config

[general]
#A cache path where the current status of vdirsyncer will be stored
status_path = "~/.config/vdirsyncer/status/"

#Create a new sync connection between two storages we will define
[pair nextcloud_contacts]
a = "nextcloud_contacts_local"
b = "nextcloud_contacts_remote"
collections = ["from a", "from b"]
metadata = ["displayname"]

# A local storage to store the contacts
[storage nextcloud_contacts_local]
type = "filesystem"
# Path to where to store the vcards
path = "~/.contacts/"
fileext = ".vcf"

# A remote storage to nextcloud
[storage nextcloud_contacts_remote]
type = "carddav"
# Can be obtained from nextcloud
url = "https://your.nextcloud.lcl/remote.php/dav/addressbooks/users/USERNAME/"
username = "USERNAME"
# Shell command which calls the external command pass and reads the password nextcloud
password.fetch = ["command", "pass", "nextcloud"]
# SSL certificate fingerprint
verify_fingerprint = "FINGERPRINT"
#Verify ssl certificate. Set to false if it is self signed and not installed on local machine
verify = true

The SSL certificate fingerprint can be obtained by the following line

echo -n | openssl s_client -connect your.nextcloud.lcl:443 | openssl x509 -noout -fingerprint

Then we can initialize our synchronisation with vdirsyncer discover. We will sync then with vdirsyncer sync To Automate the process I will run a cron job on it crontab

*/5 * * * * /usr/bin/vdirsyncer sync

The contact book is relatively easy to configure

~/.config/khard/khard.conf

[addressbooks]
# Create new contact book
[[Nextcloud]]
# Path to directory of vdirsynced contacts
path = ~/.contacts/contacts/

[general]
debug = no
default_action = list
# Set editor to edit contacts
editor = nim
merge_editor = vimdiff

# Set a few tweaks to how to display contacts
[contact table]
display = first_name
show_nicknames = no
sort = last_name

[vcard]
# Perform search in the vard files to improve performance
search_in_source_files = yes

With that configuration we can run khard show and if everything went fine, we can see a list of all our contacts.

Part 2: Receive mails with offlineimap

As a second part I want to fetch all my emails

Because we never want to store passwords in plaintext, we want to use pass again. In offlineimap we have to create a python script first, which then can provide us a method to read the password

~/.offlineimap.py

1
2
3
4
5
6
7
#!/usr/bin/env python2
from subprocess import check_out

# Python method to obtain password
# Runs the shell command "pass show mail" and return the result of it
def get_pass():
    return check_out("pass show mail", shell=True).strip("\n")

Some people might have issues with import check_out. These should install the python package subprocess32 and should replace the check_out call with check_output in the python script above.

And here is my configuration for offlineimap

~/.offlineimaprc

[general]
accounts = private
# Path to python script, which contains the method to obtain the password
pythonfile = ~/.offlineimap.py

# Create a mail syncronisation
[Account private]
localrepository = private-local
remoterepository = private-remote

# Local storage
[Repository private-local]
type = Maildir
# Path to where the mails will be stored
localfolders = ~/.cache/mail/private

# Remote storage
[Repository private-remote]
type = IMAP
# Path to Imap server
remotehost = <HOST>
remoteuser = <USER>
# Password method defined in our external python script
remotepasseval = get_pass()
remoteport=993
# Use ssl to synchronize emails
ssl = yes
# See fingerprint for vdirsyncer
cert_fingerprint = <FINGERPRINT>

# Sync all mails into their corresponding mail folders
# Not needed for local, but it might be necessary for mobile clients to have the sent emails in the corresponding folder
[mbnames]
enabled = yes
filename = ~/.cache/mail/mailboxes
header = "mailboxes"
# Set naming for the folders
peritem = "+$(accountname)s%(foldername)s"
sep = " "
footer = "\n"

We should not forget to create the directory ~/.cache/mail. With that things done we are ready to run offlineimap and probably wait quite a while until all our mails are downloaded.

Part 3: Sending mails with msmtp

As next we will configure msmtp to correctly send emails. Once we got that setup, we also can use msmtp in scripts to automate status updates or similar. I this case we have directly a parameter, which we can set to specify a shell command to optain our mail password. The configuration is not that hard either.

~/.msmtprc

account private
port PORT
host mailserver@example.com
from user@example.com
user USERNAME
passwordeval "pass show mail"
auth on
tls on
tls_starttls on
tls_certcheck on
tls_fingerprint FINGERPRINT

The usage from msmtp is rather simple. It takes stdin as a message and sends it with a given account. A hello world example would be

echo -e "Subject: Hello world\nThis a test mail from msmtp" | msmtp -a private <EMAIL>

Now we can already receive emails and send emails. But I want to be able to use my mailclient completely offline. Therefore we need to be able to write emails offline and send them as soon as we have an connection to our mailserver again. The shell script msmtpq. We can paste it basically anywhere where it is in our PATH to be executable from everywhere. I will paste it into ~/.local/bin/. The only thing left to do is to substitute every msmtp call with msmtp.sh. If an internet connection exists, it will be automatically uploaded. If not, then the mail will be queued.

  • To manage the queue, we use msmtpq.sh --g-mgmt.
  • Therefore to flush the queue, we run msmtpq.sh --q-mgmt -r.
  • With -d We can see the content of our queue. We will use this functionality later and automate the process.

The next thing we want to do is to tag our emails with notmuch.

Part 4: Tag emails with notmuch

To begin with notmuch we want to setup some basic configuration for notmuch to find its mail directory and other initial information.

~/.notmuch-config

[database]
# Path to our mails
path=~/.cache/mail

[user]
# My Name
name=Jon Doe
# My mail address
primary_email=example@example.com

[new]
# Which tags to give a new email
tags=new
ignore=

[search]
# Mails with this tags will be excluded from search
exclude_tags=deleted;spam;

[maildir]
# Synchronize mail tags with offlineimap
synchronize_flags=true

With that being said we can run notmuch new to see all our emails loaded into the notmuch database with is in our case located in ~/.cache/mail/.notmuch/.

Not we can use notmuch in the command line to manually tag emails. But in the next step we will create a script depending on our needs to do that automatically for us.

~/notmuch-hook.sh

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
#!/bin/sh
# Index all new emails
notmuch new
# Move tag new mails with unread and inbox
notmuch tag +inbox +unread -new -- tag::new
# Mark sent mails as not inbox, but sent
notmuch tag -new -inbox +sent -- from:<MYEMAIL>
# Move mails in trash out of the inbox
notmuch tag -inbox -- tag:trash
# Move mails from amazon.com out of my inbox and tag with shop
# We will have a seperate virtual mailbox in neomutt later for that
notmuch tag -inbox +shop -- from:*@amazon.com

To automatically retag all messages after offlineimap we can run a hook in offlineimap to run our notmuch script automatically after syncing

~/.offlineimaprc

[Account private]
localrepository = private-local
remoteport = remote-local
# Path to script which will be executed after mailsync
postsynchook = ~/.notmuch-hook.sh

Part 5: Mailsyncing

To automate notmuch tagging, mailsync and my oflfinemsmtp queue, we will create a syncing script. When we want to send a notification to our desktop from a cron run script, we need to specify the Xmenu dbus. For that we add to our .profile the following

In ~/.profile

touch $HOME/.cache.Xdbus
chmod 600 $HOME/.cache/Xdbus
env | grep DBUS_SESSION_BUS_ADDRESS > $HOME/.cache/Xdbus
export "export DBUS_SESSION_BUS_ADDRESS" >> $HOME/.cache/Xdbus

Here is the final mailsync script

~/.local/bin/mailsync.sh

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
#!/bin/sh

#Get the Xdbus and load it up to be able to show notifications
if [ -r "$HOME/.cache/Xdbus" ]; then
    . :$HOME/.cache/Xdbus"
fi

# Get number of emails queued to sent
num_queue=$(msmtpq.sh --q-mgmt -d | grep num | wc -l)

# Flush queue
msmtpq.sh --q-mgmt -r

#Sync imap emails
offlineimap

#Get number of new mails
num_unread=$(notmuch count tag:unread)

#When run with parameter q, then don't show notifications
if [ "$1" != "q" ]; then
    notify-send " 📧Mailsync" "Synced mail\n 📬 $num_unread new mails\n 📯 $num_queue Mails uploaded"
fi

The only thing left to do is to add a cron job to automatically synchronize emails

*/10 * * * * mailsync.sh

Part 6: Smash everything into mutt

In mutt is a lot to configure. Here is the basic configuration to get the functionality we wanted.

~/.muttrc

#Setup minimal mutt mailboxes
set folder = "~/.cache/mail/"
set mbox_type = Maildir
set spoolfile = +/
set postponed = +private/Drafts
set trash = +private/Trash
set record = +private/Sent

#Setup notmuch mailboxes
#URL to your mail folder
set nm_default_uri = "notmuch:///home/dbauer/.cache/mail"
set virtual_spoolfile=yes
virtual-mailboxes \
    " 📥Inbox"        "notmuch://?query=tag:inbox"\
    " 📬Unread"       "notmuch://?query=tag:unread"\
    " 📧Sent"         "notmuch://?query=tag:sent"\
    " 🛒Shopping"     "notmuch://?query=tag:shop"\
    " 🗄 Trash"        "notmuch://?query=tag:trah"

#Basic naming setup
set from "myemail@myserver.com"
set realname "My name"
set use_from = yes
set reply_to = yes

#Send emails via msmtpq.sh
set sendmail = "msmtpq.sh -a private"
set sendmail_wait = 0

#Sidebar configuration
set sidebar_format = "%B %?N?[%N]? %* (%S)"
set sidebar_visible = yes

#Look up contacts via khard
set query_command= "khard email --parsable --search-in-source-files %s"

#Hacky way to select the inbox in sidebar with key i
#The number of <sidebar-prev> depends on how many mailboxes we have.
#We basically just press them to reach the top in the worst case from the very bottom.
unbind index i
macro index i "<sidebar-prev><sidebar-prev><sidebar-prev><sidebar-prev>"
#Mark email as deleted with key d
macro index d "<modify-labels>+trash<enter>"
#Search emails via notmuch with /
macro index // "<vfolder-from-query>"
#Add email address to contact book with A
macro index,pager A "<pipe-message>khard add-email<return>"
#Save selected attachment with S
macro attach S <save-entry><kill-line>$HOME/Downloads/<enter>j<enter>
#General command to edit notmuch labels of selected email
macro index,pager y modify-labels
#Navigate through sidebar
bind index,pager <left> sidebar-prev
bind index,pager <right> sidebar-next
bind index,pager <space> sidebar-open