Securely using mutt with multiple Gmail Accounts

[statmonkey]

In my continuing search for obscure things that no one besides myself wants to know here is a little mutt love.  It is also another demonstration of the power of my favorite distro - everything you need is part of the standard install and is there waiting for you when you need it.  One more hat tip to Vsido, great job all!

Why do this?
I like mutt, its easy to use and configure and if I want I can always log to the browser window to get mails.  Personally though, I rarely do as mutt can do everything I need in a light, text based interface.  As always because it's there and can be done. 

Who would do it?
Well, anyone who feels the same way I do or anyone looking to streamline their email handling and save some time over the long run.  Mutt can be configured with boxes in much the way your gmail accounts are and if you have multiple accounts as I do using mutt with multiple accounts switching between accounts is much easier and faster. 

How skilled do you need to be?
Really about the only "skill" required is a solid understanding of the command line and a little patience.

What is required?

• Nano, vim, emacs, any cli document app
  mutt
  a gmail account or two
  your username and password
  gpg
  

All of these are standard with Vsido so you should have to do nothing for that distro.  For others YMMV.

Steps

• Set up secure password file
  Set up your .muttrc with your specific account info
  Add some html handling
  Remove the temp password file (unsecured version)
  Mutt away!

Setting up your password file

There are several ways to do this.  Ultimately this solution (provided here) is not without it's flaws. A really capable hacker, one who is capable of getting control of your box could take advantage of the temp file handling used here and get access to your account info.  For that matter anyone who could figure out your control password could access pgp.  In other words if you are a heinous worry wart this won't be secure enough for you.  For my purposes it provides enough that I am unconcerned.  If you desire more then you probably have a better understanding of PGP than I do and know how to use shred and a temp drive in ram.  I don't see the need for that level of tin hatted-ness. Ok, flame away. 

For security we will be setting up an encrypted file for your passwords. For my examples I will be using my old favorite nano but any similar app will work.
To start with we will need to generate a gpg key to secure our passwords file.  To do this (you should already have gpg installed if you are on Vsido)
   

Code Select Expand

gpg --gen-key
This will start the key generation process and it will prompt a few questions.  Accept the default (DSA + RSA) and then accept the maximum keylength.  It will then ask you the validity period of the key (if you are planning on a public key it should be for a finite period) I am the only one on my box (I hope) so I just chose a key period that would not expire.  Once this is done it will start asking you for several other pieces of info.  Like your name, email and maybe some comments and in the end will ask you for a pass phrase. You can put whatever you want in any of this but it is best that you fill it out correctly and use a good pass phrase.   You will need the name you entered in the very next step as well as that pass phrase when you use mutt so don't forget who you are or what your pass phrase is.  As always use something you can remember and that is secure . 
For more tips and tricks on gpg check out http://nixtricks.wordpress.com/2009/10/04/introduction-to-encryption-of-files-using-gpg/ which is where I learned this. Note some of it is out of date but the basic concepts are correct

We will need a place to put this file and for my purposes I will use a folder called .encrypt like so:
   

Code Select Expand

cd ~
   

Code Select Expand

mkdir .encrypt
   

Code Select Expand

cd .encrypt
   

Code Select Expand

nano .gmpass
In that file enter the following:
Gmail1 yourfirstpassword
Gmail2 yoursecondpassword
Gmail3 yourthirdpassword
etc. as you need and then save the file
Now we can encrypt the file with:
   

Code Select Expand

gpg --encrypt --recipient 'Your Name' .gmpass
What we are saying here is use gpg to encrypt the file .gmpass  I would suggest that you don't delete the original file .gmpass quite yet.  Just in case you made an error somewhere you can keep it around for reference.

Great, we now have an encrypted gmail password file.  Next you will need to create a file in the .encrypt folder called .tmp otherwise our temp storing of messages won't work.  Once it is there it will immediately be overwritten when you run mutt, so no worries. Create that file now with
   

Code Select Expand

touch ~/.encrypt/.tmp

Setting Up Mutt
For the most part mutt should be ready to go. Next we have to set up a folder called .mutt in your home folder to put the cache headers in.

Code Select Expand

mkdir ~/.mutt
Mutt itself should not need any outside help (like procmail, fetchmail, etc) and all we have to do is edit and create the .muttrc  Since this file is a little extensive I used geany to do it but nano or vim will work as well.  The first section until the macros is all you need to worry about adjusting.  I have attempted to put some notes in to make it easier to see.  It really only takes a few minutes.  Clearly the password section requires you to adjust according to where you put your pass file and what you named it but the rest is simple.  Just make sure the password paths are correct.

Code Select Expand

#-----------#
# Passwords #
#-----------#
# remember our passwords file .gmpass?  Well these lines find it and decrypt it using gpg. Important note the my_ part of each of these is
# critical so don't leave it out, it must be "set my_etcetera .......
set my_tmpsecret=`gpg2 -o ~/.encrypt/.tmp -d ~/.encrypt/.gmpass.gpg`
set my_gpass1=`awk '/Gmail1/ {print $2}' ~/.encrypt/.tmp`
set my_gpass2=`awk '/Gmail2/ {print $2}' ~/.encrypt/.tmp`
set my_gpass3=`awk '/Gmail3/ {print $2}' ~/.encrypt/.tmp`
set my_del=`rm -f ~/.encrypt/.tmp`

#---------------#
# Account Hooks #
#---------------#
# What you are replacing in here is "user#acct that is it the rest stays the same for each of the three accounts
# all we are doing here is creating hooks to get the mail and stay in contact with gmail.  Ummm, don't try this with yahoo :)
account-hook . "unset imap_user; unset imap_pass; unset tunnel" # unset first!
account-hook        "imaps://user1stacct@imap.gmail.com/" "\
    set imap_user   = user1stacct@gmail.com \
        imap_pass   = $my_gpass1"
account-hook        "imaps://user2ndacct@imap.gmail.com/" "\
    set imap_user   = user2ndacct@gmail.com \
        imap_pass   = $my_gpass2"
account-hook        "imaps://user3rdacct@imap.gmail.com/" "\
    set imap_user   = user3rdacct@gmail.com \
        imap_pass   = $my_gpass3"

#-------------------------------------#
# Folders, mailboxes and folder hooks #
#-------------------------------------#
# In this section you are changing the user#acct and Your Name and that is it. You could monkey with the folder locations as you wish

# Setup for user1:
set folder          = imaps://user1stacct@imap.gmail.com/
mailboxes           = +INBOX =[Gmail]/Drafts =[Gmail]/'Sent Mail' =[Gmail]/Spam =[Gmail]/Trash
set spoolfile       = +INBOX
folder-hook         imaps://user1stacct@imap.gmail.com/ "\
    set folder      = imaps://user1stacct@imap.gmail.com/ \
        spoolfile   = +INBOX \
        postponed   = +[Gmail]/Drafts \
        record      = +[Gmail]/'Sent Mail' \
        from        = 'Your Name <user1stacct@gmail.com> ' \
        realname    = 'Your Name' \
        smtp_url    = smtps://user1stacct@smtp.gmail.com \
        smtp_pass   = $my_gpass1"

# Setup for user2:
set folder          = imaps://user2ndacct@imap.gmail.com/
mailboxes           = +INBOX =[Gmail]/Drafts =[Gmail]/'Sent Mail' =[Gmail]/Spam =[Gmail]/Trash
set spoolfile       = +INBOX
folder-hook         imaps://user2ndacct@imap.gmail.com/ "\
    set folder      = imaps://user2ndacct@imap.gmail.com/ \
        spoolfile   = +INBOX \
        postponed   = +[Gmail]/Drafts \
        record      = +[Gmail]/'Sent Mail' \
        from        = 'Your Name <user2ndacct@gmail.com> ' \
        realname    = 'Your Name' \
        smtp_url    = smtps://user2ndacct@smtp.gmail.com \
        smtp_pass   = $my_gpass2"
       
# Setup for user3:
set folder          = imaps://user3rdacct@imap.gmail.com/
mailboxes           = +INBOX =[Gmail]/Drafts =[Gmail]/'Sent Mail' =[Gmail]/Spam =[Gmail]/Trash
set spoolfile       = +INBOX
folder-hook         imaps://user3rdacct@imap.gmail.com/ "\
    set folder      = imaps://user3rdacct@imap.gmail.com/ \
        spoolfile   = +INBOX \
        postponed   = +[Gmail]/Drafts \
        record      = +[Gmail]/'Sent Mail' \
        from        = 'Your Name <user3rdacct@gmail.com> ' \
        realname    = 'Your Name' \
        smtp_url    = smtps://user3rdacct@smtp.gmail.com \
        smtp_pass   = $my_gpass3"

#--------#
# Macros #
#--------#
macro index <F1> "y12<return><return>" # jump to mailbox number 12 (user1 inbox)
macro index <F2> "y6<return><return>"  # jump to mailbox number 6 (user2 inbox)
macro index <F3> "y18<return><return>"  # jump to mailbox number 18 (user3 inbox)
# Below requires added script separate from mutt for more info see the section on opening HTML files in a browser
macro index <F10> "<copy-message>/tmp/mutttmpbox\n<enter><shell-escape>~/bin/viewhtmlmail.py\n" "View HTML in browser"
macro pager <F10> "<copy-message>/tmp/mutttmpbox\n<enter><shell-escape>~/bin/viewhtmlmail.py\n" "View HTML in browser"
#-----------------------#
# Gmail-specific macros #
#-----------------------#
# to delete more than 1 message, just mark them with "t" key and then do "d" on them
macro index d ";s+[Gmail]/Trash<enter><enter>" "Move to Gmail's Trash"
# delete message, but from pager (opened email)
macro pager d "s+[Gmail]/Trash<enter><enter>"  "Move to Gmail's Trash"
# undelete messages
macro index u ";s+INBOX<enter><enter>"         "Move to Gmail's INBOX"
macro pager u "s+INBOX<enter><enter>"          "Move to Gmail's INBOX"

#-------------------------#
# Misc. optional settings #
#-------------------------#
# Check for mail every minute for current IMAP mailbox every 1 min
set timeout         = 60
# Check for new mail in ALL mailboxes every 2 min
set mail_check      = 120
# keep imap connection alive by polling intermittently (time in seconds)
set imap_keepalive  = 300
# allow mutt to open new imap connection automatically
unset imap_passive
# store message headers locally to speed things up
# (the ~/.mutt folder MUST exist! Arch does not create it by default)
set header_cache    = ~/.mutt/hcache
# sort mail by threads
set sort            = threads
# and sort threads by date
set sort_aux        = last-date-received

color normal white default
color hdrdefault brightcyan default
color signature green default
color attachment brightyellow default
color quoted green default
color quoted1 white default
color tilde blue default

The color stuff  at the end I haven't really played with so if anyone out there has a better idea let me know.  I read a couple of ways to do the password files btw but could not make sense of them.  I know this way works so there you have it.

HTML
If you would like to get some html abilities then create a file called .mailcap in the user's root like this:

Code Select Expand

nano ~/.mailcap
and enter this line:
   text/html; elinks -dump %s ; copiousoutput
You can substitute lynx if that is your preference. I will either add to this or create another post about html processing.  The mutt devs seem to have toyed with some things without my knowledge and my old python scripts aren't working at the moment.

Opening HTML files in a browser
Good news!  This function can be easily done thanks to Akkana and this post http://shallowsky.com/blog/tech/email/mutt-viewing-html-mail.html.  The bottom line is as follows.  Download the script from git hub here https://github.com/akkana/scripts/blob/master/viewhtmlmail Save it as viewhtmlmail.py in your ~/bin folder and make it executable. Then copy the following two lines and place them in the macro's section of your .muttrc

Code Select Expand

macro index <F10> "<copy-message>/tmp/mutttmpbox\n<enter><shell-escape>~/bin/viewhtmlmail.py\n" "View HTML in browser"

Code Select Expand

macro pager <F10> "<copy-message>/tmp/mutttmpbox\n<enter><shell-escape>~/bin/viewhtmlmail.py\n" "View HTML in browser"
make the file ~/bin/viewhtmlmail.py executable with chmod +x and then when you press F10 with an open html mail it will open in firefox/iceweasel.

Remove our temp file
Let's not forget the unsecured file we created earlier.  We can just wipe it away with

Code Select Expand

rm ~/.encrypt/.gmpass
Since I am thinking of it, there is no need to get nervy about wiping this out.  Should you later add or delete accounts writing a new file and re-running the command

Code Select Expand

gpg --encrypt --recipient 'Your Name' .gmpass will allow you to overwrite the gpg file.

Fire Up Mutt
When you start mutt, PGP will ask you for your passphrase.  It should also show you that it is checking the encrypted file and getting the info out after that.  Then, if all went well you should see mutt going online to get your email.  Oops, it's possible that your fist screen is empty or you don't see the mails for your online accounts?  Well that is because of the mailboxes.  If you followed my rc they will all be nicely tucked away in their own little homes.  Your machine and local stuff will come into the main box and the rest of this stuff will be accessible (most simply) by your macro's.  If you press F1 you will briefly see a list of accounts and it will open the first set of inboxes as designated by the macro.  It should then take you to that mailbox.  If you know mutt you will have no issues and if you don't just press the ? key where advised and start to explore.  It can do it all and mutt is in the true unix tradition of something that does it's thing and does it very very well.

Just to be thorough let's pull apart our macro:
      macro index <F1> "y12<return><return>" # jump to mailbox number 12 (user1 inbox)
What this is saying is that pressing F1 will take you to the 12th mailbox which will be the inbox for user1. If you look above at the boxes we created earlier and look at the way the folders are laid out in mutt it will be pretty clear how and why the macros are set to 6 and 12. Hint:you can use the y key and the c key in certain locations to prowl around the folders.

Local Mail
So where did our local mail go?  Well unless you set up boxes for it it will be in /var/mail/$USER.  I did monkey with this and found it ... "interesting" in other words I didn't like the results. I guess I have just become comfy with /var/mail/$USER.  I did just go ahead and leave it in the standard folder /var/mail/$USER. I use this box a lot for scripts and testing so I would rather lose my gmail than it.  This means that to get to it I have to use "C" and type in the path.  I personally have no issues with this but YMMV.

Glib-Object-CRITICAL -- UPDATE - RESOLVED
At lease for the moment there is a fix for this.  I no longer have this error after running the following as root:

Code Select Expand

apt-get install pinentry-curses pinentry-gtk2 pinentry-qt4 signing-party

You are most likely going to see the following:

Code Select Expand

(pinentry:22868): GLib-GObject-CRITICAL **: Object class GtkSecureEntry doesn't implement property 'editing-canceled' from interface 'GtkCellEditable'
This error from what I can understand was "fixed" in git a couple of years ago but still has not been applied.  Don't ask me I just work here .
This statement is still true. Git has pinentry 0.83 and Debian is still using pinentry 0.81.1 so much for Sid and cutting edge I have not yet found a work around and if anyone has any ideas let me know.  It seems to stem from pgp not finding or knowing where to look for pinentry-gtk2.  It is incidental if annoying and doesn't harm anything.  If I come up with a fix or anyone out there does I will add it.   

Summation
I hope someone finds this useful.  For me personally I have been doing my mail like this for some time and find it extremely effective.  I rarely use the browser for mail and that keeps my distractions to a minimum.  Well, not totally true I have tendency to be distracted by anything I find interesting, but you get my drift.  The real point here is once again, Vsido has "everything" that is needed to do what many would consider a somewhat sophisticated process and what I would consider essential.  Happy Vsido-ing.

[jedi]

@statmonkey; WOW, just wow!  You have put some truly incredible work into the info you post freely for us here.  This is why, IMO, that in the end, no other OS offers what Linux does.  Your contributions here have been incredible.  Please keep up the excellent work...

[VastOne]

Outstanding How To statmonkey, once again showing precise details and a step by step guide that is always effective

This is something I will check out... I want to get away from browser based email and this is the direction I was thinking.  I will keep you posted on how it goes

Thank you for this... 

[statmonkey]

Why thank you Jedi you made me blush.  Really not that much but if it is useful then I am happy.  I actually came back to edit it but will put the edit in this comment as well.  Writing that forced me to get off my arse and figure out the html issues I was having. So here is a bonus.

Opening HTML files in a browser
Good news!  This function can be easily done thanks to Akkana and this post http://shallowsky.com/blog/tech/email/mutt-viewing-html-mail.html.  The bottom line is as follows.  Download the script from git hub here https://github.com/akkana/scripts/blob/master/viewhtmlmail Save it as viewhtmlmail.py in your ~/bin folder and make it executable. Then copy the following two lines and place them in the macro's section of your .muttrc

Code Select Expand

macro index <F10> "<copy-message>/tmp/mutttmpbox\n<enter><shell-escape>~/bin/viewhtmlmail.py\n" "View HTML in browser"

Code Select Expand

macro pager <F10> "<copy-message>/tmp/mutttmpbox\n<enter><shell-escape>~/bin/viewhtmlmail.py\n" "View HTML in browser"
make the file ~/bin/viewhtmlmail.py executable with chmod +x and then when you press F10 with an open html mail it will open in firefox/iceweasel as a local file.  Pretty cool stuff.

[jedi]

Just set it up, and it works perfect.  Great for me, and I also am not to the "Tin Hat" stage, though with recent events being what they are, my email is no ones business but my own.  Any extra security today is great to have, and this was simple to set up following your great directions!!!  Thanks again for this!

[dizzie]

This is one example of why I love mutt so much

[statmonkey]

I will also update the original post but believe I have solved the glib error.  Apparently our original build misses some needed libs for pinentry but on site I found about enigmail they were doing essentially the same things and were not getting any errors with glib.  The only difference I could see was that they were installing enigmail, pinentry-curses pinentry-gtk2 pinentry-qt4 signing-party.  Since enigmail's function is equivalent to mutts I deduced that making sure that pinentry-curses pinentry-gtk2 pinentry-qt4 signing-party were installed would solve the issue.  Such was the case.  So if you want to get rid of the Glib-Object-CRITICAL error associated with the start of mutt with GPG just run:

Code Select Expand

apt-get install pinentry-curses pinentry-gtk2 pinentry-qt4 signing-party at least for me this ended the issues.

Additionally using the Akkana script with the latest firefox also causes a different Glib error.  This is caused by the apps api.  GLib-CRITICAL **: g_slice_set_config.  There is at present no fix for this but it is harmless ... well no fix involving FF if you want to use another browser adjust viewhtml.py accordingly. I just changed it to chromium and these errors also went away.