header

Torsten Curdt’s weblog

Backing up Gmail

For a while I have been trying to use Gmail as my primary email client. Online searching, no worries about the backup, tagging …but still it just doesn’t cut it – at least for me. I know people are happy with it – but I feel a bit stuck. No progress. The improvements they made so far just do not address the real problems IMO.

  • You cannot filter on special headers like “Precedence” and “Delivered-To”
  • You cannot filter on the virtual mailbox (myaccount+virtualmailbox@gmail.com)
  • When sending mails from a different email address the header still contains the gmail address
  • Tagging (aka labeling) and spam reporting does not work from your desktop email clients
  • There is no IMAP support
  • No offline mails …or what do you do on a plane or train?

Besides technical reasons why implementing e.g. IMAP on top of Gmail is probably not that easy I also suspect business reasons for not improving the desktop client experience of Gmail users. Who would see the ads then? The funny thing is – I would even pay for a proper email setup with IMAP and tagging and so on. The providers I found so far did not fill this gap for me.

So because all these things I decided to move my mail back to my laptop and only use Gmail for backup purposes. So first I had to get my current mails downloaded. …and I thought that would be an easy one. This post pointed me to mpop which seems to be perfectly suited for the job. Unfortunately it is not available through fink yet. But a simple standard install works fine.

./configure --prefix=/sw
make
sudo make install

The configuration is easy

# defaults
defaults
tls on

# gmail
account gmail
host pop.gmail.com
user myaccount
password myaccountpassword
keep on
only_new off
tls_starttls off
delivery mbox ~/Backup/myaccount@gmail.mbox

# Set a default account
account default : gmail

Now if you start it up it works fine. Of course given that you have enable the pop3 access for all mails in the Gmail settings. For some reason Google hands out my 4000 mails in batches of 300-600 mails. So you have to run mpop over and over again. Unfortunately after re-running mpop a couple of times I always hit the following problem.

...
retrieving message 347 of 605 (4.69 KB): 100%
retrieving message 348 of 605 (3.48 MB): 100%
retrieving message 349 of 605 (7.60 MB): 100%
mpop: command RETR 350 failed
mpop: POP3 server message: -ERR [SYS/TEMP] Temporary system problem. Please try again later.
mpop: error during mail retrieval (account default from .mpop.conf)

I am not absolutely sure whether in the end it was really just a temporary problem or not (I have seen this over a couple of days). Maybe the too frequent logins were a problem – no idea. In the end looping with a delay did the trick for me

#!/bin/sh
while [ false ] ; do
  echo "retrieving..."
  mpop -C ~/.mpop.conf
  sleep 300
done

Now comes the really fishy part. My Gmail account lists around 4000 mails but the downloaded mbox file only contains about 3700 mails. WTF? Lock in alarm – no thanks!

  • otuk
    tcurdt thank you
  • niko
    help me pls
    mpop: error during mail retrieval (account default from /home/yes/.mpoprc)
  • @otuk: read what it says!
    chmod 600 /home/otuk/.mpoprc
  • otuk
    mpop: /home/otuk/.mpoprc: must have no more than user read/write permissions
    help me please
  • I use gmail as primary e-mail for longer than I can remember and it works fine for me. No problems, although I make an occasional trip to the spambox to retrieve a false positive. But then, I read a book/magazine or talk to people in the train or plane. I also do not keep thousands of mail to re-read when I'm bored :D.

    I think the error is the result of a mechanism made to prevent people harming the system by putting on too much load.
  • Hi TC! (sorry for not writing in English! My post isn't that important :)

    Also ich bin ja immer noch total begeistert von Gmail. Gut, es dauert (gerade auf langsamen Verbindungen) echt immer ne Weile, bis die Oberfläche da ist, aber dann kann man sogar die Emails (offline) lesen und be-browsen, die man schon im Cache hat (kann natürlich auch ein Sicherheitsrisiko sein, aber mich hat's schon ein, zweimal gerettet).

    Was du mit dem Problem der Offlinemails meinst, weiß ich nicht. Man kann sie doch abrufen und dann offline seine Mails schreiben, die eigene gmail-Adresse in den BCC packen und dann gehen sogar die Threads nicht verloren durch das offline-Getippe. Oder was raffe ich hier nicht? ;)

    Die anderen Punkte (besonders den mit dem Mailen von den anderen Emailadressen!) sind echt nich so dolle und auch ich warte auf einige von ihnen schon seit geraumer Zeit...

    Einreichen, denen mal mit einem Zaunpfahl winken und hoffen, dass es besser wird! :)
    Gruß,
    Musch
  • Tim
    I agree with. I also use GMail only for backup and checking online all of my "other" email accounts on one website.
  • I've been very interested in email for quite a while now (which is why I'm not so active in the Commons these days). A few incoherent comments:

    Filtering is just a poor mans tagging. Associating meta-data to emails is much more powerful. I don't understand why Google takes such a limited view of labeling. I want to run rules and tagging engines against the full range of data in the email.

    IMAP suck. IMAP really sucks. It is a truly dreadful protocol. IMAP scales very badly. It is not the answer. A RESTful HTTP based protocol which builds on WebDAV is the way forward.

    (I think that I should stop boring you with my personal obsessions now...)
blog comments powered by Disqus