Gmail and IMAP Folders

* Warning this is a long post as I am trying to cover this topic in enough detail to make sense .. and I am not sure if it all will unless you read it all the way through. *

I finally switched over to use Mac Mail for access to my Gmail account and for the most part I like using Gmail this way but there are some very strange ways in how Google has made their IMAP integration work.  I use quite a few “labels” to keep things organized (I am a folder junky!) and when I setup Mac Mail to access Gmail via IMAP it created each label as a “folder” in my IMAP client.  This is great because now I have the same organization capability in my email client as I do on the Gmail interface.  But here is where things get interesting.  Also I need to point out that these quirks are somewhat specific to using Mac Mail as the IMAP client to Gmail since it does not allow you to select which IMAP folders to download locally – it’s all or none which is …… ahh don’t get me started !

The way that Gmail labels and IMAP folders work is this:

• Each label translates to an IMAP folder
• In Gmail if you add a label to a message it will appear in that IMAP folder
• If you add two labels to a message it will appear in TWO IMAP folders and so on
• In your IMAP client if you delete a message from an IMAP folder, on Gmail it simply removes that label – it does not delete the message
• In Gmail there is a label called “All Mail” – this is where Gmail stores a reference to any and every mail message in your account (received and sent)
• In your IMAP client there is a “All Mail” IMAP folder which will have a “copy” of every message in your Gmail account.

So now with that information and some other tests I did I found out that there is at least 2 copies of every message I receive when using my Mac Mail IMAP client.  That is when I get new messages using Mac Mail it puts the message in the “InBox” and then there is also a copy put into the “All Mail” folder.

Did I mention you cannot tell Mac Mail to NOT download the All Mail IMAP folder ?  Actually if someone reads this and knows how .. send me an email .. please

Now expand that and lets say you have some Gmail Filters configured to automatically add labels to messages as they come in.  If you add one label to a message when it is received and then download that message with your Mac Mail client you will now have 3 copies of the email on your system. One in your InBox, one in All Mail and another in the IMAP folder for that label.  WOW do you see where this is going ?  What did I do ?  Well I removed all my automatic filters now that I am using Mac Mail and will file these messages into the folders as I need to.

Now that I know this information I started to wonder what happens with messages I send/ delete from my IMAP client.  I did a simple test but before I show that here is my IMAP client configuration for mailbox behaviors.

Here you can see that I am telling Mac Mail to store nothing on the server when I send a message, create drafts or deleting a message.

The test I ran was to simply send myself an email from another account and then to send an email from my Gmail account to another account.   Here is how this looks in Mac Mail when I sent an email to someone else and then did a search for all messages with the subject of TEST ONE across all mailboxes.  3 copies !

Sent – Gmail and All Mail are on the Gmail servers with the respective filters. Sent Mail is the copy of the sent message on my local system. Actually these are all local on my system but the Sent Mail is not on Gmail.

There is no way to store it in fewer locations on Mac Mail (did I mention that .. oh yeah I did) since the way that the Gmail backend works, when you send a message it automatically puts a Sent label on it and then also an All Mail label.  You don’t see these labels though in Gmail since they seem to be hidden when you use the web interface but they correspond to the selections below in Gmail:

You will see each of these (except for Chats) as IMAP folders in your email client. ** DON’T DELETE THESE FROM IMAP ** I have no idea what would happen or even if it would allow me to but I am not going to take a chance .. maybe when I get time to setup a dummy account and play with it more

Deleting Messages

Deleting email actually works pretty good if you use the configuration that I have above.  Do not store deleted messages on the server and move them to the Trash folder.  This way you will still have a copy of the message locally in case you need to recover it but you are not pushing these email messages back and forth between you IMAP client and Gmail.  The key is that this Trash folder is not the one on Gmail but is stored on your local hard drive.  In the picture below the Trash folder highlighted is where deleted messages will go when using your IMAP client.

The Trash folder at the bottow under [Gmail] is where deleted messages will go if you are using your Gmail web interface.

Now as we mentioned before there is that pesky little All Mail hidden label that is still going to cause a problem. In your IMAP client when you delete an email from the InBox it is going to basically have the InBox label removed from it on the Gmail server but it will still reside in the All Mail folder on your IMAP client and in Gmail.

Summary

So how can you get around this ?  You can’t.  How can you manage this ?  Well if you decide to use only your IMAP client as your primary means of interacting with Gmail then you can delete all the messages from the [Gmail / All Mail] IMAP folder on your IMAP client.  This won’t delete them from your other IMAP folders, your InBox folder or Gmail server but will remove it only from the All Mail IMAP folder locally only and save some space.  Now this is not the same behaviour in Gmail.  If you delete a message from the All Mail folder there it WILL delete the message completely.  The reason is that in Gmail they actually store only one copy of the message but make it visible in many different ways using the labels.

So what is the jist of this loooong blog entry ?

• If your going to use an IMAP client use it exclusively
• Don’t store stuff on the server such as drafts / sent or delete messages unless you work from more than one computer and need to move back and forth.  If you have a desktop at the office and one at home and want to use IMAP in both places then this would be when you DO want to store these items on the server.
• If you use an IMAP client from one computer you should use an IMAP client from all computers
• Delete emails from your Sent folder on a regular basis (remember there are two sent folders – one on Gmail and one on you local system)
• Delete messages from the All Mail IMAP folder in your IMAP client ONLY – Do not do this in Gmail otherwise you will wipe out all your email.
• Use an IMAP client that allows more control over which IMAP folders to download locally like Thunderbird.  Then you can just never download the All Mail folder and it is not an issue.

Okay now I have read through this and hope that it makes sense.  I am really thinking about moving back to Thunderbird but I like all the integration that Mac Mail has with everything on the Mac.  Note to Thunderbird developers – copy the good things Mac does in Mail and ignore the stupid things they do and you will have a KILLER app.