Ever wonder why most e-mail clients hide images by default? The reason for the "display images" button is because images in an e-mail must be loaded from a third-party server. For promotional e-mails and spam, usually this server is operated by the entity that sent the e-mail. So when you load these images, you aren't just receiving an image—you're also sending a ton of data about yourself to the e-mail marketer.
Loading images from these promotional e-mails reveals a lot about you. Marketers get a rough idea of your location via your IP address. They can see the HTTP referrer, meaning the URL of the page that requested the image. With the referral data, marketers can see not only what client you are using (desktop app, Web, mobile, etc.) but also what folder you were viewing the e-mail in. For instance, if you had a Gmail folder named "Ars Technica" and loaded e-mail images, the referral URL would be "https://mail.google.com/mail/u/0/#label/Ars+Technica"—the folder is right there in the URL. The same goes for the inbox, spam, and any other location. It's even possible to uniquely identify each e-mail, so marketers can tell which e-mail address requested the images—they know that you've read the e-mail. And if it was spam, this will often earn you more spam since the spammers can tell you've read their last e-mail.
Update: email marketers claim they can get Gmail folder information, but several readers correctly pointed out that common browsers like Chrome and Firefox do not, in fact, send that information themselves as part of the referrer.
But Google has just announced a move that will shut most of these tactics down: it will cache all images for Gmail users. Embedded images will now be saved by Google, and the e-mail content will be modified to display those images from Google's cache, instead of from a third-party server. E-mail marketers will no longer be able to get any information from images—they will see a single request from Google, which will then be used to send the image out to all Gmail users. Unless you click on a link, marketers will have no idea the e-mail has been seen. While this means improved privacy from e-mail marketers, Google will now be digging deeper than ever into your e-mails and literally modifying the contents. If you were worried about e-mail scanning, this may take things a step further. However, if you don't like the idea of cached images, you can turn it off in the settings.
This move will allow Google to automatically display images, killing the "display all images" button in Gmail. Google servers should also be faster than the usual third-party image host. Hosting all images sent to all Gmail users sounds like a huge bandwidth and storage undertaking, but if anyone can do it, it's Google. The new image handling will rollout to desktop users today, and it should hit mobile apps sometime in early 2014.
There's also a bonus side effect for Google: e-mail marketing is advertising. Google exists because of advertising dollars, but they don't do e-mail marketing. They've just made a competitive form of advertising much less appealing and informative to advertisers. No doubt Google hopes this move pushes marketers to spend less on e-mail and more on Adsense.