Effective methods to protect email addresses from spammers

Publishing an email address on a web page invites more spam. Protect your address by masking it from the email harvesters (spambots) used by spammers. This article tests 50 masking methods against 23 harvesters to see which methods work to stop spammers, and which do not.

Spam, spam, spam

A 2002 U.S. Federal Trade Commission (FTC) study, Email Address Harvesting: How Spammers Reap What You Sow (PDF), posted test email addresses around the Internet and measured the amount of spam they received. 86% of all email addresses published on web pages received spam (PDF chart). One test address received spam just nine minutes after the address was first published. So, what can you do to publish your address, but keep spammers from finding it?

The FTC recommends masking your address to make it harder to find by the automated email harvesters (“spam robots” or “spambots”) used by spammers. But there have been dozens of methods proposed to mask an address. This article tests 50 of them. Email addresses masked by each method are run past a collection of harvesters to see which methods work, and which do not. The results may surprise you. Many popular email masking methods don’t work.

How to protect email addresses

To better cover all of these masking methods, I've grouped similar methods together and review each group in its own article. Each article explains the method, shows examples, and tabulates harvester test results.

• Stop spammer email harvesters by obfuscating email addresses. Mask your email address by translating it into numeric ASCII character codes or by reversing the letters in the address. Browsers unmask the obfuscated email address for site visitors, but harvesters are often left confused.
• Stop spammer email harvesters by fragmenting email addresses. Protect your email address by splitting it into separate pieces. Use multiple lines of text, replace “@” with “at”, or embed HTML tags. The pieces are easy for a visitor to re-assemble, but harder for an email harvester to find.
• Stop spammer email harvesters by inserting addresses with JavaScript or CSS. Remove your email address from the original text of a web page and insert it later after the page is loaded by using JavaScript or CSS. The protected address is visible to site visitors, but very hard for email harvesters to find.
• Stop spammer email harvesters by drawing addresses with images or Flash. Protect your email address from an email harvester’s text scanning by drawing the address instead in an image or Flash animation. Visitors can still see the email address, but spambots cannot.
• Stop spammer email harvesters by hiding web pages from the harvesters. Instead of masking individual addresses, hide the entire web page containing your email address. Harvesters can’t find the page, but your site’s visitors still can.
• Stop spammer email harvesters by blocking spammer access to the site. Block access to your web site for known spammers and the email harvesters they use, while letting legitimate visitors through without a problem.
• Stop spammer email harvesters by using a contact form. Don’t publish email addresses at all. Instead provide visitors with a contact form for sending you email. Add a CAPTCHA image or math challenge to let human visitors through while stopping the automated harvesters used by spammers.

Results

The table below summarizes the email address protection methods reviewed in these articles. Each method is graded on its effectiveness, browser support, usability, and accessibility for the disabled. A method is effective only if none of the tested email harvesters bypassed it. A method is well-supported only if all major current web browsers support it well.  And a method is usable and accessible if it is not awkward for visitors to use and it is readable by screen readers used by the visually impaired.

About two thirds of the email address protection methods tested are not effective at stopping spammers. Surprisingly, many popular methods are not effective, including:

• Obfuscating an email address with ASCII character codes.
• Replacing “@” with “at”, “(at)”, or “[at]” in an email address.
  
• Embedding HTML tags within an email address.
• Using “robots.txt”, meta tags, and URL “nofollow” attributes to stop web spiders from visiting site pages.

Overall, most email address protection methods that are effective also have poor usability and accessibility. Tricky methods sometimes don’t work in all browsers. Contact forms, login pages, Flash animations, and CAPTCHA challenges will annoy some visitors. If you obsess too much about protecting your email address from spammers, you’ll also block or annoy legitimate visitors. Spam may be the price you pay for maintaining open communications lines for your web site’s visitors.

Out of all of these methods, one method stands out as being effective, usable, accessible, functional in all web browsers, JavaScript-free, plugin-free, and easy to author and maintain:

Split the email address onto two lines

like this:

User: person
Domain: example.com

However, keep in mind that while an automated harvester may be stopped by this and some of the other methods, a human harvester will get them all. No matter how well you protect your address, you’ll probably still get some spam. Use a spam filter for your email program. A 2005 U.S. Federal Trade Commission study, Email Address Harvesting and the Effectiveness of Anti-Spam Filters (PDF), found that 95% of spam could be stopped by a spam filter.

What else can you do?

There are several more methods that I didn’t try because they are complex to set up or they won’t work (yet) in most web browsers:

• Use a disposable email address. Publish your email address as plain text and don’t worry about protecting it. When it starts getting too much spam, delete the old email address and make a new one. This can be a hassle to manage, so there are many web services available that provide disposable email addresses. The free flow-to.com service has an interesting twist. Instead of posting your email address to a web page, you post a special link to their site. The site responds with a generated one-time-use email address that, when mailed to within 24 hours, will forward the email to you. If harvested, the generated address is unlikely to be valid by the time it gets used.
• Draw an email address using Scalable Vector Graphics (SVG). SVG is a formatting language that describes 2D drawings containing lines, areas, and labels. A drawing could include an email address text label or an address drawn with lines. While Adobe has an SVG Viewer plugin, the goal is to build SVG support directly into web browsers. SVG is partially supported by some current web browsers, such as Firefox and Opera, but not yet by Safari or Internet Explorer.
• Draw an email address using Microsoft’s Silverlight plugin. This is Microsoft’s new Flash-like plugin to draw shapes on a web page. Like Adobe’s Flash, it can be used to draw an email address. As of this writing, the plugin has just been announced in a beta release. It may take years for it to become widely available.
• Draw an email address in a movie shown by Apple’s QuickTime, Real’s RealPlayer, or Microsoft’s Windows Media Player plugins. Most visitors probably have one of these movie players installed. You could create an MPEG4, AVI, QT, or WMA movie containing your email address and play it on a web page. However, the approach is pretty awkward and it will slow down page loads while the browser waits to get the movie from your web server.
• Draw an email address using Java. The Java programming language is an excellent way to create interactive applications, including those started from a Web page. However, there are technical complexities in making Java work for all web browsers and Java is way overkill for protecting a single email address.
• Draw an email address using PDF. This format is widely used for posting fully-formatted documents on the web. Adobe’s Acrobat Reader or Apple’s Preview can show these documents. While a PDF document is an awkward way to protect one email address, it would work well to protect a list of email addresses for a company contact list.
• Draw an email address using other plugins. There are many more plugins for web browsers. You could use Elsevier’s MDL plugin and draw an email address as a chemical formula. Or use a VRML or X3D plugin to draw your email address in flashy 3D. Or embed an email address in blueprints using Autocad’s DXF format, if visitors have a plugin to show it. In certain markets, these plugins are common. But on the web at-large, visitors are unlikely to have or want these plugins.
• Be obscure. Never use a popular method of protecting your email address. If a method gets too popular, spammers will implement a way around it in their email harvesters. If everybody starts using images for email addresses, spammers will add optical character recognition (OCR) to extract them. If everybody uses JavaScript schemes, spammers will add JavaScript support. So, don’t follow the crowd.

What email harvesters did I test?

Sorry, I won’t publish (or email you) the names of the email harvesters I tested. I don't want their product names on my web pages, and the unwanted search engine attention that that would bring. And I don't want their developers using my testing to show how well their products work compared to their competitors.

If you really want to do your own tests, search the web with the obvious keywords.

Beware of doing your own testing

As I collected and tested these email harvesters, I observed a few things:

• One email harvester came with a virus embedded. Installation of two harvesters was blocked by Windows when they tried to access protected memory. One email harvester’s installation tried to add an unexplained background service to Windows.
• Every email harvester must access the Internet. Are they just harvesting, or is there anything else that they are sending to and from your computer? One email harvester had a suspiciously high CPU and bandwidth usage while it was idle.
• Several email harvesters can scan a hard drive too. You can aim them at your web browser cache. Or consider how others might use them at an Internet cafe, library, or other public computer site. Or as a virus payload on your computer.
• Several email harvesters could be installed as invisible Internet Explorer plugins to scan web pages as they are browsed. Consider how that might be used at an Internet cafe.
  
• Several email harvester makers also sell Internet cafe management products. How convenient.
  
• One email harvester maker also makes a “corporate monitoring” product that does keystroke logging, provides hidden remote access to a PC’s files, and enables remote controls to start applications such that they won’t show up in the Windows task manager. That’d be a handy tool for controlling spammer zombies.
  
• There are dozens of bulk emailers for sending “newsletters” to a mailing list. Some can be started under remote control without anything showing up on the computer’s screen.
• One application at a bulk email software site helpfully emails you whenever a PC's dynamic IP address changes. That’d be handy for keeping track of spammer zombies.
• Several email harvester developers actually have “anti-spam” policies on their web site. We’re assured that they are “strictly opposed” to spam.
• My favorite harvester/mailer company tag line: “We make a better world.”

If you really must try these yourself, be careful.

Future email harvesting technology

Spammers use the same type of computer as you or I. And each year, the latest computer on the market is almost twice as fast as the year before. When a spammer upgrades their computer, they can use the faster processor to do more sophisticated text scanning and to do a better job of defeating email masking schemes. What could they do in the near future?

Text scanning has two phases: lexical analysis and syntax parsing, Lexical analysis looks for patterns in a sequence of characters, and syntax parsing looks for patterns in those patterns. Lexical analysis is fast, but syntax parsing is slower. For maximum harvesting speed, most current email harvesters use simple lexical analysis and no syntax parsing. They recognize obvious patterns of characters, but they do not look at the context around those characters. With faster computers, spammers can use smarter software to better extract email addresses. When they do, many of today’s email address protection methods will become ineffective.

For example, a regular expression used by a lexical analyzer can easily find an email address like “person@example.com.” Here’s the expression:

/[A-Z0-9_\-\.]+@[A-Z0-9_\-\.]+/gi

For most people this looks like jumbled nonsense, but to a programmer and a regular expression parser, this says “one or more letters or numbers, an @, and one or more letters or numbers”. You can test this yourself, and learn more about regular expressions, by using Rob Locher’s nice Regular Expression Tester.

Email harvesters are already doing this. It is very easy to extend this to look for “person AT example.com,” a masking technique recommended by the U.S. Federal Trade Commission back in 2005. Here’s the expression:

/[A-Z0-9_\-\.]+ *(@|AT) *[A-Z0-9_\-\.]+/gi

Email harvesters are already doing this too, so I’m not giving anything away here.

It is pretty easy to extend this to match any predictable pattern for representing an email address. All of the following can be recognized in a similar way:

person@example.com
person @ example.com
person at example.com
person(at)example.com
person[at]example.com
person@example.com
person@example.com
person%64example.com
person<!--comment-->@example.com
person<tag></tag>@example.com
person<tag>@</tag>example.com
<tag>person@</tag>example.com

Some email harvesters are already doing this. Slightly more sophisticated lexical analysis can recognize email addresses that use HTML, URL, or JavaScript character codes. And harvesters are doing this now too.

All of this can be done with fast simple lexical analysis. When you add in syntax parsing, you can find more complex patterns where the name and address are more spread out. For instance, syntax parsing can recognize the method I recommended earlier:

User:  person
Domain:  example.com

Syntax parsing also can recognize this:

A parser can easily extract rows and columns from a table. Some email harvesters can do this now.

Any predictable pattern can be parsed. It just takes a bit more computer time. And computer time is cheap and getting cheaper. The email address split on to two lines above is only safe today because it isn’t a common enough pattern yet for spammers to have bothered adding it to their harvesters.

Any email masking method based upon a predictable pattern is unsafe. Every numeric character code-based obfuscation method is unsafe. Every method that predictably spells out punctuation or adds spaces or HTML tags is unsafe. Every method that uses a well-known algorithm to scramble or encrypt the letters is unsafe. Every method that predictably spreads out an address onto multiple lines or into table cells is unsafe. If it is predictable, it is parsable by a future email harvester.

What about JavaScript? The JavaScript language was designed to be simple and fast to execute. It is possible to integrate it into an email harvester. The harvester will run slower, but computer time is cheap. So far, none of the harvesters I tested ran web page JavaScript. I expect this to change within the next few years.

Some email harvesters skip writing their own HTML parsers and just plug in to Internet Explorer. Current harvesters grab page text as it is loaded. Future harvesters may grab the text after JavaScript's “onload” functions have run (the way screen readers for the visually impaired do). When they make this change, JavaScript schemes that insert email addresses on page loads will become unsafe.

Unfortunately, almost every method used to mask email addresses also diminishes the usability and accessibility of the web. A real fix to the spam problem must involve stopping the spammers themselves, whether by legal or technical changes to the Internet.

Further reading

Studies

• Email Address Harvesting: How Spammers Reap What You Sow. This 2002 study by the U.S. Federal Trade Commission (FTC) posted 250 email addresses on web pages, message boards, chat rooms, news groups, dating services, and others, then measured the quantity of spam received at each address. After six weeks, 86% of addresses posted to web pages received spam. One email address posted to a chat room received spam within nine minutes. The type of spam message received was not related to the type of site on which the address was posted — addresses posted to a children’s news group still received spam for adult web sites and illegal drugs. The FTC recommended masking addresses by replacing “@” with “at”, however my harvester tests above show that this method is no longer effective.
• “Remove Me” Surf Results (PDF). This brief 2002 study by the U.S. Federal Trade Commission (FTC) looked at spam messages that included a link to “remove me” from the email list. 63% of those links did not function.
• Why Am I Getting All This Spam (PDF). This 2002 study by the Center for Democracy and Technology posted test email addresses to web pages, news groups, and discussion boards. 97% of the spam received came through email addresses posted on web pages. The study also posted email addresses in three forms: a plain text form, fragmented by replacing “@” with “at”, and obfuscated by using numeric ASCII character codes. All of the plain text email addresses received spam, but none of the fragmented or obfuscated email addresses received any spam. However, my email harvester testing shows that these methods used in 2002 are no longer effective today.
  
• False Claims in Spam (PDF). This 2003 study by the U.S. Federal Trade Commission (FTC) looked at the truth or falsehood of claims made in spam. 33% of the spam had a false “From” address, 22% had a false “Subject” line, and 40% had false “Message” text. Overall, 66% of all spam had false information.
  
• Email Address Harvesting and the Effectiveness of Anti-Spam Filters (PDF). This 2005 study by the U.S. Federal Trade Commission (FTC) tried to assess the effectiveness of the 2003 CAN-SPAM act that made email harvesting a crime. Again, the FTC posted plain text email addresses on web pages, message boards, blogs, chat rooms, and news groups, then measured the quantity of spam received. After five weeks, 99% of the spam received came through the email addresses posted to web pages, which meant that email harvesting was going on despite now being illegal. Masking email addresses by replacing “@” with “at” reduced spam from 6,416 messages to just one spam message. However, my email harvester testing shows that this masking method is no longer effective today. The study also looked at using spam filters in email programs and found that they blocked up to 95% of all spam.
• EMail Address Protection Study (PDF). In 2005, Damien Giry and Michael Neve at the Université Catholique de Louvain in Belgium reported on their study of email address protection methods. They posted email addresses at various depths within a site, at different locations on a web page (meta tag, head, and body), and as plain text and masked email addresses using several of the methods I discuss. They found that email addresses received spam even if they were placed deeper into a web site, whether placed in the head or body of a page, and whether or not they were masked. Email harvesters weren’t picky — they scanned anything and everything for email addresses. Addresses masked using embedded HTML tags or obfuscated with ASCII character codes still received spam, which is consistent with my tests.
• Automatic Email Munger. Daniele Raffo reports on an ongoing informal multi-year study that compares spam levels for two posted email addresses: one in plain text (see its message history) and one obfuscated using numeric ASCII character codes (see its message history). While the obfuscated email address received little or no spam in 2005, by mid-2006 it had begun to receive sporadic spam. This is consistent with my tests above which found that email harvesters can now recognize obfuscated email addresses.

Articles

• How do spammers harvest email addresses? Uri Raz provides a good list of ways in which spammers gather email addresses. Harvesting addresses from web pages is just one way.
• JavaScript Email Cloaking. Shawn Hall discusses JavaScript-based masking methods, and shows several examples of flawed masking methods. Flawed methods include obfuscation using ASCII character codes, replacing “@” with “at” or an image, or adding “nospam” to an address. I agree.
• Methods to hide email addresses from page source. Sarven Capadisli has a nice review of methods to mask email addresses, all of which are also discussed in the above articles. The site’s user comments added to his article are worth browsing to see what people are using, and what others say about the effectiveness of those methods.
• Protecting Your Email Address. William and Mari Bontrager discuss several email address protection approaches they’ve tried, and what has and has not worked. They advocate their flow-to.com disposable email address service mentioned earlier.
• Protecting your website’s email addresses from being used by spammers. Aron Roberts from the University of California, Berkeley, discusses the methods, benefits, and drawbacks of several email masking methods. I tested all but one of the methods he discusses: adding legal but unusual elements to an address URL in a “mailto” link. However, this method has very poor usability and accessibility, plus (as he notes) several browsers and email programs do not handle these unusual email addresses correctly.
• The Spam Experiment: Where do we get spam from? In 2002, Phil Bradley did an informal study in which he posted test email addresses at various sites and measured the amount of spam that he received. While he received spam through email addresses posted on web pages and in news groups, the most spam came to an email address he used when filling out an opt-out form.
• M.I.T. Spam Study Finds Instant Wealth, Sexy Coeds Just a Click Away! To offset this depressing look at spamming, check out the encouraging results in this important, um, “study.” :-)