Spammers are an odd bunch. I assume they were just probing to see if my email address was valid when I received this creative message a minute ago:
Subject: xaiedu
Fifteen birds in five firtrees, their feathers were fanned in a
fiery breeze!
But, funny little birds, they had no wings!
Today I spent some time compiling a list of all of the RFCs that our email hosting system supports. See any I might have missed?…
1939 - POP3
2449 - POP3 Extension Mechanism
3501 - IMAP4rev1
2683 - IMAP4 Implementation Recommendations
2180 - IMAP4 Multi-Accessed Mailbox Practice
2088 - IMAP4 non-synchronizing literals (LITERAL+)
2177 - IMAP4 IDLE command
2342 - IMAP4 Namespace
2087 - IMAP4 QUOTA Extension
3348 - IMAP4 Child Mailbox Extension
3502 - IMAP4 MULTIAPPEND Extension
3691 - IMAP4 UNSELECT command
2221 - IMAP4 Login Referrals
draft - IMAP4 Extension for SASL Initial Client Response
2821 - SMTP (updated rfc821)
3463 - SMTP Enhanced Mail System Status Codes
3464 - SMTP Delivery Status Notifications
2554 - SMTP Extension for Authentication
2920 - SMTP Extension for Command Pipelining
3207 - SMTP Extension for TLS
1870 - SMTP Extension for Message Size Declaration
1652 - SMTP Extension for 8bit-MIMEtransport
2476 - SMTP Message Submission (port 587)
1945 - Hypertext Transfer Protocol -- HTTP/1.0
2616 - Hypertext Transfer Protocol -- HTTP/1.1
2822 - Internet Message Format (updated rfc822)
2045..2049 - Multipurpose Internet Mail Extensions (MIME)
2183 - Content-Disposition MIME header
1894 - Message Format for Delivery Status Notifications
2298 - Message Format for Message Disposition Notifications
3503 - Message Disposition Notification profile for IMAP
3462 - Multipart/Report Content Type
3834 - Automatic Responses to Electronic Mail
2246 - The TLS Protocol Version 1.0
2595 - Using TLS with IMAP and POP3
2142 - Required abuse@ Mailbox Name
4408 - Sender Policy Framework (SPF) Version 1
An opinion held by some anti-spam experts is that "Email is Broken". These folks want to fix the spam problem by completely overhauling the Internet’s email systems and replace SMTP with new protocols that will not allow spammers to pollute the inbox. They compare email to protocols such as Usenet NNTP, which was abandoned by the average Internet user years ago because it lacked simplicity and was not a very powerful way to share information and files, when compared to other web tools such as discussion boards, BitTorrent and blogs. Or FTP which was abandoned by tech-savvy folks who know it’s security risks.
Replacing today’s email protocols with something new would be a great solution if email was indeed broken… But it is not.
Email’s success is based on the fact that it is open and simple. Everybody can understand it, so everybody uses it, and that is what makes it so powerful. Replacing the simplicity of today’s email protocols with something more secure yet more complicated, will break the very thing that makes email great.
Keep that in mind when you are building your killer anti-spam solution. We are.
Check it out… our first Webmail.us movie
Many sequels to come.
I posted an answer to this question on Monday on the Webmail blog. However, something is broken with the feedburner RSS feed on our site.
Anti-spam has been one of my areas of focus for several years, and something that I have enjoyed. Recently though, spam fighting has gotten pushed to the side with all of the other things that I am responsible for. It is painful to watch spam slip by the filters and I am definitely excited to have Mike T working on this project.
The first thing Mike is going to do is take our existing whitelist/blacklist functionality and push it up to the Postfix level, rather than the amavisd + spamassassin level. This will ensure that system-wide rules don’t override individual user or domain rules. The second thing he is going to do is give our customers the ability to whitelist/blacklist IP addresses, in addition to just sender email addresses. Next, Mike will be building a system that makes "whitelist/blacklist/greylist/unknown" decisions based on aggregate whitelist/blacklist data, sending history, mail volumes and third party sender reputation databases. This will be some really powerful stuff. After that, he will do some work on the content filters behind this sender reputation system, possibly incorporating DSPAM which learns from the spam that each individual receives.
I mentioned back in February that we are switching our POP3/IMAP proxy software from Perdition to Dovecot. These proxy servers are still in beta, because there is a bug with how Dovecot handles SSL connections. Timo (Dovecot’s author) attempted a fix a few weeks ago, but the fix introduced new problems, so we reverted back. I am hoping to get the final bugs resolved within the next couple of weeks so that we can release this out of beta.
In the mean time, we have been hard at work upgrading our backend IMAP software – also to Dovecot. Currently we run Courier-IMAP, but Courier does not handle large mail folders efficiently. Webmail, unlike desktop clients, does not have its own cache, so it relies on the IMAP server to obtain header listings and to perform sorts and searches. Courier lacks indexes that would make these operations fast. Instead it must open every message file and parse out the header information in order to return a sorted list of emails back to webmail. Dovecot on the other hand, makes heavy use of indexes. The indexes allow a folder with 10,000 messages to be sorted in less than a second, whereas Courier would take 30-60 seconds or even longer, and usually cause a timeout. The speed difference is amazing.
In order to make the switch seamless, we have configured Dovecot to run in parallel with Courier on the existing mailbox servers. We patched Dovecot to utilize Courier’s folder subscription and message UID lists, so that both systems can utilize the same maildirs. If you are interested in these patches, shoot me an email and I will send them to you.
Webmail will be the first application to switch to Dovecot. We began rolling this out server-by-server on Wednesday. We are taking our time with this rollout since it is such a big change, just in case any unforeseen problems arise. So far the issues have been minor, and easily corrected.
Once the beta proxy is bug-free, we will start migrating the front-end POP3 and IMAP systems to Dovecot. Needless to say, we are making a big commitment to this Dovecot thing. It will be at the core of what’s to come.
Phil Wainewright wrote an article on his Software as Services blog Monday on a vary familiar topic – the ironies of Software as a Service companies who operate their own data centers. I quote:
"One of the ironies of the on-demand application space is that many of the leading names operate their own data centers. This seems somewhat illogical, if not downright hypocritical. On the one hand, they ask their customers to rely on a third party to provide mission-critical business applications. But instead of doing likewise with their own infrastructure, they host in-house."
This is has been a topic of discussion at Webmail for years… Would it be cheaper buy servers and put them in a collo facility and manage them ourselves? Or could we build out our own data center here in the VT CRC? Or should we lease servers and let another company manage the hardware and network?
Our expertise is not infrastructure (even though we have people dedicated to it and that is what I spend 90% of my own time on). We are a software company with heavy infrastructure usage. We are also a customer service company. And our people are great at what they do – programming and pleasing our customers. Again quoting the article:
"To be very good at [infrastructure] at the same time as writing the software and developing the customer base is very hard. These companies are getting tripped up by things that are mission critical but they’re not really core to the development of the software."
The article goes on to say that many SaaS companies end up bringing their infrastructure back in-house once they have stabilized because:
"They simply don’t trust third party providers to offer the quality, pricing and capabilities they demand."
Believe me, I have done my homework on this topic. I have looked at collo services from Equinix, Internap; managed hosting services from ServerVault, Rackspace, DigitalNation (now Verio), DialTone, among others, and I have talked with local companies about building our own facility.
When we have these discussions, it always comes down to two things: (1) Where do we feel our internal resources can be best spent? and (2) Can we find a partner that we can trust with the rest of the stuff at an affordable cost?
We believe that we can be most effective by focusing our technical resources on innovating our software, rather than managing hardware. We have been outsourcing our infrastructure to Rackspace since 2004, and we were outsourcing to ServerVault before that. Both are great companies. Since Rackspace guarantees that our servers and network will always be up, it allows our infrastructure team to work on the system, not the hardware. We work on things such as:
I think the comment from the article that sums it up for most companies is that they simply don’t trust third party providers to give them what they need. There are some horrible hosting companies out there, I know. And even the bad ones have pretty web sites. But man… Rackspace has hands down been the best partner I have ever worked with. They are an amazing company, with great people who have earned my respect and trust; and most importantly they have given us what we need. We are rapidly approaching 100 servers with Rackspace, and I have yet to find a compelling reason to bring that in-house.
As long as Rackspace continues to scale their business relationship with us as we grow, we will not be one of those companies who deviate from their core competencies by bringing their mission critical infrastructure in-house.