The largest virus outbreak ever occurred again this week.  Our cluster of ClamAV servers handled it just fine, but that is not what this post is about.  Virus outbreaks like this cause a swarm of backscatter mail.  Backscatter is when innocent mailboxes are flooded with undeliverable mail notifications, because of the fact that viruses forge random sender addresses.  To combat this, we discard virus notifications from other servers, because ALL of them are bogus.  Viruses forge the sender address… So all you mail administrators out there – stop bouncing this crap back to the sender!

Blocking virus bounces based on Subject and other headers has worked reasonably well, but it does not block backscatter from simple SMTP rejects, because those bounces don’t contain pretty "you sent a virus" subject lines.  It also doesn’t block backscatter caused by bounced spam, because spammers love to use sender addresses as well.

The high volume of backscatter seen this week caused us to look deeper for a way to block this stuff.  And we found it…  We now tag as spam any bounce message where the original email was not sent from our email system.

The rule is: (1) if the email is from a null sender, and (2) if the email has a bounce-style Subject or From header (such as "Subject: Undeliverable Mail" or "From: Mail Delivery Subsystem"), and (3) if the body of the message does not contain our servers’ Received headers – tag it as spam.

Now our email hosting customers have even cleaner inboxes, and you can thank Sober.Y.

At Webmail.us we name all of our servers using hostnames that fall under a very generic domain name.  I am not going to list that domain name here so that Google doesn’t index it in this blog post, but I’m sure you could figure it out if you wanted to.

The reason we use such a generic domain is because a large percentage of our business comes through resellers.  And most of our resellers want their customers to think that the email service is entirely owned and operated by them – in fact we encourage this.  Our resellers own the relationship with their customer, and we have no desire to interfere with that.

So we help them hide the fact that Webmail.us is powering their email hosting service by using generic server names everywhere, even in our reverse DNS.  And if you were to check the WHOIS database for this domain you will see that all of the contact info references a third-party holding company named "Whois Privacy Protection Service, Inc."

Pretty slick, huh?

If you run Postfix, you probably know that it is the best SMTP server on the planet.  I bet it does everything you want plus more, and you know that it has very few shortcomings.  If so, I agree with you.

However, there is one Postfix shortcoming that has been on our radar to solve for quite some time.  Postfix cannot reject unknown users within domains that are an alias for another domain (i.e. domain aliases).  Postfix stubbornly accepts mail for any random email address within a domain alias, and only after it has incorrectly accepted the bogus email, does it bounce it back to the sender.  This is because Postfix’s smtpd address validation occurs prior to address  rewrites, and thus it is only an approximation.  This is a problem because a directory harvest attack on a domain alias can fill up your mail queues with bounces.

Here is how Korey solved this problem last week:

Our Postfix MySQL database structure is very simple.  It contains valid addresses, aliases, domains and domain aliases, mapped to their real address.  Here is a simplified example:

+——————–+——————–+
| email              | virtual            |
+——————–+——————–+
| support@webmail.us | support@webmail.us |
+——————–+——————–+
| @webmailus.com     | @webmail.us        |
+——————–+——————–+

In the past, this would cause our system to accept mail for junk addresses at our domain alias such as "asdf@webmailus.com".  By modifying the virtual_alias_maps query as follows, our system now rejects these invalid addresses as it should.  Maybe one day Postfix will do address rewrites prior to address validation, which would allow us to remove this complex SQL query.  Until then, this hack works great for us.  It may help you too…

/etc/postfix/main.cf:
  virtual_alias_maps =
    proxy:mysql:/etc/postfix/mysql_virtual_gate.cf,

/etc/postfix/mysql_virtual_gate.cf:
  query = select virtual from users where email = ‘%s’
      and left(virtual,1) <> ‘@’
    union
    select virtual from users where email = (
      select concat(left(‘%s’,locate(‘@’,’%s’)-1), virtual) as rewritten
      from users where email = ‘@%d’ and left(virtual,1) = ‘@’
    ) and left(virtual,1) <> ‘@’
    limit 1

Yesterday we updated our SMTP servers so that they deliver mail using a pool of 30 IP addresses.  Previously outgoing mail from all of our SMTP servers was NAT’d through a single outgoing IP address.  Using a single IP however left our system somewhat vulnerable, because if that IP address were to become temporarily blacklisted by an anti-spam group, it could affect the deliverability of all customer mail.  We have dealt with this situation in the past by switching to a new unused IP temporarily while addressing the problem.  However you can’t do that anymore because SenderBase.org and TrustedSource.org will see a huge mail volume spike, which their partners use to further block mail.

Now with the use of a pool of outgoing IP addresses, if an IP becomes temporarily blacklisted, we can pull the affected IP address out of use while we deal with the customer who got us blacklisted.

Fyi, our IPs have only been blacklisted three times during the six years we have been in business.  In both cases it was resolved with within hours.

Webmail.us customers who have SPF records for their domains should continue to
use the following value in order to always have the updated IPs
automatically included in your domain’s SPF record:

  example.com IN TXT "v=spf1 include:emailsrvr.com -all"