Here is a Christmas present from Korey. I hope you find it useful:
http://www.webmail.us/misc/blagr.php
Provides a window into our dnscache + rbldnsd system. You can see which third-party spam blacklists use (we rsync them locally), and you can do lookups on IP addresses and domains that you think might be listed.
http://www.webmail.us/misc/spamrules.php
Provides a complete list of our spam filtering rules, along with description and score. This list is auto-updated daily.
The virus stats on our home page show that 1.6% of all emails contain a virus. I know that number isn’t right because we’re only counting emails that actually make it to our virus scanners, and we have Postfix rules that reject a ton of viruses before it ever makes it that far. The real virus numbers are much higher.
Postini shows a ridiculous spike in virus traffic over the past month. Check out the “Viruses > Last Six Months” tab on their stats page. I’d like to say that I agree with Postini’s numbers based on the rejects I’ve seen at the SMTP-level since Sober.U and Sober.Y hit the Internet. But since our stats don’t take SMTP rejects into account, it is difficult to say exactly what real virus traffic we’re seeing.
I bet we can fix that 🙂
Pat forwarded me a suggestion from a customer today and I
figured I’d discuss it here, since this is a common misconception about SPF…
> I would like to see SPF-enhanced white listing. If the e-mail
> passes the SPF check, white list the e-mail message and
> continue delivery. If the e-mail message fails the SPF check,
> pass it through the normal SPAM checks and process the e-mail
> message like any other e-mail received.
Many spammers publish SPF records for their domains too, and they send their spam from the mail servers listed in their SPF. By whitelisting all mail that passes SPF checks, we would be allowing a lot of spam in. I just did a search of my spam folder and 91 spam emails passed the SPF checks, meaning the spam domain has published SPF records and the spam email was sent from that domain’s legitimate servers.
SPF is not designed for whitelisting.
Rather, SPF is designed to prevent phishing and other forgeries. If an email is sent from a server that is not listed in a domain’s SPF record, we can assume it to be a forgery and either tag it as spam or discard it. This is really the only safe way to use SPF, and this is how we use it currently.
Here are the SPF scores we are using:
score SPF_PASS -0.001
score SPF_HELO_PASS -0.001
score SPF_FAIL 8.001
score SPF_SOFTFAIL 4.001
score SPF_HELO_FAIL 5.001
score SPF_HELO_SOFTFAIL 4.001
We tag messages as spam at a score of 6.000 or higher. Fyi, the -0.001 scores exist only so that we can see in the message headers if email passed the SPF checks.
After several years of running BIND9 on our DNS caching servers we
have finally ditch it and switch to D. J. Bernstein’s dnscache. On an average day
our SMTP and Spam Filtering servers send 1400 queries per second to
each of our DNS servers during peak hours. We made the switch because as we’ve grown we have seen more reliability, performance and general weirdness
issues with BIND.
Most notably, when the BIND cache would reach about 250 MB, its
performance deteriorated noticeably. It would respond slowly and even
drop queries. I have heard this is caused by BIND’s internal data
structures not efficiently getting rid of old cache records. Instead
BIND tries to cache every record until it expires and when it does
reach some internally calculated limit, BIND starts to discard new
cache records instead of old records. This causes the server’s
performance to take a nose dive, and causes our pagers to go off…. Time to
run "service named stop; sleep 2; service named start" again!
Also, BIND didn’t efficiently cache records from our rbldnsd servers
behind it. We could never really figure out why so many requests were
reaching rbldnsd and not hitting the BIND cache. Now with dnscache, we
have a good view of exactly what it is doing and have fine tuned the
SOAs in rbldnsd so that dnscache caches our spam DB lookups exactly how
we want it. No more weirdness going on behind the scenes.
Mr. Bernstein has a lot of nasty things to say about BIND. Don’t believe all of his hype, but do trust the fact that DJB’s code is
much simpler, more reliable and possibly more powerful than BIND. BIND
is overkill for almost every use. It tries to be all things for all
systems, whereas the DJB keeps things simple and provides a different server for
each purpose. I
like simplicity. The install process was little bit awkward on a Linux system though,
with the daemon tools and stupid errno patch.
FYI our dnscache servers are AMD Athlon 3200s with 1GB RAM, and they
each are handling 1400 queries per second using only 15% CPU. Currently
we have a 100 MB cache size, but we are still tuning that.
The largest virus outbreak ever occurred again this week. Our cluster of ClamAV servers handled it just fine, but that is not what this post is about. Virus outbreaks like this cause a swarm of backscatter mail. Backscatter is when innocent mailboxes are flooded with undeliverable mail notifications, because of the fact that viruses forge random sender addresses. To combat this, we discard virus notifications from other servers, because ALL of them are bogus. Viruses forge the sender address… So all you mail administrators out there – stop bouncing this crap back to the sender!
Blocking virus bounces based on Subject and other headers has worked reasonably well, but it does not block backscatter from simple SMTP rejects, because those bounces don’t contain pretty "you sent a virus" subject lines. It also doesn’t block backscatter caused by bounced spam, because spammers love to use sender addresses as well.
The high volume of backscatter seen this week caused us to look deeper for a way to block this stuff. And we found it… We now tag as spam any bounce message where the original email was not sent from our email system.
The rule is: (1) if the email is from a null sender, and (2) if the email has a bounce-style Subject or From header (such as "Subject: Undeliverable Mail" or "From: Mail Delivery Subsystem"), and (3) if the body of the message does not contain our servers’ Received headers – tag it as spam.
Now our email hosting customers have even cleaner inboxes, and you can thank Sober.Y.
At Webmail.us we name all of our servers using hostnames that fall under a very generic domain name. I am not going to list that domain name here so that Google doesn’t index it in this blog post, but I’m sure you could figure it out if you wanted to.
The reason we use such a generic domain is because a large percentage of our business comes through resellers. And most of our resellers want their customers to think that the email service is entirely owned and operated by them – in fact we encourage this. Our resellers own the relationship with their customer, and we have no desire to interfere with that.
So we help them hide the fact that Webmail.us is powering their email hosting service by using generic server names everywhere, even in our reverse DNS. And if you were to check the WHOIS database for this domain you will see that all of the contact info references a third-party holding company named "Whois Privacy Protection Service, Inc."
Pretty slick, huh?
If you run Postfix, you probably know that it is the best SMTP server on the planet. I bet it does everything you want plus more, and you know that it has very few shortcomings. If so, I agree with you.
However, there is one Postfix shortcoming that has been on our radar to solve for quite some time. Postfix cannot reject unknown users within domains that are an alias for another domain (i.e. domain aliases). Postfix stubbornly accepts mail for any random email address within a domain alias, and only after it has incorrectly accepted the bogus email, does it bounce it back to the sender. This is because Postfix’s smtpd address validation occurs prior to address rewrites, and thus it is only an approximation. This is a problem because a directory harvest attack on a domain alias can fill up your mail queues with bounces.
Here is how Korey solved this problem last week:
Our Postfix MySQL database structure is very simple. It contains valid addresses, aliases, domains and domain aliases, mapped to their real address. Here is a simplified example:
+——————–+——————–+
| email | virtual |
+——————–+——————–+
| support@webmail.us | support@webmail.us |
+——————–+——————–+
| @webmailus.com | @webmail.us |
+——————–+——————–+
In the past, this would cause our system to accept mail for junk addresses at our domain alias such as "asdf@webmailus.com". By modifying the virtual_alias_maps query as follows, our system now rejects these invalid addresses as it should. Maybe one day Postfix will do address rewrites prior to address validation, which would allow us to remove this complex SQL query. Until then, this hack works great for us. It may help you too…
/etc/postfix/main.cf:
virtual_alias_maps =
proxy:mysql:/etc/postfix/mysql_virtual_gate.cf,
/etc/postfix/mysql_virtual_gate.cf:
query = select virtual from users where email = ‘%s’
and left(virtual,1) <> ‘@’
union
select virtual from users where email = (
select concat(left(‘%s’,locate(‘@’,’%s’)-1), virtual) as rewritten
from users where email = ‘@%d’ and left(virtual,1) = ‘@’
) and left(virtual,1) <> ‘@’
limit 1