Tag Archives: tech

Realtors: Change with us

5 Replies

Last year Beth and I bought our first house.  We used a Realtor.  Our experience was great, and I would recommend my Realtor, Rhonda Brewer, to anybody looking to buy a home in the Blacksburg area (and I have recommended her to a few co-workers).  Rhonda led us through the home buying process from start to finish.  She helped us locate properties, tour them, helped us make the winning offer in a competitive bid situation, and guided us through all of the legal crap.  Thanks Rhonda!

However… if there had of been an electronic alternative available for at least part of the buying process, I would have used it.   This is nothing against Rhonda, or my uncle Den who is a Realtor in Rhoad Island, or any Realtor for that matter.  I just think your market needs a disruptive company to come in and shake things up a bit.  I was surprised at how lo-tech the home shopping tools available are, and how closed of a system the MLS is.  Home listings are posted on the MLS for licensed Realtor eyes only several days before they are available on the public sites.  Sometimes homes are sold before they ever reach the public listings.  Come on this is 2006!  Put this information directly in the hands of your customers and find a new way to make money from it.

Also, it sucks that you have to go to a completely separate web site to find information on historical home sales in the neighborhoods you are shopping in.  Whoever opens up the MSL and integrates this data will make a killing – somehow.

Today I read about what Redfin is doing in Seattle and San Francisco.  They have opened up the MLS and integrated historical sale data, and mapping software.  And if you want they’ll even help you make an offer and help you through the legal crap.  Awesome.  Realtors are up in arms over there and sending Redfin employees threats and other nastiness.  But they shouldn’t be.  This was bound to happen.  The information should be publicly accessible, just like it is in every other efficient market.

Realtors: If you look at this change as something that is going to steal your job away, it will.  If you embrace this change and change with it, then you will survive and even gain customers.

I do not see Redfin’s make-an-offer / legal-representation service as an alternative to using a Realtor.  Realtors know their local markets well, know the neighborhoods and may always do better than an online service at this.  Redfin’s service instead is an alternative to buying a home without a Realtor, which a lot of people do.  Redfin is trying to capture a 1% commission from the home sales that are not currently giving Realtors any commission.

My predictions:

(1) Redfin or a similar company will take this business model nation-wide within the next couple of years.

(2) Traditional Realtors will be bitter about this and will quickly become obsolete.  The older generation of home buyers will continue to use Realtors, but the demand for traditional Realtors will shrink and there will be less of these folks around.

(3) Progressive Realtors will accept and embrace this change and look for ways to capitalize on it.  They will point their customers to the online tools, help them use the tools, and be there for them when it is time to make an offer.  Realtors will do less per sale, but they will be able to work more efficiently with their clients by letting their clients do the searching for what they want, rather than they Realtor doing the search as a middle-man.

(4) The buying agent commission will decrease and/or buying agents will start giving cash back to the buyer as Redfin is doing.  Currently it is 3% of the sale price.

(5) More people will use a Realtor when buying a house because of the commission decrease.

(6) Similar change will occur on the seller-side, but less rapidly because it is not nearly as needed.

Why am I getting so much spam?

1 Reply

I posted an answer to this question on Monday on the Webmail blog.  However, something is broken with the feedburner RSS feed on our site.

Anti-spam has been one of my areas of focus for several years, and something that I have enjoyed.  Recently though, spam fighting has gotten pushed to the side with all of the other things that I am responsible for.  It is painful to watch spam slip by the filters and I am definitely excited to have Mike T working on this project.

The first thing Mike is going to do is take our existing whitelist/blacklist functionality and push it up to the Postfix level, rather than the amavisd + spamassassin level.  This will ensure that system-wide rules don’t override individual user or domain rules.  The second thing he is going to do is give our customers the ability to whitelist/blacklist IP addresses, in addition to just sender email addresses.  Next, Mike will be building a system that makes "whitelist/blacklist/greylist/unknown" decisions based on aggregate whitelist/blacklist data, sending history, mail volumes and third party sender reputation databases.  This will be some really powerful stuff.  After that, he will do some work on the content filters behind this sender reputation system, possibly incorporating DSPAM which learns from the spam that each individual receives.

Watch what you put on Red Hat ES4

Leave a reply

We put some additional spam filtering servers online yesterday, and we figured we’d bump up to Red Hat ES4 (Linux 2.6.9-patched).  We had been running Red Hat ES3 (Linux 2.4.21-patched) for all of our prior spam filtering servers.  But the performance on these new ES4 servers sucked under a high load.

At first I noticed from vmstat that it was swapping a bunch of application memory when it didn’t need to, but I fixed that by:   echo 0 > /proc/sys/vm/swappiness

It also seemed that it was swapping our tmpfs partitions when there was plenty of memory, which ES3 didn’t do.  But even when turning swapoff entirely and using less than 50% of our RAM, once there were over 100 Postfix smtpd processes receiving mail, the server’s load average sky-rocketed and the server became unresponsive. Our ES3 servers handle the same load all day long with no problem.

I also noticed from iostat that the ES4 servers did a lot of reads from our root partition, which our ES3 servers do not do. I don’t know why it would do that because we use the exact same drive and config and software versions on both machines.  And plus, Postfix and amavisd/spamassassin are supposed to keep all their config data in memory.

The strangeness got stranger when I stopped either amavisd or Postfix.  With Postfix accepting incoming mail on port 25 and amavisd stopped, the reads on the root partion vanish.  With Postfix’s port 25 blocked (no incoming mail) and amavisd crunching on queued mail, the reads on the root partition vanish.  However with both running, there are heavy reads on the root partition… WTF!

Anyways, after spending more than a day on it I just wanted to let you know that ES4 sucks and we have rebuilt these machines using our trusty ES3 image and they are now operating great.

Moral

Courier –> Dovecot

2 Replies

I mentioned back in February that we are switching our POP3/IMAP proxy software from Perdition to Dovecot.  These proxy servers are still in beta, because there is a bug with how Dovecot handles SSL connections.  Timo (Dovecot’s author) attempted a fix a few weeks ago, but the fix introduced new problems, so we reverted back.  I am hoping to get the final bugs resolved within the next couple of weeks so that we can release this out of beta.

In the mean time, we have been hard at work upgrading our backend IMAP software – also to Dovecot.  Currently we run Courier-IMAP, but Courier does not handle large mail folders efficiently.  Webmail, unlike desktop clients, does not have its own cache, so it relies on the IMAP server to obtain header listings and to perform sorts and searches.  Courier lacks indexes that would make these operations fast.  Instead it must open every message file and parse out the header information in order to return a sorted list of emails back to webmail.  Dovecot on the other hand, makes heavy use of indexes.  The indexes allow a folder with 10,000 messages to be sorted in less than a second, whereas Courier would take 30-60 seconds or even longer, and usually cause a timeout.  The speed difference is amazing.

In order to make the switch seamless, we have configured Dovecot to run in parallel with Courier on the existing mailbox servers.  We patched Dovecot to utilize Courier’s folder subscription and message UID lists, so that both systems can utilize the same maildirs.  If you are interested in these patches, shoot me an email and I will send them to you.

Webmail will be the first application to switch to Dovecot.  We began rolling this out server-by-server on Wednesday.  We are taking our time with this rollout since it is such a big change, just in case any unforeseen problems arise.  So far the issues have been minor, and easily corrected.

Once the beta proxy is bug-free, we will start migrating the front-end POP3 and IMAP systems to Dovecot.  Needless to say, we are making a big commitment to this Dovecot thing.  It will be at the core of what’s to come.

Outsource your data center

5 Replies

Phil Wainewright wrote an article on his Software as Services blog Monday on a vary familiar topic – the ironies of Software as a Service companies who operate their own data centers.  I quote:

"One of the ironies of the on-demand application space is that many of the leading names operate their own data centers. This seems somewhat illogical, if not downright hypocritical. On the one hand, they ask their customers to rely on a third party to provide mission-critical business applications. But instead of doing likewise with their own infrastructure, they host in-house."

This is has been a topic of discussion at Webmail for years… Would it be cheaper buy servers and put them in a collo facility and manage them ourselves?  Or could we build out our own data center here in the VT CRC?  Or should we lease servers and let another company manage the hardware and network?

Our expertise is not infrastructure (even though we have people dedicated to it and that is what I spend 90% of my own time on).  We are a software company with heavy infrastructure usage.  We are also a customer service company.  And our people are great at what they do – programming and pleasing our customers.   Again quoting the article:

"To be very good at [infrastructure] at the same time as writing the software and developing the customer base is very hard. These companies are getting tripped up by things that are mission critical but they’re not really core to the development of the software."

The article goes on to say that many SaaS companies end up bringing their infrastructure back in-house once they have stabilized because:

"They simply don’t trust third party providers to offer the quality, pricing and capabilities they demand."

Believe me, I have done my homework on this topic.  I have looked at collo services from Equinix, Internap; managed hosting services from ServerVault, Rackspace, DigitalNation (now Verio), DialTone, among others, and I have talked with local companies about building our own facility.

When we have these discussions, it always comes down to two things: (1) Where do we feel our internal resources can be best spent?  and (2) Can we find a partner that we can trust with the rest of the stuff at an affordable cost?

We believe that we can be most effective by focusing our technical resources on innovating our software, rather than managing hardware.  We have been outsourcing our infrastructure to Rackspace since 2004, and we were outsourcing to ServerVault before that.  Both are great companies.  Since Rackspace guarantees that our servers and network will always be up, it allows our infrastructure team to work on the system, not the hardware.  We work on things such as:

  • scaling our software to take advantage of our growing number of servers
  • optimizing performance
  • automating common system management tasks
  • building monitoring and recovery tools
  • securing our systems from spam, viruses and malicious traffic
  • intelligent storage systems

I think the comment from the article that sums it up for most companies is that they simply don’t trust third party providers to give them what they need.  There are some horrible hosting companies out there, I know.  And even the bad ones have pretty web sites.  But man… Rackspace has hands down been the best partner I have ever worked with.  They are an amazing company, with great people who have earned my respect and trust; and most importantly they have given us what we need.  We are rapidly approaching 100 servers with Rackspace, and I have yet to find a compelling reason to bring that in-house.

As long as Rackspace continues to scale their business relationship with us as we grow, we will not be one of those companies who deviate from their core competencies by bringing their mission critical infrastructure in-house.