SPAM—More than you ever wanted to know about how we block it

Over the past week or two there’s been a brief bump in the amount of spam you received (if you were one of about 500 lucky individuals). C&IT now seems to have the situation under control, but I thought folks would be interested in how we attempt to control SPAM. It’s a complex process, involving what security folks call defense in depth, and, as with most modern warfare, it’s an arms race.
The Wayne Connect email system uses three different kinds of anti-spam protection, acting in serial (i.e. each one operates on the output of the previous one).
The first layer, Cisco Ironport Senderbase (known sometimes simply as Ironport, which was the name of the company before Cisco acquired it several years ago) filters out mail from any source that has a bad ‘reputation’. This machine relies on a continuously maintained national database of known spammers. That database is assembled from all the other Ironport machines located around the world.  Believe it or not, about ninety percent of the email messages that reach Wayne State are blocked at the outermost wall by this ‘appliance’.
Mail that gets through this filter is then submitted to the second layer, the Quarantine filter you probably know about. This has an algorithm to guess whether things might be spam based on various characteristics of the messages. Messages that ‘look’ suspicious to the software are placed in quarantine and you get a message every morning from the machine telling you what has been quarantined in the past 24 hours. You can then tune the system by telling it which domains (such as ‘wayne.edu’ or ‘freep.com’) you want to permit, and it establishes what is known as a whitelist.
The third layer is Wayne Connect itself, which has another algorithm, and places suspicious mail in your Junk folder. I find there is very rarely anything in there, but if something does show up, I look at it (when I remember) and either delete it or mark it as ‘not junk’ and it moves to my Inbox.
The result is that, although the occasional message slips through, over ninety percent never reaches you. And all of this is totally automatic, incidentally—no human being ever sees any message the system blocks.
In case you are wondering what happened last week, incidentally, the first layer was modified (to make it faster by doubling the number of machines it runs on, simplifying somewhat) and the new machine needed a little tinkering to get the filter to work correctly, so anybody’s mail that went through the new machine was not properly filtered for a few days.
For those of you who were spammed, you now can see what kind of stuff we normally shield you from. I was one of those who got some hair-raising messages during. I imagine you want us to keep the shields up.
Just for fun, here’s a graph produced by the anti-spam system showing what got through, and what was blocked by each layer.

Proportion of mail blocked, by category

Notice, incidentally the number in the top right-hand column. That represents messages received over a recent twenty-four hour period. Mail comes in to Wayne at a peak rate (at noon) of roughly 140,000 messages per hour!

Leave a Reply

Your email address will not be published. Required fields are marked *