Spoof proof and truth

Well!
Not my easiest working week earlier this year.
We’ve been the butt of a significant spoof email attack.
This has led to one of our sites and several of our domains being listed on antispam block lists.
Quite often this kind of block is a minor annoyance. Few systems rely on block lists for antispam in the UK, Europe or North America. But in New Zealand it is apparently a very different story.
It appears that they place a huge amount of trust in these lists and are happy to lose business mail than they are to receive spam.
I can understand the abhorence of spam. A significant part of my job is security related and inbound spam is currently running at 85-99% in the Asia Pacific region. Slightly less in the States and even less in the UK.
So we were blocked.
Next step: start procedures to be removed.
This requires contact with our customers to find out what the issue is, precisely, and then find the offending list and then why we are on it. We can then take steps to fix any perceived issue and get delisted.
An interim step is to request affected customer whitelist our domain.
So we start dialogue with customers and attempt to contact appropriate IT staff. I can honestly tell you that this issue is like chewing nails, but more painful.
To cut the long story a little shorter these IT staff weren’t very helpful. So we had to dig a lot to find a solution.
Many hours were put in by our staff in NZ to find the exact lists that were causing the issue and I must praise their attitude and fortitude under immense pressure from internal Management.
Finally, lists are too some degree found and the owners/admins contacted.

Filtering Inbound Spam
========================
Now we get into the major issue. The beligerence and down right rudeness of the list admins has to be experienced to be believed. Now I have no doubt that at one point they performed a service that was sadly lacking in email. A way of identifying spammers. And certainly as a minor marker in wider antispam they have a place.
Time and technology have moved on though. And it is exceptionally easy for global corporations to be caught up in an attack like this. It is much much better to judge each mail on it’s own merits. The major query here though is the huge amount of processing power required. Our own organisation cannot provide the huge infrastructure required and we are one of the largest of our type in the world.
So we rely on Security as a Service solution. All our inbound mail is sent through a third party antiviral / antispam service. This filters almost all spam from the inbound route. It does collect some business mail but this is normally quarantined and can be released. It’s still not perfect. It still blocks sites that are known to be spamming but does it based on a short term reputations. So it’s easy to fix a problem and then suddenly all’s hunky dory again.
This system has large pattern matching, heuristic, reputation and content filtering systems that are all used to create a likelihood score: spamminess.
Part of the reputation element is the block lists, but if the rest of the logic isn’t spammy then you find the mail gets through.
Job done!
We filter out 87% of all mail before it gets to users over all domains and all countries. Some domains this figure is over 99.9% and on some it is less than 0.01%. One domain receives ~50 legitimate mails a day and +200,000 spam.
We rarely end up blocking client or even marketing system trying to contact us and as mentioned before this is based on short term reputations so can be remedied.
But we don’t block based solely on a list.

The Fix
========================
Well, it turns out that getting removed from some of the lists is impossible as the admins refuse to see reason.
If you share an ISP with a spammer for example the whole ISP is blocked.
So we are forced to use an internal relay and masquerading out mail on another domain.
Extremely annoying. Only mitigated by the fact that we will be consolidating our mail domain.
I will be looking into SPF and protectionist procedures for our primary domains but this will take a while and has implications for many parts of the business.

– Posted using BlogPress from my iPhone