Everything you Know about Relevancy is Wrong!

"One of the joys in being a digital/email marketer is that you don’t have to rely on opinion. You can simply run some numbers and decide for yourself."

Relevance marketers versus data. Let the battle begin!

I read an article the other day from a relevance marketer – you know, the person who insists that your emails MUST be segmented and relevant in order to (a) be any good and (b) keep you out of deliverability hell.

There were two things that really confused me about the article. The first was that an email with a 60% open rate that generated 3,000 opens is better than one with a lower open rate that generated 4,500 opens. The difference in the cost of the send? Approximately $25. I don’t know about you but, as a marketer, I’d happily pay $25 to have 1500 people (who have already signed up for my list) to learn more about what I’m talking about. It works out to 1.6 cents per open. At a 10% click to open rate, it’s 16 cents per click – significantly cheaper than almost any other paid media. So while this author may “feel” that relevance is “better’, the data tells me it’s a no-brainer to send to the larger list. Even if it is less “relevant.”

The second thing that this article claimed was that larger lists will inevitably run into deliverability problems, as ISPs are actively looking at engagement as a metric for deciding who does and does not get into your inbox. Here’s my question…how can sending to people who’ve actively (or even passively) signed up for your email cause delivery problems? Can this assumption really be true?

One of the joys in being a digital/email marketer is that you don’t have to rely on opinion. You can simply run some numbers and decide for yourself. I pulled from our eDataSource inbox tracker (which is a completely awesome tool, btw) which uses a panel of several hundred thousand emails – across 335 different senders – and decided to see what the numbers have to say.

I’m making two assumptions here. The first is that open rate is a proxy for engagement. It’s not ideal but it’s the best leading indicator we have across the industry. The second is that inbox rate is a proxy for deliverability issues – good inbox rates are indicative of programs that do NOT have major deliverability issues.

The basic questions we are asking are simple – (1) do larger lists cause deliverability problems and (2) does a low open rate – a sign of “bad” relevance – cause deliverability issues?

We have three lists with three remarkably different “relevance” rates. Guess which one has deliverability issues -

List A – 4.9 MM sends, 28.9 % open rate

List B – 3.4 MM sends, 7.1% open rate

List C – 2.5 MM sends, 3.9% open rate

If you guessed C…you would be wrong.

The answer is (drum roll please)…NONE of them.

List A has an open rate 7.4 times greater than List C, yet they have nearly identical inbox rates at three of the major ISPs. List A also has WORSE metrics then list B, despite more than 4 times a greater open rate. How in the world of relevance can that possibly happen? List C is obviously spamming their users yet suffering no harm, while List A is doing a great job with engagement. Yet they all have roughly the same inbox performance and the most “engaged” list actually does slightly WORSE at Google.

The doubters out there may think I must have cherry picked these three lists just to prove a point. So let’s roll it up and look at the combined results of all 335 different mail programs.

Holy relevance, Batman! No matter what the list size, the average inbox rate metrics are almost the same across ALL groups.

I know, I know – my analysis will fall apart when we look at email programs that only sent to their most engaged customers. It must follow that less engage users equal major deliverability problems. Let’s go to the data!

What’s that? Another (nearly) flat chart? How can that be? If the relevance marketers are right, then this chart simply can’t be true. But it is. Which means only one thing….

Relevance and/or list size seems to have little to no impact on inbox rates for ISPs.

Truth be told, we have a dog in this hunt –our email list is a gateway to money. We want to send to the largest list possible that makes sense. We don’t want to blindly send with no thought (that would violate Dela Quist’s “Don’t Be Stupid” rule) but we want to send to as large a list as possible to (a) maximize our ROI and (b) keep true to the brand promise we established when users signed up for our email program.

Would we rather have super-high engagement rates? Sure! Relevance is important because it gives you more chances to “win” with your email universe. More opens equals more chances to meet both of our goals.

Will we NOT send to my less engaged universe because it’s not “relevant” and we’re afraid of the deliverability boogeyman? Not a chance. We’d rather make more money.

Will we artificially retire email names? Maybe. It depends on a combination of my sales cycle, sign-up practices, opt-out practices and merchandise mix. But I’ll use data – rather than “gut” – to drive that decision.

In looking at the source of most “relevance”-based articles, it seems the source is fairly consistent – vendors who have tools that specialize in making emails “more relevant.” Which sounds more like an attempt to convince people they need something they don’t (or just soother their email self-loathing tendencies), rather than one that is rooted in the data of what actually happens when you hit the “send” button.

You know what matters? Good list hygiene matters. Good signup policies are a must. Sending to your entire file on a regular basis matters. Your brand matters (bigger brands have it easier.) Subject lines matter. Sending more often – rather than less – also seems to matter. But the data tells me one thing - relevance (as measured by high open rates) does not matter for ensuring inbox delivery.

Send more. It’s ok.