Everything you Know about Relevancy is Wrong!

Everything you Know about Relevancy is Wrong!

"One of the joys in being a digital/email marketer is that you don’t have to rely on opinion. You can simply run some numbers and decide for yourself."

Relevance marketers versus data. Let the battle begin!

I read an article the other day from a relevance marketer – you know, the person who insists that your emails MUST be segmented and relevant in order to (a) be any good and (b) keep you out of deliverability hell.

There were two things that really confused me about the article. The first was that an email with a 60% open rate that generated 3,000 opens is better than one with a lower open rate that generated 4,500 opens. The difference in the cost of the send? Approximately $25. I don’t know about you but, as a marketer, I’d happily pay $25 to have 1500 people (who have already signed up for my list) to learn more about what I’m talking about. It works out to 1.6 cents per open. At a 10% click to open rate, it’s 16 cents per click – significantly cheaper than almost any other paid media. So while this author may “feel” that relevance is “better’, the data tells me it’s a no-brainer to send to the larger list. Even if it is less “relevant.”

The second thing that this article claimed was that larger lists will inevitably run into deliverability problems, as ISPs are actively looking at engagement as a metric for deciding who does and does not get into your inbox. Here’s my question…how can sending to people who’ve actively (or even passively) signed up for your email cause delivery problems? Can this assumption really be true?

One of the joys in being a digital/email marketer is that you don’t have to rely on opinion. You can simply run some numbers and decide for yourself. I pulled from our eDataSource inbox tracker (which is a completely awesome tool, btw) which uses a panel of several hundred thousand emails – across 335 different senders – and decided to see what the numbers have to say.

I’m making two assumptions here. The first is that open rate is a proxy for engagement. It’s not ideal but it’s the best leading indicator we have across the industry. The second is that inbox rate is a proxy for deliverability issues – good inbox rates are indicative of programs that do NOT have major deliverability issues.

The basic questions we are asking are simple – (1) do larger lists cause deliverability problems and (2) does a low open rate – a sign of “bad” relevance – cause deliverability issues?

We have three lists with three remarkably different “relevance” rates. Guess which one has deliverability issues -

List A – 4.9 MM sends, 28.9 % open rate

List B – 3.4 MM sends, 7.1% open rate

List C – 2.5 MM sends, 3.9% open rate

If you guessed C…you would be wrong.

The answer is (drum roll please)…NONE of them.

List Name

Size

Open Rate

Yahoo

AOL

Google

List A

4,193,141

28.9% 97.7% 98.0% 90.5%

List B

3,398,985

7.1% 99.0%  100.00% 98.3%

List C

2,528,591

3.9%  95.3% 100.0% 91.9%

 

List A has an open rate 7.4 times greater than List C, yet they have nearly identical inbox rates at three of the major ISPs. List A also has WORSE metrics then list B, despite more than 4 times a greater open rate. How in the world of relevance can that possibly happen? List C is obviously spamming their users yet suffering no harm, while List A is doing a great job with engagement. Yet they all have roughly the same inbox performance and the most “engaged” list actually does slightly WORSE at Google.

The doubters out there may think I must have cherry picked these three lists just to prove a point. So let’s roll it up and look at the combined results of all 335 different mail programs.Fradyaug14

Holy relevance, Batman! No matter what the list size, the average inbox rate metrics are almost the same across ALL groups.

I know, I know – my analysis will fall apart when we look at email programs that only sent to their most engaged customers. It must follow that less engage users equal major deliverability problems. Let’s go to the data!frady2aug14

What’s that? Another (nearly) flat chart? How can that be? If the relevance marketers are right, then this chart simply can’t be true. But it is. Which means only one thing….

Relevance and/or list size seems to have little to no impact on inbox rates for ISPs.

Truth be told, we have a dog in this hunt –our email list is a gateway to money. We want to send to the largest list possible that makes sense. We don’t want to blindly send with no thought (that would violate Dela Quist’s “Don’t Be Stupid” rule) but we want to send to as large a list as possible to (a) maximize our ROI and (b) keep true to the brand promise we established when users signed up for our email program.

Would we rather have super-high engagement rates? Sure! Relevance is important because it gives you more chances to “win” with your email universe. More opens equals more chances to meet both of our goals.

Will we NOT send to my less engaged universe because it’s not “relevant” and we’re afraid of the deliverability boogeyman? Not a chance. We’d rather make more money.

Will we artificially retire email names? Maybe. It depends on a combination of my sales cycle, sign-up practices, opt-out practices and merchandise mix. But I’ll use data – rather than “gut” – to drive that decision.

In looking at the source of most “relevance”-based articles, it seems the source is fairly consistent – vendors who have tools that specialize in making emails “more relevant.” Which sounds more like an attempt to convince people they need something they don’t (or just soother their email self-loathing tendencies), rather than one that is rooted in the data of what actually happens when you hit the “send” button.

You know what matters? Good list hygiene matters. Good signup policies are a must. Sending to your entire file on a regular basis matters. Your brand matters (bigger brands have it easier.) Subject lines matter. Sending more often – rather than less – also seems to matter. But the data tells me one thing - relevance (as measured by high open rates) does not matter for ensuring inbox delivery.

Send more. It’s ok.

 

Related Posts

 

Comments 2

Guest - Mohammed (website) on Wednesday, 20 August 2014 17:35

Bob, Reputation matters to ISP's these days than segmentation. If you are getting good opens and clicks then there is no need to segment mail. Why some ESP's consider to segment mail in B2C world is to getting higher user engagement on fresh data than legacy data. If you are getting good clicks NOT opens then there is no need to segment.

Bob, Reputation matters to ISP's these days than segmentation. If you are getting good opens and clicks then there is no need to segment mail. Why some ESP's consider to segment mail in B2C world is to getting higher user engagement on fresh data than legacy data. If you are getting good clicks NOT opens then there is no need to segment.
Bill McCloskey (website) on Friday, 22 August 2014 01:21

Reading this I would have to believe that you aren't having any delivery issues. Clearly you've got it nailed. Unless it's not quite as simple as you make it sound. (Link bait?

You said "Good list hygiene matters. Good signup policies are a must. Sending to your entire file on a regular basis matters. Your brand matters (bigger brands have it easier.) Subject lines matter. Sending more often – rather than less – also seems to matter." I agree, those things do matter. There are several more that you failed to mention around infrastructure and authentication. "Good list hygiene" also accounts for a wide range of items.

What we're hearing from ISPs is that they are taking into account how users respond to email. Either they're lying or your analysis is missing something.

What I think is happening is that the relevance signal is being swamped by other signals. For example, good list hygiene overlaps a lot with relevance. Poor list hygiene correlates strongly with lower open and response rates, higher bounce rates and higher complaint rates -- indicators of a lack of relevance. It's well known that if you get everything else right you can get away with low open and click rates. Similarly, if your content is relevant and desired you can get away with doing some other things poorly.

My takeaway is that it's an interesting analysis but fails to control for external variables which explains why the conclusion directly contradicts what the ISPs say they're doing.

Reading this I would have to believe that you aren't having any delivery issues. Clearly you've got it nailed. Unless it's not quite as simple as you make it sound. (Link bait? :) You said "Good list hygiene matters. Good signup policies are a must. Sending to your entire file on a regular basis matters. Your brand matters (bigger brands have it easier.) Subject lines matter. Sending more often – rather than less – also seems to matter." I agree, those things do matter. There are several more that you failed to mention around infrastructure and authentication. "Good list hygiene" also accounts for a wide range of items. What we're hearing from ISPs is that they are taking into account how users respond to email. Either they're lying or your analysis is missing something. What I think is happening is that the relevance signal is being swamped by other signals. For example, good list hygiene overlaps a lot with relevance. Poor list hygiene correlates strongly with lower open and response rates, higher bounce rates and higher complaint rates -- indicators of a lack of relevance. It's well known that if you get everything else right you can get away with low open and click rates. Similarly, if your content is relevant and desired you can get away with doing some other things poorly. My takeaway is that it's an interesting analysis but fails to control for external variables which explains why the conclusion directly contradicts what the ISPs say they're doing.
Already Registered? Login Here
Guest
Saturday, 24 August 2019
Supercharge Your Email Marketing Programs

Stay On Top of Your Game

The Free OI Newsletter Delivered to your Inbox Each Week

Subscribe to Get:

  • Cutting-Edge Email Marketing Tips and Tricks
  • Product Reviews
  • The Latest Email Marketing Jobs
  • Industry News