So, we’re beginning to enter the dog days of summer, when thoughts turn to family vacations and fun in the sun. We’ve long since stopped worrying about fitting into our swimsuits and now just want to enjoy the surf. In fact, we’re right in the thick of summer movie season with one of the most anticipated movies of the year due on our doorsteps this week. Hollywood has a lot riding on the success of the latest Batman movie, having dropped at least $250 million into making the film. And that’s before marketing expenses. Unfortunately, I see a lot of businesses taking a similar, “Hollywood blockbuster” approach to their marketing, putting all (or many of) their eggs in one basket and hoping for a monster hit. But is that the right approach? Let’s take a look.
I realize nobody likes a math quiz, particularly in the middle of the summer, but hang with me for a second. Imagine two scenarios:
- In the first one, you’ll invest your entire testing budget in a single, big-ticket initiative. If it succeeds, you expect a 30-35% increase in year-on-year profits. Of course, if it fails, you’ve blown your whole budget.
- In the second scenario, you’ll split your budget into a series of small monthly tests, each of which are expected to increase sales by about 2%-3%. Obviously, any single test could fail to produce any result, but you’ll risk only 1/12th the total budget on each.
Which is better?
Well, as you might expect, it’s kind of a trick question. You see, an increase of just 2.4% monthly, compounded over the course of the year, equals a 33% lift for the year. So, the two outcomes are roughly equivalent.
Well, except for the amount of budget you risk at any one time.
In this specific (and, to be fair, somewhat unrealistic) example, you reduce risk by spending only a small amount each month in exchange for smaller expected returns. You’re probably not going to release any blockbusters, but you’re also unlikely to experience any major flops, either.
By testing and measuring your activities, you’re not forced into a “blockbuster mentality,” where the whole gets reduced into hits and flops. Whether you call it agile marketing or simply “doing it wrong quickly” you can put your emphasis on achieving long-term results. Yes, it’s much more “tortoise” than “hare.” But try to remember who won that race.
For instance, I once conducted a test on an e-commerce site that improved conversion rate by 15% and sales by 10% simply by increasing the font size in the shopping cart. The entire test cost the company a few hundred dollars to run and resulted in roughly $10 million in annual sales. Now, this is an extreme example and one that’s not, sadly, typical. Usually most tests cost more to run–and produce less dramatic results. And, sure, I’ve seen plenty of tests that failed outright. But, the costs of those failures almost always was offset by the successes of other tests, either singly or in the aggregate.
Of course, as Stanford professor Sam Savage likes to say,
“Uncertainty is an objective feature of the universe. Risk is in the eye of the beholder.”
For instance, I fully recognize that there are times you need to swing for the fences. Not everything you’re going to do lends itself to this type of testing. Maybe your company is launching a major update of your main product, including broadcast media, press and your CEO ringing the opening bell on the New York Stock Exchange. Or you’re a startup that’s shifting customer categories completely and you need to gain attention for your brand. It’s not to say you can’t test anything in those periods, but it’s also understandable that you’re going for more of a blockbuster.
The point is, one size doesn’t fit all, at least not all the time. But by mixing testing into your existing activities, you can improve your results in a steady fashion, month after month. And, using those steady results, look back and show your boss that you still had a blockbuster year.
