Trending Now

Does experimental marketing require lots of customers?

Everyone knows what a great job the big boys do. Amazon, eBay–you can go down the list. All of these “born on the web” companies have massive feedback loops that allow them to experiment with small changes in their marketing, their user experience, their offers, and ten other things. Whenever they make a change, they see if more people are buying and decide whether the new way “works” or not. Amazon’s CTO once told me they can change a font at breakfast and know whether it was a good idea by lunchtime. But how can you do the same thing, when you don’t get nearly as many visitors as these big websites do?

In a sense, you are worried about something important. Because these sites have so many visitors, they can make a decision about their change much faster than you can about yours. And they can be more sure that they are right than you can.


Image by jonobacon via Flickr

So, you could throw up your hands, saying, “Unless our statistics reach statistical significance at the 95th percentile, we aren’t going to make decisions on them.” Or you could say, “We’ll never have a large enough sample size to make up our minds.”

Or you could take a different approach.

One is to wait longer. Accept the fact that if you get 1,000 visitors a month instead of 1,000 a minute that it will take you longer to evaluate your changes than Amazon. And that can be OK. If you have a small site, you probably don’t want to spend all your time running tests and making changes anyway—you have the rest of your business to run.

But there is another approach that you take in other areas that works here, too. Don’t worry so much about being absolutely sure of the answer. If three people walked into your store and each decided against buying the sale item you had displayed, even after they looked it over carefully, would you leave that display as is until 20 more people did the same thing just to be sure it was a clunker?

Probably not. You’d start to tinker with it, even though it’s entirely possible that those people were not representative of the population at large.

You can do the same thing with your website and your Internet marketing. If you have a page that seven out of eight abandon your site from, you can try to make a change to that page and see if you do better with the next eight people. It is very possible by taking this approach that you will sometimes act on unreliable information, but it is very unlikely that you’ll have that kind of bad luck many times in a row. By experimenting more, you are likely to move in the right direction.

So don’t let low visitor counts stop you from experimenting. Remember that if you don’t look at your statistics at all, you are experimenting anyway. You just never know what is working. At least this way, you know what is working some of the time—perhaps even most of the time.

Enhanced by Zemanta

Mike Moran

Mike Moran is a Converseon, an AI powered consumer intelligence technology and consulting firm. He is also a senior strategist for SoloSegment, a marketing automation software solutions and services firm. Mike also served as a member of the Board of Directors of SEMPO. Mike spent 30 years at IBM, rising to Distinguished Engineer, an executive-level technical position. Mike held various roles in his IBM career, including eight years at IBM’s customer-facing website,, most recently as the Manager of Web Experience, where he led 65 information architects, web designers, webmasters, programmers, and technical architects around the world. Mike's newest book is Outside-In Marketing with world-renowned author James Mathewson. He is co-author of the best-selling Search Engine Marketing, Inc. (with fellow search marketing expert Bill Hunt), now in its Third Edition. Mike is also the author of the acclaimed internet marketing book, Do It Wrong Quickly: How the Web Changes the Old Marketing Rules, named one of best business books of 2007 by the Miami Herald. Mike founded and writes for Biznology® and writes regularly for other blogs. In addition to Mike’s broad technical background, he holds an Advanced Certificate in Market Management Practice from the Royal UK Charter Institute of Marketing and is a Visiting Lecturer at the University of Virginia’s Darden School of Business. He also teaches at Rutgers Business School. He was a Senior Fellow at the Society for New Communications Research and is now a Senior Fellow of The Conference Board. A Certified Speaking Professional, Mike regularly makes speaking appearances. Mike’s previous appearances include keynote speaking appearances worldwide

Join the Discussion

Your email address will not be published. Required fields are marked *

Back to top Back to top