Everyone knows what a great job the big boys do. Amazon, eBay–you can go down the list. All of these “born on the web” companies have massive feedback loops that allow them to experiment with small changes in their marketing, their user experience, their offers, and ten other things. Whenever they make a change, they see if more people are buying and decide whether the new way “works” or not. Amazon’s CTO once told me they can change a font at breakfast and know whether it was a good idea by lunchtime. But how can you do the same thing, when you don’t get nearly as many visitors as these big websites do?
In a sense, you are worried about something important. Because these sites have so many visitors, they can make a decision about their change much faster than you can about yours. And they can be more sure that they are right than you can.
Image by jonobacon via Flickr
So, you could throw up your hands, saying, “Unless our statistics reach statistical significance at the 95th percentile, we aren’t going to make decisions on them.” Or you could say, “We’ll never have a large enough sample size to make up our minds.”
Or you could take a different approach.
One is to wait longer. Accept the fact that if you get 1,000 visitors a month instead of 1,000 a minute that it will take you longer to evaluate your changes than Amazon. And that can be OK. If you have a small site, you probably don’t want to spend all your time running tests and making changes anyway—you have the rest of your business to run.
But there is another approach that you take in other areas that works here, too. Don’t worry so much about being absolutely sure of the answer. If three people walked into your store and each decided against buying the sale item you had displayed, even after they looked it over carefully, would you leave that display as is until 20 more people did the same thing just to be sure it was a clunker?
Probably not. You’d start to tinker with it, even though it’s entirely possible that those people were not representative of the population at large.
You can do the same thing with your website and your Internet marketing. If you have a page that seven out of eight abandon your site from, you can try to make a change to that page and see if you do better with the next eight people. It is very possible by taking this approach that you will sometimes act on unreliable information, but it is very unlikely that you’ll have that kind of bad luck many times in a row. By experimenting more, you are likely to move in the right direction.
So don’t let low visitor counts stop you from experimenting. Remember that if you don’t look at your statistics at all, you are experimenting anyway. You just never know what is working. At least this way, you know what is working some of the time—perhaps even most of the time.
