Trending Now

Are you testing the right things on your website?

It doesn’t happen as much anymore, but it wasn’t long ago that every time I showed slides explaining how Amazon tests every change they make to their website, some wisenheimer would pipe up, “Yeah, well Amazon has a lot of money, so they can afford to test.”  I was forced to edify them by explaining that it’s actually the opposite: “Amazon tests, so they have a lot of money.”

Nowadays, everyone seems to have the testing religion. They test up and down, but are they testing the right things? When I talk to clients, the vast majority of their tests are about very small changes:

  • Color, placement, or text on call to action buttons
  • One image vs. another
  • One headline vs. another
  • One page layout vs. another

Now, understand, none of these tests are bad ideas. Yes, you should test these things. But my problem is that this is just about all I ever see tested. There is a whole word out there waiting for you–it just needs you to take some initiative:

  • Remove a step from your navigation to conversion
  • Add a step from to your navigation
  • Replace text with a video
  • Add text transcriptions to your videos
  • Change the marketing message

You might read these and say to yourself, “Gee, those tests are hard!” And you would be right. It takes more than a couple of minutes to think through the ramifications of these tests. Instead, I see two things simultaneously going on, both of which are wrong:

  1. All of the testing concerns the easy stuff we discussed at the top of the post. We do testing, but it’s all limited to a single page with variations that take minutes to think up.
  2. We make all the changes I described in the second list–we just don’t test any of them. Think about it. The last time you overhauled your navigation or your marketing message or redid your content types, did you test any of it? Probably not. You just did the creative work and made the change live in production and assumed you had improved things. You’re not actually testing the biggest, riskiest changes that have the most potential for improvement and the most risk of failure.

It doesn’t have to be that way.

Instead of relegating A/B testing to the simple stuff, your biggest changes are the ones to give the most testing scrutiny. Make sure you test the big stuff and you’ll benefit a lot more than from testing only the small stuff.

Mike Moran

Mike Moran is a Converseon, an AI powered consumer intelligence technology and consulting firm. He is also a senior strategist for SoloSegment, a marketing automation software solutions and services firm. Mike also served as a member of the Board of Directors of SEMPO. Mike spent 30 years at IBM, rising to Distinguished Engineer, an executive-level technical position. Mike held various roles in his IBM career, including eight years at IBM’s customer-facing website,, most recently as the Manager of Web Experience, where he led 65 information architects, web designers, webmasters, programmers, and technical architects around the world. Mike's newest book is Outside-In Marketing with world-renowned author James Mathewson. He is co-author of the best-selling Search Engine Marketing, Inc. (with fellow search marketing expert Bill Hunt), now in its Third Edition. Mike is also the author of the acclaimed internet marketing book, Do It Wrong Quickly: How the Web Changes the Old Marketing Rules, named one of best business books of 2007 by the Miami Herald. Mike founded and writes for Biznology® and writes regularly for other blogs. In addition to Mike’s broad technical background, he holds an Advanced Certificate in Market Management Practice from the Royal UK Charter Institute of Marketing and is a Visiting Lecturer at the University of Virginia’s Darden School of Business. He also teaches at Rutgers Business School. He was a Senior Fellow at the Society for New Communications Research and is now a Senior Fellow of The Conference Board. A Certified Speaking Professional, Mike regularly makes speaking appearances. Mike’s previous appearances include keynote speaking appearances worldwide

Join the Discussion

Your email address will not be published. Required fields are marked *

Back to top Back to top