Trending Now

Seth Moore of Overstock.com on multivariate testing

Last week, I wrote about an up-and-coming player in multivariate testing, SiteSpect, which allows you to test almost any aspect of your user experience, unlike the standard JavaScript testing approaches. SiteSpect also eliminates the need to modify your site at all, speeding the test process so you can Do It Wrong (more) Quickly. I conducted an e-mail interview with one of SiteSpect’s happy customers, Seth Moore, the Manager of Marketing Analytics and Testing at Overstock.com.


Me: What’s your background, Seth? How did you reach the position you are in today?
Seth: I came to Overstock two years ago from the U.S. Senate and have a background in political science and quantitative methods.
Me: How long has your company been doing A/B or multivariate testing? Did they use other vendors before SiteSpect? What were the results?
Seth: We have been doing testing for years. We tried most of the major platforms and couple of in-house platforms. All the testing platforms we have used have added value to the company. The difficulty for us was always the agility and adaptability of the platform. We always wanted to avoid development time where possible, and SiteSpect has been the best for quickly deploying tests without development changes to the Web site code.
Me: Are senior managers aware of the testing that you do? What kind of persuasion was required to get this kind of testing in place? What kind of communication do your top execs get about testing now?
Seth: A high-level involvement in testing has never been a problem for us. The CEO follows test results like some follow the NBA playoffs. In fact, there is a healthy amount of competitive banter about whose site ideas are performing the best, so everyone watches the results like a spectator sport.
Me: Why is testing so important to your company? What has been the impact to your business?
Seth: Testing is critical to making sure that Web site optimization is not “two steps forward and one step back.” Often, we find that the results are counterintuitive to what we had assumed, so now we hesitate to make site changes without testing. The Web site has, of course, improved in efficiency throughout our company’s history, but testing has accelerated our rate of improvement markedly.
Me: How did you hear about SiteSpect? What attracted you to their offering?
Seth: When SiteSpect first made their sales pitch to us, we were drawn to the idea that we could test without altering the Web site code. If development has to insert extra JavaScript on the page or make other code alterations, it creates a bottleneck in the site optimization process. SiteSpect also gives us access to the raw data from tests allowing us to do deep statistical dives and segmentation rather then just looking at aggregated data.
Me: How easy is SiteSpect to begin using? How long did it take after you signed a deal to start your first test?
Seth: SiteSpect uses regular expressions heavily, so that was the bulk of the learning curve. We were able to deploy SiteSpect quickly and had a test up immediately thereafter.
Me: How has SiteSpect lived up to what you expected? What features distinguish it from the competition?
Seth: We have been pleased with the results from SiteSpect. It is highly adaptable, easy to integrate with Omniture and other web analytics vendors, and provides a lot more statistical insight into our test results then most platforms.
Me: Is there anything about the way you do testing that my readers would like to know about that I haven’t asked about? What would you advise people who aren’t conducting tests? What would you advise people who are conducting tests but are not using SiteSpect?
Seth: All online business boils down to one simple equation: traffic x conversion rate x conversion value = revenue. The more money you invest in advertising, the more you ought be investing in making that traffic count. SiteSpect offers a sophisticated tool at a good price.
Me: Thanks so much for sharing your experience with my readers.

Avatar

Mike Moran

Mike Moran is an expert in digital marketing, search technology, social media, text analytics, web personalization, and web metrics, who, as a Certified Speaking Professional, regularly makes speaking appearances. Mike’s previous appearances include keynote speaking appearances worldwide. Mike serves as a senior strategist for Converseon, an AI powered consumer intelligence technology and consulting firm. He is also a senior strategist for SoloSegment, a marketing automation software solutions and services firm. Mike also serves as a member of the Board of Directors of SEMPO. Mike spent 30 years at IBM, rising to Distinguished Engineer, an executive-level technical position. Mike held various roles in his IBM career, including eight years at IBM’s customer-facing website, ibm.com, most recently as the Manager of ibm.com Web Experience, where he led 65 information architects, web designers, webmasters, programmers, and technical architects around the world. Mike's newest book is Outside-In Marketing with world-renowned author James Mathewson. He is co-author of the best-selling Search Engine Marketing, Inc. (with fellow search marketing expert Bill Hunt), now in its Third Edition. Mike is also the author of the acclaimed internet marketing book, Do It Wrong Quickly: How the Web Changes the Old Marketing Rules, named one of best business books of 2007 by the Miami Herald. Mike founded and writes for Biznology® and writes regularly for other blogs. In addition to Mike’s broad technical background, he holds an Advanced Certificate in Market Management Practice from the Royal UK Charter Institute of Marketing and is a Visiting Lecturer at the University of Virginia’s Darden School of Business. He also teaches at Rutgers Business School. He is a Senior Fellow at the Society for New Communications Research. Mike worked at ibm.com from 1998 through 2006, pioneering IBM’s successful search marketing program. IBM’s website of over two million pages was a classic “big company” website that has traditionally been difficult to optimize for search marketing. Mike, working with Bill Hunt, developed a strategy for search engine marketing that works for any business, large or small. Moran and Hunt spearheaded IBM’s content improvement that has resulted in dramatic gains in traffic from Google and other internet portals.

Join the Discussion

Your email address will not be published. Required fields are marked *

Back to top