I’ve written before about multivariate testing, the best way of optimizing your Web site to drive conversions. I was lucky today to speak with the president of ReallyGreatRate.com, Matt Schaub, who told me his story of how he’s improved his Web marketing through multivariate testing. If you’re not using this technique, read on to find out how it works.
ReallyGreatRate.com is an online financial lead generation company that connects lenders to people looking for student loans, debt consolidation loans, and home mortgages. The company does all of its marketing online, passing loan applicants to up to four lenders (getting compensated for each valid lead passed). You can imagine how valuable it is to increase the number of leads passed, which requires convincing more people to complete the online application form. 
But how do you do that? Everyone has an opinion of what should be changed. Multivariate testing boils those opinions down to testable sizes. Matt started with the submission form you see above. He worked with Optimost, a leading multivariate testing vendor, to identify 13 separate areas on the page that could be modified, and came up with seven or eight different treatments for each area.
For example, the masthead at the top of the page was one such area. Matt worked with Optimost to come up with several variations, such as changing the location of the image to the left side, changing the words in the tag line, and many others.
Optimost ran several rounds of testing to cycle through each variant, identifying the ones that seemed to outperform the rest—the ones that correlated with more loan applicants pressing that “Submit” button. Within four months, a winning design was chosen. 
It doesn’t look all that different, does it? It’s a bit less cluttered on the left side, removing some things that proved more distracting than helpful, and the most noticed text in the middle of the screen has been reworded to emphasize starting the process. But it looks very similar to what Matt already had.
Here’s what wasn’t so similar: Form completion went up 116%! A new form that didn’t look all that much different than the old one had dramatically different results. Now, no expert could look at these two forms and know for sure which would test better. And, remember, thousands of different variants were tested, which no expert could do either.
So now Matt is happy he’s got the best one, right? Wrong. “I’m starting a new round of testing this week,” he told me. “I’m going to constantly change my form. You have to! What works now may not work next month.”
When I tell people to “Do it wrong quickly,” this is what I am driving at. Experiment. Measure. Keep what is working and change all of it, working or not. Matt advised all Web marketers, “Never get complacent.”
