Biznology
Where business and technology create a winning customer experience.

searcherJared Spool is one of the few user experience pros who focuses on Web site search. And when he says something, I pay attention. His post yesterday crams into one article more about site search behavior than most people will ever know. But I have one disagreement with him. It’s rare, but it happens.

 Jared says that his research shows that people using site search want just one answer rather than a list of great search results. I’m certain that he is interpreting his research correctly, but I wonder if there might be some selection bias at work. Here’s why I say that.

My friend Andrei Broder worked with a few other luminaries years back to identify several different types of searches: navigational, informational, and transactional. This was a great insight, because it was generally thought that all searches were informational—the searcher desired a screen-full of useful search results to choose from. Andrei and colleagues proved that some searches are attempts to do something (transactional) or go somewhere (navigational) and they typically need the right result at the top.

And many Web site searches do require just one correct answer, as Jared points out. But not all of them. In my work at ibm.com, I noticed that the most preliminary searches often were informational ones. Someone might search for “e-mail archiving case studies”—they don’t want to get just one. Now, sure, if you have a page on your site that lists every blessed e-mail archiving case study, that would be a great #1 result, but you usually don’t have that kind of aggregation page for every possible query.

Searchers would not want your “Content Management Case Studies” page as #1, even if that list included every e-mail archiving case study, because it also includes too many other irrelevant case studies. Instead, searchers would love a list of case studies that match the query. They could scan through that list and click several results, drinking in that practical information they crave.

If I am right, then why did Jared’s study show otherwise? It might have to do with sample bias. When search tests are defined, they often focus on popular queries. Those queries are far more likely to have a single page that aggregates the information in one place, simply because so many people are interested in the subject. I suspect that if unpopular, unusual search queries were studied—and those queries are the bulk of the volume—you might see a different result. I can’t prove it however, so I’d love to see more information about what kinds of queries were tested.

Jared is smart to tell folks that not all queries are informational. He might even be on solid ground to say that most are not. I just think that a good portion of them are.
So, do listen to Jared. About almost everything. But just understand that some searches are informational searches, where a rich list of choices is exactly what your customer needs.

Tweet about this on TwitterShare on FacebookShare on Google+Pin on PinterestShare on LinkedInEmail this to someone
Mike Moran

About Mike Moran

Mike Moran has a unique blend of marketing and technology skills that he applies to raise return on investment for large marketing programs. Mike is a former IBM Distinguished Engineer and a senior strategist at Converseon, a leading social consultancy. Mike is the author of two books on digital marketing, an instructor at several leading universities, as well as a Senior Fellow at the Society for New Communications Research.

9 replies to this post
  1. People may just need one “right” answer, but the “right” answer for you may not be the answer I’m looking for, or the page I’m looking for may not exist. Since the engines cannot read between lines (or between words) to see exactly what the person is looking for from the search term, it’s good to have multiple results.
    Still, I think results on 2nd page and after are waste. If engines limit the number of search results to just top 10, it would save time and energy. ;)

  2. Nice post! And thanks for all the nice words.
    For what’s it worth (and because WP trackbacks may not be working with MT), I’ve posted a detailed response here.
    There was one point I didn’t address in my response. I didn’t talk about the way we collected our data.
    It’s possible there’s a bias issue that is affecting our results, but I think it’s a low probability. The supposition I came to was from a collection of studies of people doing real tasks on real sites.
    None of the studies involved using top queries. Instead, they were all watching folks use the sites for things they needed to achieve on their own.
    So, if there’s a bias, it’s unlikely it was introduced by the method by which we looked at the Search data. And since this is across multiple studies with a variety of sites (from e-commerce to medical information), I don’t think it’s a selection problem.
    As for Andrei’s taxonomy, I think that’s great for dividing up queries, but I’m not sure it’s a good way to think about goals. Queries are an artifact of the crude type-something-into-a-box-and-cross-your-fingers approach to effective searches.
    In our work, queries don’t really match goals very well. They are just a manifestation of the user’s inability to clearly state what they are really trying to do.
    Think of the average query as the equivalent of a three-year-old complaining of stomach cramps. They just don’t have the skills to adequately describe what they need.
    Keep up the excellent work…
    Jared

  3. Motoko, I think that is the point of view that people typically take–the one Jared is trying to debunk. I think he has a good point but it doesn’t apply as universally as he thinks.
    Jared, I would never try to poke holes in your methodology (mostly because I don’t understand user testing methodology all that much). Perhaps the problem isn’t your selection bias, but mine. Maybe my years at IBM have caused me to think that the situation I cited was common, when perhaps it is uncommon. I thought that sites with long sales cycles that require lots of information would naturally have these kinds of searches, but maybe IBM was quite unusual in that respect. Thanks, Jared, for clarifying your methodology and reinforcing the point you made in the study. It’s news to me nad that makes it an even more interesting piece of work.

  4. “If engines limit the number of search results to just top 10, it would save time and energy. ;)”
    Is that a joke? That’s got to be a joke but just in case it isn’t I think that would obviously be a bad idea. People love to ‘surf’ the net. Sometimes folks will find the information they need and THEN continue to browse just for the sake of browsing. Especially woman. It’s like window shopping.

  5. Thanks for the comment, Chris. I think when someone puts a smiley face after a comment, it’s generally a good idea to interpret it as joke.
    But the research does show that very few searchers go to the second page of search results, and that number seems to dwindle with each passing year, as they decide to do another search instead. And I’ve never seen any research that shows women to be more likely to go to the second page, but perhaps you have? In the coming days of demographic targeting, any research that clearly shows such patterns could be helpful.

  6. I didn’t notice the smiley thing! LOL. I figured it was a joke but you never know. No I have seen no research about woman going to page two more than men but it’s a browsing thing.
    When you go shopping with your wife or girlfriend isn’t it different then when a guy shops solo? Woman want to browse more it seems to me.
    I had a conversation with someone the other day who really only gets online to download music or watch movies from limewire. He didn’t know there was a second page!!! LMAO!!!
    I think most regular people who browse the net automatically assume that the top ranking result is, for some reason, the ‘best’ one.

  7. Hi Mike…
    I think it depends upon the type of search more..If I want to buy something then I will try to check the prices at different sites quoting the different rates and discounts. If I get the same product at lesser prices at different sites, then I would like to try out these different sites for sure..And dats the reason I would like to have a lot of search results in my search..

Webinars

video

Yesterday, our author Andrew Schulkind presented our latest Biznology webinar about content marketing. If you've ever been greeted by a deafening silence after...