Do people ask you why your Web site’s search doesn’t use Google? That’s frustration talking—they didn’t find what they wanted and they think that Google will magically fix that. Are they right? Read this month’s Biznology newsletter—Can Google Fix Your Web Site’s Search?—to find out.
Just 34% of Web Site searchers find what they are looking for. Why are searchers so disappointed in Web site searches? Blame it on Google. Oh yes, some Web site search engines are just plain bad, but the biggest problem is one of expectations—Google raised ’em and searchers expect your Web site search to work as well as Google. Why doesn’t it? And why shouldn’t you just replace your site search engine with Google? Wouldn’t that fix everything?
Probably not. The sad truth is that Google (or any good search engine) may do a better job searching the whole Web than it does searching a single Web site. To find out why this is so, we need to examine what makes Google and other Web search engines seem so magical.
The Web’s Content Marketplace
When you think about Google searching the billions of pages on the Web for the best answer to a searcher’s query, it sounds hard to find the right result, but the truth is that many correct results exist for most queries. If any of the authors on any Web site did a good job of optimizing those “correct answer” pages for search, then Google will likely find that answer.
Think about a search for “HDTV”—perhaps the searcher is at the very beginning of the purchase process and is not even sure what they are, but everyone seems to be talking about them. A Google search for “HDTV” will bring up places to buy them, but will also show informational pages about HDTV.
How did Google find that perfect page on that perfect site among the billions of pages on millions of sites? Google’s two main techniques do the trick:
- Keywords. Google looks at the words in the query and the words on all those billions of pages to find some pages that seem to match the query very closely.
- Links. Because millions of pages have the word “HDTV” on them, the keywords aren’t enough. Google looks at which pages have the most links to them—links with “HDTV” as the anchor text—and which pages have links from the most authoritative places.
Focusing on keywords and on links works well for queries across the entire Web because of the content marketplace. You can think of the Web as a place in which each page competes in a market of attention. Authors optimize their pages for searchability by repeating popular search keywords as part of their writing style. In addition, these pages attract links when they are well-written and contain useful information.
When pages compete this way, searchers win, because Google can find at least one good answer for almost any query. It has billions of pages to choose from and it needs only one or two to have optimized content or strong relevant links. As long as a couple of pages among the billions match, the searcher gets the right answer.
Your Site’s Content Monopoly
Contrast Google’s seat atop the content marketplace with your poor old site search engine. At first blush, it seems easier to find the searcher’s answer among the pages on a single Web site, because there are so many fewer pages to look at. But it isn’t.
Suppose our HDTV searcher started at Sharp’s Web site, because he has been very happy with the Sharp big screen TV he bought a few years ago. Searching for “HDTV” on Sharp might not bring up any explanation of HDTV at all, just a list of models, which would prove a frustrating result for a searcher just trying to learn what all the HDTV fuss is about.
But is it really the Web site search engine that is at fault here? Would Google’s site search products do any better at searching Sharp’s site? Most likely they would not, because site search is much harder than Google’s full Web search. Unlike Google’s search of the entire Web, where many Web sites could provide the correct answer, on your corporate Web site only one group is allowed to provide the answer, in this case the HDTV group. There’s no competition. If that one department does not optimize their content, the site search engine won’t find the answer. Sharp’s HDTV group may have neglected to provide any introductory information about HDTV, or may have done a poor job optimizing that page for search.
But the site search engine’s job is harder still. As we explained earlier, the content marketplace of the Web allows Google to examine links to pages anywhere on the Web to find relevant answers, but this content marketplace does not exist on a corporate Web site. Your corporate search spider does not even see these links—it crawls only pages on your own site, not pages all over the Web that link to your site. So if the world at large has voted with its links that one of your pages is a great answer for a certain query, your site search engine won’t know it because it is limited to analyzing links between pages on your site, not all inbound links from other Web sites. And the Sharp team that works on microwave ovens has little reason to ever link to Sharp’s HDTV pages. Most corporate Web sites have few links from one part of its site to another, providing little grist for a search engine’s link analysis mill.
Such is the danger of a content monopoly—all information coming from only one site. The proof of this comes from Google itself, which sells its technology to corporations for use in their site searches. Experts can debate whether Google’s site search products are better or worse than others in the market, but it is far from the leading seller. Like any product, some customers have tried it and brought in a replacement. Google may sell a good product, but it is no instant cure to the woes of site search.
How Can You Fix Site Search?
Google’s technology won’t fix your site search because no one’s technology will—at least not by itself. Oh, Google and plenty of other search engines do a good job, but most of what ails site search is not about technology—it’s about your content. You have two basic ways to improve your site’s content:
- Top-down. Determine the most popular queries for your Web site’s search and ensure that they provide correct results, as recently explained in a Biznology blog entry called “Improving Your Web Site’s Top Searches.”
- Bottom-up. Optimize every page on your site, so that you provide good results for even the most unique queries. You can borrow a page from search marketing to improve your site search, by making sure every page is indexed, and that every page has its content optimized.
Search technology is very important, but no technology can find pages on a corporate Web site that don’t get indexed or that don’t even use the searcher’s words on the page. Replacing your site search engine might improve your results, but almost every site could improve results with its current search engine by fixing its content. If you improve your content, you may not have to answer any more questions about why you don’t use Google on your Web site.
