Steve Arnold says yes. Who’s Steve Arnold? He’s both the president of Arnold Information Technology and an author who spoke at The Search Engine Meeting yesterday, the same conference I spoke at on Monday. Apparently my talk, which I considered rather uncontroversial, piqued Steve’s interest, because he referred to it a half dozen times during his own talk, describing the techniques that I advocated as “spamming.” I don’t know anyone that agrees with him, but he says his charge speaks to the generational divide in search technologists.
I began working in publishing and search in the late ’70s, so I should clearly be part of that older generation. I clearly recall search engines such as IBM’s STAIRS, which, while interesting at the time, pale in comparison to what we expect of search engines today. A large database back in the 1970s was a few hundred thousand documents, which is the number of documents returned by an average Google search today. Searchers had to understand what was in the database, what nomenclature was used, and the Boolean syntax to enter a search query. I was part of the team that developed the first commercial search engine that used linguistics (so that the word “mice” matches “mouse”), which is part of every search facility in use today. While we were all pioneers back then, I don’t think even Steve wants to return to those days.
But Steve does seem to pine for the days when all documents were manually classified by trained librarians and researchers. He rails against folksonomies, such as del.icio.us, and bemoans the “manipulation” that content providers can exercise over search results, highlighting an article in his talk where digg was exposed as susceptible to trickery. In response to a question from the audience after his talk, he agreed that Google’s results are “corrupt.” And, honestly, he has a point. There’s nothing magical about folksonomies giving you the right answer—for popular subjects, they may do just fine, but a researcher-tagged database might do a lot better for other subjects. And neither Google nor any other search engine gives you objectively relevant results, but that is because there is no such thing as objective relevance—relevance is in the eye of the beholder. The bottom line is that any system undoubtedly has its Achilles’ heel, where someone out to manipulate results can do so. A good system is hard to manipulate and strives to improve every day in that respect. In my opinion, Google, Yahoo!, MSN Search, and many other systems qualify as good systems in that respect. So Steve has a point, but only because objective relevance does not exist.
But Steve goes too far when he describes every technique that makes content attractive to a search engine as “spam,” as he did in his talk. In fact, he made it clear that he considered the advice I gave to folks to optimize their content to be advising them to spam the search engines. Now, I suppose folks can disagree on the meaning of the word “spam,” but to me, the search engines’ terms of service define spam. The search engines describe many techniques that go beyond that ethical line, techniques that they consider to be spam, everything from keyword stuffing to link farms to cloaking. Bill Hunt and I spend lots of space in our book, Search Engine Marketing, Inc., explaining what these techniques are, why they are bad for searchers, search engines, and even search marketers. These techniques are unethical and we don’t advocate them in any way.
But Steve doesn’t accept the standard definition of spam, expanding “spam” (in his mind) to include anything that makes your content more attractive to search engines—including all the legitimate techniques of search engine optimization. By doing so, Steve impugns the ethics of any search marketer pursuing efforts that are perfectly acceptable to the search engines. He dilutes the term “spam” and dissipates the outrage possible for real spam techniques. He makes a mockery of the work that Google’s Matt Cutts and other search engine employees do to combat genuine spam, by extension calling into question why their terms of service are so lenient. If any organic search marketing technique can be called “spam,” the term ceases to have any meaning.
Steve especially took issue with my advice that people “add keywords to their content” so that search engines would find their pages. (I advised this in the context of Web site search, for which there is no spam issue, but I also mentioned that this technique helps your rankings with Internet search engines such as Google.) I showed that IBM’s page on Product Lifecycle Management went from #175 in Google to #1 when we did a few simple things such as adding those keywords to the page. (The keywords were missing completely originally, so adding them to page hardly constitutes the spam technique of keyword stuffing.) Steve stated that merely adding keywords to a page is spamming. I don’t know why IBM’s leading offering for Product Lifecycle Management ought to be relegated to #175 just because we didn’t understand what search engines are looking for, but that appears to be what Steve is advocating. Steve went so far as to say that search engine optimization techniques are leading to the “end of relevance.” (This will come as a shock to the hordes of searchers that rely on these results each day because they are more convenient and often more comprehensive than the libraries and manually-tagged databases of yore.) Andrew McKay of FAST commented later in the day that Steve’s outlook was the “most defeatist thing he had heard in a long time—just because not all problems have been solved is there a reason to give up.”
In a private discussion after his talk, I challenged Steve on his definition of spam, pointing out that his definition is one that even the search engines don’t share. Steve did not defend his view in any way, merely saying, “I apologize” but he also went on to say that he will continue to use the word “spam” the same way in the future. As I continued to ask him why he would keep using the term “spam” to include ethical behavior, he continued to remind me that he had apologized, but offered no insight into his reasoning. Honestly, when someone apologizes for doing something but says that he is going to continue to do it, it’s not clear how heart-felt that is.
Steve also seems nostalgic about the days of manual tagging by experts, but those systems seem every bit as susceptible to manipulation as any other. There are scattered reports of payola to Open Directory editors for listing Web sites in that directory. Whether they are true or not, it’s clear that having human beings involved in the tagging process does not eliminate the possibility of manipulation.
Steve would have you believe that we are somehow worse off with the search we have today than the researcher’s walled garden of yesteryear, but I don’t see how. Clearly, it would be troubling if tobacco companies dominated the search results for “smoking,” but they don’t. Pharmaceutical companies don’t dominate the results for diseases, and political parties don’t hold sway on public issues. In fact, the Web, and search on the Web especially, has given voice to folks without big money behind them in a way that no other media has.
When pressed on his worldview that demonizes ethical search optimization behavior, Steve described himself as coming from the camp of manually indexing content, saying, “I am 64 years old and not about to change” and, “If I was 30, maybe I would have to adapt to these new things.” Perhaps. But he certainly has adopted that new-fangled terminology of “spam,” so it would be nice if he used the word properly instead of getting attention by creating artificial controversy. As someone old enough to remember the old days, I’m here to tell you they weren’t all that good. I’d also like to think that we could discuss the changes that have occurred in search over the years without questioning people’s character, but to each his own. I hope that my experience that goes back over 25 years helps inform me about the new things you see today, rather than causing me to wax nostalgic. I also hope I never feel I am so old that I can’t change my opinion in the light of a changing world.
