For ranking algorithms, AdAge is no Google

As someone who has worked on ranking algorithms, I know how difficult they are to develop. No, check that. They are easy to develop. They are hard to develop only if you want them to actually work. All this was brought to mind recently when Advertising Age (“AdAge” to the in crowd). changed the ranking algorithm for its Power 150, the well-known rating mechanism for all marketing blogs, including this one. I’ve proudly displayed it on the Biznology blog for years, but I’m thinking about taking it down from the site permanently, because I just don’t understand how this revamped algorithm is accurate. And when your algorithm loses that trust, things can shift in a hurry. The bottom line is that AdAge is no Google when it comes to ranking algorithms.

Google has probably spent a little more time on its ranking algorithm than AdAge, of course, and it is certainly more complex. Google’s algorithm actually has two parts–the query factors and the page factors. The query factors are about what search word you typed in the box, while the page factors try to identify which pages are quality pages from quality sites. It’s those page factors that matter for this discussion, because it’s exactly the same problem that AdAge tries to solve with its Power 150 rankings.

But that’s where the similarity ends, because Google doesn’t publish its ranking algorithm’s components, and Google certainly doesn’t explain how the algorithm changes over time. AdAge explains exactly what its ranking factors are, and was up front about changing them recently.

Unfortunately, not everyone is excited about the changes, including me. This blog careened almost 400 spots in a single day, from the high 200s to the high 600s in ranking.  (For those of you outside of the marketing blog echo chamber, even though it is called the AdAge 150, it ranks far more blogs than that.) Because our blog would seem to have been just as good one day as the next, that seemed a bit odd to me. I have reached out to the folks at AdAge to find out what is going on, but have not been favored with a reply.

After a little investigation, I found some discussions on how AdAge has changed its blog ranking algorithm, replacing two important parts of the algorithm, PostRank and Collective Intellect, who both withdrew the APIs that AdAge has been counting on. To me, PostRank was the most of important of these two, using complex algorithms to determine the level of engagement and sharing of the posts. Both sources were considered to be quite accurate, which is why AdAge was using them. Under the old algorithm, Biznology bounced around some, but it has been steadily improving for the last six months as we have upgraded our look and feel, moved to a dedicated site, added subscribers and visitors, and (the biggest improvement of all) added several new writers that have diversified the writing from listening to just me most days. We’ve always had good writers on Biznology, but I wrote at least three times a week–now we have more diverse opinions from more writers that build on the great folks we’ve always had. I thought we were moving in the right direction.

But suddenly, AdAge does not agree with me. I have looked at our AdAge ranking every day for years as a kind of quick gauge of how we were doing, and I know that many of our industry did. That is, until a couple of weeks ago, when the new algorithm debuted.

Now, AdAge had no choice but to change the algorithm because two of the sources they relied on were disappearing. So, they did not have the option of leaving things alone. But the problem is what they replaced them with. Instead of those two rich sources of data, they are now using two rather simple ones: the number of shares on Facebook and tweets on Twitter.

And thus the plummet for Biznology. Apparently, our readers share our content in these venues less than in other venues, so the blog’s ranking has dropped like a stone.  Now, just the fact that our blog dropped in rankings is no reason to declare the new algorithm flawed. Even when it drops over 300 places. For every blog that dropped, another one rises and that favored blog writer probably thinks the new algorithm is great.

But I have seen complaints from others on the new algorithm. One especially well-written piece alleges that one high-ranking blog under the new AdAge ranking system has not been updated since 2009. If true, that’s not high praise for the new algorithm. I’ve casually looked at the new list of top blogs and I’ve seen significant drops for several that I think are very good.

Because of all of that, I am considering removing the AdAge Power 150 badge from Biznology as others have done. It’s sad, because it has been a badge of quality that I have been proud of, but it’s hard to believe in this ranking anymore.

Why am I bringing all this up? Because search marketers face the same wrath of Google every day as the ranking algorithm fluctuates. And while we might rant and rave about the vagaries, Google must live with the changes that it makes. Google must test them to be sure that they are really improvements. And if Google starts to lose its way with its ranking algorithm, people will stop trusting the results, just as with AdAge. And that isn’t good for Google or AdAge.

The recent woes of AdAge are but a small glimpse into what Google goes through every day. They face changes in behavior of spammers, which a good ranking algorithm must cope with. (It’s possible that spam tweets and Facebook sharing might account for moribund blogs rising in the AdAge rankings.) They face changes in data availability, just as AdAge did, most recently when Twitter stopped allowing Google access to its firehose, which you can bet was part of its ranking algorithm. It’s one reason that Google developed Google+, to try to get at that realtime data again without depending on Twitter.

It costs an enormous amount to continually update and check that ranking algorithm at Google–money that AdAge undoubtedly does not have to spend on its ranking algorithm.  And the real irony in all of this? The reason that PostRank withdrew its public API is that it has itself been acquired by Google. Google now has all that data to use in its own ranking algorithm, sharing it with no one.

So, when it comes to ranking algorithms, AdAge is certainly no Google. I wonder if we should just rate all marketing blogs by PageRank and let it go at that. Oh wait– Google doesn’t show up-to-date PageRank ratings publicly any more either. Maybe we’ll have to go back to paying attention to what our readers tell us and give up the marketing blog ranking game entirely. That doesn’t seem like a bad thing, either.