Last month, I argued that your marketing investments should not be influenced by measurement limitations. The higher you get in the marketing funnel, the harder it is to demonstrate the value of your content marketing. At base, awareness marketing content is a great branding vehicle. But brand value is a qualitative measure that doesn’t often get the investment it deserves. What other ways can we measure the value of awareness marketing?
There are attribution models that can help you demonstrate lead generation from awareness marketing. But they are imperfect for a very simple reason: for a large number of leads, we legally cannot track our users with the kind of granularity needed to prove that our efforts cause leads to happen. Much of the time, the best we can do is demonstrate the correlation between running an awareness campaign and the leads that happen downstream from these activities.
Observer Effects in Digital Marketing
I call these issues observer effects, because they remind me of what happens in science when our very measurements affect what we are measuring. An example is in quantum mechanics. Heisenberg proved that our very observation of objects changes the way they behave. Perhaps a more relevant example is in social science, where the existence of observers changes the way their subjects behave.
Legal limits
In digital marketing, we are constantly running into these walls for two primary reasons: legal and behavioral. Most countries have laws that prevent companies from gathering personally identifiable information (PII) and using it to target individuals. These so-called do-not-track laws (about which I have written in InformIT) define hard limits on our measurements.
In my job for IBM, for example, we design all our cookies to expire when a user leaves our site. We do this to prevent ourselves from gathering PII for those users. But this prevents us from measuring our repeat visitors, which are tagged as “null” referrers–a geeky way of saying they’re not coded as anything in particular. Calling visits from bookmarks or autocomplete typing in the browser “null referrers” is a derogatory term meaning “we don’t care.” We seem to only care about tactics that are tied to marketing investments, and null referrers cannot be tied to such tactics. Yet, we also know these are some of our most engaged visitors. They convert at an alarmingly high rate. But we can’t tell from what drive-to tactic they first found our site. And we desperately want to.
Behavioral limits
IBM is probably the most careful company on the planet when it comes to legal and regulatory issues. Some companies take a different interpretation of the do-not-track laws than we do. Their cookies do not expire. Some companies continue to gather user data as they navigate and search. When users find out about this, they get angry. The reason for the laws, I hope, is consumer pressure to prevent companies from violating user privacy.
Users hate being tracked. They hate having to opt out of marketing. They especially hate having to opt out of marketing that seems creepily tuned to their past digital behaviors. For example, I hate it when I have to close out of a pop-up ad from Amazon for a product that I’ve already purchased. I lost track of the number of times I’ve had to do this for my own books. And users don’t just hate this kind of targeting because it is often badly done. They hate it even when it’s done well.
If you are inclined to flaunt privacy laws, it rarely works, because users hate having their privacy violated. So tracking them changes their behavior. And that adds a lot of uncertainty to your results.
How Do We Measure With Observer Effects?
Part of the answer is we can’t measure our users with absolute certainty. We have to accept that digital marketing measurement is not all science. It’s almost as much art as science. If we embrace the art, we can get better information on our users, and ultimately serve them better without violating their privacy. Rather than describing the science, which is well documented all over the place, I thought I’d save some of your time and attention by merely mentioning two of the arts we need to embrace to do this.
Correlation measurements
Marketers have used these metrics as long as marketing has been a thing. “We ran this TV ad and we suddenly saw an uptick in visits to the store.” In offline media, this still is the model. But because we can gather all kinds of transactional data in online media, it is often dismissed as an invalid way to demonstrate the value of digital marketing activities.
But it need not be so. And in some cases, correlation is simply the best we can do. So let’s embrace it and try to make it more accurate. For example, why not tie online and offline media together by putting digital artifacts (hashtags, short URLs, etc.) into offline ads, and measuring the results. It’s still correlation, but it is more clear than simply, “we ran this ad and we had a corresponding spike in traffic.”
My favorite thing to do along these lines is to map null referrers to search referrers. I have noticed a regular correlation here: Whenever our organic search referrals go up, our null referrals go up about the same amount. If it doesn’t, this is a negative signal similar to high bounce rates. If your content is deep enough, it will take multiple sessions for an engaged user to consume it all. Given limited time slots, they return to the best sites. If they don’t, it’s an indication that we could do a better job. To prove that, we should probably do some multivariate testing. But low null referrers is a red flag.
Brand value measurements
Sometimes the best thing to learn, especially about awareness sites, is how these pages influence users to develop trust in your company, which is another way of talking about brand value. We can measure this with social sentiment and other digital means. But we can also measure this by simply asking them. I’m talking about surveys on or off your site that are built with the best social science in them.
One of the challenges of surveys is making it easy for users to give you feedback without needing to opt out. In my experience, opt-in surveys get insignificant results. But if you force users to opt out of a pop-up survey, you will tend to measure the people who are motivated to give you feedback. Considering that users hate having to opt out, more users will tend to be motivated to give you negative feedback. This can be helpful in finding and solving user problems with your site. But it doesn’t tend to lead to accurate brand value measurement.
I recommend taking a cue from blogs and building moderated comments into your site design. User motivations for commenting are also somewhat skewed, but less so than opt-out surveys. And because they’re open-ended, you give them the control over the form of their answers. You can send these through semantic and sentiment algorithms to make brand value judgements about them. But that science is rapidly maturing.
There are many more artful ways of learning about your users without violating their privacy. But the most important thing is to enable your executives to accept both the art and the science of digital measurement. If you do a good job with this, you will likely see your budgets for awareness marketing grow.
