Hindsight 20/20 Series: 3. Metrics – social business friend or foe?

This is part 3 of the Hindsight 20/20 series (here are the links to Part 1, “Your choice of social platform does matter”, and 2, “Beware of the social digital divide(s)”). This month’s post deals with the thorny issue of relying on metrics to assess the success of internal enterprise social platforms. Are metrics important? If not, why bother? If so, which ones tell you what is really happening? Naturally, full books can be written on this subject, so the modest objective of this post is to zoom in on typical questions that may come up when implementing your social platform. Here are some of the topics that you may want to consider throughout your implementation. 1. Why bother? Are metrics important? Isn’t “social” the only way to go? You, and many of the early adopters in your organization, may think that the need for a social platform is so obvious that you don’t need explanations. However, chances are that for the vast majority of potential users, the jury on the actual benefits of going social is still out.

There are lots of missed predictions when it comes to innovative technologies. Just take a look at what IBM predicted back in 2008 about innovations that would change the way people work, live and play over the next five years. While some of them are becoming a reality, others are still not more than wishful thinking. Thus, it’s not reasonable to expect others to just believe that a social business platform is as crucial to large organization as offices, desks, phones and computers are. Collecting data to support your point of view will help you in the long term. 2. OK, so data is important, but which metrics should I collect? At a minimum, you should collect all data that you easily have access to. While that sounds like a very wasteful process, it’s very important that you adopt a comprehensive data strategy. Due to the relative novelty of social technologies in an enterprise context, the first few years of implementation resemble scientists trying to figure out the unknown: the deep oceans, outer space, or variations at geological scales. At the beginning, you don’t know exactly what you are looking for, so you aim for everything that’s possible to gather within your resource boundaries. After a while, you’ll start seeing patterns, and can narrow down your analysis to the metrics that are really telling you a story. The good news is that mature social platforms can be your friend: they will automatically collect a vast array of metrics, and they will already have a number of canned reports for you, which are likely to address your most basic needs. However, each organization is different, and you’ll need to augment that basic reporting framework with your own. Make sure you partner with the folks running your enterprise reporting solutions and big data analysis tools: you’ll be amazed about how much information you have hidden in your content and logs. 3. I did that and don’t have much to show for it. What am I missing? In the first couple of years, most of your data will tell you roughly how well you are doing in terms of adoption growth and vitality, but not much else. You will need to rely heavily on surveys and story telling to better understand what is really happening out there. So, make sure you establish a baseline via a user survey. Understand what the major challenges are that your organization faces on the communications and collaboration fronts. From that point on, run quarterly surveys to monitor if progress is being made. Avoid survey fatigue by making sure users are not requested to answer the same survey more than once a year. Also, create a mechanism for users to share their success stories with others. If you feel bold, encourage them to share failures too: there is a lot to be learned from initiatives that came short from expectations. 4. My numbers are all over the place. Why is that happening? Metrics for a social business platform must be assessed on a use case by use case basis. Some, such as employee communications, are enterprise wide, and are looking for broad viewership and high participation. Others, such as project collaboration, tend to be limited to a subset of users, and may have a small number of power users and a large number of “lurkers”. You may even have “seeding the machine” use cases: content that is posted for reference or future use, and has low visibility initially, but may be important in the future when users are looking for information. Thus, a community with content that has lots of views but few comments is not necessarily a failure. For example, lots of views but few new content posted is a desired usage profile for support communities,  as it typically means that most of the questions have already been answered, and can be easily found via search. Likewise, a community can have lots of activity but not necessarily generate much business value, if all that people are doing are talking about the weekend on sports or how cute their cats are. Of course, this type of “non-business” community has a role in the overall implementation, but if that’s the majority of the activity in your social platform, your numbers may look good, but you may not be realizing much of the potential business value. 5. I still can’t tell if this is working or not. What gives? Where you are in the life cycle matters: metrics that are important on the first year may not be relevant in the long term. Also, after a year or two, you are expected to have more dead, inactive or so-so communities than wildly successful ones. That is not an indication that things have failed. It’s similar to how the consumer Internet is very successful despite the fact that most websites out there are also moribund or have very few viewers. As time goes by, you should focus on the few communities that made the transition from experimental to established places for conversations and collaboration, and leverage their best practices to revitalize the ones that are lukewarm and enhance the chances of success for the new ones. Also, you may want to invest more on the use cases had the best returns so far, and divest from the ones that demand too much effort with very few results. Having worked as a performance engineer for years, I have always advocated the importance of substantiating recommendations based on actual numbers. I’ve often quoted the fictitious Lazarus Long character to support that view: “If it can’t be expressed in figures, it is not science; it is opinion” (source). However, while one can tell without a shade of doubt that a piece of Java code is misbehaving and be very prescriptive about what the solution is, social platforms don’t typically benefit from such a deterministic approach. On the other side, investing in social platforms is a significant endeavor, and should not be just a leap of faith. Many social media enthusiasts brush aside the importance of metrics, and often quote Plato to defend their position: “A good decision is based on knowledge and not on numbers”. What they fail to consider is that Plato also said: “Numbers are the highest degree of knowledge.  It is knowledge itself” (in Epinomis). The right balance, as usual, is somewhere in between, so this may be the most appropriate quote to rely on: “99 percent of all statistics only tell 49 percent of the story” (“Gents with no Cents).