Why wrong measurements can be bad for your community’s health…
In my update on the 2008 Tribalization of Business study on business communities that we are doing with Deloitte and The Society for New Communications Research last week – I pointed out how some companies are totally misaligning their measurements of community effectiveness with their goals.
As you will see from the slides, many companies measure effectiveness by looking at page views and time spent on the site. Yet not one company listed ad revenue as a goal for the community – which is what page views and time spent on the site would be good for. Let’s assume that your goal is to have a support community – one in which people can help one another or get help from some your employees. If you could deliver the support in a way that never required people to come to your site, you would still achieve your goals. In fact, if you build your community so that people do not have to come to it, chances are that you will have more people participating in it. There are only so many destinations that a person will visit on a regular basis, and chances that your business community becomes one of them are fairly slim.
Another interesting wrong-headed metric-related finding from the study is that a majority of respondents found that “getting people to engage” was one of the biggest obstacles to making a community work. Now if you have a small community, chances are that you could get a fairly high engagement rate. The larger your community becomes, however, the more its profile will resemble that of large public communities – 1% of hardcore contributors, 10% of active users and 80-90% of lurkers. Now does that mean that the lurkers do not get value from your community? In the case of the customer support community, lurkers who do not contribute could still find the help they need and feel better about you than if they had not found it and also save you the cost of a call into the call center. So measuring community effectiveness by measuring engagement is just not a representative metric of community success.
Now the real issue with all this is that if you have a community development team who is being measured by those wrong-headed metrics, they will invariably develop bad behaviors in order to maximize these metrics. They could in fact develop community features that will stand in the way of success for your communities, or close down communities that are in fact doing really well.
If you missed it, there is a dynamic conversation on managing communities going on right now…Chris Brogan kicked it off and Nancy White wrote some interesting musings and also kept track of many of the other interesting links.
You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.










May 2nd, 2008 at 5:22 am
Speaking about the engagement one, I feel it’s always that way. The current number of people coming to my blog by way of RSS and directly is around 6,000, and my current average comments on a post is around 40. So what’s that, less than 1%? Engagement. But then, is commenting the ONLY engagement? Are links just as good? Follow on posts?
It’s a tricky number, as you’ve pointed out. Great post.
May 28th, 2008 at 11:18 am
Using the metrics that you’re talking about to lure media buyers is standard, yet, as you said, they’re not necessarily accurate. If we develop standardized, third-party online media auditing (as is done with print media), advertisers can be assured that their messages are reaching the right people in accurate numbers. Our team just built http://www.buysafemedia.com and I think it has some useful information on the subject of print media auditing. Do you think we apply the same principles to the Internet?
July 22nd, 2008 at 1:11 pm
[...] Why wrong measurements can be bad for your community’s health [...]
August 10th, 2008 at 8:57 pm
Hi Francios,
I couldn’t agree with you more. Success metrics and measurement is one of the bigger challenges today. Web managers and web analytics groups as they exist today are just extended over to the community management tasks and they follow their traditional methods.
Also business owners take those metrics in exactly the same fashion as they do for traditional web media and apply it to community decision making.
The reality here is that success metrics for any community is truly unique to the end goals of that community, mostly revolving around relationships and unless those are not clearly defined and accepted, any measurement applied will most likely be incorrect.
Do you have any examples of successful measurements used in this space?
Thanks again for some truly insightful articles.
Sav