Yesterday I talked about my questions about the methodology used in the 2012 Nonprofit Social Network Benchmark Report. Today, I’m going to talk about Problem #2.
Calculation of “value” based on self-reported approximates
Whoa, that sounds like a mouthful huh? So, what do I mean? Well, they have some very, very interesting statistics about the value of Facebook likes and the cost of Facebook likes/Twitter followers. Here are the two I’m focusing on:
- “The Average Value of a Facebook Like is $214.81 over the 12 months following acquisition.”
- “The Average Cost of a Facebook Like is $3.50 and a Twitter Follower costs $2.05.”
These are two very powerful and very interesting statistics. This is something many, many people have been wondering and trying to figure out. As soon as I saw these statistics I wondered about how they were calculated. I wondered this because my whole dissertation is focused on how nonprofits can evaluate the value of their social media efforts – and ROI is a big factor in that. It’s very difficult to calculate the value of a like or cost of a follower because there are many factors to take into consideration, a few of them are: Was this person a previous donor? Are you 100% certain they donated because of Facebook? Are you 100% certain they increased their donation because of Facebook? Are they more active with your organization because of Facebook and how do you know that? How much staff time do all of your staff spend on social media? These are just a few questions that are important to consider.
So, I was thinking some complex analysis or detailed questions must have been used to find those statistics. Then I reviewed their appendix which included survey questions. Here are the two questions that informed the two stats on cost/value:
“13. What is the average cost for your organization to acquire a new member Facebook Like/Fan or Twitter Follower?”
“20.For new supporters acquired via Facebook, what is the average value of the supporter to your organization over the 12 months following acquisition?”
Yikes. That’s what I thought when I read those. Not because they are bad questions – they are great. But those were the only two questions I saw on this. That means the numbers they used were 100% self-reported. It also means that people could have answered these in a bunch of different ways. Some respondents might not have included staff costs – while others included staff and direct costs. There were no qualifiers. That means the data may not be reliable because we don’t know how people were answering. It seems as though these were new questions this year, hopefully next year these are beefed up a bit to ensure better data.
Note: In the report, under the statistic about average cost they do have a caveat about the numbers being self-reported. None the less, this is an important problem to note with these data statistics.
Enjoy this post? Read about Problem #1, Problem #3, Problem #4, and the Official Response from NTEN/Common Knowledge about the report Part 1 and Part 2.
Photo Credit: Ryantron
{ 2 comments… read them below or add one }
You are raising valid and excellent questions here, Kristen. Most people reading survey results aren’t familiar with research methods and statistics, and as a result, take the facts at face value.
I’ve seen many surveys that have misleading questions. They are skewed from the get go.
I’ll bet if you ask for the raw data, they might share it. Worth a try.
Thanks Elaine. I agree, most people read reports, news articles, etc and believe the information is true of all nonprofits – it doesn’t help many blogs, etc are reporting the survey stats as facts and not clarifying that the information isn’t generalizable.
I’d love to get my hands on the raw data, but I’m pretty confident they won’t give it to me. They still haven’t told me how they did their data analysis – so I doubt they’ll give me their data.