There’s a Sucker Born every Minute- Esp. in Social Media Measurement

I said it yesterday and I will say it again today: You are a breath of fresh air when it comes to social media measurement and analysis… and common sense, for that matter.

This is the level of conversation I expect to have with real professionals. More importantly, this is the level of conversation companies deserve from their Social Media teams.

This is absolutely fantastic.

You hooked up with the best IMHO when it comes to measuring Social Media ROI, Olivier understands exactly what needs to happen and cares passionately that others are misleading business owners and the like.

I asked the question this morning on Twitter, Do you respect Social Media or is it all about you?

So many are here ONLY to make a quick buck at the expense of those who make our world work. I can only say thank you for talking about this and I hope you will continue to fight the good fight.

My Moto is: How Many People Are Better Off Because You Lived?

Many Blessings,

Owen

Anna, this is fantastic and I can hardly wait to see where you take this. But, one thing is for sure, as traditional Business Intelligence and Predictive Analytics systems absorb Social Media Monitoring results, the value proposition for this entire marketplace changes.
Thanks again for this and I’m looking forward to more!

Bravo!

“Because any good data nerd knows that the inputs should be guided by your companies internal goals and not by some one with a white board and a video on youtube.”

Wonderful.

I also love the reference to the no-man’s land of reporting.

This is an excellent framework.

MOAR! I say, MOAR!

You mean there are people out there who take advantage of other people because they don’t understand something? Gasp! ROI is so basic, it’s silly. The ‘problem’ with it in social media is that it takes time to measure. When a new technology comes down the pipe, everyone thinks it’s a quick fix to results. And it’s not. It’s just like everything else – it takes time.

Nice post.

Great post! You’ve said exactly what I’ve been thinking about when it comes to the issue of social media measurement. Monitoring is only one part of the equation, but unfortunately not enough people recognize that.

The one thing that I think is still missing is a formula that enables us to assign or calculate a $ value to measurement metrics.

Thoughts?

Anna, this is outstanding work. I’d add a group bracket to the bottom of the measurement chart for Reputation, Revenue, Reduced Cost, closing the outcome loop. Of course, in measurement, that’s the Holy Grail, eh? We’ll continue to be seen more as artists than business people until we figure out how much impact each input has on these main outcomes. The marketing side will have an easier time of doing so, whilst those of us on the PR side will need to study a bit harder.

Thanks so much for this effort!
Sean
@commammo
(member of Institute for PR measurement commission…)

Found this via a tweet from Olivier – first time I’ve seen your blog and I’ll certainly be back. This is an excellent approach that dispenses with the squishy, rewritten ROI definitions that have become all too commonplace lately (return on engagement! return on interaction! return on kittens and rainbows!).

A lot of communications/PR folks are suddenly finding themselves responsible for social media at their respective organizations. Often these are people who aren’t very quantitatively-inclined (I’d venture most PR majors didn’t minor in math or statistics). We’re used to thinking and framing concepts with words. Many of us weren’t so great at defining and measuring the ROI of our traditional communications programs, let alone social media ones. So it perhaps becomes a natural extension to apply our word-based comfort zone to measuring social media and leads us focus on things like fans, followers, engagement or comments instead of dollars earned or saved. That’s not wrong in the sense that those things aren’t important and shouldn’t be considered (monitored) but it is wrong in that they shouldn’t be equated with or substituted for ROI.

So while you argue that “adding a qualitative aspect to a previously quant based world” has thrown people for a loop in terms of social media, I’d also argue that the opposite is true – social media is frequently being handled by people who are more comfortable with qualitative measures than quantifiable ones, and that’s what’s leading people to corrupt definitions and measures of ROI to make it fit their qualitative mindset.

Excellent chart, great discussion starter and I’m looking forward to the next in this series.

@amymengel

@jodi- I have some really good ideas on how to generally look at the financial realities of SM, but I really feel that you have to create the magic equation on a business by business basis.

@commammo- I think the overall reputation metric is a tricky 1 because I think it’s often a just a buzz word. I would say reputation could easily fall under sentiment analysis.. but that’s my thoughts. As for the other two if I was being specific I would loop them under the ROI/ROI forecasting section. Thoughts?

Anna, I think the next step in calculating true value is creating a generic equation or formula that can then be tailored to each organization’s specific needs. Perhaps you already have one in mind?

When I first started my career in PR (yikes – almost 20 years ago), we used to use Ad Equivalency metrics to measure PR value. I think a similar kind of formula/equation may be a starting point in helping us understand the true ROI of social media.

This Computerworld article on SM measurement is also a good read: http://bit.ly/2unDM3.

@JodiEchakowitz

Great post Anna. Agree with your key point about the difference in monitoring and measuring. Key distinction to be made. The Social Media analytics space is already complex and crowded with many monitoring tools, but few unified back-end tracking tools. Your framework is a good start. Wonder if tools will evolve that really put everything together in one dashboard, one data mashup. Not easy.

Where does integration with site Web Analytics tools take place in your framework? Metric Analytics?

@HartHooton

No bitter rage here. 😉

Monitoring and measurement are so easily confused because a person feels like they are being active when monitoring and believe they are producing results. Of course, those “results” already existed before the data mining, but the person monitoring feels as if they have delivered.

Measurement is an afterthought to the purchase of a tool. Tools are just that…tools. Completely useless tied to strategic objectives. It is a matter of who is going the distance and who is going to question the results.

Looking forward to follow-ups of this post…

Lauren Vargas
Community Manager at Radian6
@VargasL

@AnnaOBrien
Thanks to @justinkistner for pointing me to your site. Nice work being done here! I look forward to follow-up posts.

I’ll be the first to admit that I have a lot to learn when it comes to measurement and social media. But this kind of discussion is in some ways antithetical to the notion of progress being made in the move from old methods of communication to new.

Measurement used to be part of polling and data collection when it was not really feasible to put a question to an entire community unless you were holding an election and had the money to do so.

Today you can ask a community a question publicly and “listen” to the response by collecting and filtering through online channels. The answer you get may not be what you expect, or what you want to hear.

The real test of an organization’s or company’s character is what it does with that answer.

It’s like the difference between finding out someone’s position on a topic from a third party so that you can say what they want to hear and thereby manipulate them, versus asking them directly and listening (or not listening) to the response.

Most corporations are pushing an agenda – namely, putting profits above all else (think externalities, things corporations won’t worry about unless forced to do so by an outside agency or group) – so they fall into the former camp. Measurement becomes a lot less complicated when you are willing to ask a question directly and listen to the answer.

I have written extensively on this subject and agree with you.

HOWEVER …

Don’t forget to keep a real-world perspective on things. Measurement can be very difficult and expensive. For a small company or non-profit, it may be impossible. In those cases, “monitoring” might be a very appropriate strategy, especially if the cost of engagement is near-zero to begin with.

Over time, the ability to measure will go up and the cost will come down, but in the meantime, “monitoring” is an appropriate strategy for some institutions, as long as they don’t try to pass it off as ROI.

@MarkSchaffer I am not implying that monitoring is not valuable or that measurement takes precedent over it. And I agree with your point, for most situations (esp. applicable to small businesses) you don’t need a lot of complex analytics. Like you say, “monitoring is an appropriate strategy for some institutions, as long as they don’t try to pass it off as ROI.”

@MarkMclaren I wanted to progress on the above and address your issue that this post is “antithetical to the notion of progress being made in the move from old methods of communication to new.”

Let me be blunt, many people see quantitative analytics as primarily based on survey data. Traditionally that has played a part in some aspects, but there are multitudes of other types of data that can be used in advanced measurement. Some types of social media measurement, specifically neuro analytics & sentiment analysis, feed off the valuable customer feedback data you mentioned.

On the small scale it’s easy to see what your loyal brand followers think of your brand. However, major brands are struggling to manage the floods of consumer feedback they receive daily; as time passes the quantity of available data is only going to grow. We have to create a feasible way to address these needs.

Last but not least, let me say this once and for all. I am not valuing one side of this chart more than the other. I am simply saying they are different.

Anna, love this post. It begs the question what are the theories in the theory-based section? Some people say it’s social psychologists who most study influence so the best ideas will likely come from that field.

Taking a step back, part of the problem is that the marketers involved are focused on micro channels and not the big picture, in addition to not being very quant-oriented in the first place. Kevin Hillstrom (follow @minethatdata) has a great post on the mindset difference between Catalog Marketers (big picture P&L) and Online Marketers (focused too much on each micro channel and the next thing):

http://minethatdata.com/blog/2009/09/online-marketers-vs-catalog-marketers.html

Great to see your tweets move to the blogosphere.

Anna — agree that “reputation” has been at least co-opted into a buzzword in some circles, but in ongoing brand research that many companies do, it’s a relatively simple matter to include both types of media in the inputs set of questions. Yes, this does depend on surveys of one kind or another, and occasionally qualitative interviews.

As for adding revenue and cost reduction to the forecasting section, that may be my bad — the word “forecast” is throwing me. In establishing the program, I’d think the benchmark needed to be set first — a retrospective look at whether there are any relationships between increased revenue, decreased costs and our various communication activities. That might set the expectations for potential impact, and inform the forecast model (maybe that’s what your talkin’ about?!)

The sentiment analysis might reveal patterns of thinking among, for example, commenters or tweeters, but do those comments or tweet affect the sentiment of others? That’s what I mean be looking for impact on reputation.

Mark is right — getting this type of measurement right can be costly, which is why I always start with objectives before recommending courses of action.

Good discussion!
Sean
@commammo

AMEN…AMEN…AMEN!!!! It’s in the secret sauce…the delicate balance of all three – Monitoring, Reporting, Measurement – to get to the strategic goal oriented objectives. Present and Future.

Thanks for taking the time to really dissect this and then share! BRAVO!

I would like to thank you for this excellent post, that is helping all of us to push things forward.

Since Ben Kunz from Mediassociates invited me to answer to this post, i did an extensive answer on my Posterous here:
http://branislavperic.posterous.com/my-take-on-the-social-media-monitoringmeasure

This is great. NOW can we get some Truth in Advertising rules out there. I am constantly pitched by monitoring companies that advertise their “measurement services” and claim to “measure reputation.” These people are capitalizing on the interest in measurement but intentionally misleading people and contributing to the confusion in the marketplace. Thanks for doing your part to educate that marketplace.

Thank you for starting this conversation in such an astute way.

I’m looking forward to, more than anything else, your delineation of the terms intuition, needs, theory, monitoring, measurement, reporting, and management.

Keeping it simple:
WOW!