Many panels at the conferences I go to end up being drab affairs with softball questions, way too much agreement and way too little insight. Which why I was pleasantly surprised by the Measuring Conversational Media panel at the conversational marketing summit yesterday. The panelists were:
- James Lamberti, SVP, Search & Technology, comScore
- Steve Rubel, SVP & Director of Insights, Edelman Digital
- Don Springer, President & CEO, Collective Intellect
Debra Aho Williamson from eMarketer, who was the moderator, didn’t have to ask too many hard questions as the panelists were more than happy to disagree with each other. There were three key issues:
- The difference between measurement and insight
- Approaches to measurement
- The role of standards
“Measurement doesn’t necessarily lead to insight” was a point Steve Rubel made a few times. He was implying that much of the new-fangled data collection and measurement focused on social & conversational media brought little value and insight to brand marketers. That may be true in general but in fast-changing environments it is rather difficult to say a priori what data one must collect and what analysis one must perform to achieve insight. Insight tends to be serendipitous and in my experience I’ve found that serendipity tends to correlate well with data availability. It’s hard to see what you can’t look at.
comScore’s Lamberti was emphatic that measurement doesn’t need to be reinvented. Although conversational media and traditional media are rather different on a number of axes–reach vs. engagement campaigns, large vs. low volume, targeting value vs. targeting advocates, pushed messaging vs. dialog, etc.–the fundamental measurement is the same. Don Springer’s take was that one does, in fact, need fundamental advancements in measurenment in order to track key metrics such as number of conversations, share of voice and sentiment change. I think I fall on the side of Springer here. What’s the equivalent of sentiment change measurement in traditional Web analytics? Tracking star ratings or equivalents is insufficient since they won’t tell you want a blogger is thinking about a product or brand.
Steve Rubel and Don Springer really got into it when they discussed standards. Steve’s take was that w/o standards much of the measurement conversations are meaningless as they are about apples vs. oranges. At one point Don said “standards stifle innovation.” Both positions have truth in them but that’s topic for a longer post. I have to catch a flight back to Boston.
Update: the panel was also covered by PaidContent.
I have found that creating a “swarm” of data metrics around the topic of what is most important to my business (paid subscriptions) has been helpful. Some metrics are directly related, others less so. But insights do come from making unexpected connections between different data sets. Interesting and insightful correlations can come out of some pretty unusual places.
Have to agree with Steve Rubel’s comment that, “Measurement doesn’t necessarily lead to insight”. Often measurements can lead to missed opportunities due to misinterpretation.
But what’s the alternative? No measurement?
Don, this seems like a smart strategy. My econometrics professor said “data speak to you at 3am.” He was right about that, in addition to “data are plural.” Sometimes insight requires a bit of nudging. Data can help with that. The challenge is not letting the existing data and measurement system constrain the thinking.
On the basis of your insights above I can see where James Lamberti is coming from. Okay, so much of it is new and traditioanl media measurement techniques can’t process the impact of Twitter, YouTube etc. But its the same people, the same audience, just doing things differently.
I wonder if there is no effective solution at present. I don’t think the market has any particularly capable solution and I am not sure the users are educated enough to recognise it even if it did. In 2-4 year when the semantic solutions begin to roll it might be very different.
Measurement and data do not always lead to insight, I agree.
But no measurement and no data always lead to no insight. Period.
(Or, sorry, I forgot that certain brand gurus get insight form divine enlightenment. The rest of us mortals need neasurement and data).
It’s what you measure, how you measure it and, not least, what you do with all the data at hand – that determines insight (or the lack of it).
As to standards and innovation, Picasso has said it best: “By all means, break all the rules – as soon as you know them!” He could paint a square face with 3 eyes and that was innovation, because he could draw a perfectly realistic, academic portrait before that. To break rules, there must be something to break in the first place, a departure point, a coordinate system.
In this sense, standards enable innovation, in addition to other mundane but vital practicality they are needed for.
Just MHO, it’s others’ sovereign right to differ 🙂