Is There Really Something Wrong With CGM Research?

Following is my latest OnlineSpin column in MediaPost, which is a response to public comments made by Bill Neal, a highly respected market research expert, about consumer-generated media. For me, this piece was uncharacteristically defensive, but I feel it is important to clarify for the record what I believe are misguided comments about a subject I’m passionate about and am submersed in professionally. I hope that Bill will accept my criticism as healthy feedback, and consider continuing a constructive conversation with me and others. I also encourage further public debate here on AttentionMax, as well as on the OnlineSpin blog, which has begun here. Here’s my piece:

Is There Really Something Wrong With CGM Research? 

By Max Kalehoff, June 16, 2006

Toby Bloomberg recently published a Q&A with Bill Neal, the “Godfather of Marketing Research,” about the role of consumer-generated media in market research. Bill is an accomplished researcher, and surely smarter and more experienced than me. Despite his credentials, his assertions unfortunately reveal an absence of even the most basic understanding of CGM.

I will address some of these misguided assertions, but I must first acknowledge my own biases: I’m a huge proponent of CGM research and an occasional pundit on the limitations of “pure scientific, white-glove social research.” I also make my living at a firm that measures and analyzes CGM.

With my disclosures made, here are some of Bill’s comments on CGM, along with my feedback:

1. CGM Is Not Representative

“In many ways it [CGM] combines the worst elements of non-scientific research – self selection and advocacy – both positive and negative. That is, those out there in the Internet world who are generating their own media are self-motivated to do so and are not representative of any defined population of buyers. And, given the fact that they have taken a public position on a particular product or service, it means that they more often than not have exceptional or non-typical attitudes about those products and services…”

CGM is biased by self-selection and advocacy, and that’s precisely why it’s so valuable! CGM creators are, by definition, a highly defined population of hand-raisers–the ones so engaged with a category or product that they talk about it without prompting. These are the most elusive individuals, though highly valuable. They are your super consumers, your early adopters, your extreme loyalists and detractors, who have disproportionate influence on broader discussion and ideation. The argument that CGM is weak because it happens to be effective at identifying powerful stakeholders outside of the average is without merit. The problem with marketing today is that it too often constricts itself to the average, where findings are diluted, insightful nuances ignored and early indicators forgotten.

2. CGM Is Neither Credible Nor Reliable

“The information they [consumers] generate may be true, or not true – there is no way to discern which. Therefore, the information generated by those folks is neither credible nor reliable.”

Readers, participants and researchers in social media determine CGM credibility, reliability and influence according to a huge range of factors. For example: Am I familiar with the person or his affiliation? Is that person known to be trustworthy? How often does that person speak, and how often do others in the community respond? How often do others seek out someone precisely because that person is perceived as credible? How often is that person linked or referred? Are there norms and benchmarks, and historical indicators of trust?

Moreover, truthfulness often is irrelevant. Why? Because CGM acts just like media! It’s become one of the most prolific sources of online content. It’s attractive to search engines because of its robust text and numerous links, thereby enabling passionate information seekers and speakers to find one another. CGM tends to be compelling, refreshing and believable, and it is successfully competing for attention against all the other messages marketers continue to throw against the wall. Perceived credibility is why e-commerce sales increase when products are juxtaposed with consumer reviews.

3. CGM Includes Viral Discussion Which Marketers May Not Like

“But marketers must keep in mind that a few influencers can generate a great number of product mentions if they decide to feature a particular product or service in their blogs. And these things can get out of hand very quickly, signaling a problem that’s really not a problem to the vast majority of customers.”

What matters is not whether things get out of hand, but that there are influential consumers creating CGM, who spark ongoing, viral dialogue around products or services. This is CGM acting not only as media, but as influential news media.

4. I Don’t Know A Lot About CGM Research Firms, But I Do Know The Problems

“I don’t know a lot about them [CGM research firms] and have not used them in my consulting practice…But to the best of my understanding, they are primarily counting product/service mentions and, in some cases identifying the major sources of those mentions…I’ve already talked about the problems with simply counting the number of brand ‘hits’ and how that can be so misleading.”

CGM research firms are analyzing billions of public discussions to deliver a wide range of quantitative and qualitative research applicable in nearly every marketing situation. These measurements go far beyond product mentions and brand hits, and do dive deep into sophisticated consumer segmentation and insights, social influence mapping, and media measurements.


Published by Max Kalehoff

Father, sailor and marketing executive.

Leave a comment